Kristian Nielsen <knielsen@knielsen-hq.org> writes:
Stewart Smith <stewart@flamingspork.com> writes:
Idea: if txn touches more than one database, record as metadata the position of other db, so slave just has to wait for at least that id to be applied in other database before applying this txn in this source. This seems like a relatively simple solution to the problem (unless i've missed something).
Yes, this sounds like an interesting idea. More generally, for a given transaction with some global transaction ID, we can optionally log as extra metadata a list of other global transaction IDs that slave must wait for before executing this transaction.
This would allow master to write information about dependencies between transactions in the same stream. This could be used to help pipeline and parallelise slave transaction execution within the same stream. Basically, you start applying a batch of transactions at once and just COMMIT in the correct order. If you know they're non conflicting, this should all go fine. If they are conflicting, then try again but in series. This (of course) works a whole lot better when you get rid of statement based replication. (this is what i've been advocating for a while, and it's possible to do entirely on the slave, but the master could help a bit if it has any knowledge).
In your example, there would be one element in the list, the global transaction ID of the last transaction touching "other db". And it could be used for lots of different other cases as well. Seems kind of nice.
Yeah... it seems like a decent idea, I've been meaning to implement something like it for a few years now... a million and one other more important/urgent patches :) -- Stewart Smith