Re: [Maria-developers] Interaction between rpl_slave_state and rpl_binlog_state
Andrei Elkin <andrei.elkin@mariadb.com> writes:
I would also raise another but relevant topic of maintaining
gtid_"executed"_pos
which is an union of all GTID executed regardless of their arrival method. E.g some of foreign (to the recipient server) domains gtid:s may
The master potentially could be (made) interested in such table should create a replication mode without necessary binlogging on the server.
Sorry, I don't follow. Do you mean to create a new table of GTID positions, which will be updated by all transactions, whether originating locally or replicated by a slave thread? Or do you mean to have the mysql.gtid_slave_pos table updated also by locally originated transactions? Or do you mean to have a new system variable @@gtid_executed_pos which is constructed from the existing binlog and mysql.gtid_slave_pos_table, similar to @@gtid_current_pos, but maybe including more GTIDs somehow?
This is not really something new as it's exactly how mysql implemented gtid bookkeeping.
I think MySQL originally kept track of GTIDs only in the binlog, right? And binlog was required to be enabled for GTID to work? I do remember seeing something that relaxed this requirement, but I haven't followed the details. - Kristian.
I would also raise another but relevant topic of maintaining
gtid_"executed"_pos
which is an union of all GTID executed regardless of their arrival method. E.g some of foreign (to the recipient server) domains gtid:s may
The master potentially could be (made) interested in such table should create a replication mode without necessary binlogging on the server.
Sorry, I don't follow.
My fault, sorry! I should've referred straight to synchronous replication ... where master maintains an "immediate" (without binlog proxy) constant connection to slaves (when it breaks the slave would have to take snapshot, or find a binlog service e.g on some other slave). [In such a case the master connection would still collect transaction statement till prepare or commit to send them to slaves at those points.] So here replicated transactions need identification on the binlog-less master to display its execution status (which would matter when configuration changes and a new master is reelected). I am not dwelling into specifics too deep now, it's just to exemplify such use case.
Do you mean to create a new table of GTID positions, which will be updated by all transactions, whether originating locally or replicated by a slave thread?
This one.
Or do you mean to have the mysql.gtid_slave_pos table updated also by locally originated transactions?
Or do you mean to have a new system variable @@gtid_executed_pos which is constructed from the existing binlog and mysql.gtid_slave_pos_table, similar to @@gtid_current_pos, but maybe including more GTIDs somehow?
This is not really something new as it's exactly how mysql implemented gtid bookkeeping.
I think MySQL originally kept track of GTIDs only in the binlog, right? And binlog was required to be enabled for GTID to work?
True.
I do remember seeing something that relaxed this requirement, but I haven't followed the details.
Like we update `gtid_slave_pos` on slave by the slave applier on the binlog-less server, so they keep updating a general `gtid executed` table by any incl the user connection and slave applier. Cheers, Andrei
andrei.elkin@pp.inet.fi writes:
where master maintains an "immediate" (without binlog proxy) constant connection to slaves (when it breaks the slave would have to take snapshot, or find a binlog service e.g on some other slave). [In such a case the master connection would still collect transaction statement till prepare or commit to send them to slaves at those points.] So here replicated transactions need identification on the binlog-less master to display its execution status (which would matter when configuration changes and a new master is reelected).
Aha, so delegate the binlogging task to a slave. I guess there needs to be a binlog *somewhere* for GTID to make sense. I wonder if it would be possible to avoid introducing a new table (for example updating the mysql.gtid_slave_pos table on the master). Having two different places (binlog and mysql.gtid_slave_pos) is already a source of considerable complexity in the MariaDB GTID, something I would have liked to avoid. But I felt that it is was really important to allow a slave to use GTID without --log-slave-updates (hence the need for the table), and also that adding a mandatory extra table operation to every transaction on the master was unacceptable (hence the binlog). Another fundamental design detail of MariaDB GTID is that it supports out-of-order sequence numbers in the binlog. Again, this is a source of considerable complexity. This was needed so that 10.0 could enable GTID by default on the master, so that a simple CHANGE MASTER is sufficient to start using GTID after an update. For example, an existing master-master ring that is upgraded to 10.0 will not have gtid_domain_id set and thus will generate out-of-order GTIDs in the binlog. The implementation is very carefully designed to always be able to find the correct binlog position from a GTID even in case of out-of-order sequence numbers. A consequence is that in MariaDB, it is not in general possible to compare GTIDs without reference to the binlog. This can also easily affect things like extending gtid_current_pos or this idea of binlogless master. I wonder how well understood this generally is by users and developers - and if not, it is a good sign that it is too complicated... My point is - maybe it would be an idea to introduce a fully-strict GTID mode now. In fully-strict mode, GTIDs are guaranteed to be in-order (within each domain) in every binlog in the system. Ie. there would be a marker at the start of every binlog file (GTID_LIST perhaps) that this server is fully-strict and any attempt to disable fully-strict (without CHANGE MASTER) or connect a fully-strict slave would give an error. Then in fully-strict mode sequence numbers are strictly monotonic (within a domain), and GTIDs can be compared simply on sequence numbers. And then a lot of more advanced GTID features can be implemented in a simpler and more intuitive (for the user) way. For example, having GTIDs stored in multiple places is less of a problem, since unification is trivial by just taking the max sequencenumber within each domain. - Kristian.
participants (2)
-
andrei.elkin＠pp.inet.fi -
Kristian Nielsen