Re: [Maria-developers] MariaDB 10 and MaxScale binlog router

i don't honestly remember, but we tend to make a disable switch for all changes we do... will check once i get around to it...(have to clear backlog after conference visit now) /Jonas On Thu, Apr 16, 2015 at 8:10 PM, Jean-François Gagné < jeanfrancois.gagne@booking.com> wrote:

Hi Jonas, will it be possible to disable the creation/resource consumption of this index ? As I am planning to stick to file/position negotiation, I would like to avoid consuming resource consumption for features that I do not need. Many thanks,

Jean-François Gagné

On 2015-04-15 15:28, Jonas Oreland wrote:

thx for reminding me kristian about the gtid index patch!

/Jonas

On Fri, Apr 3, 2015 at 12:50 PM, Kristian Nielsen < knielsen@knielsen-hq.org> wrote:

...

Jean-François Gagné writes:

...

...

...

(2) SELECT binlog_gtid_pos('mst-bin.000001',310) This is used by the slave to obtain the correct GTID position corresponding to the position at which it is starting, when it is connecting in non-GTID mode.

When the slave has this information, it becomes easy for the DBA to switch to a new master using GTID:

STOP SLAVE; CHANGE MASTER TO master_host='new_master', master_use_gtid=slave_pos; START MASTER;

This works even if the slave was not using GTID mode prior to the CHANGE MASTER, thanks to that SELECT binlog_gtid_pos().

Is this really needed ?

Well, it's needed in the general case to be able to get the correct GTID position to automatically switch to a different master using GTID, as above.

It is not _really_ needed in the sense that the DBA can just manually SET GLOBAL gtid_slave_pos='<position>' instead. Or the DBA might not have any need for using GTID in the first place.

One of the primary goals of MariaDB GTID was to make it easy to start using it, that is why this was implemented.

...

My understanding is that the SQL_THREAD will remember the GTID of the last executed transaction, which make the GTID provided by "SELECT binlog_gtid_pos('mst-bin.000001',310)" quickly obsolete. This SELECT

...

There might be a subtility with multiple DomainIDs that I am missing: that SELECT might return the GTIDs of all write domains up to that position... Again, this needs to read all the binlog up to that

Correct. The GTID position in the general case has one GTID per replication domain id. And since some of those domains may have no replicated transactions for a long time, the binlog_gtid_pos() call is used to fetch the full position.

...

Moreover, I guess that the GTID returned by this function should not be the GTID of the transaction at this position, but the GTID of the previous transaction. This needs to read from the beginning of the binary logs (reading a binary log backward is not possible to my knowledge). If the binlog file size is 100 GB... (you can see my point I think). Also, if the previous position is not in the same

Yes, you are right, there will be a need to scan the most recent binlog. So there is the potential for a performance regression, more so with large binlog files and/or frequent slave connects.

The intention was to have an index on the binlog files so that the GTID position can be found quickly (both for the gtid_slave_pos() call in the non-GTID case, and for GTID connect). But pressure to get the feature out meant it was released without, and I agree that this was unfortunate.

Jonas Oreland said he has a patch already that implements this, and that will be contributed soon...

...

write domain as the current transaction, the transaction from the right write domain (and all the other transaction from the other write domains) must be found. This looks terribly inefficient.

...

write domain is in the header of the binlog). I would prefer to forbid the slave to use automatic positioning (with GTID) until it had read the header of the next binlog. Basically, I prefer to push

That could be reasonable.

The counter-argument is that we need binlog indexing anyway for GTID mode. And with binlog indexes, the overhead for binlog_gtid_pos() will be negligible. So we could avoid the complications of introducing new kinds of states of a slave ("has a valid GTID position" vs. "does not have a valid GTID position").

In any case, if this causes a performance regression in practice, we will find some solution. Waiting to get the position from the header of the next binlog as you suggested, or some option to disable the binlog_gtid_pos() call, or something.

...

The need for that "SELECT binlog_gtid_pos(...)" makes it very hard to implement the MariaDB Slave Protocol in the Binlog Server is a "simple" way. If it is not needed in the protocol, I would prefer to simplify the slave protocol than to complexify the Binlog Server.

The binlog server can probably just ignore that call. It should not cause problems - only the automatic switch to GTID mode will not work, but I think the binlog server does not support GTID anyway.

- Kristian.

_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp