Ian Gilfillan <ian@mariadb.org> writes:
In order to document this properly (and in response to
Great! Thanks for looking into this.
https://jira.mariadb.org/browse/MDEV-13236) , I want to clarify a few things:
The main documentation is here, I'm sure it can be improved or made more visible: https://mariadb.com/kb/en/mariadb/mysqlgtid_slave_pos-table/
As I understand it: * the variable is dynamic, so changes becomes effective immediately
Yes. Note that changing it requires that all slaves are stopped. So in that sense, it only takes effect once slaves are running again.
* tables are created immediately whenever a transaction is seen from a specified engine (listed in the variable)
Yes, for some value of "immediately". As mentioned in https://mariadb.com/kb/en/mariadb/mysqlgtid_slave_pos-table/, the creation is asynchroneous. What this means is that, the need to create a table is discovered inside record_gtid(), which is deep in the replication of the transaction when it is time to commit it. It is not a good place to try to create a table. So the transaction continues without the new table, and the table creation is started in a background thread. So most likely the table will only be available a bit later, after that transaction (and possibly a few other) have been replicated. Usually, the table should be created in a fraction of a second. Also note that thanks to two-phase commit, the system is behaving correctly (in terms of crash safety and consistency) also for the transactions that complete prior to the new table being available.
* the manual creation of the tables was an initial design suggestion, and there should be no need to ever create a table manually.
Yes, generally there should be no need to ever create a table manually. The initial table is created by mysql_install_db (or mysql_upgrade), and --gtid-pos-auto-engines can create any other tables required. The one situation I can think of where manual action makes sense is as explained in the documentation, in a system using primarily MyISAM, but the database was created as InnoDB. In that case it can make sense to ALTER the table to MyISAM (but not to create a table manually).
I don't have 10.3.1 running yet to test, but some questions: * what happens if slave threads are not stopped before changing the variable? Just delay the creation of the table, or can something unpleasant happen from a table being created in the middle of something?
As explained in the documentation, this results in an error that slaves must be stopped before changing the variable, similar to many other replication variables.
* what happens if a storage engine is later removed from the variable? If the table still exists, I presume the existence of the table takes priority and the table is still used - the variable is just used to check the existence of a table and create it if needs be, not determine its use?
Yes. The variable only concerns the _creation_ of the table. The _use_ of the table is automatic whenever the table is there.
* what benefit does the variable being empty (the default) provide? I saw the comment about the principle of least surprise, but in what way could it being populated and the tables existing be negative? In other words, why would one not simply recommend setting this immediately?
As I wrote before, I do not have any strong opinions on this. The feature is designed to work well in all cases. Generally a default of --gtid-pos-auto-engines=innodb,tokudb,rocksdb should be beneficial. It was always a sore spot in the original GTID implementation that there was no really good answer to which engine the mysql.gtid_slave_pos table should be created with, and it has caused real problem as well. If the mysql.gtid_slave_pos table was created as MyISAM, and --gtid-pos-auto-engines=innodb,tokudb,rocksdb set as default, then there will be a decent solution to this issue. There are also some arguments in favour of an empty default, at least initially until more testing has been done. MariaDB replication is extremely complex code, there can be bugs. As you say, there is the principle of least surprise. New tables magically springing into life during replication does have some element of unpleasant surprise. And do not forget that this feature is mostly/only useful to users that want to use TokuDB and RocksDB (both still relatively new), _and_ at the same time use InnoDB, in a performance-critical setting. Not a common usecase. Technically, setting a non-empty default is just putting the right default in sql/mysqld.cc, the code is already written to handle it. The current empty default is based on the feedback from the users who took an interest in the feature so far.
* If there is a benefit to it being empty (and therefore no multiple tables created), will there be a mechanism to remove a table beyond manually deleting?
There is currently no such mechanism, and there should not be a need to remove a table, I think. There _is_ code that tries to correcly handle if a table is manually removed. Such removal is best done with slave threads stopped. And the user would be responsible that the rows in the table are either no longer needed, or alternatively that they are migrated to the other tables. Generally, it should not be recommended to ever do such manual removal, just like it is not a good idea to manually remove or otherwise mess with other tables in the mysql schema. That is why it is not documented - just like the effect of removing other mysql.* tables is not documented. I hope this helps. Feel free to ask again if you have more questions or if something was unclear, and thanks for helping improve the documentation. - Kristian.
On 03/07/2017 15:15, Kristian Nielsen wrote:
I have now pushed the code to 10.3. It should appear in an upcoming MariaDB 10.3.1 release, IIUC.
Following the discussion so far, the default for --gtid-pos-auto-engines is currently empty. It can be easily changed later (eg. to innodb,tokudb,rocksdb) simply by changing the default value in sql/mysqld.cc. The code will allow this default setting even if eg. tokudb or rocksdb is not enabled.
(There might be a need to adjust a couple test cases that specially manipulates mysql.gtid_slave_pos in devious ways).
Let me know in case of any problems or if there are comments/questions.
- Kristian.
_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp