- developers - lists.mariadb.org

Re: [Maria-developers] MariaDB 10.x GTID
by Kristian Nielsen 11 Apr '14

11 Apr '14

Jan Lindström <jplindst(a)mariadb.org> writes: > some background, lets assume we have two galera clusters that are > geographically distributed (lets say for example that cluster A is on Europe > and cluster B on USA). Inside a cluster all nodes are set up so that server_id > and domain_id are the same and we will use strict GTID mode (for a example on > cluster A server_id = 1 and domain_id =1 and in cluster B server_id = 2 and > domain_id = 2). In cluster we naturally use Galera replication. Additionally, > we have asynchronous replication (MySQL replication) between these clusters. > 1) We would like to have some special domain_id (i.e. GTID) for transactions > that are executed only on certain cluster e.g. on cluster A. This would mean > that there is some dynamic option to enable "local" transaction execution mode, > where transactions are executed on cluster A but not on cluster B. I know that > this kind of feature does not exists but is it even possible ? Basically, this > would mean for MySQL replication point of view a fact that if GTID is formed > like this special GTID, they are not replicated to slave, but they are written > to binlog. It sounds like what you want is some option on the slave that would cause it to ignore all GTIDs with some specific domain id, similar to how it is currently possible to filter on server id. That seems a reasonable request; such option does not exist currently though. Another option is to use the @@skip_replication variable. Set this for the sessions doing the "local" transactions. Then set replicate_events_marked_for_skip to either FILTER_ON_SLAVE or FILTER_ON_MASTER, as you like. This is specifically designed to allow to log certain transactions to the binlog but not replicate them, so it sounds like it is what you asked for. > 2) There will be some challenge in maintaining the sequence number for GTID > correctly: for global GTID purposes, we can just count commits, and skip every > local transaction commit, but, we must also initialize the sequence for every > node joining the cluster. In MySQL-5.6 this happens by copying binlog files > (only header part needed) to joining node. I wonder if same strategy works for > MGC-10, is MariaDB GTID is stored in binlog header ? Yes, the GTID is stored in the binlog header. It is also stored in a file master-bin.state. This file is read at server start to initialise the GTID sequence numbers. However, if the file is not found for any reason, the information is extracted from the last binlog file (the master-bin.state is just a cache). However, you need the whole binlog file (if master-bin.state is missing), not just the header. I am sure it must be the same for MySQL-5.6 - you need to scan to the end of the binlog file, as there may be extra GTIDs logged in the file that were not there when the header was written. Of course, just after server start, the most recent binlog file is empty, so just the header works. It sounds like you would want to just copy over the master-bin.state file. However, this whole approach sounds wrong to me. The current GTID state is a cluster-wide property, not a local property, I do not understand why you want to maintain it locally. As I understand it, Galera decides on commit order. The certification process decides for parallel transactions T1 and T2 which one will be committed before the other. The GTID sequence has to match the commit order. So the GTIDs need to be generated by Galera, at the point in the code where the commit order is decided. This Galera code then also needs to do the binlog writing, so that binlog order matches GTID order and commit order. With Galera handling GTID number allocations, I do not understand why you would need to worry about binlog headers. But maybe I have misunderstood, I do not know Galera in details. Maybe you could explain how Galera ensures matching commit order, binlog order, and GTID order? Or alternatively, explain how things work without such ensurance? > 3) Independent table (in data dictionary) or file for replication filters. I > can change the replication filters dynamically, but if the server crash and i > don?t report to my.cnf it?s lost. Therefore, there is a need to store > replication filters persistently and these should be loaded automatically when > server is restarted. Currently we do not have this, replication filters are configuration options, and those are stored in the config file, not in dynamic tables or files. The server_id filters are set by CHANGE MASTER though, IIRC, so they are stored in master.info / relay_log.info. > 4) Dynamic binlog. Currently options are read-only. This we do not have. There has been talk of allowing it under global read lock; this might work, but it would require some work to carefully check all of the code for possible issues. There are a lot of places in the code where expensive synchronisation/locking is avoided based on the knowledge that the binlog can not be enabled or disabled on-the-fly. Hope this helps, - Kristian.

1 0

[Maria-developers] debugging MDEV-5932, wrong result set for tokudb
by Rich Prohaska 10 Apr '14

10 Apr '14

Hello, TokuDB generates an empty result set for a select with sub-select, but generated a non-empty result set for the same query when creating a new table. The result set should be the same in both cases. What is going on here? InnoDB computes an empty result set for both cases. I traced the problem to the tokudb::index_read function. The tokudb::index_read does not handle the null byte key prefix. Unfortunately, I do not understand how the null byte (byte 0) in the key handed to the ::index_read function is supposed to work. Most of the time, the value of this byte is 0. Sometimes it is set to 1. How should the storage engine work WRT this? BTW, this problem exists with TokuDB in MySQL 5.5.36, MySQL 5.6.16. I have not been able to consistently see this on MariaDB 5.5.36 for some reason. In any case, I am interested in how the handler's index_read function is supposed to process the null byte. Thanks

2 1

[Maria-developers] Buildbot machines
by Rasmus Johansson 09 Apr '14

09 Apr '14

Hi, I discussed both with Dbart and Serg last week about the situation with buildbot. Kristian and Elena also had similar discussion on IRC today. To sum it up, there seems to be two aspects to it; 1) more machines, i.e. slaves could/should be added immediately and 2) some development work could/should be done on buildbot to support some other ways of building, like automatically spinning up and down platform specific virtual machines in a public cloud somewhere whenever they are needed. I looked into alternatives for the 1) immediate need of more machines and thought. I found a pretty interesting alternative from OVH.COM, who offers dedicated servers to a very reasonable price. Here is a good example of what they can offer: https://www.ovh.com/us/dedicated-servers/infra/2014-EG-32.xml . I would be prepared to give it a try. Do you think the specs are good enough to help with the load buildbot currently has? BR, Rasmus -- Rasmus Johansson, VP Engineering MariaDB | t: +358 50 499 9589 | Skype: ratzpo

3 5

Re: [Maria-developers] [Commits] Rev 3991: MDEV-5138 Numerous test failures in "mtr --ps --embedded"
by Sergei Golubchik 09 Apr '14

09 Apr '14

Hi, Alexey! On Apr 02, Alexey Botchkov wrote: > > Still, another question, why did you have to rename net_store_data() to > > net_store_data_cs() ? > > That's personal. > I really don't like functions with exactly same names - makes code harder to > understand and debug. > Do you belive it's better not to rename these here? That's up to you. Sometimes I create functions with the same name, sometimes I don't. For example, tdc_acquire_share() - there are few functions with the same name, because they do exactly the same. Actually, one of tdc_acquire_share() is a real function, others are inline convenience wrappers - they eventually call the "real" one. But tdc_acquire_share_shortlived() - while also only a convenience wrapper - has a different name to emphasize that it has different semantics. But anyway, that's your patch. If you'd like to rename - go ahead. Regards, Sergei

1 0

[Maria-developers] INT thread variables in mysql and mariadb (all versions as far as i know)
by Rich Prohaska 09 Apr '14

09 Apr '14

suppose that I have a THDVAR_INT variable with min = -1 and max = 1, and i set it to -1. how come select @@var returns 4294967295 instead of -1?

2 1

[Maria-developers] Error in "make install"
by sriram patil 09 Apr '14

09 Apr '14

Hello all, I have had built and installed the mariadb source code before, no hassles whatsoever. I was trying to install it again but getting the following error when trying to "make install" ....... -- Installing: /usr/local/mysql/mysql-test/././var/tmp -- Installing: /usr/local/mysql/mysql-test/././var/tmp/mysqld.2 -- Installing: /usr/local/mysql/mysql-test/././var/tmp/mysqld.1.sock CMake Error at mysql-test/cmake_install.cmake:36 (FILE): file INSTALL cannot copy file "/home/sriram/Downloads/mariadb/10.0/mysql-test/./var/tmp/mysqld.1.sock" to "/usr/local/mysql/mysql-test/././var/tmp/mysqld.1.sock". Call Stack (most recent call first): cmake_install.cmake:119 (INCLUDE) make: *** [install] Error 1 I have changed the owner and group of the folder "/usr/local/mysql", so root access is not an issue. I am using Ubuntu 12.04. Did anyone get such error while installing and was able to solve it? Thanks, Sriram

2 3

[Maria-developers] Vision for next 10.x minor releases
by Honza Horak 08 Apr '14

08 Apr '14

Hi guys, first, congratulations for the 10.0 release, well done! Even though it can look insane, I'm already wondering about next minor releases ;) Especially if you have already some idea about 10.1 and later. Specifically, I'm wondering if these releases will still be somehow tight to MySQL's release cycle (say 10.1 will be somehow following 5.7) or if you plan totally independent release schedule like slower/faster the release cadence? In other words, since MySQL changed only minor version and MariaDB bumped the major (we all know the reason), I'm wondering if something changed in thinking about major/minor versioning -- like if the difference between 10.0 and 10.1 will still be considered the same like between say 5.3 and 5.5. I don't expect you can give us any dates, but some expectations, if 10.1 can be expected in months or rather years? If that was already discussed somewhere, please, just point me to the correct place. Thanks, Honza

2 2

Re: [Maria-developers] Rev 4077: MDEV-4856 SQL_ERROR_LOG shows 1146 errors which didnt appear in mysql client
by Sergei Golubchik 07 Apr '14

07 Apr '14

Hi, Holyfoot! On Feb 22, holyfoot(a)askmonty.org wrote: > revno: 4077 > revision-id: holyfoot(a)askmonty.org-20140222173543-xkzlp5rpga41evur > parent: svoj(a)mariadb.org-20140218065405-vw3bfhvhrfjsc6hh > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-4856 > timestamp: Sat 2014-02-22 21:35:43 +0400 > message: > MDEV-4856 SQL_ERROR_LOG shows 1146 errors which didnt appear in mysql client. > The fill_schema_table() function used to call get_table_share() > for a table name in WHERE then clear the error list. That way > plugins receive the superfluous error notification if it happens > in it. Also the problem was that error handler didn't prevent > the suppressed error message from logging anyway as the logging > happens in THD::raise_condition before the handler call. > Fixed by adding the Warnings_only_error_handler before the call. > raise_condition() also fixed. The Internal_error_handler class > was exteded with virtual functions to handle storing the error > message. This looks too complex to me. Why would you extend the base Internal_error_handler class - no other error handler did that or needed that, fwiw. I'd simply push an error handler, store the first message there and access it in get_schema_tables_record() via THD::get_internal_handler(). Regards, Sergei

2 4

[Maria-developers] md5sum mariadb-10.0.10.tar.gz
by Jakub QB Dorňák 07 Apr '14

07 Apr '14

Hi guys, I am getting mariadb-10.0.10.tar.gz with md5 checksum of d41d8cd98f00b204e9800998ecf8427e, while it should be 14ce22b8197d4eae88d237776d47220f. Thank You for any help, QB

2 2

[Maria-developers] Getting started
by sriram patil 04 Apr '14

04 Apr '14

Hi Sergei, I have got myself familiarized with parsing and execution source code. I wanted to start working on the source code a little bit. But I could not find any open bugs on MariaDB launchpad. It would really help if you could give me a few open bugs or enhancements to start with. Thanks, Sriram

2 1