- developers - lists.mariadb.org

[Maria-developers] Standard way for time->datetime cast
by Alexander Barkov 06 Mar '14

06 Mar '14

Hi Sergei, (my mailer failed during the previous attempt. writing again, sorry if you get this twice) Please review the patch implementing the standard (and MySQL-5.6) compatible way of time to datetime conversion. Thanks.

2 3

[Maria-developers] Parallel replication benchmarks
by Kristian Nielsen 06 Mar '14

06 Mar '14

I've done a set of benchmarks for parallel replication on the same machine I used previously for my group commit benchmarks, http://kristiannielsen.livejournal.com/16382.html The code tested is the newest code in the bzr repository and what will be in 10.0.9 (this is significantly improved from what is in 10.0.8). I plan to write up a blog post about it in a couple of days with nice graphs, but meanwhile Axel asked me to summarise in this mail. I tested with sysbench 0.5, using oltp.lua (medium-sized transactions) and update_index.lua (minimal transactions with just a single primary-key update per transaction). I used 10M rows, 16GB buffer pool and 2 * 1.9 GB redo logs. This is with a single table. I tested simply by preparing the binlog on the master, then setting up an already prepared slave and doing START SLAVE UNTIL the end of the log. The error log then shows the time spent for the slave to catch up. I tested everyting in GTID mode, as that is the recommended mode for parallel replication (though my guess is that old-style replication will be much the same, there isn't much difference in the code between what is done to actually execute events). Node that these tests are for in-order parallel replication. All commits on all slaves happen in the same order as on the master; the use of parallel replication is invisible to applications. This is in contrast to eg. MySQL 5.6 multi-threaded slave, which requires the application to partition its data into independent schemas. Here are the prelimiary results, in number of seconds for the slave to catch up (lower is better) versus number of threads (--slave-parallel-threads, 0 means not using parallel replication): For oltp.lua. 48 threads used to generate the load on the master, and --binlog-commit-wait-count=12 --binlog-commit-wait-usec=10000 to allow the server to delay a commit by up to 10 milliseconds in order to get more group commit and thus more opportunities for parallel apply on the slave: A: --log-slave-updates --sync-binlog=1 --innodb-flush-log-at-trx-commit=1 B: --skip-log-slave-updates --innodb-flush-log-at-trx-commit=1 C: --skip-log-bin --innodb-flush-log-at-trx-commit=2 D: --log-bin=master-bin --sync-binlog=0 --innodb-flush-log-at-trx-commit=0 #thr A B C D 0 1065 869 193 202 2 361 432 147 161 4 221 264 118 121 8 135 177 103 107 12 114 153 104 105 16 109 140 104 107 24 111 139 107 105 32 111 136 99 109 48 111 126 108 109 64 111 121 99 111 We see here a 2-10 times speedup from parallel replication. The master has around 12 transactions in every group commit, which provides good opportunities for parallelism on the slave. Note that parallel replication is especially effective when the binlog is enabled and crash-safe (--sync-binlog=1 --innodb-flush-log-at-trx-commit=1). This is because parallel replication can run the commit of one transaction in parallel with any other transaction, even if the two transactions would otherwise conflict. This makes group commit especially effective. In fact, this manages to more or less completely eliminate any penalty for enabling crash-safe binlog on the slave, which is quite nice. Note also that disabling the binlog actually tends to make things _slower_, not faster, when using parallel replication. I believe this is due to MDEV-5802, which may be worth fixing for 10.0. Here are results for update_simple.lua with 48 threads on the master. This produced around 13 transactions per group commit on the master, with no --binlog-commit-wait-count to delay commits: A: --log-slave-updates --sync-binlog=1 --innodb-flush-log-at-trx-commit=1 B: --skip-log-slave-updates --innodb-flush-log-at-trx-commit=1 C: --skip-log-bin --innodb-flush-log-at-trx-commit=2 #thr A B C 0 931 899 271 2 546 653 258 4 365 494 176 8 261 365 203 12 247 350 197 16 233 336 207 24 242 316 209 32 237 292 194 48 235 270 208 64 228 249 195 Again we get a good speedup from parallel replication, even though with such small transactions, there is less opportunity for improvement, as the actual work for transactions is rather small compared to the overhead for managing the replication of each event. And again, the ability to utilise group commit effectively provides the biggest benefit. Finally, I tried a test of update_index.lua where I ran the load on the master single-threadedly. This creates a binlog with _no_ opportunities for parallelism from group commits - each transaction needs to be executed on its own by the slave, as we do not know for sure that they will not conflict on row locks. However, due to the possibility to run the commits in parallel (and hence get group commit on the slave), we still see some speedup even here when --sync-binlog=1 and --innodb-flush-log-at-trx-commit=1. When binlog and innodb sync is disabled, parallel replication makes things slower due to the overhead of thread communication: A: --log-slave-updates --sync-binlog=1 --innodb-flush-log-at-trx-commit=1 B: --skip-log-slave-updates --innodb-flush-log-at-trx-commit=1 C: --skip-log-bin --innodb-flush-log-at-trx-commit=2 #thr A B C 0 1075 949 270 2 597 673 319 4 443 623 334 8 407 588 349 12 393 544 336 16 391 536 352 24 - 492 336 32 389 472 358 48 389 419 344 64 391 399 354 So overall, results look very good, especially for slaves with binlog enabled. (And binlog disabled could turn out better if MDEV-5802 is fixed). Let me know if there are any questions, and I'll be happy to answer them. - Kristian.

1 0

[Maria-developers] A problem of DDL apply in parallel replication
by nanyi607rao 06 Mar '14

06 Mar '14

hi kristian , Maybe you have encountered this problem and fixed it, that is a DDL statement Query event has a Gtid event ahead, but has no end_event (like xid_event, commit or rollback) behind. so in the latest codes of parallel replication, count_queued_event_groups will increase but count_committing_event_groups will not for DDL. it would lead next event_group wait forever I guess. 2014-03-05 nanyi607rao

2 2

Re: [Maria-developers] gsoc replication tasks
by Kristian Nielsen 06 Mar '14

06 Mar '14

Sergei Golubchik <serg(a)mariadb.org> writes: > I'd like to discuss what replication tasks we might want to be > implemented in this year's google summer of code. I don't really have the bandwidth for handling any GSoC at the moment. But here are a bunch of replication tasks that should be suitable for a newcomer, in approximate increasing order of difficulty: MDEV-4776 FR: Consider showing GTID-based 'until' value in slave status MDEV-5119 Events from a GTID-incapable master inherit domain ID of the slave MDEV-4987 Sort by domain_id when list of GTIDs are output MDEV-5289 master server starts slave parallel threads MDEV-4989 Support for GTID in mysqlbinlog MDEV-4990 Implement simple failover script using GTID MDEV-5262 Missing retry after temp error in parallel replication I would be happy to assist anyone that wants to work seriously on one of these, GSoC or not. I'm available on IRC now, and most of the day if you want to discuss more... - Kristian.

1 0

Re: [Maria-developers] [JIRA] (MDEV-5497) Test suite test connect.fix fails
by Alexander Barkov 05 Mar '14

05 Mar '14

Hello Olivier, You changed status to "fixed", but I cannot see any fixes in 10.0-connect. Are you going to push a fix soon? Thanks. On 03/05/2014 03:36 PM, Olivier Bertrand (JIRA) wrote: > > [ https://mariadb.atlassian.net/browse/MDEV-5497?page=com.atlassian.jira.plug… ] > > Olivier Bertrand commented on MDEV-5497: > ---------------------------------------- > > Sorry Elena, I did not read your comment entirely before answering. Your analyze was quite good, just missing the reason why this was happening. > >> Test suite test connect.fix fails >> --------------------------------- >> >> Key: MDEV-5497 >> URL: https://mariadb.atlassian.net/browse/MDEV-5497 >> Project: MariaDB Development >> Issue Type: Bug >> Affects Versions: 10.0.7, 10.0.8 >> Environment: Slackware Linux 14.1 + kernel 3.12.6 >> Reporter: Duncan Roe >> Assignee: Olivier Bertrand >> Priority: Minor >> Fix For: 10.0.9 >> >> Time Spent: 1 hour >> Remaining Estimate: 0 minutes >> >> {noformat} >> cd /usr/mysql-test; ./mtr connect.fix >> {noformat} >> (as root) gives me an error report as below. I am keeping the mysql-test directory, so please let me know if you'd like anything else from it. >> {noformat} >> Logging: ./mtr connect.fix >> vardir: /usr/mysql-test/var >> Checking leftover processes... >> Removing old var directory... >> Creating var directory '/usr/mysql-test/var'... >> Checking supported features... >> MariaDB Version 10.0.7-MariaDB >> - SSL connections supported >> Collecting tests... >> Installing system database... >> Using server port 43317 >> ============================================================================== >> TEST RESULT TIME (ms) or COMMENT >> -------------------------------------------------------------------------- >> worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 >> worker[1] mysql-test-run: WARNING: running this script as _root_ will cause some tests to be skipped >> connect.fix [ fail ] >> Test ended at 2014-01-01 19:32:13 >> CURRENT_TEST: connect.fix >> --- /usr/mysql-test/plugin/connect/connect/r/fix.result 2013-12-23 20:35:03.000000000 +1100 >> +++ /usr/mysql-test/plugin/connect/connect/r/fix.reject 2014-01-01 19:32:13.000000000 +1100 >> @@ -80,7 +80,7 @@ >> name city birth hired >> John Boston 2 1986-01-05 2010-06-02 >> Henry Boston 0 1987-06-07 2008-04-01 >> -George San Jose 1 1981-08-01 2010-06-02 >> +George San Jose 1 1981-07-31 2010-06-02 >> Sam Chicago 2 1979-11-02 2007-10-10 >> James Dallas 1 1992-05-03 2009-12-14 >> Bill Boston 1 1986-09-01 2008-02-10 >> @@ -96,7 +96,7 @@ >> name city birth hired >> John Boston 2 1986-01-05 2010-06-02 >> Henry Boston 0 1987-06-07 2008-04-01 >> -George San Jose 1 1981-08-01 2010-06-02 >> +George San Jose 1 1981-07-31 2010-06-02 >> Sam Chicago 2 1979-11-02 2007-10-10 >> James Dallas 1 1992-05-03 2009-12-14 >> Bill Boston 1 1986-09-01 2008-02-10 >> @@ -112,7 +112,7 @@ >> name city birth hired >> John Boston 2 1986-01-05 2010-06-02 >> Henry Boston 0 1987-06-07 2008-04-01 >> -George San Jose 1 1981-08-01 2010-06-02 >> +George San Jose 1 1981-07-31 2010-06-02 >> Sam Chicago 2 1979-11-02 2007-10-10 >> James Dallas 1 1992-05-03 2009-12-14 >> Bill Boston 1 1986-09-01 2008-02-10 >> @@ -128,7 +128,7 @@ >> name city birth hired >> John Boston 2 1986-01-05 2010-06-02 >> Henry Boston 0 1987-06-07 2008-04-01 >> -George San Jose 1 1981-08-01 2010-06-02 >> +George San Jose 1 1981-07-31 2010-06-02 >> Sam Chicago 2 1979-11-02 2007-10-10 >> James Dallas 1 1992-05-03 2009-12-14 >> Bill Boston 1 1986-09-01 2008-02-10 >> mysqltest: Result content mismatch >> - saving '/usr/mysql-test/var/log/connect.fix/' to '/usr/mysql-test/var/log/connect.fix/' >> -------------------------------------------------------------------------- >> The servers were restarted 0 times >> Spent 0.000 of 6 seconds executing testcases >> Failure: Failed 1/1 tests, 0.00% were successful. >> Failing test(s): connect.fix >> The log files in var/log may give you some hint of what went wrong. >> If you want to report this error, please read first the documentation >> at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html >> mysql-test-run: *** ERROR: there were failing test cases >> {noformat} > > > > -- > This message was sent by Atlassian JIRA > (v6.2-OD-10-004-WN#6253) >

2 1

Re: [Maria-developers] Active Directory authentication plugin - GSoC 2014
by Sergei Golubchik 05 Mar '14

05 Mar '14

Hi, Talina! On Feb 28, talinashrotriya(a)gmail.com wrote: > Hello Sir, > > My name is Talina, presently studying Computer Science from India. > > I am interested to work on the “Active Directory authentication plugin”project. I am good at C/C++ , MYSQL and also have good knowledge about Active directory. Could you suggest me a few links to get started with? > Eager to start work. > > Thanks! We don't have this project on the https://mariadb.com/kb/en/google-summer-of-code-2014/ anymore. I will add it back when we have a mentor for it. Regards, Sergei

1 0

[Maria-developers] MariaDB & MySQL community event 3rd of April 2014 in Santa Clara
by Michael Widenius 05 Mar '14

05 Mar '14

Hi! I am happy to announce that the MariaDB Foundation is organising a MariaDB & MySQL community event in Santa Clara on Thursday the 3rd of April. The venue is the Hilton Santa Clara hotel, a short walk from the Percona Live 2014 event. You can find more information on my blog at: http://monty-says.blogspot.fi/2014/03/mariadb-mysql-community-event-2014-in… We already got some very interesting talks and speakers, but I would like to see even more talk proposals. It's ok to propose a talk that it's also hold at Percona Live. As this event is free, I expect a totally different group of attendees, so that should be fine! When we have got a nice list of proposed talks, we will post them here so that everyone has a chance of commenting of which talks should be accepted. (Sorry, but this time we don't have time to create a proper selection committee so we have to do this a bit rough) Personally I will talk about MariaDB 10.0, which should be declared stable by then. Hope to see many MariaDB developers and users in Santa Clara! You can sign up for the event and/or dinner at: http://www.eventbrite.com/e/mariadb-mysql-community-event-2014-tickets-1080… Regards, Monty PS: Feel free to discuss the event on this email list! We want to have as many as possible participating in making this the best possible MariaDB and MySQL event.

1 0

[Maria-developers] regarding the project in gsoc 2014
by nitin goel 05 Mar '14

05 Mar '14

Sir/Ma'am, I would like to bring to your notice that i used 'mysql' to drop and create tables to understand project better but would like to know more specifications for the project 'create or replace, create if not exists, and drop if exists'. I have been working in C/C++ since 2010 and would like to learn more from you. Please also mention how to contact the mentor for the project and proceed further on project. Thanks Nitin Goel

2 1

[Maria-developers] MariaDB slave_parallel_threads and MySQL slave_parallel_workers
by Rasmus Johansson 05 Mar '14

05 Mar '14

Hi, I received a question that I wanted to share here to get some insight on. MariaDB 10.0 has a global setting named "slave_parallel_threads" and MySQL 5.7 has the same parameter named "slave_parallel_workers". Should slave_parallel_workers be an alias to slave_parallel_threads in MariaDB 10.0 to be more compatible with MySQL? BR, Rasmus -- Rasmus Johansson, VP Engineering MariaDB | t: +358 50 499 9589 | Skype: ratzpo

2 1

[Maria-developers] [GSoC] memcached priorities
by Tan Tran 04 Mar '14

04 Mar '14

Hi all, I'm looking to do my GSoC project with MariaDB, specifically the suggestion to "Port the InnoDB memcached interface to MariaDB" on the page https://mariadb.com/kb/en/google-summer-of-code-2014/ . But the project description links to a JIRA bug at https://mariadb.atlassian.net/browse/MDEV-4674 , where the author suggests that we support memcached through TCP/UDP instead, with no mention of the InnoDB plugin. I figure that the InnoDB plugin is more important, since it's a MySQL 5.6 feature, it's widespread in industry, and the lack of a memcache interface is one of only a handful X's for MariaDB on http://www.skysql.com/blogs/max-mether/mysql-56-vs-mariadb-100 . Additionally, even without TCP/UDP support, one could just code the memcache logic manually anyway, whereas the InnoDB integration is more sophisticated and feature-rich and would be a good selling point for SkySQL. So I'd like to start working on a proposal. Can any devs weigh in on which aspect of memcache is more desirable for MariaDB? If it sounds likes a good, straightforward GSoC project that nobody else has claimed yet? Thanks, Tan Tran

1 0