- developers - lists.mariadb.org

[Maria-developers] Tools/suggestions for MySQL 5.6 merge
by Kristian Nielsen 06 Dec '11

06 Dec '11

I have been doing a bit of preparation/planning for the upcoming merge of MySQL 5.6. I think the 5.6 merge (and 5.5 merge) is very important for MariaDB. There is much stuff in there that we really need to make available to our users (and probably some better left out). If we fail to keep the merges going, it will leave the community in a very sorry situation; I do not see anyone else stepping up to this task. My idea is that we could merge MySQL 5.6 piece-by-piece. Several other developers mentioned this idea at the Athens meeting: - It makes it much easier to selectively merge or not merge stuff. For example there are probably large chunks of optimiser changes that we do not want to merge (or maybe merge only the testcases for). On the other hand we probably want all InnoDB stuff. - I am hopeful it makes the merge much more managable when done in smaller pieces. Large reorganisations of code (header refactoring, copyright changes) should be easier when not intermixed with other changes. Merges from MySQL 5.5 -> MySQL 5.6 are probably also easier to deal with when done in isolation (I assume we can get conflicts for these, though we already have the equivalent changes in MariaDB 5.5). - Easier to share the task between developers. One developer can do a single piece in a few days, then pass the baton when he needs to switch to other tasks (or just gets tired). Complex merges of large feature pushes can be null-merged to allow main merge to progress, and in parallel another developer can work on reworking the patch into the MariaDB code. For some changes I think we will even be able to do the actual merge in parallel. - Easier to plan and understand how far the merge is progressing. Eg. if we merge M weeks worth of 5.6 pushes every N days, that allows to estimate when we will catch up with Oracle development - Easier debugging of test failures; by keeping the trees green between pieces merged, new test failures will be known to be caused by relatively smaller sets of changes, simplifying debugging. I have done some work to facilitate this kind of piecewise merge. As it turns out, it is not trivial to determine suitable pieces to merge. Oracle developers use no discipline for the direction they do merges in, so the sequence of pushes into lp:mysql-server ("trunk") is not immediately apparent. So I took the time to reconstruct the list of pushes from reading through the history; this was not overly hard to do. I next wrote a script that scans the history based on this list of pushes, and further annotates each push with some useful information: - Size of diff for the patch - Number of changesets merged (if a merge) - Mark if this is a merge from MySQL 5.5 - One-line short comment for the push, either hand-crafted for selected items, or else extracted from the commit message. - Revision ID (including developer info) It should be fairly simple to scan this list from the bottom, collecting chunks of related or manageable pushes and merging them as one piece. Eg. A sequence of small pushes could be merged as one, or a sequence of pushes by one developer, or a single large feature push. Here is an example extract of such an annotated list: 0 2* tor.didriksen(a)oracle.com-20111101065343-ha3d79h1ayk9j7x9 [5.5] Null merge from 5.1/5.5 13 andrei.elkin(a)oracle.com-20111031135232-q13pfsaieo8ujekj Minor test adjustment 791 alexander.nozdrin(a)oracle.com-20111031115220-01la8vj7ssmcm0mc Bug#61392 0 2* tor.didriksen(a)oracle.com-20111031091354-8p6s03s1zxerrp8p [5.5] Null merge Bug#12406055 from 5.1/5.5 5380 15 jorgen.loland(a)oracle.com-20111031081122-dnufl9r4li0i3ni6 [opt] Larger merge from optimiser tree 134 2* andrei.elkin(a)oracle.com-20111028212219-cdcuh37i70k9o4ms [5.5] Merge Bug#56299 (?) from 5.5 451 2 andrei.elkin(a)oracle.com-20111028130851-mz4p2mg28y14wb7k merge from trunk to local tree. 43 sergey.glukhov(a)oracle.com-20111028094528-enre87osw9u4qjy7 [opt] Bug#32254 post-merge fix (optimiser) 48 tor.didriksen(a)oracle.com-20111028092548-a3gjubgmwuhdunfj Bug#11747423 post-merge fix 114 tor.didriksen(a)oracle.com-20111028091835-xqavfrxkc50gbsxq "Default value for sort_buffer_size changed from 2M to 256K" The first column is diff size, the second is number of changesets merged (or empty for non-merge changesets). A "*" marks pushes that are only merges from MySQL 5.5. I pushed the script to lp:mariadb-tools, in /mysql-5.6-merge-stuff/. It also includes the full output in 5.6-annotated.txt (it takes 15-20 minutes for bzr to chew through the history on a fast multi-core machine). I hope this will be useful for the merge. Comments/suggestions/questions very welcome! - Kristian.

1 0

[Maria-developers] MariaDB 5.1.60 and 5.2.10
by Daniel Bartholomew 06 Dec '11

06 Dec '11

All, MariaDB 5.1.60 is nearly ready to release. There are only a couple of mirrors which don't have the files. The changelog and release notes pages are here: * http://kb.askmonty.org/en/mariadb-5160-release-notes * http://kb.askmonty.org/en/mariadb-5160-changelog Unless something comes up, I will probably activate the download page for this release and 5.2.10 in a couple of hours. This will be followed by a combined announce email and so forth. Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

1 0

[Maria-developers] MariaDB 5.2.10
by Daniel Bartholomew 05 Dec '11

05 Dec '11

All, The MariaDB 5.2.10 release is just about ready to go. Most of the packages have been mirrored. We're still waiting on Windows packages (we're having some issues with the Windows build server), but they will hopefully be available soon. This brings up the question: Should we delay the release until we have Windows packages or add them after the fact? The Release Notes and Changelog for the release are here: - http://kb.askmonty.org/en/mariadb-5210-release-notes - http://kb.askmonty.org/en/mariadb-5210-changelog Please let me know ASAP if there is anything that should be mentioned in the Release Notes that currently is not (or jump in and add it directly to the page yourself). Serg: The PAM note in the release notes is pretty sparse. Can you add some more to it? Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

1 0

Re: [Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 3120: Added suppressions
by Sergei 03 Dec '11

03 Dec '11

Hi, Monty! Can you please explain how it is possible to have "different number rows in the log" ? On Dec 03, noreply(a)launchpad.net wrote: > ------------------------------------------------------------ > revno: 3120 > committer: Michael Widenius <monty(a)askmonty.org> > branch nick: maria-5.1 > timestamp: Sat 2011-12-03 20:29:15 +0200 > message: > Added suppressions > Fixed feedback_plugin_send to not generate a random number of lines. > modified: > mysql-test/t/feedback_plugin_send.test > mysql-test/valgrind.supp > support-files/compiler_warnings.supp > > === modified file 'mysql-test/t/feedback_plugin_send.test' > --- mysql-test/t/feedback_plugin_send.test 2011-11-13 12:28:35 +0000 > +++ mysql-test/t/feedback_plugin_send.test 2011-12-03 18:29:15 +0000 > @@ -17,7 +17,16 @@ > perl; > $log_error= $ENV{'MYSQLTEST_VARDIR'} . '/log/mysqld.1.err'; > open(LOG, '<', $log_error) or die "open(< $log_error): $!"; > - /feedback plugin:.*/ && print "$&\n" while $_=<LOG>; > + > + # Get the first few rows (as there may be different number rows in the log) > + $i= 0; > + while ($_=<LOG>) > + { > + if (/feedback plugin:.*/) > + { > + print "$&\n"; > + break if ($i++ >= 3); > + } > + } > close LOG; > EOF > - > Regards, Sergei

1 0

[Maria-developers] 5.3: optimizer status update and test requests
by Sergei Petrunia 01 Dec '11

01 Dec '11

Hi Elena, A quick update on what has been happening in 5.3 optimizer recently. The text is FYI, requests for QA are marked with 'REQUEST': == Semi-joins: no non-PS crashes, optimizations refactoring == When I was fixing BUG#887468, I have made some refactorings to semi-join optimization code and pushed them into 5.3-main (revno: 3316, revision-id: psergey(a)askmonty.org-20111126082752-u8s9tv928lc67c8o) This change has grown to be more than just refactoring: - EXPLAIN now shows "Start temporary" at different position (documentation pending) - The optimizer may pick certain plans that were not picked before. REQUEST#1: please re-run semi-join optimization tests for the new server. == Discovery of effects of @@optimizer_prune_level == I have (yesterday) found https://bugs.launchpad.net/maria/+bug/898747. In a nutshell, default value of @@optimizer_prune_level=1 causes the optimizer to prune away some semi-join query plans. There can be different views about how serious BUG#898747 is, but its effect is that certain range of query plans were not tested before. REQUEST#2: re-run semi-join optimization tests with @@optimizer_prune_level=0 This can be joined with REQUEST#1 (however, if you find bugs, we're interested to know whether they have been present before the revno:3316 patch). To be continued... BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 2

[Maria-developers] Proposal: change SUBQUERY to MATERIALIZE
by Sergei Petrunia 01 Dec '11

01 Dec '11

2 1

Re: [Maria-developers] Proposal: change SUBQUERY to MATERIALIZE
by Sergei Petrunia 01 Dec '11

01 Dec '11

1 0

Re: [Maria-developers] Proposal: change SUBQUERY to MATERIALIZE
by Sergei Petrunia 01 Dec '11

01 Dec '11

1 0

[Maria-developers] MYSQL_THDVAR and SET GLOBAL
by MARK CALLAGHAN 30 Nov '11

30 Nov '11

Is there an easy way to prevent SET GLOBAL ... from being done for plugin variables declared via MYSQL_THDVAR_...? For the following variable I can do SET GLOBAL innodb_fake_changes=1 and that sets a global variable. static MYSQL_THDVAR_BOOL(fake_changes, PLUGIN_VAR_OPCMDARG, "In the transaction after enabled, UPDATE, INSERT and DELETE only move the cursor to the records " "and do nothing other operations (no changes, no ibuf, no undo, no transaction log) in the transaction. " "This is to cause replication prefetch IO. ATTENTION: the transaction started after enabled is affected.", check_fake_changes, NULL, FALSE); -- Mark Callaghan mdcallag(a)gmail.com

3 2

[Maria-developers] MariaDB 5.5 binary tarball packages and libaio
by Kristian Nielsen 30 Nov '11

30 Nov '11

Hi Wlad, I got some progress with Buildbot and producing binary tarball packages for MariaDB 5.5, using your suggestions. But I encountered a problem. I used -DBUILD_CONFIG=mysql-release and installed libaio-dev. But then the server fails like this: ./bin/mysqld: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory Of course - as libaio.so must be installed by users. I do not think it is appropriate that users need to install libaio.so to use the binary tarballs (as those are supposed to work anywhere - on the other hand, for .deb for example we _should_ use libaio of course, and dependencies will take care that it is installed). So what to do? For now I've tried to build without libaio for the generic tarballs (-DIGNORE_AIO_CHECK=1). So binaries will work; to get aio, users can build themselves or install real packages. Or do you have another idea? Do you know what MySQL does? One option could be statically linking libaio. Now in general, static linking is a can of worms with many pitfalls, however for just libaio it might work (libaio.so is statically linked on my Debian Wheezy). However, I am not sure if this is safe in general, nor do I know how to make cmake link just libaio statically? Any ideas? - Kristian.

4 4