December 2011 - developers - lists.mariadb.org

[Maria-developers] Please fix: gis-precise crashes on solaris
by Sergei Petrunia 09 Dec '11

09 Dec '11

Hi Alexey, I'm looking at 5.3 tree and see main.gis-precise to repeatably crash on solaris: http://buildbot.askmonty.org/buildbot/builders/sol-sparc-32/builds/1518/ste… We don't have a solaris machine but the test failure has a stack trace. Could you please look into this? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Please do a test of 5.3 with these new @@optimizer_switch settings
by Sergei Petrunia 08 Dec '11

08 Dec '11

Hi Elena, We're going to make the following enabled by default: @@optimizer_switch='semijoin_with_cache=on,outer_join_with_cache=on' @@join_cache_level=2 Could you please make a test run with these settings? It is important to test revision>=3337 (just pushed it), because it has a change in the optimizer which makes it choose semijoin+join_cache execution plans much more often. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] WL#245 New (by Sanja): EXISTS to IN transformation
by worklog-noreply＠askmonty.org 07 Dec '11

07 Dec '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: EXISTS to IN transformation CREATION DATE..: Wed, 07 Dec 2011, 12:51 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Client-BackLog TASK ID........: 245 (http://askmonty.org/worklog/?tid=245) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Transform queries like: ... WHERE EXISTS (SELECT 1 FROM inner_table WHERE inner_table.field = 2*outer_table.field AND maybe_something_else)... and ... WHERE NOT EXISTS (SELECT 1 FROM inner_table WHERE inner_table.field = 2*outer_table.field AND maybe_something_else)... into ... WHERE 2*outer_table.field IN (SELECT inner_table.field FROM inner_table WHERE 1 = 1 AND maybe_something_else).. ... WHERE 2*outer_table.field IS NULL OR 2*outer_table.field IN (SELECT inner_table.field FROM inner_table WHERE inner_table.field IS NOT NULL AND maybe_something_else).. To allow optimizations made for IN/ALL/ANY subqueries. Conversion is possible only if: 1)real NULL is not important (top element of WHERE/ON AND/OR list, i.e. NULL equal to FALSE) 2)the subquery has the only dependence which we bring out of it 3)the subquery is simple (has no aggregate function, GROUp BY, ORDER BY, LIMIT HAVING and so on) For NOT EXISTS conversion the subquery should be marked that its left part can't be NULL. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] No coverage for cases where materialization failed to create temptable?
by Sergei Petrunia 06 Dec '11

06 Dec '11

Hi Timour, I was working on BUG#868908, which involves having different kind of handling for cases where Materialization is not applicable because temp table had the wrong properties. So, I've set a goal to catch such cases: === modified file 'sql/item_subselect.cc' --- sql/item_subselect.cc 2011-12-06 21:03:00 +0000 +++ sql/item_subselect.cc 2011-12-06 21:11:16 +0000 @@ -4182,6 +4182,8 @@ bool subselect_hash_sj_engine::init(List */ if (tmp_table->s->keys == 0) { + //fprintf(stderr, "Q: %s\n", current_thd->query()); + DBUG_ASSERT(0); DBUG_ASSERT( tmp_table->s->uniques || tmp_table->key_info->key_length >= tmp_table->file->max_key_length() || Surprisingly, running mysql-test-run t/subselect*.test with this didn't produce any failures. Is this poor test coverage? Or, this branch is now deadcode? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Tools/suggestions for MySQL 5.6 merge
by Kristian Nielsen 06 Dec '11

06 Dec '11

I have been doing a bit of preparation/planning for the upcoming merge of MySQL 5.6. I think the 5.6 merge (and 5.5 merge) is very important for MariaDB. There is much stuff in there that we really need to make available to our users (and probably some better left out). If we fail to keep the merges going, it will leave the community in a very sorry situation; I do not see anyone else stepping up to this task. My idea is that we could merge MySQL 5.6 piece-by-piece. Several other developers mentioned this idea at the Athens meeting: - It makes it much easier to selectively merge or not merge stuff. For example there are probably large chunks of optimiser changes that we do not want to merge (or maybe merge only the testcases for). On the other hand we probably want all InnoDB stuff. - I am hopeful it makes the merge much more managable when done in smaller pieces. Large reorganisations of code (header refactoring, copyright changes) should be easier when not intermixed with other changes. Merges from MySQL 5.5 -> MySQL 5.6 are probably also easier to deal with when done in isolation (I assume we can get conflicts for these, though we already have the equivalent changes in MariaDB 5.5). - Easier to share the task between developers. One developer can do a single piece in a few days, then pass the baton when he needs to switch to other tasks (or just gets tired). Complex merges of large feature pushes can be null-merged to allow main merge to progress, and in parallel another developer can work on reworking the patch into the MariaDB code. For some changes I think we will even be able to do the actual merge in parallel. - Easier to plan and understand how far the merge is progressing. Eg. if we merge M weeks worth of 5.6 pushes every N days, that allows to estimate when we will catch up with Oracle development - Easier debugging of test failures; by keeping the trees green between pieces merged, new test failures will be known to be caused by relatively smaller sets of changes, simplifying debugging. I have done some work to facilitate this kind of piecewise merge. As it turns out, it is not trivial to determine suitable pieces to merge. Oracle developers use no discipline for the direction they do merges in, so the sequence of pushes into lp:mysql-server ("trunk") is not immediately apparent. So I took the time to reconstruct the list of pushes from reading through the history; this was not overly hard to do. I next wrote a script that scans the history based on this list of pushes, and further annotates each push with some useful information: - Size of diff for the patch - Number of changesets merged (if a merge) - Mark if this is a merge from MySQL 5.5 - One-line short comment for the push, either hand-crafted for selected items, or else extracted from the commit message. - Revision ID (including developer info) It should be fairly simple to scan this list from the bottom, collecting chunks of related or manageable pushes and merging them as one piece. Eg. A sequence of small pushes could be merged as one, or a sequence of pushes by one developer, or a single large feature push. Here is an example extract of such an annotated list: 0 2* tor.didriksen(a)oracle.com-20111101065343-ha3d79h1ayk9j7x9 [5.5] Null merge from 5.1/5.5 13 andrei.elkin(a)oracle.com-20111031135232-q13pfsaieo8ujekj Minor test adjustment 791 alexander.nozdrin(a)oracle.com-20111031115220-01la8vj7ssmcm0mc Bug#61392 0 2* tor.didriksen(a)oracle.com-20111031091354-8p6s03s1zxerrp8p [5.5] Null merge Bug#12406055 from 5.1/5.5 5380 15 jorgen.loland(a)oracle.com-20111031081122-dnufl9r4li0i3ni6 [opt] Larger merge from optimiser tree 134 2* andrei.elkin(a)oracle.com-20111028212219-cdcuh37i70k9o4ms [5.5] Merge Bug#56299 (?) from 5.5 451 2 andrei.elkin(a)oracle.com-20111028130851-mz4p2mg28y14wb7k merge from trunk to local tree. 43 sergey.glukhov(a)oracle.com-20111028094528-enre87osw9u4qjy7 [opt] Bug#32254 post-merge fix (optimiser) 48 tor.didriksen(a)oracle.com-20111028092548-a3gjubgmwuhdunfj Bug#11747423 post-merge fix 114 tor.didriksen(a)oracle.com-20111028091835-xqavfrxkc50gbsxq "Default value for sort_buffer_size changed from 2M to 256K" The first column is diff size, the second is number of changesets merged (or empty for non-merge changesets). A "*" marks pushes that are only merges from MySQL 5.5. I pushed the script to lp:mariadb-tools, in /mysql-5.6-merge-stuff/. It also includes the full output in 5.6-annotated.txt (it takes 15-20 minutes for bzr to chew through the history on a fast multi-core machine). I hope this will be useful for the merge. Comments/suggestions/questions very welcome! - Kristian.

1 0

[Maria-developers] MariaDB 5.1.60 and 5.2.10
by Daniel Bartholomew 06 Dec '11

06 Dec '11

All, MariaDB 5.1.60 is nearly ready to release. There are only a couple of mirrors which don't have the files. The changelog and release notes pages are here: * http://kb.askmonty.org/en/mariadb-5160-release-notes * http://kb.askmonty.org/en/mariadb-5160-changelog Unless something comes up, I will probably activate the download page for this release and 5.2.10 in a couple of hours. This will be followed by a combined announce email and so forth. Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

1 0

[Maria-developers] MariaDB 5.2.10
by Daniel Bartholomew 05 Dec '11

05 Dec '11

All, The MariaDB 5.2.10 release is just about ready to go. Most of the packages have been mirrored. We're still waiting on Windows packages (we're having some issues with the Windows build server), but they will hopefully be available soon. This brings up the question: Should we delay the release until we have Windows packages or add them after the fact? The Release Notes and Changelog for the release are here: - http://kb.askmonty.org/en/mariadb-5210-release-notes - http://kb.askmonty.org/en/mariadb-5210-changelog Please let me know ASAP if there is anything that should be mentioned in the Release Notes that currently is not (or jump in and add it directly to the page yourself). Serg: The PAM note in the release notes is pretty sparse. Can you add some more to it? Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

1 0

Re: [Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 3120: Added suppressions
by Sergei 03 Dec '11

03 Dec '11

Hi, Monty! Can you please explain how it is possible to have "different number rows in the log" ? On Dec 03, noreply(a)launchpad.net wrote: > ------------------------------------------------------------ > revno: 3120 > committer: Michael Widenius <monty(a)askmonty.org> > branch nick: maria-5.1 > timestamp: Sat 2011-12-03 20:29:15 +0200 > message: > Added suppressions > Fixed feedback_plugin_send to not generate a random number of lines. > modified: > mysql-test/t/feedback_plugin_send.test > mysql-test/valgrind.supp > support-files/compiler_warnings.supp > > === modified file 'mysql-test/t/feedback_plugin_send.test' > --- mysql-test/t/feedback_plugin_send.test 2011-11-13 12:28:35 +0000 > +++ mysql-test/t/feedback_plugin_send.test 2011-12-03 18:29:15 +0000 > @@ -17,7 +17,16 @@ > perl; > $log_error= $ENV{'MYSQLTEST_VARDIR'} . '/log/mysqld.1.err'; > open(LOG, '<', $log_error) or die "open(< $log_error): $!"; > - /feedback plugin:.*/ && print "$&\n" while $_=<LOG>; > + > + # Get the first few rows (as there may be different number rows in the log) > + $i= 0; > + while ($_=<LOG>) > + { > + if (/feedback plugin:.*/) > + { > + print "$&\n"; > + break if ($i++ >= 3); > + } > + } > close LOG; > EOF > - > Regards, Sergei

1 0

[Maria-developers] 5.3: optimizer status update and test requests
by Sergei Petrunia 01 Dec '11

01 Dec '11

Hi Elena, A quick update on what has been happening in 5.3 optimizer recently. The text is FYI, requests for QA are marked with 'REQUEST': == Semi-joins: no non-PS crashes, optimizations refactoring == When I was fixing BUG#887468, I have made some refactorings to semi-join optimization code and pushed them into 5.3-main (revno: 3316, revision-id: psergey(a)askmonty.org-20111126082752-u8s9tv928lc67c8o) This change has grown to be more than just refactoring: - EXPLAIN now shows "Start temporary" at different position (documentation pending) - The optimizer may pick certain plans that were not picked before. REQUEST#1: please re-run semi-join optimization tests for the new server. == Discovery of effects of @@optimizer_prune_level == I have (yesterday) found https://bugs.launchpad.net/maria/+bug/898747. In a nutshell, default value of @@optimizer_prune_level=1 causes the optimizer to prune away some semi-join query plans. There can be different views about how serious BUG#898747 is, but its effect is that certain range of query plans were not tested before. REQUEST#2: re-run semi-join optimization tests with @@optimizer_prune_level=0 This can be joined with REQUEST#1 (however, if you find bugs, we're interested to know whether they have been present before the revno:3316 patch). To be continued... BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 2

[Maria-developers] Proposal: change SUBQUERY to MATERIALIZE
by Sergei Petrunia 01 Dec '11

01 Dec '11

2 1