February 2011 - developers - lists.mariadb.org

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 13 Apr '11

13 Apr '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa > Ichikawa > Sent: Sonntag, 6. Februar 2011 09:32 > To: Vladislav Vaintroub > Cc: Philip Stoev; maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. Hello, Kazuhisa > No :-). My experience with MySQL installers is limited to fresh > install. This is the first time I tried 'upgrade' and my expectation > was purely imaginary. Right, there is no upgrade functionality in MySQLs installers up-to-now (nor I heard or any planning for it recently). But I noticed that both you and Philip had quite similar requests for the changed UI in case of "upgrade possible". Both of you recommended changes at the "Database Properties" dialog, I think it is wrong place, by the time you enter this dialog it is too late, you should not have entered this dialog, as the intention was not "create new", but "upgrade existing" anyway.. So since both of my users have pointed to usability issues, I added a new dialog that comes after license agreement that and tells user that upgradable instances were found and please make a choice between installing a new instance and not installing a new instance(optional upgrade one or more existing ones). This sounds almost like your first suggestion, but in own dialog. This dialog comes right before feature selection, and only thing it might do is to unselect "Database Instance" feature in the following dialog . Technically, it duplicates functionality of the feature selection tree. So what... I hope it adds some clarity for the user - at least tells the user that his existing instances *can* be upgraded. Or maybe it won't help that much, since user never looks at the UI anyway :) (nice article about that is here http://www.codinghorror.com/blog/2009/10/treating-user-myopia.html ) Anyway, I uploaded the newest msis to the usual place, if somebody wants to play with it. Thanks, Vladislav.

2 4

[Maria-developers] MWL#116: Efficient group commit, ready for code review
by Kristian Nielsen 04 Apr '11

04 Apr '11

Hi Serg, Please find attached the patch for code review of MWL#116: Efficient group commit. I went through all your comments in the architecture review and updated the worklog description and the code accordingly. I hope I got everything, otherwise I hope you have the patience to point out any omissions once again! The patch is pushed here if you want to see the complete code: lp:~maria-captains/maria/mariadb-5.1-mwl116 Once the code passes code review, I think the plan is to include this feature in MariaDB 5.3. Please let me know if you need anything else from me for the review. - Kristian.

2 3

[Maria-developers] Review of replication/group commit work
by Kristian Nielsen 28 Mar '11

28 Mar '11

Hi Monty, Thanks for offering to review my replication worklogs. Here are pointers to the various worklogs and patches, and suggestions for order to things in etc. Some background information is in this series on my blog: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html http://kristiannielsen.livejournal.com/12810.html There is a tree on launchpad based on 5.2 with all the patches merged, if you want to look at the actual code in one piece: lp:~maria-captains/maria/mariadb-5.2-rpl ---- The main worklog, and the place to start, is MWL#116: http://askmonty.org/worklog/Server-Sprint/?tid=116 I think you'll have to read through the worklog/design first (it already had architecture review by Serg) in order to properly review the code. The patch for MWL#116 is here: https://lists.launchpad.net/maria-developers/msg03693.html It also has links to a lauchpad tree with just this patch. This worklog extends the storage engine API with a facility to efficiently coordinate commit order among transaction participants, and also implements group commit for binlog. ---- The next worklog is MWL#136: http://askmonty.org/worklog/Server-RawIdeaBin/?tid=136 Here is the commit mail: https://lists.askmonty.org/pipermail/commits/2010-November/000735.html (also this one, but it just renames some variables to bettter names): https://lists.askmonty.org/pipermail/commits/2010-November/000741.html This worklog makes START TRANSACTION WITH CONSISTENT SNAPSHOT, well, consistent. It also uses this for the binlog to provide non-blocking mysqldump --single-transaction --master-data. ---- The last worklog is MWL#163: http://askmonty.org/worklog/Server-Sprint/?tid=163 Here is the commit mail: https://lists.askmonty.org/pipermail/commits/2010-November/000711.html This is a re-write of a Facebook patch. Mark has mentioned this on several occasions as being important for them to fix a hot-spot. It implements an option to release InnoDB row locks earlier, before committing to binlog and storage engine (which is not safe in general, but can be safe in certain applications such as Facebook presumably). ---- That's it for now! - Kristian.

2 1

[Maria-developers] request for feedback on live schema changes patch
by Zardosht Kasheff 18 Mar '11

18 Mar '11

Hello all, In another thread, there was a discussion about what it would take to allow the storage engine to support live schema changes in mysql and mariadb. I said that I was working porting this existing feature from MySQL Cluster to MySQL 5.1.46. I would like to show the work I have done so far and solicit feedback. Attached is the diff based off of 5.1.46. Before looking at the patch, please be aware of the following things: - I am not an expert in this code. As a result, I had to use an approach of that minimizes the risk of introducing regressions. \ - My main interest is to get something safe that works and can be integrated in 5.1. As a result, kept the current mysql_alter_table functionality in place (as old_mysql_alter_table), and added MySQL Cluster's as new_mysql_alter_table. For an engine to get new_mysql_alter_table to be used, the engine must expose an alter table flag HA_GENERAL_ONLINE. - Another reason why I kept old_mysql_alter_table was that I did not want to deal with engines that implement handler::add_index and make sure that they work properly under the new API. Also, I did not want to port alter table changes related to partitioned tables. - I have run the current 5,1 mysql test suite (that is in the mysql-test directory of source tarballs), and verified that using old_mysql_alter_table and new_mysql_alter_table works fine with them. That being said, I would love to hear feedback on this patch, and would also like to understand what (if anything) can be done to get this into MariaDB 5.1. I realize that may not be possible, but if it is, I would like to try. Thanks -Zardosht

2 5

[Maria-developers] MWL#90 combined patch for review
by Sergey Petrunya 08 Mar '11

08 Mar '11

Hi Monty, Please find attached the complete patch of MWL#90 for review. The tree with the code is on launchpad/buildbot: https://code.launchpad.net/~maria-captains/maria/5.3-subqueries-mwl90 http://buildbot.askmonty.org/buildbot/grid?branch=5.3-subqueries-mwl90 and it has no failures other than those also found in mainline 5.3. The merge from 5.3-main was about a week ago. BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 5

[Maria-developers] Pbxt loses rows
by Michael Widenius 08 Mar '11

08 Mar '11

Hi! This is a bug that has been around a long time but not yet fixed (at least not in 1.0.11 which is in the MariaDB 5.1 tree). Once in a while when running the pbxt.insert test, pbxt fails to find all rows in the table: pbxt.insert [ fail ] Test ended at 2011-02-25 16:44:03 CURRENT_TEST: pbxt.insert --- /home/my/maria-5.1-merge/mysql-test/suite/pbxt/r/insert.result 2011-02-10 22:57:21.000000000 +0200 +++ /home/my/maria-5.1-merge/mysql-test/suite/pbxt/r/insert.reject 2011-02-25 16:44:03.030856755 +0200 @@ -298,7 +298,7 @@ insert into t2 select t1.* from t1, t2 t, t3 where t1.id1 = t.id2 and t.id2 = t3.id3; select count(*) from t2; count(*) -25500 +25495 drop table t1,t2,t3; create table t1 (a int, b int); insert into t1 (a,b) values (a,b); Rerunning the test usually works It would be really good to get this fixes ASAP as it's not good if a storage engines looses rows randomly... The above happend in my latest 5.1 tree on Linux. Regards, Monty

2 2

Re: [Maria-developers] [Commits] Rev 2905: Merge Percona patch MWL#47 into mariadb 5.2-percona. in http://bazaar.launchpad.net/~maria-captains/maria/5.2
by Kristian Nielsen 03 Mar '11

03 Mar '11

knielsen(a)knielsen-hq.org writes: > message: > Merge Percona patch MWL#47 into mariadb 5.2-percona. > > This patch adds options to annotate the binlog (and the mysqlbinlog > output) with the original SQL query for queries that are logged > using row-based replication. Serg raised the very relevant issue about what to do about the similar feature in MySQL 5.6: http://forge.mysql.com/worklog/task.php?id=4033 http://forge.mysql.com/worklog/task.php?id=5404 The Oracle team has worked more on this since last I looked, and the two features are now very similar. Here are the main differences, as I see it: - The MySQL 5.6 feature is enabled by --binlog_rows_query_log_events, while the MWL#47 option is called --binlog-annotate-rows-events. - MWL#47 has --replicate-annotate-rows-events which makes the slave receive and binlog the annotations. The MySQL feature always receives annotation events if present in the binlog (so replication 5.6->pre-5.6 will break if annotations are enabled). The MySQL feature binlogs the annotations based on the same --binlog-annotate-rows-events option as the master. - MWL#47 has a mysqlbinlog option --skip-annotate-rows-events, MySQL 5.6 does not. - MySQL 5.6 mysqlbinlog can log the annotations with BINLOG statements so that they are preserved with `mysqlbinlog | mysql`; MWL#47 does not have this. - MWL#47 uses binlog event type 160, MySQL 5.6 uses an ignorable event of type 29. As far as I can tell, the actual event format is identical, except for the LOG_EVENT_IGNORABLE_F flag (which says it's ok for a slave to ignore this event if it doesn't know about it, however there is no server version that knows the LOG_EVENT_IGNORABLE_F but not the MySQL annotation event). - I think there are a couple of issues with the MySQL patch (newlines in queries breaking mysqlbinlog output, duplicated annotations for INSERT DELAYED). There might be issues with the MWL#47 patch also, of course. So the question is how to prepare for when we merge MySQL 5.6. An easy option could be to just ditch MWL#47 (and possibly backport the MySQL 5.6 patch). However, as I understand it, the original sponsor of MWL#47 is already using it, and so we may have to remain compatible with the existing MWL#47 implementation. Can we get some definite information on that? Another option might be to simply change the MWL#47 to use the same event type as MySQL (the event format is the same, and we can add the LOG_EVENT_IGNORABLE_F). And replace the MWL#47 options --binlog-annotate-rows-events and --replicate-annotate-rows-events with the MySQL --binlog_rows_query_log_events option (though I think the MySQL option name is not good). But again, I am not sure if this is a valid option given possible existing usage of MWL#47. The third option I can think of is to leave MWL#47 as it is. Then when we get to merge MySQL 5.6, we will basically drop the MySQL part, doing the following to remain backwards compatible with MySQL: - Make the --binlog_rows_query_log_events option set --binlog-annotate-rows-events and --replicate-annotate-rows-events. - Make a slave read a MySQL rows_query_log_event as a MWL#47 annotation event. - Merge over the bit of the MySQL patch that makes mysqlbinlog output annotation events using a BINLOG '...' statement. This should make upgrade from MySQL 5.6 -> MariaDB fully work. Using MySQL5.6+ with MariaDB will also work, only MySQL slaves will not receive MariaDB master annotation events. The third option seems to preserve the existing MWL#47 most faithfully and does not seem to need much work in terms of the eventual merge, so this is what I suggest as the best way forward. But I am open to other (justified) opinions. - Kristian.

2 1

[Maria-developers] Igor please review: [Commits] Rev 2926: BUG#724275: Crash in JOIN::optimize in maria-5.3 in file:///home/psergey/dev2/5.3-2/]
by Sergey Petrunya 01 Mar '11

01 Mar '11

Hello Igor, Could you please review the below: ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: commits(a)mariadb.org X-Mailer: mail (GNU Mailutils 1.2) Date: Tue, 1 Mar 2011 00:30:02 +0300 (MSK) Subject: [Commits] Rev 2926: BUG#724275: Crash in JOIN::optimize in maria-5.3 in file:///home/psergey/dev2/5.3-2/ At file:///home/psergey/dev2/5.3-2/ ------------------------------------------------------------ revno: 2926 revision-id: psergey(a)askmonty.org-20110228212959-hl3selurcdou7d22 parent: igor(a)askmonty.org-20110227181411-k81vmi35o0no21ab committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: 5.3-2 timestamp: Tue 2011-03-01 00:29:59 +0300 message: BUG#724275: Crash in JOIN::optimize in maria-5.3 - Make equality-substitution-for-ref-access code in JOIN::optimize() treat join_tab->ref.key_copy correctly (in the way create_ref_for_key() has filled it). === modified file 'mysql-test/r/join.result' --- a/mysql-test/r/join.result 2010-10-30 13:07:45 +0000 +++ b/mysql-test/r/join.result 2011-02-28 21:29:59 +0000 @@ -1251,3 +1251,37 @@ Handler_read_rnd_next 1 DROP TABLE t1, t2; End of 5.1 tests +# +# BUG#724275: Crash in JOIN::optimize in maria-5.3 +# +create table t1 (a int); +insert into t1 values (1),(2); +insert into t1 select * from t1; +create table t2 (a int, b int, key(a,b)); +insert into t2 values (1,1),(1,2),(1,3),(1,4),(2,5),(2,6),(2,7),(2,8),(2,9); +insert into t2 select * from t2; +insert into t2 select * from t2; +insert into t2 select * from t2; +create table t3 (a int, b int, key(a)); +insert into t3 values (1,1),(2,2); +select * from +t3 straight_join t1 straight_join t2 force index(a) +where t2.a=1 and t2.b=t1.a and t1.a=t3.b and t3.a=1; +a b a a b +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +1 1 1 1 1 +drop table t1,t2,t3; === modified file 'mysql-test/t/join.test' --- a/mysql-test/t/join.test 2010-10-30 13:07:45 +0000 +++ b/mysql-test/t/join.test 2011-02-28 21:29:59 +0000 @@ -941,3 +941,26 @@ DROP TABLE t1, t2; --echo End of 5.1 tests + +--echo # +--echo # BUG#724275: Crash in JOIN::optimize in maria-5.3 +--echo # + +create table t1 (a int); +insert into t1 values (1),(2); +insert into t1 select * from t1; + +create table t2 (a int, b int, key(a,b)); +insert into t2 values (1,1),(1,2),(1,3),(1,4),(2,5),(2,6),(2,7),(2,8),(2,9); +insert into t2 select * from t2; +insert into t2 select * from t2; +insert into t2 select * from t2; + +create table t3 (a int, b int, key(a)); +insert into t3 values (1,1),(2,2); +select * from + t3 straight_join t1 straight_join t2 force index(a) +where t2.a=1 and t2.b=t1.a and t1.a=t3.b and t3.a=1; + +drop table t1,t2,t3; + === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2011-02-27 18:14:11 +0000 +++ b/sql/sql_select.cc 2011-02-28 21:29:59 +0000 @@ -1056,6 +1056,7 @@ */ for (JOIN_TAB *tab= join_tab + const_tables; tab < join_tab + tables; tab++) { + uint key_copy_index=0; for (uint i=0; i < tab->ref.key_parts; i++) { @@ -1071,13 +1072,14 @@ { *ref_item_ptr= ref_item; Item *item= ref_item->real_item(); - store_key *key_copy= tab->ref.key_copy[i]; + store_key *key_copy= tab->ref.key_copy[key_copy_index]; if (key_copy->type() == store_key::FIELD_STORE_KEY) { store_key_field *field_copy= ((store_key_field *)key_copy); field_copy->change_source_field((Item_field *) item); } } + key_copy_index++; } } _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message ----- -- BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Buildbot Outage
by Adam M. Dutko 01 Mar '11

01 Mar '11

It is possible the buildbots I sponsor are offline today: mariadb-brs, adutko-centos-amd64 and adutko-ultrasparc3. The transformer that feeds our house was hit by lightning last night. It was still operational this morning but I believe the power company is replacing it today.

1 0

[Maria-developers] Fix for PBXT for problem with 'repair-pending'
by Michael Widenius 25 Feb '11

25 Feb '11

Hi! We have got some failures of the following type in pushbuild: pbxt.mysqlslap w2 [ fail ] Test ended at 2011-02-25 14:37:57 CURRENT_TEST: pbxt.mysqlslap mysqltest: At line 43: query 'drop database pbxt' failed: 1010: Error dropping database (can't rmdir './pbxt', errno: 39) The reason is that at the end of the test there was left in the pbxt directory a file "repair-pending" with a reference to some old already deleted table. There was nothing else in the directory. My assumption (and the code seams to agree with it) is that when you delete a table, you don't remove the table from the repair-pending file even if it had been recorded there. Here is a patch that fixes this: === modified file 'storage/pbxt/src/table_xt.cc' --- storage/pbxt/src/table_xt.cc 2010-11-26 22:37:34 +0000 +++ storage/pbxt/src/table_xt.cc 2011-02-25 14:40:53 +0000 @@ -1755,6 +1755,8 @@ xtPublic void xt_drop_table(XTThreadPtr tab_close_mapped_files(self, tab); tab_delete_table_files(self, tab_name, tab_id); + /* Remove table from "repair-pending" */ + xt_tab_table_repaired(tab); ASSERT(xt_get_self() == self); if ((te_ptr = (XTTableEntryPtr) xt_sl_find(self, db->db_ Regards, Monty

2 1