December 2010 - developers - lists.mariadb.org

[Maria-developers] MWL#116: Efficient group commit, ready for code review
by Kristian Nielsen 04 Apr '11

04 Apr '11

Hi Serg, Please find attached the patch for code review of MWL#116: Efficient group commit. I went through all your comments in the architecture review and updated the worklog description and the code accordingly. I hope I got everything, otherwise I hope you have the patience to point out any omissions once again! The patch is pushed here if you want to see the complete code: lp:~maria-captains/maria/mariadb-5.1-mwl116 Once the code passes code review, I think the plan is to include this feature in MariaDB 5.3. Please let me know if you need anything else from me for the review. - Kristian.

2 3

[Maria-developers] request for feedback on live schema changes patch
by Zardosht Kasheff 18 Mar '11

18 Mar '11

Hello all, In another thread, there was a discussion about what it would take to allow the storage engine to support live schema changes in mysql and mariadb. I said that I was working porting this existing feature from MySQL Cluster to MySQL 5.1.46. I would like to show the work I have done so far and solicit feedback. Attached is the diff based off of 5.1.46. Before looking at the patch, please be aware of the following things: - I am not an expert in this code. As a result, I had to use an approach of that minimizes the risk of introducing regressions. \ - My main interest is to get something safe that works and can be integrated in 5.1. As a result, kept the current mysql_alter_table functionality in place (as old_mysql_alter_table), and added MySQL Cluster's as new_mysql_alter_table. For an engine to get new_mysql_alter_table to be used, the engine must expose an alter table flag HA_GENERAL_ONLINE. - Another reason why I kept old_mysql_alter_table was that I did not want to deal with engines that implement handler::add_index and make sure that they work properly under the new API. Also, I did not want to port alter table changes related to partitioned tables. - I have run the current 5,1 mysql test suite (that is in the mysql-test directory of source tarballs), and verified that using old_mysql_alter_table and new_mysql_alter_table works fine with them. That being said, I would love to hear feedback on this patch, and would also like to understand what (if anything) can be done to get this into MariaDB 5.1. I realize that may not be possible, but if it is, I would like to try. Thanks -Zardosht

2 5

[Maria-developers] The query cache patch fixed by me and then Monty (most changes by him)
by Oleksandr Byelkin 07 Jan '11

07 Jan '11

Hi! Here is changes which was done: - Fixed switching off of the query cache -- memory freed now -- after completely switching of it could be switched on again - Cleaned up saving the query without comments and space - Added all the missing test cases (and checked results) - Added more symbols to LEX to make parsing faster (not part of this thing, but I thought it was better to get this done now as I was looking at the code) - Fixed all the issues that I had in my review of query parsing - Cleaned up mysql-tests -- Made test results easier to read -- removed 'percona' prefix from tests names -- Made test result files notable smaller - Added 'thd' to query_cache::invalidate calls, to remove some current_thd usages - Added variable thd->query_cache_is_valid, that is set to 1 if query_cache could possible read query from the query_cache -- If not, there is no reason to put the query in the cache The above change allowed me to remove safe_to_cache_queries from send_result_to_client() and make this part of the code not depending in lex_start() - simplified a couple of functions to use pointers instead of index - Fixed bug where SQL_NO_CACHE was not detected if followed by tab, newline or return i> that's about it 2 Vadim & Oleg: If you OK with changes above and the patch then it will be included in our tree.

2 3

[Maria-developers] Logoff trigger for MariaDB
by Oli Sennhauser 31 Dec '10

31 Dec '10

Hi all, As suggested by Sergei we should discuss this here. Copying the login trigger stuff to a place in the code where I thought it makes sense I have created a logout trigger. This gives the capability to track a session from its beginning to its end and collects all the data from each session. With this capability we can start thinking about auditing of MySQL users like it is needed/wanted from big companies. I am NOT a developer and I was just lucky that it worked out how I did it. So I am not of much help for an other approach as Sergei hinted in his comment. Now: free to discuss... http://www.fromdual.com/mysql-logon-and-logoff-trigger-for-auditing Regards, Oli -- FromDual - Vendor independent and neutral MySQL consulting. Website http://www.fromdual.com/ Skype fromdual Jabber fromdual(a)swissjabber.ch Yahoo IM fromdual Xing https://www.xing.com/profile/Oliver_Sennhauser LinkedIn http://ch.linkedin.com/in/shinguz Twitter http://twitter.com/shinguz

2 2

Re: [Maria-developers] [Fwd: [Commits] Rev 2998: Fixed LP bug #639935 (bug #58727). in file:///home/igor/maria/maria-5.1-bug693935/]
by Oleksandr Byelkin 24 Dec '10

24 Dec '10

Hi! 24 Dec 2010, в 06:45, Igor Babaev написал(а): > Sanja, > > Please review this fix ASAP. > We need it to move on with 5.3 The patch is OK, but I would prefer to have comment before Item::quick_fix_field() that this method is only for optimizer needs, it can be used only for certain Items controlled by optimizer and maybe add that only Item_func_* need recursion for this. But you could write comments much better then I so it is up to you how write the comment. > > Regards. > Igor. > > -------- Original Message -------- > Subject: [Commits] Rev 2998: Fixed LP bug #639935 (bug #58727). in > file:///home/igor/maria/maria-5.1-bug693935/ > Date: Fri, 24 Dec 2010 06:42:05 +0200 (EET) > From: <igor(a)askmonty.org> > Reply-To: maria-developers(a)lists.launchpad.net > To: <commits(a)mariadb.org> > > At file:///home/igor/maria/maria-5.1-bug693935/ > > ------------------------------------------------------------ > revno: 2998 > revision-id: igor(a)askmonty.org-20101224044150-p9yja1e87riwjh7g > parent: monty(a)askmonty.org-20101221172350-wqtf0t6evpc0un4b > committer: Igor Babaev <igor(a)askmonty.org> > branch nick: maria-5.1-bug693935 > timestamp: Thu 2010-12-23 20:41:50 -0800 > message: > Fixed LP bug #639935 (bug #58727). > When the optimizer creates items out of other items it does > not have to call the fix_fields method. Usually in these > cases it calls quick_fix_field() that just marks the > created item as fixed. If the created item is an Item_func > object then calling quick_fix_field() works fine if the > arguments of the created functional item are already fixed. > Otherwise some unfixed nodes remain in the item tree and > it triggers an assertion failure whenever the item is > evaluated. > > Fixed the problem by making the method quick_fix_field > virtual and providing an implementation for the class > Item_func objects that recursively calls the method > for unfixed arguments of any functional item. [skip]

1 0

Re: [Maria-developers] pre-allocating binlog to speed up sync_binlog=1
by Sergei Golubchik 23 Dec '10

23 Dec '10

Hi, Arjen! [moving to the public list] On Dec 14, Arjen Lentz wrote: > Hi all > > Can we adopt/implement http://forge.mysql.com/worklog/task.php?id=4925 in > MariaDB? > The benchmark info is in the item, and looks quite interesting. > The author tested it using a separate tool to preallocate the binlog, but > it should be straightforward implementing this inside the server, that'd > also be much easier to manage without extra fuss for a DBA. You mean - before it gets into 5.6 and we merge it the normal way? Or you mean - with a different implementation (w/o mysqlbinlogalloc) ? Regards, Sergei P.S. These are the changesets: final version: http://lists.mysql.com/commits/113309 http://lists.mysql.com/commits/113306 http://lists.mysql.com/commits/113307 review comments: http://lists.mysql.com/commits/116965 http://lists.mysql.com/commits/121478

4 6

Re: [Maria-developers] [Commits] Rev 2959: New table creation/renaming block added if old encoded table present. in file:///home/bell/maria/bzr/work-maria-5.1-CREATE/
by Sergei Golubchik 20 Dec '10

20 Dec '10

Hi, sanja! On Oct 22, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.1-CREATE/ > > ------------------------------------------------------------ > revno: 2959 > revision-id: sanja(a)askmonty.org-20101021220232-eqso4ey46985wfzf > parent: sergii(a)pisem.net-20101020105843-7ybuqyc9n5328tie > committer: sanja(a)askmonty.org > branch nick: work-maria-5.1-CREATE > timestamp: Fri 2010-10-22 01:02:32 +0300 > message: > New table creation/renaming block added if old encoded table present. ok to push Regards, Sergei

1 0

[Maria-developers] Proposals for improving BKA/MRR visibility in EXPLAIN and diagnostics
by Sergey Petrunya 17 Dec '10

17 Dec '10

Hello, Below are some ideas on how to make DS-MRR/BKA easier to work with for the users (= those who don't run mysqld under debugger). Questions are marked with 'Q:' but any comments are welcome. Better EXPLAIN -------------- Philip has complained numerous times that it is not shown in EXPLAIN whether DS-MRR will use key sorting. Currently, MRR alone is show like this: MariaDB [test]> explain select * from t1 where key1<30; +----+-------------+-------+-------+---------------+------+---------+------+------+----------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+-------+---------------+------+---------+------+------+----------------------------------+ | 1 | SIMPLE | t1 | range | key1 | key1 | 5 | NULL | 1 | Using index condition; Using MRR | +----+-------------+-------+-------+---------------+------+---------+------+------+----------------------------------+ MRR with BKA are shown like this: MariaDB [test]> explain select * from t2,t1 where t2.a=t1.key1; +----+-------------+-------+------+---------------+------+---------+-----------+------+------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+------+---------------+------+---------+-----------+------+------------------------------------+ | 1 | SIMPLE | t2 | ALL | NULL | NULL | NULL | NULL | 10 | Using where | | 1 | SIMPLE | t1 | ref | key1 | key1 | 5 | test.t2.a | 1 | Using join buffer (flat, BKA join) | +----+-------------+-------+------+---------------+------+---------+-----------+------+------------------------------------+ The suggestion is to - get rid of "Using MRR" word, - instead, show "Sort rowids" and/or "Sort keys". - Show the above two whenever rowid-ordered and/or key-ordered retrieval is performed. The above examples will look as follows: MariaDB [test]> explain select * from t1 where key1<30; +----+-------------+-------+-------+---------------+------+---------+------+------+-----------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+-------+---------------+------+---------+------+------+-----------------------------------------------+ | 1 | SIMPLE | t1 | range | key1 | key1 | 5 | NULL | 1 | Using index condition; Sort keys; Sort rowids | +----+-------------+-------+-------+---------------+------+---------+------+------+-----------------------------------------------+ MRR with BKA are shown like this: MariaDB [test]> explain select * from t2,t1 where t2.a=t1.key1; +----+-------------+-------+------+---------------+------+---------+-----------+------+------------------------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+------+---------------+------+---------+-----------+------+------------------------------------------------------------+ | 1 | SIMPLE | t2 | ALL | NULL | NULL | NULL | NULL | 10 | Using where | | 1 | SIMPLE | t1 | ref | key1 | key1 | 5 | test.t2.a | 1 | Using join buffer (flat, BKA join); Sort keys; Sort rowids | +----+-------------+-------+------+---------------+------+---------+-----------+------+------------------------------------------------------------+ Q: any comments? Counters -------- Like it is done with other kinds of table accesses, we want to have counter-based way of analyzing of what has been happening around MRR. The most obvious are: 1. Handler_multi_range_read_init_count status variable This will tell how many multi_range_read_init() calls have been made that used non-default MRR implementation, i.e. one will be able to see how many times real MRR scans were performed. Q: this counter doesn't show how many times key sorting/rowid sorting/both strategies were used. Is that ok? One could argue that information about strategy choice is not in high demand as MRR strategy choice is based mostly on system settings and DDLs. 2. Handler_multi_range_read_next_count status variable This will tell how many records were returned by MRR to the upper layer. Q: MRR does index and rnd_pos scans under the hood, and these scans do increase counters. This means that, for a single row returned by MRR, multiple counters will be incremented. Less obvious suggestions: 3. It would be useful to have an idea about whether DS-MRR had sufficient buffer space to operate. One can get a rough picture by adding counters of key/rowid sort operations: - Handler_mrr_rowid_sort_count - Handler_mrr_key_sort_count If Handler_mrr_key_sort_count == Handler_multi_range_read_init_count then we've had enough buffer space. If key_sort_count is two times greater than the number of init operations, then one could conclude that on average two buffer refills were needed and increasing buffer size up to 2x would be beneficial. 4. Other possible things - Index condition pushdown check count, true/false ratio - Some statistics to globally check if the optimizer's guesses about required sizes for key/rowid parts of the buffer were any good. Q: do we see the need for any of the above? BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

3 3

[Maria-developers] WL#171 New (by Monty): new free MariaDB client library
by worklog-noreply＠askmonty.org 14 Dec '10

14 Dec '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: new free MariaDB client library CREATION DATE..: Tue, 14 Dec 2010, 19:51 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Client-Sprint TASK ID........: 171 (http://askmonty.org/worklog/?tid=171) VERSION........: WorkLog-4.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Create a new MariaDB/MySQL/Drizzle client library that is Apache, BSD or LGPL. This task is to correct the limitations of the old lgpl client library, MWL#134, as outlined in http://kb.askmonty.org/v/lgpl-mysql-client-library-32358. Minimum requirements are: * Should be binary link level compatible with the current MySQL 5.1 client library. To use it, one would just have to relink the application with the new library. No re-compilation of the source should be necessary. ** It should be able to connect to all current versions of MariaDB, MySQL and Drizzle. * It should support all the new functionality in MariaDB's client library: ** New functions ** Pluggable authentication ** The license should be one of BSD/LGPL/Apache The different development options are: * Use the old LGPL MySQL library from 3.23 as a base. * Use Drizzles client library as base. * Use one of the free MySQL client libraries used by some of the languages as a base. * Writing a new one from scratch. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

Re: [Maria-developers] [Commits] Rev 2867: Fix LP BUG#685411 in file:///home/tsk/mprog/src/5.3-lpb-685411/
by Oleksandr Byelkin 14 Dec '10

14 Dec '10

Hi! On 14.12.2010 14:08, timour(a)askmonty.org wrote: > At file:///home/tsk/mprog/src/5.3-lpb-685411/ > > ------------------------------------------------------------ > revno: 2867 > revision-id: timour(a)askmonty.org-20101214120805-vafrxb6i59u9gyef > parent: igor(a)askmonty.org-20101211205039-b1alcmyethjpddzu > committer: timour(a)askmonty.org > branch nick: 5.3-lpb-685411 > timestamp: Tue 2010-12-14 14:08:05 +0200 > message: > Fix LP BUG#685411 Ok, to push. [skip]

1 0