- developers - lists.mariadb.org

[Maria-developers] WL#191 New (by Knielsen): status variables for last InnoDB binlog position
by worklog-noreply＠askmonty.org 29 Mar '11

29 Mar '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: status variables for last InnoDB binlog position CREATION DATE..: Tue, 29 Mar 2011, 09:06 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 191 (http://askmonty.org/worklog/?tid=191) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: In every commit, InnoDB/XtraDB stores the binlog position in the system tablespace (in a transactional-safe way). At server start, this position is printed to the error log. This is useful after taking a backup to provision a slave, as it provides the values to use for CHANGE MASTER TO. It would be useful if these values printed to the error log were also directly available to SQL as read-only status variables: innodb_startup_binlog_master_file innodb_startup_binlog_master_pos This would be generally useful. When this is implemented, the test case main.group_commit_binlog_pos can be fixed to not require `grep` and `tail`, and so work on Windows with no posix tools installed. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] Improved table elimination
by Lars Rönnbäck 29 Mar '11

29 Mar '11

When doing some comparisons of query optimization in Microsoft SQL Server and MariaDB I noticed that there is a second type of table elimination which MariaDB has not yet implemented. It depends on foreign keys being declared, and can under some circumstances substantially improve performance. Rather than to write a lengthy discussion, here is a test script for MariaDB 5.2.5 in which the queries with the condition "where ... is not null" should result in this type of table elimination: http://pastebin.com/s68RwgG2 This screencast shows the table elimination in effect in MSSQL: http://www.screenr.com/tPG We rely on table elimination to get better performance in 3NF-views and functions used in our Anchor Modeling (6NF-table) technique. However, the need for table elimination has previously come from denormalized modeling, where it is used to speed up queries against star schemas (e g eliminating dimension tables). It should therefore not only be appreciated by our crowd, if implemented in full. We were very happy to see the 'outer join table elimination' implemented and are wondering if someone could put in the effort to add the 'foreign key table elimination'? Since everything we do with Anchor Modeling is Open Source, it would be nice to have an Open Source database that fully supports the technique. Support in other databases is described here, if anyone is interested: http://www.anchormodeling.com/?page_id=22 Thanks, Lars

1 0

[Maria-developers] WL#190 New (by Yakatz): Optional parameters support in stored procedures
by worklog-noreply＠askmonty.org 28 Mar '11

28 Mar '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Optional parameters support in stored procedures CREATION DATE..: Mon, 28 Mar 2011, 20:26 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 190 (http://askmonty.org/worklog/?tid=190) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: This bug has been open on MySQL since 2005. In a nutshell, support PROCEDUREs like this: CREATE PROCEDURE ListUsers ( @userID int, @authorized bit = NULL ) AS.... ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] Review of replication/group commit work
by Kristian Nielsen 28 Mar '11

28 Mar '11

Hi Monty, Thanks for offering to review my replication worklogs. Here are pointers to the various worklogs and patches, and suggestions for order to things in etc. Some background information is in this series on my blog: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html http://kristiannielsen.livejournal.com/12810.html There is a tree on launchpad based on 5.2 with all the patches merged, if you want to look at the actual code in one piece: lp:~maria-captains/maria/mariadb-5.2-rpl ---- The main worklog, and the place to start, is MWL#116: http://askmonty.org/worklog/Server-Sprint/?tid=116 I think you'll have to read through the worklog/design first (it already had architecture review by Serg) in order to properly review the code. The patch for MWL#116 is here: https://lists.launchpad.net/maria-developers/msg03693.html It also has links to a lauchpad tree with just this patch. This worklog extends the storage engine API with a facility to efficiently coordinate commit order among transaction participants, and also implements group commit for binlog. ---- The next worklog is MWL#136: http://askmonty.org/worklog/Server-RawIdeaBin/?tid=136 Here is the commit mail: https://lists.askmonty.org/pipermail/commits/2010-November/000735.html (also this one, but it just renames some variables to bettter names): https://lists.askmonty.org/pipermail/commits/2010-November/000741.html This worklog makes START TRANSACTION WITH CONSISTENT SNAPSHOT, well, consistent. It also uses this for the binlog to provide non-blocking mysqldump --single-transaction --master-data. ---- The last worklog is MWL#163: http://askmonty.org/worklog/Server-Sprint/?tid=163 Here is the commit mail: https://lists.askmonty.org/pipermail/commits/2010-November/000711.html This is a re-write of a Facebook patch. Mark has mentioned this on several occasions as being important for them to fix a hot-spot. It implements an option to release InnoDB row locks earlier, before committing to binlog and storage engine (which is not safe in general, but can be safe in certain applications such as Facebook presumably). ---- That's it for now! - Kristian.

2 1

[Maria-developers] review: [Commits] Rev 2863: MWL#163: release of row locks in InnoDB during prepare in http://bazaar.launchpad.net/~maria-captains/maria/5.1
by Michael Widenius 28 Mar '11

28 Mar '11

Hi! >>>>> "knielsen" == knielsen <knielsen(a)knielsen-hq.org> writes: knielsen> At http://bazaar.launchpad.net/~maria-captains/maria/5.1 knielsen> ------------------------------------------------------------ knielsen> revno: 2863 knielsen> revision-id: knielsen(a)knielsen-hq.org-20101103160708-88w8p8ibe9iaoggj knielsen> parent: knielsen(a)knielsen-hq.org-20101102074027-jlnx0rnbdpovl122 knielsen> committer: knielsen(a)knielsen-hq.org knielsen> branch nick: work-5.1-mwl163 knielsen> timestamp: Wed 2010-11-03 17:07:08 +0100 knielsen> message: knielsen> MWL#163: release of row locks in InnoDB during prepare knielsen> Port the Facebook patch for releasing InnoDB row locks early to the knielsen> MWL#116 framework. knielsen> === modified file 'storage/xtradb/handler/ha_innodb.cc' knielsen> --- a/storage/xtradb/handler/ha_innodb.cc 2010-11-01 14:41:09 +0000 <cut> Spent 30 minutes checking the innodb code and things looked ok. Regards, Monty

1 0

[Maria-developers] Writeup of how to run my group commit benchmark
by Kristian Nielsen 28 Mar '11

28 Mar '11

Hi, For anyone interested in trying to redo my group commit benchmarks on more interesting hardware than what I have available, here is a writeup of the steps needed to run it. The benchmarks are described here: http://kristiannielsen.livejournal.com/15739.html http://kristiannielsen.livejournal.com/15515.html 1. Compile gypsy. This is used to generate the load. I use a specific version as the latest in trunk seemed to not build on non-windows last I tried: bzr branch '-rrevid:shane@mysql.com-20100224065101-jk3s5sro2p914nvp' lp:gypsy cd gypsy make This requires `mysql_config` in the path to get includes/libraries for libmysqlclient. It also needs openssl headers/libraries (eg. install libssl-dev package). 2. Install MariaDB with group commit enabled. Packages/sources are available for this here: http://kb.askmonty.org/v/mariadb-52-replication-feature-preview These packages are however missing the --binlog-optimize-thread-scheduling option that I used to compare the two ways to schedule threads in the above blog posts. To get that option, grab this bzr tree and build: bzr branch lp:~knielsen/maria/mariadb-5.1-mwl116-threadscheduling cd mariadb-5.1-mwl116-threadscheduling BUILD/compile-pentium64-max 3. Configure and start mysqld This involves `make install` and `mysql_install_db` when building from source, check README/INSTALL for details. For configuration, set whatever parameters are interesting to test. Eg. set InnoDB buffer pool size and log file/buffer size to something sensible, set innodb_flush_log_at_trx_commit=1 and sync_binlog=1. Also create a mysql account and a database with suitable access permissions. 4. Create the table to use: mysql --host=127.0.0.1 -utest -ptestpass test MariaDB [test]> create table t (a int primary key, b int) engine=innodb; 5. Create the script for Gypsy to generate the load. Just create a file simpleload.gypsy containing this single line: ----------------------------------------------------------------------- p|1|REPLACE INTO t (a,b) VALUES (? MOD 1000000, ? MOD 1000000)|int,int ----------------------------------------------------------------------- 6. Run the bemchmark. I run it with `script` to capture the output (yeah, I know it's crude). The `gypsy` binary is found in the directory where it was compiled in step (1). script for i in 1 4 8 12 16 24 32 48 64 96 128 ; do gypsy --host=127.0.0.1 --user=test --password=testpass --database=test --queryfile=simpleload.gypsy --duration=180 --threads=$i ; done Adjust host/user/password as appropriate for the installation of mariadb. 7. That should be it! This is obviously a rather crude benchmark, sorry about that, I did not have experience with something like sysbench or tcp-c, would be interesting to see results from such benchmarks also. - Kristian.

1 0

Re: [Maria-developers] Bundle more relevant cnf files in MariaDB package
by Arjen Lentz 28 Mar '11

28 Mar '11

Hi Haidong On 12/03/2011, at 7:56 AM, Haidong Ji wrote: > I alluded to this in IRC maybe a couple of weeks back. That is, it is > really to have intelligent default for MariaDB servers. there already is a baseline my.cnf which is installed by the DEB (Debian/Ubuntu) packages. You'll find it in the debian build subdirs. It can a) be updated with more parameters b) also be put into RPM, tarball and other builds. I would recommend not redoing it from scratch, as a number of practical issues have to be considered for this kind of default config, and reinventing the wheel there is just not necessary. Simply extend what's already there! > In particular, the bundled sample my.cnf files, under > /usr/local/mysql/support-files, such as my-small.cnf, my-medium.cnf, > my-large.cnf, and my-huge.cnf, are laughable in today's world. yep they are crap and should be removed. However, since they've already been replaced in the real world this is a separate issue. The my-*.cnf are not proof of "we need to create a good sample config" but rather a "this is old junk that should be removed from the bzr tree." The hindrance there is that some builds currently use them, so simply deleting the files is not sufficient. I'll see if I can find those dependencies, and create a modified tree for merging. Cheers, Arjen. -- Arjen Lentz, Exec.Director @ Open Query (http://openquery.com) Remote expertise & maintenance for MySQL/MariaDB server environments. Follow us at http://openquery.com/blog/ & http://twitter.com/openquery _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss(a)lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

1 0

[Maria-developers] Pushing/merging sample my.cnf
by Haidong Ji 25 Mar '11

25 Mar '11

All, I've created my trunk on launchpad for MariaDB, and am ready to make some updates to the sample my.cnf file(s) packaged with MariaDB. I have a few questions regarding how to proceed: 1. I've seen work log creation and assignment in MariaDB developer mailing list. Is work log creation necessary for changes like this, where no C/C++ source code is touched? If yes, any guidelines will be appreciated; 2. As this is more packaging related, what are the testing requirement? 3. What about peer review? I've blogged this and also just sent another email for more input. 4. After I make the change on my machine, build it, and push the changes back to Lauchpad, I will do a merge request. 5. Anything else? Comments/suggestions? Thanks, Haidong "Alex" Ji

2 4

[Maria-developers] review: [Commits] Rev 2946: Fix LP BUG#715738 in file:///home/tsk/mprog/src/5.3/
by Michael Widenius 23 Mar '11

23 Mar '11

Hi! >>>>> "timour" == timour <timour(a)askmonty.org> writes: timour> At file:///home/tsk/mprog/src/5.3/ timour> ------------------------------------------------------------ timour> revno: 2946 timour> revision-id: timour(a)askmonty.org-20110322132154-moeuyey4saatnm5x timour> parent: igor(a)askmonty.org-20110315193048-akp4yq0y079dkkx2 timour> committer: timour(a)askmonty.org timour> branch nick: 5.3 timour> timestamp: Tue 2011-03-22 15:21:54 +0200 timour> message: timour> Fix LP BUG#715738 timour> Analysis: timour> A query with implicit grouping is one with aggregate functions and timour> no GROUP BY clause. MariaDB inherits from MySQL an SQL extenstion timour> that allows mixing aggregate functions with non-aggregate fields. timour> If a query with such mixed select clause produces an empty result timour> set, the meaning of aggregate functions is well defined - either timour> NULL (MIN, MAX, etc.), or 0 (count(*)). However the non-aggregated timour> fields must also have some value, and the only reasonable value in timour> the case of empty result is NULL. <cut> === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2011-03-09 13:47:59 +0000 +++ b/sql/sql_lex.cc 2011-03-22 13:21:54 +0000 @@ -1667,6 +1667,11 @@ void st_select_lex::init_select() cond_value= having_value= Item::COND_UNDEF; inner_refs_list.empty(); full_group_by_flag= 0; + n_sum_items= 0; + n_child_sum_items= 0; + select_n_sum_item= 0; + select_n_nonsum_item= 0; + ok, but I am not totally happy with that we get even more things that needs to be initialized for every select (we already initialize a lot of things...) Please change the names of 'n_' to something more understandable. 'number_of_' or 'use the _count' prefix that we use in other places. === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2011-03-09 13:47:59 +0000 +++ b/sql/sql_yacc.yy 2011-03-22 13:21:54 +0000 @@ -1338,7 +1338,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** ev_alter_on_schedule_completion opt_ev_rename_to opt_ev_sql_stmt %type <ulong_num> - ulong_num real_ulong_num merge_insert_types + ulong_num real_ulong_num merge_insert_types remember_n_sum_func %type <ulonglong_number> ulonglong_num real_ulonglong_num size_number @@ -7019,31 +7019,43 @@ select_item_list: ; select_item: - remember_name select_item2 remember_end select_alias + remember_n_sum_func remember_name select_item2 remember_end select_alias { THD *thd= YYTHD; - DBUG_ASSERT($1 < $3); + DBUG_ASSERT($2 < $4); - if (add_item_to_list(thd, $2)) + if (thd->lex->current_select->n_sum_items > $1) + ++thd->lex->current_select->select_n_sum_item; + else + ++thd->lex->current_select->select_n_nonsum_item; Not sure if the counting of select_n_nonsum_item is correct. For example, we may have an item: SELECT MAX(field)+const_field from t1 ... In this case 'MAX(field)+const_field' will be counted as select_n_sum_item even if it should be counted also as select_n_nonsum_item. Not sure we can do this test properly at the parsing stage... Regards, Monty

1 0

[Maria-developers] WL#189 New (by Psergey): Pending bulk renames in server codebase
by worklog-noreply＠askmonty.org 22 Mar '11

22 Mar '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Pending bulk renames in server codebase CREATION DATE..: Tue, 22 Mar 2011, 16:31 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Client-BackLog TASK ID........: 189 (http://askmonty.org/worklog/?tid=189) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: This task is to file things that we intend to rename in the server codebase but can't do it just yet because of the merge problems it will create. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0