- developers - lists.mariadb.org

Re: [Maria-developers] MDEV-5755 - Scalability: let LOCK_open and THR_LOCK::mutex be normal mutexes
by Sergei Golubchik 03 Mar '14

03 Mar '14

Hi, Sergey! On Feb 27, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 4011 > revision-id: svoj(a)mariadb.org-20140227141521-fgj5b0h3qbijyojh > parent: svoj(a)mariadb.org-20140220072533-l3au3y7mrsyclfpb > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Thu 2014-02-27 18:15:21 +0400 > message: > MDEV-5755 - Scalability: let LOCK_open and THR_LOCK::mutex be normal mutexes > > Normal mutexes scale better than adaptive at high thread count. > At low thread count they behave similarly. To summarize Axel results: at low concurency normal can be ~1% slower. at high concurrency r/o - normal are up to 25% faster. at high concurrency r/w - normal are up to 10% slower. That last result bothers me a litte. Regards, Sergei

2 1

[Maria-developers] MDEV-5081 - Simple performance improvement for MariaDB
by Sergey Vojtovich 28 Feb '14

28 Feb '14

Hi Sergei, do you know if we ship any binaries with fast mutex (-DMY_PTHREAD_FASTMUTEX) enabled or officially suggest it? What was it's intention and what loads it was supposed to make faster? I'm trying to understand if fastmutex is worth fixing. It looks like duplication of PTHREAD_MUTEX_ADAPTIVE_NP, so we have double spinning when they're enabled. According to my recent benchmarks it shown worst performance. Thanks, Sergey

3 5

[Maria-developers] Keystone authentication plugin GSoc 14
by Abhishek Solanki 28 Feb '14

28 Feb '14

Hello MariaDB members , I am interested to work on "Keystone authentication plugin" project , i have a good knowledge of C/C++ and OpenStack. So can anyone help me how to get started with this project, so that i cold dig something before applying for this project. -- Thanks, Abhishek

2 1

[Maria-developers] Debian build / multi-architecture related bugs filed
by Otto Kekäläinen 27 Feb '14

27 Feb '14

Hello! The Debian build status can be seen at https://buildd.debian.org/status/package.php?p=mariadb-5.5 (and for MySQL at https://buildd.debian.org/status/package.php?p=mysql-5.5) The sparc build issue already got solved but for the others I filed the following bugs (as each failure seems to have different cause): https://mariadb.atlassian.net/browse/MDEV-5708 https://mariadb.atlassian.net/browse/MDEV-5707 https://mariadb.atlassian.net/browse/MDEV-5706 Could somebody please look into these? I assume that some of them might be easy if MySQL already has them solved, then all that needs to be done is to copy a patch from MySQL upstream or from MySQL patches in Debian (see http://anonscm.debian.org/gitweb/?p=pkg-mysql/mysql-5.5.git;a=tree;f=debian…) (MariaDB Debian builds are pulled from http://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-5.5.git and mirrored at https://github.com/ottok/mariadb-5.5) Also, here are two QA issues that are related. I am not sure what to do with them: http://qa.debian.org/bls/packages/m/mariadb-5.5.html

1 1

[Maria-developers] Problem with debug_sync, THD::enter_cond(), and kill ...
by Kristian Nielsen 26 Feb '14

26 Feb '14

I got a crash in my parallel replication test case, and this one turned out to be caused by a fundamental problem in debug_sync. I am not sure how to solve it, so I wanted to explain the issue in case others have some ideas. The crash happens in THD::awake() when a thread is killed: mysql_mutex_lock(&mysys_var->mutex); ... int ret= mysql_mutex_trylock(mysys_var->current_mutex); mysql_cond_broadcast(mysys_var->current_cond); if (!ret) mysql_mutex_unlock(mysys_var->current_mutex); The problem is that mysys_var->current_mutex changed between the trylock() and the unlock(), so we unlock a different mutex than the one we locked - ouch! The mutex changed because of this code in debug_sync.cc: if (thd->mysys_var) { old_mutex= thd->mysys_var->current_mutex; old_cond= thd->mysys_var->current_cond; thd->mysys_var->current_mutex= &debug_sync_global.ds_mutex; thd->mysys_var->current_cond= &debug_sync_global.ds_cond; } There is no mutex protection here. So this means that it is not safe to use debug_sync inside enter_cond()/exit_cond() if kill will be used on the thread at that point. Since I have a lot of tests for parallel replication where I test exactly for correct error handling when killing threads at various points in the parallel processing, it is not too surprising that I would eventually be hit by this :) I am going to avoid using debug_sync inside enter_cond()/exit_cond() for now. But any ideas for how to solve this properly? This is particularly nasty to be hit by, as it is not at all obvious that this use of debug_sync should be a problem. And it is not exactly easy to guess from the failure what the real problem is - assuming one can even reproduce the failure, the window of opportunity in the race is after all rather quite small. - Kristian.

3 4

[Maria-developers] How handle_rpl_parallel_thread rollback partial transaction when STOP SLAVE In parallel replication
by nanyi607rao 26 Feb '14

26 Feb '14

hi Kristian, I have a quesion about parallel replication. In handle_rpl_parallel_thread(), a worker thread has got a whole transaction events, but it only apply partial events. this thread wouldl skip left events when do STOP SLAVE(is that rigth?) ,because sql_worker_killed() return true. but it seems that partial transaction won't be rollbacked for wait_for_prior_commit() always return false. do me wrong? or how would it rollback that partial transaction. 2014-02-25 nanyi607rao

2 3

[Maria-developers] GSOC 2014 participation
by Suyash Loiwal 26 Feb '14

26 Feb '14

Hello I am Suyash Loiwal from India, currently pursuing my B.Tech from Manipal institute of Technology, Manipal. I would like to take a project under your organization. Can someone please guide me how to start? Thanks Suyash

2 1

[Maria-developers] MDEV-4309: Review for 201402181003_sanja_copy_calls_patch.diff
by Sergey Petrunia 25 Feb '14

25 Feb '14

Hi Sanja, My input for 201402181003_sanja_copy_calls_patch.diff: * memcpy_field_possible() should have a function comment mentioning that it checks whether memcpy_field_value() can be used. * field_conv_incompatible() should have a function comment mentioning that it is a function that is used instead of field_conv() when we know in advance that memcpy_field_possible(...)==false. Ok to push after this is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MariaDB 5.5.36 stable and MariaDB Galera 10.0.7 alpha release prep
by Daniel Bartholomew 25 Feb '14

25 Feb '14

All, I've completed most of the prep work for the MariaDB 5.5.36 (stable) and MariaDB Galera Cluster 10.0.7 (alpha) releases. Current plan is to activate and announce these releases together on Monday, 24 Feb before noon US Eastern Time (barring any last-minute issues). Draft changelogs and release notes are at the following locations: MariaDB 5.5.36 stable - https://mariadb.com/kb/en/mariadb-5536-release-notes/ - https://mariadb.com/kb/en/mariadb-5536-changelog/ MariaDB Galera Cluster 10.0.7 alpha - https://mariadb.com/kb/en/mariadb-galera-cluster-1007-release-notes/ - https://mariadb.com/kb/en/mariadb-galera-cluster-1007-changelog/ Please let me know ASAP of any additions, changes, typos, etc.... The 5.5.36 release notes in particular are very generic at the moment, so I'm very interested in additions to that page (please jump in and add!). The files for these releases are gradually making their way to MariaDB mirrors world wide. The primary mirror has them now if you want to get a jump on the competition: http://ftp.osuosl.org/pub/mariadb/mariadb-galera-10.0.7 http://ftp.osuosl.org/pub/mariadb/mariadb-5.5.36 Obilgatory note #1: The plan is for these files to be the officially released files without any changes, but that may change if something unexpected comes up (and such things have happened in the past). For those that would like to test the MariaDB Galera Cluster 10.0.7-alpha yum and apt repositories prior to the actual release, because that's just how you roll, the following config entries should get you going (you'll need to copy/paste/edit for whatever distro you are actually using/testing/playing-around-with): # - - Ubuntu - - - - - - - - - - - - - - - - # MariaDB 10.0.8 Ubuntu test mariadb.list w/MGC 10.0.7-alpha deb http://ftp.osuosl.org/pub/mariadb/mariadb-10.0.8/repo/ubuntu saucy main # - - Debian - - - - - - - - - - - - - - - - - # MariaDB 10.0.8 Debian test mariadb.list w/MGC 10.0.7-alpha deb http://ftp.osuosl.org/pub/mariadb/mariadb-10.0.8/repo/debian wheezy main # - - Red Hat - - - - - - - - - - - - - - - - # MariaDB 10.0.8 RedHat MariaDB-test.repo w/ MGC 10.0.7-alpha [mariadb] name = MariaDB baseurl = http://yum.mariadb.org/10.0.8-upd/rhel6-amd64 gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB gpgcheck=1 # - - Fedora - - - - - - - - - - - - - - - - - # MariaDB 10.0.8 Fedora MariaDB-test.repo w/ MGC 10.0.7-alpha [mariadb] name = MariaDB baseurl = http://yum.mariadb.org/10.0.8-upd/fedora20-amd64 gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB gpgcheck=1 Obligatory note #2: These test alpha repos, if you use them, should be switched to the official ones (as produced by the repository configuration generator) after the release is officially out. Obligatory disclaimer #1: These test alpha repos are _completely_ untested. They may not work at all. Obligatory disclaimer #2: MariaDB Galera Cluster 10.0.7 is an ALPHA release. You have been warned. You're on your own. Good luck and Godspeed. :) Thanks. -- Daniel Bartholomew MariaDB

4 4

[Maria-developers] MySQL Optimizer Trace Questions
by Christian Rober 25 Feb '14

25 Feb '14

Hello All, I work for TokuTek as a Storage Engine developer and I have been using the, relatively, new optimizer trace functionality. I have found the results very helpful when combined with analyzing the source code, but there a few questions I want to ask. Here is some output from the trace for a recent SELECT query on an InnoDB table: At the end of the trace, it shows the index chosen by, what I presume, is the optimizer: ... "considered_execution_plans": [ { "plan_prefix": [], "table": "`reports` `a`", "best_access_path": { "considered_access_paths": [ { "access_type": "ref", "index": "reports_group_id", "rows": 1.55e6, "cost": 411057, "chosen": true }, ... ] }, "cost_for_plan": 411057, "rows_for_plan": 1.55e6, "chosen": true } ] }, ... Here are the cost results for some rejected alternative indexes, (note: the row count for two of them is similar to the chosen index): ... "analyzing_range_alternatives": { "range_scan_alternatives" ... "rows": 2377102, "cost": 2.81e6, ... "rows": 1554164, "cost": 1.84e6, ... "rows": 2346664, "cost": 2.78e6, ... "rows": 1554164, "cost": 1.84e6, ... Here is the cost analysis for a table scan, from the beginning of the trace (still not as good as the above chosen index, but close on cost): ... "table": "`reports` `a`", "range_analysis": { "table_scan": { "rows": 3108329, "cost": 655076 }, ... Related to the fact that the table scan analysis is dominated by the large row count, and probably rejected as a valid plan compared to the chosen index for that very reason: 1. How is the row count more than merely informational to the optimizer? I had the impression the row count did not directly contribute to the optimizer's decision as much as the estimated cost, or that the row count was already accounted for/factored into in the final cost measurement. 2. Related to that question, what is cost exactly? Is there a unit of measurement (ex: Time, Disk Seeks, etc.)? 3. Is cost more important than row count or equally important? 4. What is the genesis of the cost variable displayed in the trace? I assume it is just a guess on the respective storage engine's part of how expensive (in terms of disk access, processing, time, etc.) getting the rows will be for the query. thanks in advance, Christian

2 1