- developers - lists.mariadb.org

[Maria-developers] Reviews of parallel replications
by Michael Widenius 10 Dec '13

10 Dec '13

Hi! Now continuing with reviews of the parallel replication code. Will take one patch at a time with a target to have everything reviewed this week. Review of revno 3678: Parallel replication: error handling. ------------------------------------------------------------ revno: 3678 committer: knielsen(a)knielsen-hq.org branch nick: work-10.0-mdev4506 timestamp: Mon 2013-10-14 15:28:16 +0200 message: MDEV-4506: Parallel replication: error handling. Add an error code to the wait_for_commit facility. Now, when a transaction fails, it can signal the error to any subsequent transaction that is waiting for it to commit. The waiting transactions then receive the error code back from wait_for_prior_commit() and can handle the error appropriately. Also fix one race that could cause crash if @@slave_parallel_threads were changed several times quickly in succession. <cut> === modified file 'sql/handler.cc' --- sql/handler.cc 2013-10-13 21:24:05 +0000 +++ sql/handler.cc 2013-10-14 13:28:16 +0000 @@ -1458,10 +1458,11 @@ int ha_commit_one_phase(THD *thd, bool a transaction.all.ha_list, see why in trans_register_ha()). */ bool is_real_trans=all || thd->transaction.all.ha_list == 0; + int res; DBUG_ENTER("ha_commit_one_phase"); - if (is_real_trans) - thd->wait_for_prior_commit(); - int res= commit_one_phase_2(thd, all, trans, is_real_trans); + if (is_real_trans && (res= thd->wait_for_prior_commit())) + DBUG_RETURN(res); + res= commit_one_phase_2(thd, all, trans, is_real_trans); DBUG_RETURN(res); Another way to write the above is: if (is_real_trans && !(res= thd->wait_for_prior_commit())) res= commit_one_phase_2(thd, all, trans, is_real_trans); DBUG_RETURN(res); <cut> I was checking how ha_commit_one_phase() was used. In ha_commit_trans() or trans_xa_commit(), if ha_commit_one_phase() fails, it will not run rollback. Is this right? I would have assumed that on failure, ha_commit_one_phase() should run ha_rollback_trans() before returning. === modified file 'sql/log.cc' @@ -7844,7 +7844,8 @@ int TC_LOG_MMAP::log_and_order(THD *thd, mysql_mutex_unlock(&LOCK_prepare_ordered); } - thd->wait_for_prior_commit(); + if (thd->wait_for_prior_commit()) + return 0; What does the return 0 mean here? Is it an error? Is thd->is_fatal_error assumed to be set if this happens? A comment here would not be totally wrong... @@ -5633,7 +5634,7 @@ wait_for_commit::~wait_for_commit() void -wait_for_commit::wakeup() +wait_for_commit::wakeup(int wakeup_error) { /* We signal each waiter on their own condition and mutex (rather than using @@ -5649,6 +5650,7 @@ wait_for_commit::wakeup() */ mysql_mutex_lock(&LOCK_wait_commit); waiting_for_commit= false; + this->wakeup_error= wakeup_error; Consider adding an assert that 'this->wakeup_error' is 0. Good way to ensure we don't accidently call this function twice... (as things would be bad if we first would call it with an error and then with 0) mysql_mutex_unlock(&LOCK_wait_commit); mysql_cond_signal(&COND_wait_commit); } Regards, Monty

1 0

[Maria-developers] (MDEV-4403) Attempting to use cassandra storage engine causes "service 'my_snprintf_service' interface version mismatch"
by Sergei Golubchik 10 Dec '13

10 Dec '13

Hi. I don't see my commit emails on the list, so here you are: first patch is the bug fix itself. second - cleanup of the existing code after the bug fix. Regards, Sergei

2 1

[Maria-developers] Coverity
by Christian Convey 10 Dec '13

10 Dec '13

Hi guys, I'm afraid I won't have the time available that I'd hope to, for analyzing MariaDB with Coverity Prevent. Would anyone like to take over from me the Coverity MariaDB account? Thanks, Christian

2 1

[Maria-developers] plugin doesn't load
by AugustQ 06 Dec '13

06 Dec '13

Hi, just one question: I have a simple plugin (a minimal storage engine) that does not load. When I try to load it I get this error-message: MariaDB [(none)]> install plugin dummy soname 'ha_dummy'; ERROR 1126 (HY000): Can't open shared library '/home/august/MariaDB/pgm/lib/plugin/ha_dummy.so' (errno: 2 undefined symbol: _ZN9ha_myisam5cloneEPKcP11st_mem_root) Can you please give me a tip. Thanks. Environment is MariaDB 10.0.4. The same code works without any problems in MySQL (5.5.8 - just for a test). The code is a minimal engine that just does nothing, I want to use it as a template for further work. this is the code in the form that it is working (the relevant parts): in the header-file: class ha_dummy: public handler in the code: ha_dummy::ha_dummy(handlerton *hton, TABLE_SHARE *table_arg) :handler(hton, table_arg) now I change this by modifying the inheritance-order: in the header file: added: #include "../myisam/ha_myisam.h" #include "../myisam/myisamdef.h" modified: class ha_dummy: public ha_myisam in the code: /* Constructor */ ha_dummy::ha_dummy(handlerton *hton, TABLE_SHARE *table_arg) :ha_myisam(hton, table_arg) It compiles but I cannot load it. What's the problem with st_mem_root? Thanks AugustQ PS: as written before: both versions work with MySQL.

3 4

Re: [Maria-developers] [Commits] Rev 3732: Fixed bug mdev-5382 in file:///home/igor/maria/maria-5.3-mdev5382/
by Oleksandr Byelkin 06 Dec '13

06 Dec '13

Hi! 05.12.2013 21:13, Igor Babaev пишет: > At file:///home/igor/maria/maria-5.3-mdev5382/ > > ------------------------------------------------------------ > revno: 3732 > revision-id: igor(a)askmonty.org-20131205191320-pfja761po2x9p9ri > parent: bar(a)mnogosearch.org-20131203110843-1bsdu0hcrpzhie8w > committer: Igor Babaev <igor(a)askmonty.org> > branch nick: maria-5.3-mdev5382 > timestamp: Thu 2013-12-05 11:13:20 -0800 > message: > Fixed bug mdev-5382 > When marking used columns the function find_field_in_table_ref() erroneously > called the walk method for the real item behind a view/derived table field > with the second parameter set to TRUE. > This erroneous code was introduced in 2006. OK to push! > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] RQG crashes tokudb due to unexpected storage engine API call sequence
by Rich Prohaska 06 Dec '13

06 Dec '13

We are debugging a crash induced by RQG in the tokudb storage engine. The cause of the crash is a mismatch in assumptions about the sequence of storage engine API calls. We see: tokudb::external_lock creates a txn tokudb::index_init creates a cursor on a index db with the txn tokudb::index_read use the cursor to try to read a row, and returns an error tokudb_commit commits the txn (called from sql_parse.cc:4584) tokudb::index_end tries to close the cursor and crashes since the txn has been committed. (called from the unit.cleanup() call at sql_parse.cc:4592) The current tokudb software does not expect the commit to occur before index_end is called. Is this assumption correct? Since mariadb (and i suspect mysql) does this, it appears that the storage engine should expect commit before index_end. Any comments? Thanks Rich Prohaska

2 2

[Maria-developers] MDEV-5345 - Deadlock between mysql_change_user(), SHOW VARIABLES and INSTALL PLUGIN
by Sergey Vojtovich 05 Dec '13

05 Dec '13

Hi Sergei, I'm afraid I couldn't find easy fix for this deadlock. The question is if it is worth to spend time for more complex (and possibly less stable) solution. Details below. Locking profile --------------- INSTALL PLUGIN: LOCK_plugin -> LOCK_system_variables_hash (w) SHOW VARIABLES: LOCK_system_variables_hash (r) -> LOCK_global_system_variables change_user(): LOCK_global_system_variables -> LOCK_plugin INSTALL PLUGIN -------------- Needs LOCK_plugin to register new plugin. Needs LOCK_system_variables_hash to register plugin variables. Doesn't seem to need both locks at the same time. Similar lock order is used in a few places. SHOW VARIABLES -------------- Needs LOCK_system_variables_hash to iterate system_variable_hash. Needs LOCK_global_system_variables to read variable value. Does seem to need both locks at the same time. This lock order is not used in other places. change_user() ------------- Needs LOCK_global_system_variables to read global_system_variable.table_plugin. Needs LOCK_plugin to my_plugin_lock(table_plugin). Does seem to need both locks at the same time. Similar lock order is used in a few places. "SHOW VARIABLES" looks correct. So I have only two ideas: - Fix INSTALL PLUGIN so it releases LOCK_plugin before registering variables. Sounds like the best solution, but there are a few more things to fix, e.g. UNINTALL PLUGIN. - Store default storage engine as a string, not Sys_var_plugin. Thanks, Sergey

2 3

[Maria-developers] Build fail of MariaDB10 with TokuDB enabled
by darx＠sent.com 03 Dec '13

03 Dec '13

On the (eventual) road to successfully building MariaDB10 + TokuDB, Building cd src/mariadb bzr log | head ------------------------------------------------------------ revno: 3911 committer: Alexander Barkov <bar(a)mnogosearch.org> branch nick: maria-10.0 timestamp: Tue 2013-11-26 10:53:21 +0400 message: Fixing malformed data in mysql-test/std_data/Index.xml ------------------------------------------------------------ revno: 3910 committer: Sergey Vojtovich <svoj(a)mariadb.org> on uname -a Linux dev1 3.7.10-1.16-desktop #1 SMP PREEMPT Fri May 31 20:21:23 UTC 2013 (97c14ba) x86_64 x86_64 x86_64 GNU/Linux lsb_release -ar LSB Version: core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch Distributor ID: openSUSE project Description: openSUSE 12.3 (x86_64) Release: 12.3 Codename: Dartmouth with gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-suse-linux/4.8/lto-wrapper Target: x86_64-suse-linux Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.8 --enable-ssp --disable-libssp --disable-plugin --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --enable-linker-build-id --enable-linux-futex --program-suffix=-4.8 --without-system-libunwind --with-arch-32=i586 --with-tune=generic --build=x86_64-suse-linux --host=x86_64-suse-linux Thread model: posix gcc version 4.8.2 20131016 [gcc-4_8-branch revision 203692] (SUSE Linux) cmake --version cmake version 2.8.12.1 applying a needed src patch cd /usr/local/src/mariadb # BUG @ http://www.linuxfromscratch.org/blfs/view/svn/server/mariadb.html sed -i "s/srv_buf_size/srv_sort_buf_size/" storage/innobase/row/row0log.cc after cmake, with gcc c++11 support NOT enabled, mkdir bld cd bld cmake .. \ ... \ -DWITHOUT_TOKUDB=1 build completes successfully, make ... ldd sql/mysqld linux-vdso.so.1 (0x00007fffb9dff000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9031e86000) libaio.so.1 => /lib64/libaio.so.1 (0x00007f9031c84000) libz.so.1 => /lib64/libz.so.1 (0x00007f9031a6e000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f9031832000) libssl.so.1.0.0 => /usr/local/ssl/lib64/libssl.so.1.0.0 (0x00007f90315c6000) libcrypto.so.1.0.0 => /usr/local/ssl/lib64/libcrypto.so.1.0.0 (0x00007f90311da000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f9030fd6000) libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f9030cce000) libm.so.6 => /lib64/libm.so.6 (0x00007f90309d0000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f90307b9000) libc.so.6 => /lib64/libc.so.6 (0x00007f903040b000) /lib64/ld-linux-x86-64.so.2 (0x00007f90320a2000) repeating, but with TokuDB build enabled, cd bld cmake .. \ ... \ - -DWITHOUT_TOKUDB=1 + -DWITHOUT_TOKUDB=0 build now fails, make ... [ 29%] Built target perfschema Scanning dependencies of target tokuportability_static_conv [ 29%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/huge_page_detection.cc.o [ 29%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/file.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/memory.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/os_malloc.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/portability.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_assert.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_crash.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_fair_rwlock.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_path.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_pthread.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_time.cc.o Linking CXX static library libtokuportability_static_conv.a [ 30%] Built target tokuportability_static_conv [ 30%] Generating tokuportability_static_depends.cc Scanning dependencies of target tokuportability_static [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static.dir/tokuportability_static_depends.cc.o Linking CXX static library libtokuportability_static.a /usr/bin/ar: creating /usr/local/src/mariadb/bld/storage/tokudb/ft-index/portability/libtokuportability_static.a [ 30%] Built target tokuportability_static Scanning dependencies of target tokuportability [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/huge_page_detection.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/file.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/memory.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/os_malloc.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/portability.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_assert.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_crash.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_fair_rwlock.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_path.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_pthread.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_time.cc.o Linking CXX shared library libtokuportability.so [ 31%] Built target tokuportability Scanning dependencies of target logformat [ 31%] Building CXX object storage/tokudb/ft-index/ft/CMakeFiles/logformat.dir/logformat.cc.o Linking CXX executable logformat [ 31%] Built target logformat Scanning dependencies of target generate_log_code [ 31%] Generating log_code.cc, log_print.cc, log_header.h ./logformat: error while loading shared libraries: libtokuportability.so: cannot open shared object file: No such file or directory make[2]: *** [storage/tokudb/ft-index/ft/log_code.cc] Error 127 make[1]: *** [storage/tokudb/ft-index/ft/CMakeFiles/generate_log_code.dir/all] Error 2 make: *** [all] Error 2 yes, I'm aware that opensuse is not a currently "supported platform" for TokuDB. If that's a dealbreaker, I'd like to know MariaDB project's position, if any, on getting that support in place. If it's not, I'd like to keep trying to get it working, here. darx

4 15

Re: [Maria-developers] MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables
by Sergei Golubchik 03 Dec '13

03 Dec '13

Hi, Sergey! On Aug 27, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 3807 > revision-id: svoj(a)mariadb.org-20130827121233-xh1uyhgfwbhedqyf > parent: jplindst(a)mariadb.org-20130823060357-pww92qxla7o8iir7 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Tue 2013-08-27 16:12:33 +0400 > message: > MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables > > - tc_acquire_table and tc_release_table do not access > TABLE_SHARE::tdc.used_tables anymore > - in tc_acquire_table(): release LOCK_tdc after we relase LOCK_open > (saves a few CPU cycles in critical section) > - in tc_release_table(): if we reached table cache threshold, evict > to-be-released table without moving it to unused_tables. unused_tables > must be empty at this point (added assertion). I don't understand what you're doing here, could you elaborate? E.g. explain in a changeset comment what you've done, why you introduced a new list for tables (all_share), what is its semantics, etc. Regards, Sergei

3 12

[Maria-developers] Building TOKUDB into 5.5.34 from sources
by md＠rpzdesign.com 03 Dec '13

03 Dec '13

I am trying to build 5.5.34 from source on Linux x64 (Fedora 14). I was looking at the readme files and could not find anything about building 5.5.34 from source and including TokuDB (which has the source code in the /storage/TokuDB/ directory) into the build. I was looking for something like: -DWITH_TOKUDB_STORAGE_ENGINE=ON but did not stumble across anything. Anybody got some ideas? Thanks, Marco Here is my build script: rm -f CMakeCache.txt make clean cmake \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=/mysql \ -DENABLED_PROFILING=OFF \ -DENABLE_DEBUG_SYNC=ON \ -DENABLE_GCOV=OFF \ -DINSTALL_LAYOUT=STANDALONE \ -DMYSQL_DATADIR=/mysql/data \ -DINSTALL_INCLUDEDIR=include/mysql \ -DINSTALL_UNIX_ADDRDIR=/mysql/mysql.sock \ -DUSE_ARIA_FOR_TMP_TABLES=ON \ -DWITHOUT_PBXT_STORAGE_ENGINE=ON \ -DWITHOUT_SERVER=OFF \ -DWITH_ARCHIVE_STORAGE_ENGINE=OFF \ -DWITH_EMBEDDED_SERVER=OFF \ -DWITH_FAST_MUTEXES=OFF \ -DWITH_FEDERATEDX_STORAGE_ENGINE=OFF \ -DWITH_FEEDBACK=OFF \ -DWITH_LIBWRAP=OFF \ -DWITH_PIC=ON \ -DWITH_PARTITION_STORAGE_ENGINE=OFF \ -DWITH_PERFSCHEMA_STORAGE_ENGINE=ON \ -DWITH_SPHINX_STORAGE_ENGINE=OFF \ -DWITH_SSL=OFF \ -DWITH_UNIT_TESTS=OFF \ -DWITH_VALGRIND=OFF \ -DWITH_XTRADB_STORAGE_ENGINE=ON \ -DWITHOUT_OQGRAPH=ON \ -DWITH_ZLIB=bundled make (Sorry, I posted this question on maria.com knowledge base open questions also) https://mariadb.com/kb/en/cmake-settings-to-include-tokudb-in-5534/

3 2

[Maria-developers] Movable conditions and semi-joins
by Sergei Petrunia 28 Nov '13

28 Nov '13

Hi! The below is an extension of the skype discussion. I wrote down the limitation imposed by FirstMatch, and also covered other Semi-join strategies. <contents> 1. SJ-Materialization 2. FirstMatch 3. Duplicate Elimination 4. Loose Scan </contents> == 1. SJ-Materialization == The general rule is: If the condition is originally attached inside an SJM nest, one should not move it outside. Conditions that are originally outside should not be moved inisde the SJM nest. The reason for the rule is that field values for inner tables are only available inside the SJM nest. Correspondingly, values of fields of outer tables are not available at materialization step. The only possible exception is when the condition depends only on columns that are in the IN-equality. Then, it could be moved. I don't think this is worth handling for the first milestone. == 2. FirstMatch == Consider a query select ... from ot1, nt1, nt2 where expr(ot1) in (select ... from it1, it2, ... where ...) and a join order ot1 FirstMatch(it1, it2) nt1 nt2 It is possible to move cond(otX) to table itX or ntX. There is a problem with moving condition attached to itX (will call it SUBQ_COND. this ocndition depends on table itX and maybe on ot1) to table ntX. Consider join execution execution: ot1.row0 ot1.row1 -+- it1.row1 -- it2.row1 | +- it1.row2 -- it2.row2 ->- (F1)-->-- nt1.row1 (F2)--<-- FirstMatch check is at point (F2). At that point, we know that ot1.row1 has had a match in the subquery, and the execution returns to table ot1. If SUBQ_COND is moved to table ntX, then there may be a situation where we will get to point (F2) and assume that the subquery had a match, while actually it hasn't. A suggestion by Igor: at point (F2) we should check the last return value of SUBQ_COND (actually, of all conditions that were moved like SUBQ_COND was) It is actually more complicated: what if there is COND(ot1, nt1) such that COND(ot1.ro1, nt1.row1) = FALSE? Then, we will never evaluate SUBQ_COND(ot1.row1, ...). What if never evaluate SUBQ_COND(ot1.row1, ...) but before we have evaluated SUBQ_COND(ot1.row0, ...)? We will have to track what we have evaluated for the current row of ot1.row1. == 3. Duplicate Elimination == Duplicate Elimination removes duplicate matches using a temporary table. In order to work correctly, filtering needs to be performed before the weedout is done. Let's again use an example. select ... from ot1, nt1, nt2 where expr(ot1) in (select ... from it1, it2, ... where ...) and a join order (DUPS-W-START) it1 ot1 it2 (DUPS-W-WEEDOUT) nt1 nt2 at DUPS-W-START, all rows are deleted from the temp. table. at DUPS-W-WEEDOUT, we take ot1.rowid and attempt to write it into the temp table. Basic idea: If a condition SUBQ_COND(itX, ...) is moved to the right across the (DUPS-W-WEEDOUT) point, wrong query results will occur. Example: Suppose, SUBQ_COND(it2) is moved to nt1, and join execution proceeds as follows: it1.row1 --- ot1.row1 --- it2.row1 -- DUPS-W-WEEDOUT -- nt1 | - We reach DUPS-W-WEEDOUT with (it1.row1, ot1.row1, it2.row1). - The temp. table is empty, so rowid(ot1.row1) is written into temp.table, and Weedout doesn't weed it out. - At nt1, SUBQ_COND(it2.row1) is evaluated and found to be false. join execution returns back. - Suppose, tables it2 and ot1 have no more matching rows, so execution returns to it1. (the same diagram with a few more steps) it1.row1 --- ot1.row1 --- it2.row1 -- DUPS-W-WEEDOUT -- nt1 | it1.row2 --- ot1.row1 --- it2.row2 -- DUPS-W-WEEDOUT - We reach DUPS-W-WEEDOUT with (it1.row2, ot1.tow1, it2.rowX). - The temptable already has rowid(ot1.row1). The record combination is filtered out. if SUBQ_COND(it2.row2)=TRUE, then the result will miss a row. == 4. Loose Scan == Loose Scan is similar to First Match - it does short-cutting (but also makes forward jumps on the driving table). Short-cutting imposes a limitation similar to FirstMatch - conditions that depend on subquery's tables cannot be moved out of LooseScan's range. BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] maria-tokudb-engine-10.0 deb package (raring) - if files are empty where are sources?
by Paul Hewson 27 Nov '13

27 Nov '13

Hello, I'm trying to follow the instructions on https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/ which imply there is more tuned version of tokudb for mariadb than I might get by following instructions at tokutek. However, I have ubuntu 13.04 (raring), and despite some mismatch in numbers (I have maria 10.0 rather than 5.5) I am getting an empty file when I try to apt-get install mariadb-tokudb-engine-10.0 I found a bit of discussion about this point, but got the impression it had been fixed. Does it mean it was fixed and the debs were removed? If so, do you mind if I ask where I get the mariadb tokutek sources. Again the webpage https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/ implies only some parts (most parts) of totudb have been merged into mariadb. Thanks, sorry for troubling you with naive issues like this. Paul ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 2

[Maria-developers] Malformed XML in mysql-test/std_data/Index.xml added in revision 3873
by Jeremy Cole 26 Nov '13

26 Nov '13

Bar, Regarding your change: http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/3873/mysql-… I was wondering if it's intentional that mysql-test/std_data/Index.xml now contains malformed XML. It wasn't mentioned in your commit message and the test nor XML files do not mention it specifically. Although utf8_5624_5_bad is supposed to be "bad" I didn't think it was supposed to be *that* "bad" (mismatched "p" vs "a" tags): $ xmllint --noout mysql-test/std_data/Index.xml mysql-test/std_data/Index.xml:126: parser error : Opening and ending tag mismatch: p line 126 and a <reset>a-a4</reset>xxx04</a> ^ mysql-test/std_data/Index.xml:127: parser error : Opening and ending tag mismatch: p line 127 and a <reset>a-aa5</reset>xxx05</a> ^ mysql-test/std_data/Index.xml:128: parser error : Opening and ending tag mismatch: p line 128 and a <reset>a-aaa6</reset>xxx06</a> ^ mysql-test/std_data/Index.xml:129: parser error : Opening and ending tag mismatch: p line 129 and a <reset>a-aaaa7</reset>xxx07</a> ^ mysql-test/std_data/Index.xml:130: parser error : Opening and ending tag mismatch: p line 130 and a <reset>a-aaaaa8</reset>xxx08</a> ^ mysql-test/std_data/Index.xml:131: parser error : Opening and ending tag mismatch: p line 131 and a <reset>a-aaaaaa9</reset>xxx09</a> ^ mysql-test/std_data/Index.xml:132: parser error : Opening and ending tag mismatch: p line 132 and a <reset>a-aaaaaa10</reset>xxx10</a> ^ Regards, Jeremy

2 1

[Maria-developers] Force PK option and global lock for DDLs
by erkan yanar 26 Nov '13

26 Nov '13

Ahoi, I would like to propagate two ideas (did it on #irc some times but still some love is missing) 1. Force PK option. I would like to have an option to force PK for (innodb) tables. Forcing is quite easy (a poc from 2011): http://linsenraum.de/erkules/2011/03/this-table-type-requires-a-primary-key… Why? RBR and so Galera would benefit from such an option. 2. Global DDL Lock Without Global DDL Lock i.e. ALTER TABLE, DROP TABLE etc. brings you in trouble while doing a --single-transaction dump. You can use flush tables with read lock, but then you need not to do a single-transaction dump anyway. Even xtrabackup got troubles: https://bugs.launchpad.net/percona-xtrabackup/+bug/722638 I got to confess I think ignoring tablespace ids is the right soltution. Even I think this are simple options. I believe they would be quite welcome to a lot of dba's out there. Regards Erkan -- über den grenzen muß die freiheit wohl wolkenlos sein

1 1

Re: [Maria-developers] [Commits] Rev 3908: MDEV-5277 - Ensure that all MySQL 5.6 options are supported
by Sergei Golubchik 25 Nov '13

25 Nov '13

Hi, Sergey! On Nov 25, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 3908 > revision-id: svoj(a)mariadb.org-20131125144940-nhdqjt2eisqaa7pb > parent: svoj(a)mariadb.org-20131114140000-5eq8gopqu5giwvqq > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-mdev5277 > timestamp: Mon 2013-11-25 18:49:40 +0400 > message: > MDEV-5277 - Ensure that all MySQL 5.6 options are supported by the > MariaDB 10.0 server Looks ok to me, thanks! Regards, Sergei

1 0

Re: [Maria-developers] Rev 3985: MDEV-5321 Calling mysql_library_end accesses freed memory; dumps memory to display
by Sergei Golubchik 25 Nov '13

25 Nov '13

Hi, Holyfoot! On Nov 24, holyfoot(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 3985 > revision-id: holyfoot(a)askmonty.org-20131124184915-spkpy6qt8vdbluaz > parent: igor(a)askmonty.org-20131123023813-0mgl217lyyyysv5b > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-5321 > timestamp: Sun 2013-11-24 22:49:15 +0400 > message: > MDEV-5321 Calling mysql_library_end accesses freed memory; dumps memory to display. > Don't call the vio_end() in the clean_up() in EMBEDDED mode. > Call vio_end() before the end_embedded_server(). ok to push Regards, Sergei

1 0

Re: [Maria-developers] Rev 3908: MDEV-5277 - Ensure that all MySQL 5.6 options are supported
by Sergei Golubchik 25 Nov '13

25 Nov '13

Hi, Sergey! On Nov 20, Sergey Vojtovich wrote: > revno: 3908 > revision-id: svoj(a)mariadb.org-20131120113753-vuxj3v4jt9cvccdy > parent: svoj(a)mariadb.org-20131114140000-5eq8gopqu5giwvqq > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-mdev5277 > timestamp: Wed 2013-11-20 15:37:53 +0400 > message: > MDEV-5277 - Ensure that all MySQL 5.6 options are supported by the > MariaDB 10.0 server > === modified file 'mysql-test/r/mysqld--help.result' > --- a/mysql-test/r/mysqld--help.result 2013-11-03 20:26:44 +0000 > +++ b/mysql-test/r/mysqld--help.result 2013-11-20 11:37:53 +0000 > @@ -4,6 +4,9 @@ > --defaults-file=# Only read default options from the given file #. > --defaults-extra-file=# Read this file after the global files are read. > > + --abort-slave-event-count[=name] > + MySQL 5.6 compatibility option. Use > + '--debug-abort-slave-event-count' instead. > --allow-suspicious-udfs Do we really want all these do-nothing compatibility options to show up in --help? May be not? The goal of this MDEV was not to fail with an error when starting on mysqld's my.cnf file, no one needs --help for that. > Allows use of UDFs consisting of only one symbol xxx() > without corresponding xxx_init() or xxx_deinit(). That > @@ -136,10 +151,16 @@ > --deadlock-timeout-short=# > Short timeout for the two-step deadlock detection (in > microseconds) > + --default-authentication-plugin[=name] > + MySQL 5.6 compatibility option. To be implemented in > + later 10.x versions. Remove "10.x" please, just "in later versions". We don't want to promise that all these options will be implemented in 10.x series. > === modified file 'storage/innobase/handler/ha_innodb.cc' > --- a/storage/innobase/handler/ha_innodb.cc 2013-11-13 21:58:19 +0000 > +++ b/storage/innobase/handler/ha_innodb.cc 2013-11-20 11:37:53 +0000 > @@ -15750,6 +15750,12 @@ static MYSQL_SYSVAR_BOOL(purge_stop_now, > PLUGIN_VAR_OPCMDARG, > "Set purge state to STOP", > NULL, purge_stop_now_set, FALSE); > + > +static my_bool innodb_log_checkpoint_now; > +static MYSQL_SYSVAR_BOOL(log_checkpoint_now, innodb_log_checkpoint_now, > + PLUGIN_VAR_OPCMDARG | PLUGIN_VAR_NOSYSVAR, > + "MySQL 5.6 compatibility option. To be implemented in later 10.x versions.", > + NULL, NULL, FALSE); > #endif /* UNIV_DEBUG */ Why is that (and everything else in ha_innodb.cc) necessary? We'll merge with the latest InnoDB, we will have all its variables. Regards, Sergei

2 3

[Maria-developers] MariaDB-10.0-beta sysbench results
by Axel Schwenke 23 Nov '13

23 Nov '13

Hi, I have run another sysbench with in-memory workload to check where we are with respect to mutex contention. The benchmark is sysbench OLTP (using the bzr trunk of sysbench with LUA support). 10 mio rows in a single table (about 2.5G .ibd file). InnoDB buffer pool set to 32G and warmed up. I have run the test on two different machines. First on our old trusty facebook2 machine with 8 cores/16 threads in two NUMA zones. Second on the fat lizard2 machine with 32 cores/64 threads in 4 NUMA zones. As expected, the heavy contention of the LOCK_open mutex is more visible on the bigger machine (I have not done any detailed analysis in this benchmark, but before while doing benchmarks for MDEV-5081) I notice a visible performance gain in MariaDB-10.0 over MariaDB-5.5 at 32 threads. This is probably due to the work on the InnoDB kernel mutex (impossible to prove since 5.5 doesn't have mutex instrumentation). At higher concurrency however the LOCK_open mutex kicks in and performance is clearly dominated by that mutex alone (it flattens out for all 5.5 and 10.0 versions of MariaDB) The good news is, that 10.0.4, 10.0.5 and 10.0.6 are quite close. There is a small regression from 10.0.4 to 10.0.5, but IMHO this isn't worth investigation while the LOCK_open issue is not fixed. Attached: diagrams. OpenOffice spreadsheets. XL

6 7

[Maria-developers] get is now a reserved word in mariadb 10 ?
by Jocelyn Fournier 21 Nov '13

21 Nov '13

Hi, I have a query with a column named "get". i.e. SELECT get FROM my_table... It works perfectly in 5.5, but in 10.0.6 it fails : ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'get FROM my_table' at line 1 I assume "get" is a new reserved word, probably inherited from MySQL 5.6 memcached support, although memcached is not supported in MariaDB ? (it's mentionned here : http://dev.mysql.com/doc/mysqld-version-reference/en/mysqld-version-referen… ) Is there a specific KB entry for MariaDB 10.0.6 reserved words ? (I only see an entry for 5.5 here : https://mariadb.com/kb/en/reserved-words/ ) Thanks and regards, Jocelyn Fournier

4 4

[Maria-developers] Prep for MariaDB 5.5.34
by Daniel Bartholomew 21 Nov '13

21 Nov '13

I'm currently in the process of preparing for the MariaDB 5.5.34 release. Draft changelog and release notes are here: - https://mariadb.com/kb/en/mariadb-5534-release-notes/ - https://mariadb.com/kb/en/mariadb-5534-changelog/ As always, fixes, additions, and such are welcome. Buildbot is still working on building the packages and when it's finished I'll get them prepped, signed, and uploaded to the mirrors with the plan to release tomorrow morning US Eastern Time. Thanks. -- Daniel Bartholomew MariaDB

1 0

[Maria-developers] MariaDB 10.0.6 using more file descriptors than 5.5 ?
by Jocelyn Fournier 20 Nov '13

20 Nov '13

Hi, I've just migrated MariaDB (+tokudb) from 5.5.33a to 10.0.6, and I've encountered a '[ERROR] Error in accept: Too many open files' error during a backup. As this was not occuring with MariaDB 5.5, is this expected to have 10.0.6 using much more file descriptors ? (open_files_limit was set to about 8100 with 5.5, I raised it to 65535 now to fix this issue). Thanks, Jocelyn Fournier

3 3

[Maria-developers] MDEV-5277 - Ensure that all MySQL 5.6 options are supported by the MariaDB 10.0 server
by Sergey Vojtovich 20 Nov '13

20 Nov '13

Hi! I'm gathering 5.6 options that are not present in 10.0 and the way we want to handle them. Could you check options listed below and if you're aware of their destiny indicate if options: * not used or needed in MariaDB * to be implemented in later 10.x versions * tuned via different 10.x specific options The following options exist in 5.6 but not in 10.0: * default-tmp-storage-engine * log-raw * default_authentication_plugin * ignore-db-dir * binlog-max-flush-queue-time * binlog-row-image * explicit-defaults-for-timestamp * master-info-repository * relay-log-info-repository * binlog-rows-query-log-events * binlog-order-commits * log-throttle-queries-not-using-indexes * end-markers-in-json * optimizer-trace * optimizer-trace-features * optimizer-trace-offset * optimizer-trace-limit * optimizer-trace-max-mem-size * eq-range-index-dive-limit * server-uuid * server-id-bits * slave-rows-search-algorithms * table-open-cache-instances * slave-allow-batching * slave-checkpoint-period * slave-checkpoint-group * slave-parallel-workers * slave-pending-jobs-size-max * enforce-gtid-consistency * gtid-mode * disconnect-on-expired-password * sha256-password-private-key-path (sha256_password plugin) * sha256-password-public-key-path (sha256_password plugin) The following options exist in 5.5 and 5.6 but not in 10.0: * abort-slave-event-count => debug-abort-slave-event-count * disconnect-slave-event-count => debug-disconnect-slave-event-count * exit-info => debug-exit-info * max-binlog-dump-events => debug-max-binlog-dump-events * sporadic-binlog-dump-fail => debug-sporadic-binlog-dump-fail * new * skip-new * skip-stack-trace The following options were added after 5.6.10: * innodb-buffer-pool-evict // UNIV_DEBUG * innodb-ft-total-cache-size * innodb-ft-result-cache-limit * innodb-log-compressed-pages * innodb-log-checkpoint-now // UNIV_DEBUG * rpl-stop-slave-timeout * validate-user-plugins The following NO_CMD_LINE options exist in 5.6 but not in 10.0 (won't add them): * gtid-executed * gtid-next * gtid-next-list * gtid-owned * gtid-purged * log-bin-basename * relay-log-basename * transaction-allow-batching Thanks, Sergey

4 12

Re: [Maria-developers] Rev 3974: MDEV-5182 - build of 10.0.4/r3863 fails @ 'cmake'
by Sergei Golubchik 20 Nov '13

20 Nov '13

Hi, Sergey! On Nov 18, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 3974 > revision-id: svoj(a)mariadb.org-20131118114801-1kny2cvx8kpjsw7t > parent: jplindst(a)mariadb.org-20131115132442-5d4hqg5idjta2dni > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5-mdev5182 > timestamp: Mon 2013-11-18 15:48:01 +0400 > message: > MDEV-5182 - build of 10.0.4/r3863 fails @ 'cmake' with > -DINSTALL_SYSCONFDIR/-DDEFAULT_SYSCONFDIR specified > > The reason for a bug is in support-files/CMakeLists.txt that tries to > install files in INSTALL_SYSCONF2DIR, without checking whether this > variable is actually defined. > > Normally INSTALL_SYSCONF2DIR is always defined if INSTALL_SYSCONFDIR is > defined, and support-files/CMakeLists.txt assumes that. But when one > specifies INSTALL_SYSCONFDIR manually on the command line, this > assumption might be broken. Looks ok to me. An alternative solution would be to define INSTALL_SYSCONF2DIR if INSTALL_SYSCONFDIR is defined. Regards, Sergei

2 1

[Maria-developers] Open Questions in the Knowledge Base
by Ian Gilfillan 19 Nov '13

19 Nov '13

There are quite a few unanswered questions in the Knowledge Base https://mariadb.com/kb/en/+questions/ If anyone knowledgeable in those areas can help answer some of these, even just a request for more info, that would be great :) thanks, ian

2 1

Re: [Maria-developers] Accessing rpl_global_gtid_binlog_state is not consistently protected
by Kristian Nielsen 19 Nov '13

19 Nov '13

Pavel Ivanov <pivanof(a)google.com> writes: > sql/log.cc has highly inconsistent pattern of accessing > rpl_global_gtid_binlog_state. In some places it's protected by mutex > rpl_global_gtid_binlog_state.LOCK_binlog_state, in some places it's > protected by global mutex LOCK_rpl_gtid_state, and in some places it's > not protected by any mutex at all. This is a bug that should be fixed, > right? Yes, agree, thanks for finding this. > I can file this in JIRA if you want, just wanted to let you know faster. I've filed it as MDEV-5306. > I've noticed this problem after one of our AddressSanitizer testing > bots reported that rpl_binlog_state::bump_seq_no_if_needed (called > from MYSQL_BIN_LOG::bump_seq_no_counter_if_needed) was using memory > from hash after it was already freed by rpl_binlog_state::reset > (called by MYSQL_BIN_LOG::reset_logs). Can you test again with the latest patch I've just pushed to 10.0-base? I looked briefly into making an mtr test case to catch this, but I did not see any easy way to do that. I went through the code to try and check all uses of rpl_binlog_state for proper locking, but it would be nice to get a confirmation that I did not miss anything. - Kristian.

3 2