- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 3916: MDEV-5403 - Reduce usage of LOCK_open: tc_count
by Sergei Golubchik 10 Dec '13

10 Dec '13

Hi, Sergey! Not a review, only a comment on TDC_atomic part: 1. This is perfectly generally useful, not TDC specific at all. I'd consider moving it to my_atomic.h under #ifdef __cplusplus 2. These are supposed to be trivial inlined functions. I don't think you should use inheritance and virtual methods here. Everything should be in the template. I understand that function names (add64/add32/etc) are a bit tricky. Two approaches: pass them as template parameters (like in template <typename T, ADD, CAS, ...>) or - perhaps, if you put this into my_atomic.h you'll be able to generate the necessary code without using my_atomic_add32/etc ? Regards, Sergei

2 1

Re: [Maria-developers] [svoj@mariadb.org: [Commits] Rev 3807: MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables in lp:maria/10.0]
by Sergei Golubchik 10 Dec '13

10 Dec '13

Hi, Sergey! Looks pretty much ok. See few comments/questions below. On Oct 24, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 3807 > revision-id: svoj(a)mariadb.org-20130827121233-xh1uyhgfwbhedqyf > parent: jplindst(a)mariadb.org-20130823060357-pww92qxla7o8iir7 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Tue 2013-08-27 16:12:33 +0400 > message: > MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables > > - tc_acquire_table and tc_release_table do not access > TABLE_SHARE::tdc.used_tables anymore ok > - in tc_acquire_table(): release LOCK_tdc after we relase LOCK_open > (saves a few CPU cycles in critical section) This is a bit counter-intuitive. Your change is to hold the lock more than necessary. Does it give a measurable performance improvement? > - in tc_release_table(): if we reached table cache threshold, evict > to-be-released table without moving it to unused_tables. unused_tables > must be empty at this point (added assertion). Why unused_tables must be empty at this point? > === modified file 'sql/sql_base.cc' > --- a/sql/sql_base.cc 2013-08-14 08:48:50 +0000 > +++ b/sql/sql_base.cc 2013-08-27 12:12:33 +0000 > @@ -370,7 +372,7 @@ void free_io_cache(TABLE *table) > > void kill_delayed_threads_for_table(TABLE_SHARE *share) > { > - TABLE_SHARE::TABLE_list::Iterator it(share->tdc.used_tables); > + TABLE_SHARE::All_share_tables_list::Iterator it(share->tdc.all_tables); In this function, I'd suggest to add some flag to be able to skip the mutex and the loop altogether - as in the absolute majority of cases there will be no delayed insert thread to kill. Either use a global flag to know if there's a delayed thread running, or create a per-share flag. > TABLE *tab; > Regards, Sergei

2 4

[Maria-developers] Reviews of parallel replications
by Michael Widenius 10 Dec '13

10 Dec '13

Hi! Now continuing with reviews of the parallel replication code. Will take one patch at a time with a target to have everything reviewed this week. Review of revno 3678: Parallel replication: error handling. ------------------------------------------------------------ revno: 3678 committer: knielsen(a)knielsen-hq.org branch nick: work-10.0-mdev4506 timestamp: Mon 2013-10-14 15:28:16 +0200 message: MDEV-4506: Parallel replication: error handling. Add an error code to the wait_for_commit facility. Now, when a transaction fails, it can signal the error to any subsequent transaction that is waiting for it to commit. The waiting transactions then receive the error code back from wait_for_prior_commit() and can handle the error appropriately. Also fix one race that could cause crash if @@slave_parallel_threads were changed several times quickly in succession. <cut> === modified file 'sql/handler.cc' --- sql/handler.cc 2013-10-13 21:24:05 +0000 +++ sql/handler.cc 2013-10-14 13:28:16 +0000 @@ -1458,10 +1458,11 @@ int ha_commit_one_phase(THD *thd, bool a transaction.all.ha_list, see why in trans_register_ha()). */ bool is_real_trans=all || thd->transaction.all.ha_list == 0; + int res; DBUG_ENTER("ha_commit_one_phase"); - if (is_real_trans) - thd->wait_for_prior_commit(); - int res= commit_one_phase_2(thd, all, trans, is_real_trans); + if (is_real_trans && (res= thd->wait_for_prior_commit())) + DBUG_RETURN(res); + res= commit_one_phase_2(thd, all, trans, is_real_trans); DBUG_RETURN(res); Another way to write the above is: if (is_real_trans && !(res= thd->wait_for_prior_commit())) res= commit_one_phase_2(thd, all, trans, is_real_trans); DBUG_RETURN(res); <cut> I was checking how ha_commit_one_phase() was used. In ha_commit_trans() or trans_xa_commit(), if ha_commit_one_phase() fails, it will not run rollback. Is this right? I would have assumed that on failure, ha_commit_one_phase() should run ha_rollback_trans() before returning. === modified file 'sql/log.cc' @@ -7844,7 +7844,8 @@ int TC_LOG_MMAP::log_and_order(THD *thd, mysql_mutex_unlock(&LOCK_prepare_ordered); } - thd->wait_for_prior_commit(); + if (thd->wait_for_prior_commit()) + return 0; What does the return 0 mean here? Is it an error? Is thd->is_fatal_error assumed to be set if this happens? A comment here would not be totally wrong... @@ -5633,7 +5634,7 @@ wait_for_commit::~wait_for_commit() void -wait_for_commit::wakeup() +wait_for_commit::wakeup(int wakeup_error) { /* We signal each waiter on their own condition and mutex (rather than using @@ -5649,6 +5650,7 @@ wait_for_commit::wakeup() */ mysql_mutex_lock(&LOCK_wait_commit); waiting_for_commit= false; + this->wakeup_error= wakeup_error; Consider adding an assert that 'this->wakeup_error' is 0. Good way to ensure we don't accidently call this function twice... (as things would be bad if we first would call it with an error and then with 0) mysql_mutex_unlock(&LOCK_wait_commit); mysql_cond_signal(&COND_wait_commit); } Regards, Monty

1 0

[Maria-developers] (MDEV-4403) Attempting to use cassandra storage engine causes "service 'my_snprintf_service' interface version mismatch"
by Sergei Golubchik 10 Dec '13

10 Dec '13

Hi. I don't see my commit emails on the list, so here you are: first patch is the bug fix itself. second - cleanup of the existing code after the bug fix. Regards, Sergei

2 1

[Maria-developers] Coverity
by Christian Convey 09 Dec '13

09 Dec '13

Hi guys, I'm afraid I won't have the time available that I'd hope to, for analyzing MariaDB with Coverity Prevent. Would anyone like to take over from me the Coverity MariaDB account? Thanks, Christian

2 1

[Maria-developers] plugin doesn't load
by AugustQ 06 Dec '13

06 Dec '13

Hi, just one question: I have a simple plugin (a minimal storage engine) that does not load. When I try to load it I get this error-message: MariaDB [(none)]> install plugin dummy soname 'ha_dummy'; ERROR 1126 (HY000): Can't open shared library '/home/august/MariaDB/pgm/lib/plugin/ha_dummy.so' (errno: 2 undefined symbol: _ZN9ha_myisam5cloneEPKcP11st_mem_root) Can you please give me a tip. Thanks. Environment is MariaDB 10.0.4. The same code works without any problems in MySQL (5.5.8 - just for a test). The code is a minimal engine that just does nothing, I want to use it as a template for further work. this is the code in the form that it is working (the relevant parts): in the header-file: class ha_dummy: public handler in the code: ha_dummy::ha_dummy(handlerton *hton, TABLE_SHARE *table_arg) :handler(hton, table_arg) now I change this by modifying the inheritance-order: in the header file: added: #include "../myisam/ha_myisam.h" #include "../myisam/myisamdef.h" modified: class ha_dummy: public ha_myisam in the code: /* Constructor */ ha_dummy::ha_dummy(handlerton *hton, TABLE_SHARE *table_arg) :ha_myisam(hton, table_arg) It compiles but I cannot load it. What's the problem with st_mem_root? Thanks AugustQ PS: as written before: both versions work with MySQL.

3 4

Re: [Maria-developers] [Commits] Rev 3732: Fixed bug mdev-5382 in file:///home/igor/maria/maria-5.3-mdev5382/
by Oleksandr Byelkin 05 Dec '13

05 Dec '13

Hi! 05.12.2013 21:13, Igor Babaev пишет: > At file:///home/igor/maria/maria-5.3-mdev5382/ > > ------------------------------------------------------------ > revno: 3732 > revision-id: igor(a)askmonty.org-20131205191320-pfja761po2x9p9ri > parent: bar(a)mnogosearch.org-20131203110843-1bsdu0hcrpzhie8w > committer: Igor Babaev <igor(a)askmonty.org> > branch nick: maria-5.3-mdev5382 > timestamp: Thu 2013-12-05 11:13:20 -0800 > message: > Fixed bug mdev-5382 > When marking used columns the function find_field_in_table_ref() erroneously > called the walk method for the real item behind a view/derived table field > with the second parameter set to TRUE. > This erroneous code was introduced in 2006. OK to push! > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] RQG crashes tokudb due to unexpected storage engine API call sequence
by Rich Prohaska 05 Dec '13

05 Dec '13

We are debugging a crash induced by RQG in the tokudb storage engine. The cause of the crash is a mismatch in assumptions about the sequence of storage engine API calls. We see: tokudb::external_lock creates a txn tokudb::index_init creates a cursor on a index db with the txn tokudb::index_read use the cursor to try to read a row, and returns an error tokudb_commit commits the txn (called from sql_parse.cc:4584) tokudb::index_end tries to close the cursor and crashes since the txn has been committed. (called from the unit.cleanup() call at sql_parse.cc:4592) The current tokudb software does not expect the commit to occur before index_end is called. Is this assumption correct? Since mariadb (and i suspect mysql) does this, it appears that the storage engine should expect commit before index_end. Any comments? Thanks Rich Prohaska

2 2

[Maria-developers] MDEV-5345 - Deadlock between mysql_change_user(), SHOW VARIABLES and INSTALL PLUGIN
by Sergey Vojtovich 04 Dec '13

04 Dec '13

Hi Sergei, I'm afraid I couldn't find easy fix for this deadlock. The question is if it is worth to spend time for more complex (and possibly less stable) solution. Details below. Locking profile --------------- INSTALL PLUGIN: LOCK_plugin -> LOCK_system_variables_hash (w) SHOW VARIABLES: LOCK_system_variables_hash (r) -> LOCK_global_system_variables change_user(): LOCK_global_system_variables -> LOCK_plugin INSTALL PLUGIN -------------- Needs LOCK_plugin to register new plugin. Needs LOCK_system_variables_hash to register plugin variables. Doesn't seem to need both locks at the same time. Similar lock order is used in a few places. SHOW VARIABLES -------------- Needs LOCK_system_variables_hash to iterate system_variable_hash. Needs LOCK_global_system_variables to read variable value. Does seem to need both locks at the same time. This lock order is not used in other places. change_user() ------------- Needs LOCK_global_system_variables to read global_system_variable.table_plugin. Needs LOCK_plugin to my_plugin_lock(table_plugin). Does seem to need both locks at the same time. Similar lock order is used in a few places. "SHOW VARIABLES" looks correct. So I have only two ideas: - Fix INSTALL PLUGIN so it releases LOCK_plugin before registering variables. Sounds like the best solution, but there are a few more things to fix, e.g. UNINTALL PLUGIN. - Store default storage engine as a string, not Sys_var_plugin. Thanks, Sergey

2 3

[Maria-developers] Build fail of MariaDB10 with TokuDB enabled
by darx＠sent.com 03 Dec '13

03 Dec '13

On the (eventual) road to successfully building MariaDB10 + TokuDB, Building cd src/mariadb bzr log | head ------------------------------------------------------------ revno: 3911 committer: Alexander Barkov <bar(a)mnogosearch.org> branch nick: maria-10.0 timestamp: Tue 2013-11-26 10:53:21 +0400 message: Fixing malformed data in mysql-test/std_data/Index.xml ------------------------------------------------------------ revno: 3910 committer: Sergey Vojtovich <svoj(a)mariadb.org> on uname -a Linux dev1 3.7.10-1.16-desktop #1 SMP PREEMPT Fri May 31 20:21:23 UTC 2013 (97c14ba) x86_64 x86_64 x86_64 GNU/Linux lsb_release -ar LSB Version: core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch Distributor ID: openSUSE project Description: openSUSE 12.3 (x86_64) Release: 12.3 Codename: Dartmouth with gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-suse-linux/4.8/lto-wrapper Target: x86_64-suse-linux Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.8 --enable-ssp --disable-libssp --disable-plugin --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --enable-linker-build-id --enable-linux-futex --program-suffix=-4.8 --without-system-libunwind --with-arch-32=i586 --with-tune=generic --build=x86_64-suse-linux --host=x86_64-suse-linux Thread model: posix gcc version 4.8.2 20131016 [gcc-4_8-branch revision 203692] (SUSE Linux) cmake --version cmake version 2.8.12.1 applying a needed src patch cd /usr/local/src/mariadb # BUG @ http://www.linuxfromscratch.org/blfs/view/svn/server/mariadb.html sed -i "s/srv_buf_size/srv_sort_buf_size/" storage/innobase/row/row0log.cc after cmake, with gcc c++11 support NOT enabled, mkdir bld cd bld cmake .. \ ... \ -DWITHOUT_TOKUDB=1 build completes successfully, make ... ldd sql/mysqld linux-vdso.so.1 (0x00007fffb9dff000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9031e86000) libaio.so.1 => /lib64/libaio.so.1 (0x00007f9031c84000) libz.so.1 => /lib64/libz.so.1 (0x00007f9031a6e000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f9031832000) libssl.so.1.0.0 => /usr/local/ssl/lib64/libssl.so.1.0.0 (0x00007f90315c6000) libcrypto.so.1.0.0 => /usr/local/ssl/lib64/libcrypto.so.1.0.0 (0x00007f90311da000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f9030fd6000) libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f9030cce000) libm.so.6 => /lib64/libm.so.6 (0x00007f90309d0000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f90307b9000) libc.so.6 => /lib64/libc.so.6 (0x00007f903040b000) /lib64/ld-linux-x86-64.so.2 (0x00007f90320a2000) repeating, but with TokuDB build enabled, cd bld cmake .. \ ... \ - -DWITHOUT_TOKUDB=1 + -DWITHOUT_TOKUDB=0 build now fails, make ... [ 29%] Built target perfschema Scanning dependencies of target tokuportability_static_conv [ 29%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/huge_page_detection.cc.o [ 29%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/file.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/memory.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/os_malloc.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/portability.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_assert.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_crash.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_fair_rwlock.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_path.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_pthread.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_time.cc.o Linking CXX static library libtokuportability_static_conv.a [ 30%] Built target tokuportability_static_conv [ 30%] Generating tokuportability_static_depends.cc Scanning dependencies of target tokuportability_static [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static.dir/tokuportability_static_depends.cc.o Linking CXX static library libtokuportability_static.a /usr/bin/ar: creating /usr/local/src/mariadb/bld/storage/tokudb/ft-index/portability/libtokuportability_static.a [ 30%] Built target tokuportability_static Scanning dependencies of target tokuportability [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/huge_page_detection.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/file.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/memory.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/os_malloc.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/portability.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_assert.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_crash.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_fair_rwlock.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_path.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_pthread.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_time.cc.o Linking CXX shared library libtokuportability.so [ 31%] Built target tokuportability Scanning dependencies of target logformat [ 31%] Building CXX object storage/tokudb/ft-index/ft/CMakeFiles/logformat.dir/logformat.cc.o Linking CXX executable logformat [ 31%] Built target logformat Scanning dependencies of target generate_log_code [ 31%] Generating log_code.cc, log_print.cc, log_header.h ./logformat: error while loading shared libraries: libtokuportability.so: cannot open shared object file: No such file or directory make[2]: *** [storage/tokudb/ft-index/ft/log_code.cc] Error 127 make[1]: *** [storage/tokudb/ft-index/ft/CMakeFiles/generate_log_code.dir/all] Error 2 make: *** [all] Error 2 yes, I'm aware that opensuse is not a currently "supported platform" for TokuDB. If that's a dealbreaker, I'd like to know MariaDB project's position, if any, on getting that support in place. If it's not, I'd like to keep trying to get it working, here. darx

4 15