- developers - lists.mariadb.org

[Maria-developers] Coverity
by Christian Convey 09 Dec '13

09 Dec '13

Hi guys, I'm afraid I won't have the time available that I'd hope to, for analyzing MariaDB with Coverity Prevent. Would anyone like to take over from me the Coverity MariaDB account? Thanks, Christian

2 1

[Maria-developers] plugin doesn't load
by AugustQ 06 Dec '13

06 Dec '13

Hi, just one question: I have a simple plugin (a minimal storage engine) that does not load. When I try to load it I get this error-message: MariaDB [(none)]> install plugin dummy soname 'ha_dummy'; ERROR 1126 (HY000): Can't open shared library '/home/august/MariaDB/pgm/lib/plugin/ha_dummy.so' (errno: 2 undefined symbol: _ZN9ha_myisam5cloneEPKcP11st_mem_root) Can you please give me a tip. Thanks. Environment is MariaDB 10.0.4. The same code works without any problems in MySQL (5.5.8 - just for a test). The code is a minimal engine that just does nothing, I want to use it as a template for further work. this is the code in the form that it is working (the relevant parts): in the header-file: class ha_dummy: public handler in the code: ha_dummy::ha_dummy(handlerton *hton, TABLE_SHARE *table_arg) :handler(hton, table_arg) now I change this by modifying the inheritance-order: in the header file: added: #include "../myisam/ha_myisam.h" #include "../myisam/myisamdef.h" modified: class ha_dummy: public ha_myisam in the code: /* Constructor */ ha_dummy::ha_dummy(handlerton *hton, TABLE_SHARE *table_arg) :ha_myisam(hton, table_arg) It compiles but I cannot load it. What's the problem with st_mem_root? Thanks AugustQ PS: as written before: both versions work with MySQL.

3 4

Re: [Maria-developers] [Commits] Rev 3732: Fixed bug mdev-5382 in file:///home/igor/maria/maria-5.3-mdev5382/
by Oleksandr Byelkin 05 Dec '13

05 Dec '13

Hi! 05.12.2013 21:13, Igor Babaev пишет: > At file:///home/igor/maria/maria-5.3-mdev5382/ > > ------------------------------------------------------------ > revno: 3732 > revision-id: igor(a)askmonty.org-20131205191320-pfja761po2x9p9ri > parent: bar(a)mnogosearch.org-20131203110843-1bsdu0hcrpzhie8w > committer: Igor Babaev <igor(a)askmonty.org> > branch nick: maria-5.3-mdev5382 > timestamp: Thu 2013-12-05 11:13:20 -0800 > message: > Fixed bug mdev-5382 > When marking used columns the function find_field_in_table_ref() erroneously > called the walk method for the real item behind a view/derived table field > with the second parameter set to TRUE. > This erroneous code was introduced in 2006. OK to push! > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] RQG crashes tokudb due to unexpected storage engine API call sequence
by Rich Prohaska 05 Dec '13

05 Dec '13

We are debugging a crash induced by RQG in the tokudb storage engine. The cause of the crash is a mismatch in assumptions about the sequence of storage engine API calls. We see: tokudb::external_lock creates a txn tokudb::index_init creates a cursor on a index db with the txn tokudb::index_read use the cursor to try to read a row, and returns an error tokudb_commit commits the txn (called from sql_parse.cc:4584) tokudb::index_end tries to close the cursor and crashes since the txn has been committed. (called from the unit.cleanup() call at sql_parse.cc:4592) The current tokudb software does not expect the commit to occur before index_end is called. Is this assumption correct? Since mariadb (and i suspect mysql) does this, it appears that the storage engine should expect commit before index_end. Any comments? Thanks Rich Prohaska

2 2

[Maria-developers] MDEV-5345 - Deadlock between mysql_change_user(), SHOW VARIABLES and INSTALL PLUGIN
by Sergey Vojtovich 04 Dec '13

04 Dec '13

Hi Sergei, I'm afraid I couldn't find easy fix for this deadlock. The question is if it is worth to spend time for more complex (and possibly less stable) solution. Details below. Locking profile --------------- INSTALL PLUGIN: LOCK_plugin -> LOCK_system_variables_hash (w) SHOW VARIABLES: LOCK_system_variables_hash (r) -> LOCK_global_system_variables change_user(): LOCK_global_system_variables -> LOCK_plugin INSTALL PLUGIN -------------- Needs LOCK_plugin to register new plugin. Needs LOCK_system_variables_hash to register plugin variables. Doesn't seem to need both locks at the same time. Similar lock order is used in a few places. SHOW VARIABLES -------------- Needs LOCK_system_variables_hash to iterate system_variable_hash. Needs LOCK_global_system_variables to read variable value. Does seem to need both locks at the same time. This lock order is not used in other places. change_user() ------------- Needs LOCK_global_system_variables to read global_system_variable.table_plugin. Needs LOCK_plugin to my_plugin_lock(table_plugin). Does seem to need both locks at the same time. Similar lock order is used in a few places. "SHOW VARIABLES" looks correct. So I have only two ideas: - Fix INSTALL PLUGIN so it releases LOCK_plugin before registering variables. Sounds like the best solution, but there are a few more things to fix, e.g. UNINTALL PLUGIN. - Store default storage engine as a string, not Sys_var_plugin. Thanks, Sergey

2 3

[Maria-developers] Build fail of MariaDB10 with TokuDB enabled
by darx＠sent.com 03 Dec '13

03 Dec '13

On the (eventual) road to successfully building MariaDB10 + TokuDB, Building cd src/mariadb bzr log | head ------------------------------------------------------------ revno: 3911 committer: Alexander Barkov <bar(a)mnogosearch.org> branch nick: maria-10.0 timestamp: Tue 2013-11-26 10:53:21 +0400 message: Fixing malformed data in mysql-test/std_data/Index.xml ------------------------------------------------------------ revno: 3910 committer: Sergey Vojtovich <svoj(a)mariadb.org> on uname -a Linux dev1 3.7.10-1.16-desktop #1 SMP PREEMPT Fri May 31 20:21:23 UTC 2013 (97c14ba) x86_64 x86_64 x86_64 GNU/Linux lsb_release -ar LSB Version: core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch Distributor ID: openSUSE project Description: openSUSE 12.3 (x86_64) Release: 12.3 Codename: Dartmouth with gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-suse-linux/4.8/lto-wrapper Target: x86_64-suse-linux Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.8 --enable-ssp --disable-libssp --disable-plugin --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --enable-linker-build-id --enable-linux-futex --program-suffix=-4.8 --without-system-libunwind --with-arch-32=i586 --with-tune=generic --build=x86_64-suse-linux --host=x86_64-suse-linux Thread model: posix gcc version 4.8.2 20131016 [gcc-4_8-branch revision 203692] (SUSE Linux) cmake --version cmake version 2.8.12.1 applying a needed src patch cd /usr/local/src/mariadb # BUG @ http://www.linuxfromscratch.org/blfs/view/svn/server/mariadb.html sed -i "s/srv_buf_size/srv_sort_buf_size/" storage/innobase/row/row0log.cc after cmake, with gcc c++11 support NOT enabled, mkdir bld cd bld cmake .. \ ... \ -DWITHOUT_TOKUDB=1 build completes successfully, make ... ldd sql/mysqld linux-vdso.so.1 (0x00007fffb9dff000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9031e86000) libaio.so.1 => /lib64/libaio.so.1 (0x00007f9031c84000) libz.so.1 => /lib64/libz.so.1 (0x00007f9031a6e000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f9031832000) libssl.so.1.0.0 => /usr/local/ssl/lib64/libssl.so.1.0.0 (0x00007f90315c6000) libcrypto.so.1.0.0 => /usr/local/ssl/lib64/libcrypto.so.1.0.0 (0x00007f90311da000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f9030fd6000) libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f9030cce000) libm.so.6 => /lib64/libm.so.6 (0x00007f90309d0000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f90307b9000) libc.so.6 => /lib64/libc.so.6 (0x00007f903040b000) /lib64/ld-linux-x86-64.so.2 (0x00007f90320a2000) repeating, but with TokuDB build enabled, cd bld cmake .. \ ... \ - -DWITHOUT_TOKUDB=1 + -DWITHOUT_TOKUDB=0 build now fails, make ... [ 29%] Built target perfschema Scanning dependencies of target tokuportability_static_conv [ 29%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/huge_page_detection.cc.o [ 29%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/file.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/memory.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/os_malloc.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/portability.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_assert.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_crash.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_fair_rwlock.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_path.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_pthread.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static_conv.dir/toku_time.cc.o Linking CXX static library libtokuportability_static_conv.a [ 30%] Built target tokuportability_static_conv [ 30%] Generating tokuportability_static_depends.cc Scanning dependencies of target tokuportability_static [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability_static.dir/tokuportability_static_depends.cc.o Linking CXX static library libtokuportability_static.a /usr/bin/ar: creating /usr/local/src/mariadb/bld/storage/tokudb/ft-index/portability/libtokuportability_static.a [ 30%] Built target tokuportability_static Scanning dependencies of target tokuportability [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/huge_page_detection.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/file.cc.o [ 30%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/memory.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/os_malloc.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/portability.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_assert.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_crash.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_fair_rwlock.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_path.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_pthread.cc.o [ 31%] Building CXX object storage/tokudb/ft-index/portability/CMakeFiles/tokuportability.dir/toku_time.cc.o Linking CXX shared library libtokuportability.so [ 31%] Built target tokuportability Scanning dependencies of target logformat [ 31%] Building CXX object storage/tokudb/ft-index/ft/CMakeFiles/logformat.dir/logformat.cc.o Linking CXX executable logformat [ 31%] Built target logformat Scanning dependencies of target generate_log_code [ 31%] Generating log_code.cc, log_print.cc, log_header.h ./logformat: error while loading shared libraries: libtokuportability.so: cannot open shared object file: No such file or directory make[2]: *** [storage/tokudb/ft-index/ft/log_code.cc] Error 127 make[1]: *** [storage/tokudb/ft-index/ft/CMakeFiles/generate_log_code.dir/all] Error 2 make: *** [all] Error 2 yes, I'm aware that opensuse is not a currently "supported platform" for TokuDB. If that's a dealbreaker, I'd like to know MariaDB project's position, if any, on getting that support in place. If it's not, I'd like to keep trying to get it working, here. darx

4 15

Re: [Maria-developers] MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables
by Sergei Golubchik 03 Dec '13

03 Dec '13

Hi, Sergey! On Aug 27, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 3807 > revision-id: svoj(a)mariadb.org-20130827121233-xh1uyhgfwbhedqyf > parent: jplindst(a)mariadb.org-20130823060357-pww92qxla7o8iir7 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Tue 2013-08-27 16:12:33 +0400 > message: > MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables > > - tc_acquire_table and tc_release_table do not access > TABLE_SHARE::tdc.used_tables anymore > - in tc_acquire_table(): release LOCK_tdc after we relase LOCK_open > (saves a few CPU cycles in critical section) > - in tc_release_table(): if we reached table cache threshold, evict > to-be-released table without moving it to unused_tables. unused_tables > must be empty at this point (added assertion). I don't understand what you're doing here, could you elaborate? E.g. explain in a changeset comment what you've done, why you introduced a new list for tables (all_share), what is its semantics, etc. Regards, Sergei

3 12

[Maria-developers] Building TOKUDB into 5.5.34 from sources
by md＠rpzdesign.com 02 Dec '13

02 Dec '13

I am trying to build 5.5.34 from source on Linux x64 (Fedora 14). I was looking at the readme files and could not find anything about building 5.5.34 from source and including TokuDB (which has the source code in the /storage/TokuDB/ directory) into the build. I was looking for something like: -DWITH_TOKUDB_STORAGE_ENGINE=ON but did not stumble across anything. Anybody got some ideas? Thanks, Marco Here is my build script: rm -f CMakeCache.txt make clean cmake \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=/mysql \ -DENABLED_PROFILING=OFF \ -DENABLE_DEBUG_SYNC=ON \ -DENABLE_GCOV=OFF \ -DINSTALL_LAYOUT=STANDALONE \ -DMYSQL_DATADIR=/mysql/data \ -DINSTALL_INCLUDEDIR=include/mysql \ -DINSTALL_UNIX_ADDRDIR=/mysql/mysql.sock \ -DUSE_ARIA_FOR_TMP_TABLES=ON \ -DWITHOUT_PBXT_STORAGE_ENGINE=ON \ -DWITHOUT_SERVER=OFF \ -DWITH_ARCHIVE_STORAGE_ENGINE=OFF \ -DWITH_EMBEDDED_SERVER=OFF \ -DWITH_FAST_MUTEXES=OFF \ -DWITH_FEDERATEDX_STORAGE_ENGINE=OFF \ -DWITH_FEEDBACK=OFF \ -DWITH_LIBWRAP=OFF \ -DWITH_PIC=ON \ -DWITH_PARTITION_STORAGE_ENGINE=OFF \ -DWITH_PERFSCHEMA_STORAGE_ENGINE=ON \ -DWITH_SPHINX_STORAGE_ENGINE=OFF \ -DWITH_SSL=OFF \ -DWITH_UNIT_TESTS=OFF \ -DWITH_VALGRIND=OFF \ -DWITH_XTRADB_STORAGE_ENGINE=ON \ -DWITHOUT_OQGRAPH=ON \ -DWITH_ZLIB=bundled make (Sorry, I posted this question on maria.com knowledge base open questions also) https://mariadb.com/kb/en/cmake-settings-to-include-tokudb-in-5534/

3 2

[Maria-developers] Movable conditions and semi-joins
by Sergei Petrunia 27 Nov '13

27 Nov '13

Hi! The below is an extension of the skype discussion. I wrote down the limitation imposed by FirstMatch, and also covered other Semi-join strategies. <contents> 1. SJ-Materialization 2. FirstMatch 3. Duplicate Elimination 4. Loose Scan </contents> == 1. SJ-Materialization == The general rule is: If the condition is originally attached inside an SJM nest, one should not move it outside. Conditions that are originally outside should not be moved inisde the SJM nest. The reason for the rule is that field values for inner tables are only available inside the SJM nest. Correspondingly, values of fields of outer tables are not available at materialization step. The only possible exception is when the condition depends only on columns that are in the IN-equality. Then, it could be moved. I don't think this is worth handling for the first milestone. == 2. FirstMatch == Consider a query select ... from ot1, nt1, nt2 where expr(ot1) in (select ... from it1, it2, ... where ...) and a join order ot1 FirstMatch(it1, it2) nt1 nt2 It is possible to move cond(otX) to table itX or ntX. There is a problem with moving condition attached to itX (will call it SUBQ_COND. this ocndition depends on table itX and maybe on ot1) to table ntX. Consider join execution execution: ot1.row0 ot1.row1 -+- it1.row1 -- it2.row1 | +- it1.row2 -- it2.row2 ->- (F1)-->-- nt1.row1 (F2)--<-- FirstMatch check is at point (F2). At that point, we know that ot1.row1 has had a match in the subquery, and the execution returns to table ot1. If SUBQ_COND is moved to table ntX, then there may be a situation where we will get to point (F2) and assume that the subquery had a match, while actually it hasn't. A suggestion by Igor: at point (F2) we should check the last return value of SUBQ_COND (actually, of all conditions that were moved like SUBQ_COND was) It is actually more complicated: what if there is COND(ot1, nt1) such that COND(ot1.ro1, nt1.row1) = FALSE? Then, we will never evaluate SUBQ_COND(ot1.row1, ...). What if never evaluate SUBQ_COND(ot1.row1, ...) but before we have evaluated SUBQ_COND(ot1.row0, ...)? We will have to track what we have evaluated for the current row of ot1.row1. == 3. Duplicate Elimination == Duplicate Elimination removes duplicate matches using a temporary table. In order to work correctly, filtering needs to be performed before the weedout is done. Let's again use an example. select ... from ot1, nt1, nt2 where expr(ot1) in (select ... from it1, it2, ... where ...) and a join order (DUPS-W-START) it1 ot1 it2 (DUPS-W-WEEDOUT) nt1 nt2 at DUPS-W-START, all rows are deleted from the temp. table. at DUPS-W-WEEDOUT, we take ot1.rowid and attempt to write it into the temp table. Basic idea: If a condition SUBQ_COND(itX, ...) is moved to the right across the (DUPS-W-WEEDOUT) point, wrong query results will occur. Example: Suppose, SUBQ_COND(it2) is moved to nt1, and join execution proceeds as follows: it1.row1 --- ot1.row1 --- it2.row1 -- DUPS-W-WEEDOUT -- nt1 | - We reach DUPS-W-WEEDOUT with (it1.row1, ot1.row1, it2.row1). - The temp. table is empty, so rowid(ot1.row1) is written into temp.table, and Weedout doesn't weed it out. - At nt1, SUBQ_COND(it2.row1) is evaluated and found to be false. join execution returns back. - Suppose, tables it2 and ot1 have no more matching rows, so execution returns to it1. (the same diagram with a few more steps) it1.row1 --- ot1.row1 --- it2.row1 -- DUPS-W-WEEDOUT -- nt1 | it1.row2 --- ot1.row1 --- it2.row2 -- DUPS-W-WEEDOUT - We reach DUPS-W-WEEDOUT with (it1.row2, ot1.tow1, it2.rowX). - The temptable already has rowid(ot1.row1). The record combination is filtered out. if SUBQ_COND(it2.row2)=TRUE, then the result will miss a row. == 4. Loose Scan == Loose Scan is similar to First Match - it does short-cutting (but also makes forward jumps on the driving table). Short-cutting imposes a limitation similar to FirstMatch - conditions that depend on subquery's tables cannot be moved out of LooseScan's range. BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] maria-tokudb-engine-10.0 deb package (raring) - if files are empty where are sources?
by Paul Hewson 27 Nov '13

27 Nov '13

Hello, I'm trying to follow the instructions on https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/ which imply there is more tuned version of tokudb for mariadb than I might get by following instructions at tokutek. However, I have ubuntu 13.04 (raring), and despite some mismatch in numbers (I have maria 10.0 rather than 5.5) I am getting an empty file when I try to apt-get install mariadb-tokudb-engine-10.0 I found a bit of discussion about this point, but got the impression it had been fixed. Does it mean it was fixed and the debs were removed? If so, do you mind if I ask where I get the mariadb tokutek sources. Again the webpage https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/ implies only some parts (most parts) of totudb have been merged into mariadb. Thanks, sorry for troubling you with naive issues like this. Paul ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 2