- developers - lists.mariadb.org

[Maria-developers] Can there be a better storage engine API?
by MARK CALLAGHAN 12 Sep '13

12 Sep '13

The storage engine API has never been my favorite part of MySQL. I haven't written a storage engine but I used to maintain a custom storage engine and migrated that from 4.0 to 5.0 so I had some experience with the API. Is there a different API that would make it easier for innovation to come to MySQL/MariaDB? For example there is a lot going on in the write-optimized database space. Until recently we had nothing for MySQL open-source. Now we have open source TokuDB (great news). I haven't read their source but I have been told they use something that was derived from the BerkeleyDB API. Would the BerkeleyDB API make it easier to get new storage engines into MySQL? For example, we don't have an engine optimized for in-memory workloads. And while InnoDB will have great cache hit rates when all data is in-memory. It might write much more data to disk than required. Do we need a better API or is the current code good enough? I hope to hear from people who have done serious storage engine work. -- Mark Callaghan mdcallag(a)gmail.com

7 10

[Maria-developers] updating a string session variable from a storage engine
by Rich Prohaska 12 Sep '13

12 Sep '13

i have a MYSQL_THDVAR_STR session variable in my storage engine. i want to update its value from the storage engine. it was suggested that i use "THDVAR(x)=new_value" to do it. i noticed that update_func_str is NOT called, so i wonder about the contract. is the caller responsible for memory management? i used PLUGIN_VAR_MEMALLOC for the var def. thanks

3 2

[Maria-developers] Problems building debug version of MariaDB 5.5.32
by Weldon Whipple 11 Sep '13

11 Sep '13

Hello, I'm having problems building a Debug version of MariaDB 5.5.32. (It's a new problem, I think; I'm almost positive it was working about two weeks ago.) Here is what I'm doing to build MariaDB: $ cmake -DCMAKE_BUILD_TYPE=Debug . $ make package At 69% complete, I get about 14 lines like this: [ 69%] Building CXX object unittest/mysys/CMakeFiles/explain_filename-t.dir/explain_filename-t.cc.o Linking CXX executable explain_filename-t `.text._ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function' referenced in section `.note.stapsdt' of ../../sql/libpartition.a(ha_partition.cc.o): defined in discarded section `.text._ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function[handler::ha_index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function)]' of ../../sql/libpartition.a(ha_partition.cc.o) followed by: collect2: ld returned 1 exit status make[2]: *** [unittest/mysys/explain_filename-t] Error 1 make[1]: *** [unittest/mysys/CMakeFiles/explain_filename-t.dir/all] Error 2 make: *** [all] Error 2 Some comments: 1. I'm running on CentOS 6.4: $ uname -a Linux weldon 2.6.32-358.18.1.el6.x86_64 #1 SMP Wed Aug 28 17:19:38 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux 2. I've tried "cmake -DRPM=centos6 . " (without Debug specified) and it works just fine. 3. I found the line in (282:) in ./CMakeLists.txt that reads "ADD_SUBDIRECTORY(unittest/mysys)" and commented it out. Then the build continued until it was trying to link mysqld ... and failed with the similar errors at 99% completion. (I actually WANT mysqld, so I couldn't comment it out too :-)) 4. Something I read indicates that MariaDB requires a GCC build against recent binutils. yum seems to indicate that both GCC and binutils are up-to-date. 5. The cmake option I'm using is the one for 5.5+ as instructed at https://mariadb.com/kb/en/compiling-mariadb-for-debugging/ 6. Yes, I also tried the old ./BUILD/compile-pentium64-debug-max (but it failed similarly) I THINK it's probably an incompatible version of some build tool on my development box. Thanks in advance for any advice. Weldon Whipple

2 1

[Maria-developers] MDEV 5007 - help me! it's a old mdev closed as not a bug but don't solve the problem
by Roberto Spadim 10 Sep '13

10 Sep '13

Hi guys! I posted the MDEV-5007 and it's a old problem from MDEV-3795 (the same problem) It (MDEV-3795) was closed as a not a bug, but i think it's a bug, but i didn't explained well at the first time I changed from a mysql server to a mariadb server and the result order by isn't executed, i tried myisam,aria,innodb,memory engines and all of them return the same problem (order by not executed) On the wrong query i don't see the "using filesort" at explain, and at the right query (with right order by) it show "using filesort" Could anyone help me? or at least EXPLAIN why the order by shouldn't be executed on the first query, and maybe a workaround using sql_optimizer variable or something like it? it's a closed software and i need to contact developer to change the query, that's why i'm talking about the sql_optimizer variable instead of rewrite the query Thanks -- Roberto Spadim

2 3

[Maria-developers] Rollback causes assertion with Galera 10.0
by Jan Lindström 10 Sep '13

10 Sep '13

Hi, Background: I merged 10.0 to lp:~maria-captains/maria/maria-10.0-galera/, merge result is pushed to the tree. In testing I hit repeatable to following assertion #0 0x00007f1ff987f037 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007f1ff9882698 in __GI_abort () at abort.c:90 #2 0x00007f1ff9877e03 in __assert_fail_base (fmt=0x7f1ff99ce578 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x109b4ff "empty()", file=file@entry=0x109b4d8 "/home/jan/mysql/galera-10.0/sql/log.cc", line=line@entry=294, function=function@entry=0x10a01e0 <binlog_cache_data::reset()::__PRETTY_FUNCTION__> "void binlog_cache_data::reset()") at assert.c:92 #3 0x00007f1ff9877eb2 in __GI___assert_fail (assertion=0x109b4ff "empty()", file=0x109b4d8 "/home/jan/mysql/galera-10.0/sql/log.cc", line=294, function=0x10a01e0 <binlog_cache_data::reset()::__PRETTY_FUNCTION__> "void binlog_cache_data::reset()") at assert.c:101 #4 0x00000000009497d4 in binlog_cache_data::reset (this=0x7f1f7403ad60) at /home/jan/mysql/galera-10.0/sql/log.cc:294 #5 0x0000000000949b6a in binlog_cache_mngr::reset (this=0x7f1f7403ab90, do_stmt=false, do_trx=true) at /home/jan/mysql/galera-10.0/sql/log.cc:462 #6 0x0000000000935a4c in binlog_truncate_trx_cache (thd=0x45b7d40, cache_mngr=0x7f1f7403ab90, all=true) at /home/jan/mysql/galera-10.0/sql/log.cc:1991 #7 0x0000000000936123 in binlog_rollback (hton=0x3891fb0, thd=0x45b7d40, all=true) at /home/jan/mysql/galera-10.0/sql/log.cc:2180 #8 0x000000000086b750 in ha_rollback_trans (thd=0x45b7d40, all=true) at /home/jan/mysql/galera-10.0/sql/handler.cc:1616 #9 0x00000000007a0c5f in trans_rollback (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/transaction.cc:321 #10 0x00000000006718b2 in mysql_execute_command (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:4634 #11 0x0000000000677d52 in mysql_parse (thd=0x45b7d40, rawbuf=0x7f1f74005288 "ROLLBACK", length=8, parser_state=0x7f1fe83bb610) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:6816 #12 0x00000000006774d3 in wsrep_mysql_parse (thd=0x45b7d40, rawbuf=0x7f1f74005288 "ROLLBACK", length=8, parser_state=0x7f1fe83bb610) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:6647 #13 0x00000000006688cd in dispatch_command (command=COM_QUERY, thd=0x45b7d40, packet=0x44807c1 "ROLLBACK ", packet_length=9) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:1451 #14 0x00000000006678a3 in do_command (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:1077 #15 0x000000000078ba4d in do_handle_one_connection (thd_arg=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_connect.cc:1404 #16 0x000000000078b78d in handle_one_connection (arg=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_connect.cc:1311 #17 0x0000000000b0c20a in pfs_spawn_thread (arg=0x45713a0) at /home/jan/mysql/galera-10.0/storage/perfschema/pfs.cc:1853 #18 0x00007f1ffa423f8e in start_thread (arg=0x7f1fe83bc700) at pthread_create.c:311 #19 0x00007f1ff9941e1d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) We go from log.cc::binlog_rollback : else if (ending_trans(thd, all) || (!(thd->variables.option_bits & OPTION_KEEP_LOG) && (!stmt_has_updated_non_trans_table(thd) || WSREP_FORMAT(thd->variables.binlog_format) != BINLOG_FORMAT_STMT) && (!cache_mngr->trx_cache.changes_to_non_trans_temp_table() || WSREP_FORMAT(thd->variables.binlog_format) != BINLOG_FORMAT_MIXED))) error= binlog_truncate_trx_cache(thd, cache_mngr, all); end there to cache_mngr->reset(false, true); and there is DBUG_ASSERT(empty()); bool empty() const { return pending() == NULL && my_b_tell(&cache_log) == 0; } At empty() this is what does not hold : my_b_tell(&cache_log) == 0; // not sure what this means At test output there is at the end: # 2013-09-09T10:03:21 [6620] Query: UPDATE LOW_PRIORITY IGNORE `table100_int_autoinc` AS X SET `col_char_12` = 220 WHERE X . `col_char_12_key` IN ( 2427781120 , 1 , 'i' , 'm' , 'o' ) ORDER BY `col_char_12`,`col_char_12_key`,`col_int`,`col_int_key`,`pk` LIMIT 8 failed: 1213 Deadlock found when trying to get lock; try restarting transaction. Further errors of this kind will be suppressed. # 2013-09-09T10:04:14 [6620] Query: DELETE LOW_PRIORITY IGNORE FROM `table10_key_pk_parts_2_int_autoinc` WHERE `col_char_12_key` IN ( 4200660992 , 1646198784 , 3888709632 , 32 , 'c' ) ORDER BY `col_char_12`,`col_char_12_key`,`col_int`,`col_int_key`,`pk` LIMIT 1 failed: 1205 Lock wait timeout exceeded; try restarting transaction. Further errors of this kind will be suppressed. Unfortunately, general log contains so many statements from different connections that they are not really helpful for getting repeatable test case. R: Jan -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>

2 4

[Maria-developers] AIX "make fails" - related to: MDEV-4941
by Michael Felt 09 Sep '13

09 Sep '13

AIX make fails with syntax errors. One has been fixed, per https://mariadb.atlassian.net/browse/MDEV-4941. Rather than submit error by error I thought it would be more productive to provide the output from make -i. After trying, but not seeing it 'attach', I am going to try attaching the output of make -i here. regards, Michael

2 2

[Maria-developers] Some clarifications requested for C client
by Peter Laursen 09 Sep '13

09 Sep '13

This is probably for Georg Richter (and/or @Wlad). I have a few qeustion! 1) https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Fix for CONC-44: LOAD DATA INFILE can't open utf16le encoded filenames" But we we can stil not specify UTF16LE as the CHARSET, I think? Say I have a file *named* "ÄÜÑБГД.csv". (both Western special characters and cyrillic characters in the file name). This file I will be able to read now (using revision 80), right? But if file *content* is this single line (one row, two columns) ÄÜÑ,БГД and if it is encoded as 'Windows Unicode'/utf16le I still will need to recode the file to utf8 before executing it with LOAD DATA (as UTF16LE still cannot be specified for CHARSET in LOAD DATA), right? Please confirm that I understand this! 2) https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Added support for embedded (sqlite)". This happened by beginning of this year but I just noticed now. Now what can we actually do with SQLite and the MariaDB C-API and what we the purpose of this revision? Is it to make it easier to write client applications using SQLite for internal storage? Or to make the C client code 'a generic API' for connecting to both MySQL (of all flavors) and SQLite? Is the the code/files that that were added different from offical SQLite downloads from http://www.sqlite.org/? I am not able to find any documentation anywhere. Also not a blog or whatever either. There must be some reason to add sqlite3 code?! 3) This seems to be the latest revision: https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Added dyamic column api; Added character set conversion" ('dyamic' is misspelled BTW!) Won't we need some documentation here? The claim that MariaDB's connector "has exactly the same API as the MySQL Connector/C for MySQL 5.5" ( https://kb.askmonty.org/en/mariadb-client-library-for-c/) cannot be true now as it adds funtionalities for managing MariaDB ' dymamic column waht oracle's API has not? The API from Oracle's library is now a *subset* or MariaDB API (or MariaDB's is a *superset* or Oracle). So much I can figure out, but should I need to code with it I would need description of functions' syntax, expected results and error listings. Also I am not able to figure out what the "character set conversion" referred to is doing and why it was added. 4) Why does the changelog here https://kb.askmonty.org/en/mariadb-client-library-for-c-100-changelog/ stop at revision 77? -- Peter Laursen -- Webyog

2 3

Re: [Maria-developers] Rev 3883: MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 08 Sep '13

08 Sep '13

Hi, Holyfoot! Just one question, see below. Otherwise ok. On Sep 08, holyfoot(a)askmonty.org wrote: > revno: 3883 > revision-id: holyfoot(a)askmonty.org-20130908092209-xar471jadl7lmwzo > parent: sergii(a)pisem.net-20130907203634-htu89dqmwe5gqwy5 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 55-4472 > timestamp: Sun 2013-09-08 14:22:09 +0500 > message: > MDEV-4472 Audit-plugin. Server-related part of the task. > file_logger became the service. > Data like query_id now are sent to the audit plugin. > Fix for MDEV-4770 ported from 10.0. > Fix added for the read_maria_plugin_info(). > Log rotation can be disabled with 'set rotations=0'. > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-08 09:22:09 +0000 > @@ -16,7 +16,7 @@ > > #include "my_global.h" > #include <my_sys.h> > -#include "service_logger.h" > +#include <mysql/service_logger.h> > #include <my_pthread.h> > > extern MYSQL_PLUGIN_IMPORT char *mysql_data_home; I believe we agreed to keep file_logger.c completely clean from dynamic-plugin-related code. Did you forget to remove the line above? Regards, Sergei

1 0

Re: [Maria-developers] MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 08 Sep '13

08 Sep '13

Hi, Holyfoot! On Sep 05, holyfoot(a)askmonty.org wrote: > At file:///home/hf/wmar/55-4472/ > > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-05 12:47:00 +0000 > @@ -16,10 +16,12 @@ > > #include "my_global.h" > #include <my_sys.h> > -#include "service_logger.h" > +#include <mysql/service_logger.h> > #include <my_pthread.h> > > +#ifndef _WIN32 > extern MYSQL_PLUGIN_IMPORT char *mysql_data_home; > +#endif /*!_WIN32*/ > > #ifdef HAVE_PSI_INTERFACE > /* These belong to the service initialization */ > @@ -51,6 +53,12 @@ LOGGER_HANDLE *logger_open(const char *p > unsigned int rotations) > { > LOGGER_HANDLE new_log, *l_perm; > + //char *local_mysql_data_home; > +//#ifdef _WIN32 > + //local_mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); > +//#else > + //local_mysql_data_home= mysql_data_home; > +//#endif /*_WIN32*/ I think you can completely remove this and the #ifndef above. For your MySQL plugin you can simply #include this in the plugin body like extern char *mysql_data_home; #include "file_logger.c" and later in your plugin_init function: #ifdef _WIN32 mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); #endif This way you will keep file_logger.c completely free of the dynamic plugin hacks. > /* > I don't think we ever need more rotations, > @@ -168,6 +176,29 @@ int logger_vprintf(LOGGER_HANDLE *log, c > int logger_rotate(LOGGER_HANDLE *log) > { > int result; I believe you've added recently a test to disable rotation if number of rotations is 0? Regards, Sergei

2 1

[Maria-developers] first contribution
by AugustQ 06 Sep '13

06 Sep '13

Hi, after some struggling I got this message from Bazaar: Created new stacked branch referring to /+branch-id/633549. I hope everything worked fine. If not let me know so that I can correct it. I created a branch to 10.0.4 of MariaDB and named it: contrib_AQ_01. I've uploaded this some minutes before. For some time I played with the source of MySQL 5.5.8. I've taken some of my modifications to MariaDB 10.0.4 and want to give them to you for further discussions. To say it first: this is not production-ready-code, it's a proof of a concept. My modifications are made for the MyIsam-engine in the handling of a full table-scan with a table of dynamic sized records. The reason for uploading this is: during my tests I executed statements like SELECT * FROM table INTO OUTFILE 'x.x'; With my test-data this took about 20 seconds with the original code and 12 seconds with my modifications. I've tested my code with a single table which contains 10 Mio records, a total size of 1GB. As i've written before: it's a proof of concept. I've not implemented all the possible cases. I wanted to know if a full implementation is a complete waste of time or if it is a bit useful (in this case I will do it). Some more details: in mi_dynrec.c there is a funtion _mi_get_block_info() and i've added a funtion AQ_mi_get_block_info(), these 2 functions do the same thing but in a different way. In my version of this function I've implemented only those case that I needed for my tests. Because I started with examining the source of MySQL I modified the code in the MyIsam-engine and applied these to the MyIsam-engine in MariaDB. For staying compatible with MySQL I think it's better to use these modifications in the Maria-engine of MariaDB, but only if you agree with my approach. This is what I modified: in myisamdef.h I added: #define USE_CODE_OF_AQ converting this line in a comment allowed me to use the old code, a define with this identifier activates my code after a recompile. This allowed me to do some comparisons. You will find my modifications in myisamdef.h, mi_dynrec.c and mi_extra.c if you look for this identifier. Please look at my approach. If you agree I will continue with this and check all the cases which i didn't touch for now. thank you AugustQ

1 0