- developers - lists.mariadb.org

[Maria-developers] updating a string session variable from a storage engine
by Rich Prohaska 12 Sep '13

12 Sep '13

i have a MYSQL_THDVAR_STR session variable in my storage engine. i want to update its value from the storage engine. it was suggested that i use "THDVAR(x)=new_value" to do it. i noticed that update_func_str is NOT called, so i wonder about the contract. is the caller responsible for memory management? i used PLUGIN_VAR_MEMALLOC for the var def. thanks

3 2

[Maria-developers] Problems building debug version of MariaDB 5.5.32
by Weldon Whipple 11 Sep '13

11 Sep '13

Hello, I'm having problems building a Debug version of MariaDB 5.5.32. (It's a new problem, I think; I'm almost positive it was working about two weeks ago.) Here is what I'm doing to build MariaDB: $ cmake -DCMAKE_BUILD_TYPE=Debug . $ make package At 69% complete, I get about 14 lines like this: [ 69%] Building CXX object unittest/mysys/CMakeFiles/explain_filename-t.dir/explain_filename-t.cc.o Linking CXX executable explain_filename-t `.text._ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function' referenced in section `.note.stapsdt' of ../../sql/libpartition.a(ha_partition.cc.o): defined in discarded section `.text._ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function[handler::ha_index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function)]' of ../../sql/libpartition.a(ha_partition.cc.o) followed by: collect2: ld returned 1 exit status make[2]: *** [unittest/mysys/explain_filename-t] Error 1 make[1]: *** [unittest/mysys/CMakeFiles/explain_filename-t.dir/all] Error 2 make: *** [all] Error 2 Some comments: 1. I'm running on CentOS 6.4: $ uname -a Linux weldon 2.6.32-358.18.1.el6.x86_64 #1 SMP Wed Aug 28 17:19:38 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux 2. I've tried "cmake -DRPM=centos6 . " (without Debug specified) and it works just fine. 3. I found the line in (282:) in ./CMakeLists.txt that reads "ADD_SUBDIRECTORY(unittest/mysys)" and commented it out. Then the build continued until it was trying to link mysqld ... and failed with the similar errors at 99% completion. (I actually WANT mysqld, so I couldn't comment it out too :-)) 4. Something I read indicates that MariaDB requires a GCC build against recent binutils. yum seems to indicate that both GCC and binutils are up-to-date. 5. The cmake option I'm using is the one for 5.5+ as instructed at https://mariadb.com/kb/en/compiling-mariadb-for-debugging/ 6. Yes, I also tried the old ./BUILD/compile-pentium64-debug-max (but it failed similarly) I THINK it's probably an incompatible version of some build tool on my development box. Thanks in advance for any advice. Weldon Whipple

2 1

[Maria-developers] MDEV 5007 - help me! it's a old mdev closed as not a bug but don't solve the problem
by Roberto Spadim 10 Sep '13

10 Sep '13

Hi guys! I posted the MDEV-5007 and it's a old problem from MDEV-3795 (the same problem) It (MDEV-3795) was closed as a not a bug, but i think it's a bug, but i didn't explained well at the first time I changed from a mysql server to a mariadb server and the result order by isn't executed, i tried myisam,aria,innodb,memory engines and all of them return the same problem (order by not executed) On the wrong query i don't see the "using filesort" at explain, and at the right query (with right order by) it show "using filesort" Could anyone help me? or at least EXPLAIN why the order by shouldn't be executed on the first query, and maybe a workaround using sql_optimizer variable or something like it? it's a closed software and i need to contact developer to change the query, that's why i'm talking about the sql_optimizer variable instead of rewrite the query Thanks -- Roberto Spadim

2 3

[Maria-developers] Rollback causes assertion with Galera 10.0
by Jan Lindström 10 Sep '13

10 Sep '13

Hi, Background: I merged 10.0 to lp:~maria-captains/maria/maria-10.0-galera/, merge result is pushed to the tree. In testing I hit repeatable to following assertion #0 0x00007f1ff987f037 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007f1ff9882698 in __GI_abort () at abort.c:90 #2 0x00007f1ff9877e03 in __assert_fail_base (fmt=0x7f1ff99ce578 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x109b4ff "empty()", file=file@entry=0x109b4d8 "/home/jan/mysql/galera-10.0/sql/log.cc", line=line@entry=294, function=function@entry=0x10a01e0 <binlog_cache_data::reset()::__PRETTY_FUNCTION__> "void binlog_cache_data::reset()") at assert.c:92 #3 0x00007f1ff9877eb2 in __GI___assert_fail (assertion=0x109b4ff "empty()", file=0x109b4d8 "/home/jan/mysql/galera-10.0/sql/log.cc", line=294, function=0x10a01e0 <binlog_cache_data::reset()::__PRETTY_FUNCTION__> "void binlog_cache_data::reset()") at assert.c:101 #4 0x00000000009497d4 in binlog_cache_data::reset (this=0x7f1f7403ad60) at /home/jan/mysql/galera-10.0/sql/log.cc:294 #5 0x0000000000949b6a in binlog_cache_mngr::reset (this=0x7f1f7403ab90, do_stmt=false, do_trx=true) at /home/jan/mysql/galera-10.0/sql/log.cc:462 #6 0x0000000000935a4c in binlog_truncate_trx_cache (thd=0x45b7d40, cache_mngr=0x7f1f7403ab90, all=true) at /home/jan/mysql/galera-10.0/sql/log.cc:1991 #7 0x0000000000936123 in binlog_rollback (hton=0x3891fb0, thd=0x45b7d40, all=true) at /home/jan/mysql/galera-10.0/sql/log.cc:2180 #8 0x000000000086b750 in ha_rollback_trans (thd=0x45b7d40, all=true) at /home/jan/mysql/galera-10.0/sql/handler.cc:1616 #9 0x00000000007a0c5f in trans_rollback (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/transaction.cc:321 #10 0x00000000006718b2 in mysql_execute_command (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:4634 #11 0x0000000000677d52 in mysql_parse (thd=0x45b7d40, rawbuf=0x7f1f74005288 "ROLLBACK", length=8, parser_state=0x7f1fe83bb610) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:6816 #12 0x00000000006774d3 in wsrep_mysql_parse (thd=0x45b7d40, rawbuf=0x7f1f74005288 "ROLLBACK", length=8, parser_state=0x7f1fe83bb610) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:6647 #13 0x00000000006688cd in dispatch_command (command=COM_QUERY, thd=0x45b7d40, packet=0x44807c1 "ROLLBACK ", packet_length=9) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:1451 #14 0x00000000006678a3 in do_command (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:1077 #15 0x000000000078ba4d in do_handle_one_connection (thd_arg=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_connect.cc:1404 #16 0x000000000078b78d in handle_one_connection (arg=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_connect.cc:1311 #17 0x0000000000b0c20a in pfs_spawn_thread (arg=0x45713a0) at /home/jan/mysql/galera-10.0/storage/perfschema/pfs.cc:1853 #18 0x00007f1ffa423f8e in start_thread (arg=0x7f1fe83bc700) at pthread_create.c:311 #19 0x00007f1ff9941e1d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) We go from log.cc::binlog_rollback : else if (ending_trans(thd, all) || (!(thd->variables.option_bits & OPTION_KEEP_LOG) && (!stmt_has_updated_non_trans_table(thd) || WSREP_FORMAT(thd->variables.binlog_format) != BINLOG_FORMAT_STMT) && (!cache_mngr->trx_cache.changes_to_non_trans_temp_table() || WSREP_FORMAT(thd->variables.binlog_format) != BINLOG_FORMAT_MIXED))) error= binlog_truncate_trx_cache(thd, cache_mngr, all); end there to cache_mngr->reset(false, true); and there is DBUG_ASSERT(empty()); bool empty() const { return pending() == NULL && my_b_tell(&cache_log) == 0; } At empty() this is what does not hold : my_b_tell(&cache_log) == 0; // not sure what this means At test output there is at the end: # 2013-09-09T10:03:21 [6620] Query: UPDATE LOW_PRIORITY IGNORE `table100_int_autoinc` AS X SET `col_char_12` = 220 WHERE X . `col_char_12_key` IN ( 2427781120 , 1 , 'i' , 'm' , 'o' ) ORDER BY `col_char_12`,`col_char_12_key`,`col_int`,`col_int_key`,`pk` LIMIT 8 failed: 1213 Deadlock found when trying to get lock; try restarting transaction. Further errors of this kind will be suppressed. # 2013-09-09T10:04:14 [6620] Query: DELETE LOW_PRIORITY IGNORE FROM `table10_key_pk_parts_2_int_autoinc` WHERE `col_char_12_key` IN ( 4200660992 , 1646198784 , 3888709632 , 32 , 'c' ) ORDER BY `col_char_12`,`col_char_12_key`,`col_int`,`col_int_key`,`pk` LIMIT 1 failed: 1205 Lock wait timeout exceeded; try restarting transaction. Further errors of this kind will be suppressed. Unfortunately, general log contains so many statements from different connections that they are not really helpful for getting repeatable test case. R: Jan -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>

2 4

[Maria-developers] AIX "make fails" - related to: MDEV-4941
by Michael Felt 09 Sep '13

09 Sep '13

AIX make fails with syntax errors. One has been fixed, per https://mariadb.atlassian.net/browse/MDEV-4941. Rather than submit error by error I thought it would be more productive to provide the output from make -i. After trying, but not seeing it 'attach', I am going to try attaching the output of make -i here. regards, Michael

2 2

[Maria-developers] Some clarifications requested for C client
by Peter Laursen 09 Sep '13

09 Sep '13

This is probably for Georg Richter (and/or @Wlad). I have a few qeustion! 1) https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Fix for CONC-44: LOAD DATA INFILE can't open utf16le encoded filenames" But we we can stil not specify UTF16LE as the CHARSET, I think? Say I have a file *named* "ÄÜÑБГД.csv". (both Western special characters and cyrillic characters in the file name). This file I will be able to read now (using revision 80), right? But if file *content* is this single line (one row, two columns) ÄÜÑ,БГД and if it is encoded as 'Windows Unicode'/utf16le I still will need to recode the file to utf8 before executing it with LOAD DATA (as UTF16LE still cannot be specified for CHARSET in LOAD DATA), right? Please confirm that I understand this! 2) https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Added support for embedded (sqlite)". This happened by beginning of this year but I just noticed now. Now what can we actually do with SQLite and the MariaDB C-API and what we the purpose of this revision? Is it to make it easier to write client applications using SQLite for internal storage? Or to make the C client code 'a generic API' for connecting to both MySQL (of all flavors) and SQLite? Is the the code/files that that were added different from offical SQLite downloads from http://www.sqlite.org/? I am not able to find any documentation anywhere. Also not a blog or whatever either. There must be some reason to add sqlite3 code?! 3) This seems to be the latest revision: https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Added dyamic column api; Added character set conversion" ('dyamic' is misspelled BTW!) Won't we need some documentation here? The claim that MariaDB's connector "has exactly the same API as the MySQL Connector/C for MySQL 5.5" ( https://kb.askmonty.org/en/mariadb-client-library-for-c/) cannot be true now as it adds funtionalities for managing MariaDB ' dymamic column waht oracle's API has not? The API from Oracle's library is now a *subset* or MariaDB API (or MariaDB's is a *superset* or Oracle). So much I can figure out, but should I need to code with it I would need description of functions' syntax, expected results and error listings. Also I am not able to figure out what the "character set conversion" referred to is doing and why it was added. 4) Why does the changelog here https://kb.askmonty.org/en/mariadb-client-library-for-c-100-changelog/ stop at revision 77? -- Peter Laursen -- Webyog

2 3

Re: [Maria-developers] Rev 3883: MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 08 Sep '13

08 Sep '13

Hi, Holyfoot! Just one question, see below. Otherwise ok. On Sep 08, holyfoot(a)askmonty.org wrote: > revno: 3883 > revision-id: holyfoot(a)askmonty.org-20130908092209-xar471jadl7lmwzo > parent: sergii(a)pisem.net-20130907203634-htu89dqmwe5gqwy5 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 55-4472 > timestamp: Sun 2013-09-08 14:22:09 +0500 > message: > MDEV-4472 Audit-plugin. Server-related part of the task. > file_logger became the service. > Data like query_id now are sent to the audit plugin. > Fix for MDEV-4770 ported from 10.0. > Fix added for the read_maria_plugin_info(). > Log rotation can be disabled with 'set rotations=0'. > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-08 09:22:09 +0000 > @@ -16,7 +16,7 @@ > > #include "my_global.h" > #include <my_sys.h> > -#include "service_logger.h" > +#include <mysql/service_logger.h> > #include <my_pthread.h> > > extern MYSQL_PLUGIN_IMPORT char *mysql_data_home; I believe we agreed to keep file_logger.c completely clean from dynamic-plugin-related code. Did you forget to remove the line above? Regards, Sergei

1 0

Re: [Maria-developers] MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 08 Sep '13

08 Sep '13

Hi, Holyfoot! On Sep 05, holyfoot(a)askmonty.org wrote: > At file:///home/hf/wmar/55-4472/ > > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-05 12:47:00 +0000 > @@ -16,10 +16,12 @@ > > #include "my_global.h" > #include <my_sys.h> > -#include "service_logger.h" > +#include <mysql/service_logger.h> > #include <my_pthread.h> > > +#ifndef _WIN32 > extern MYSQL_PLUGIN_IMPORT char *mysql_data_home; > +#endif /*!_WIN32*/ > > #ifdef HAVE_PSI_INTERFACE > /* These belong to the service initialization */ > @@ -51,6 +53,12 @@ LOGGER_HANDLE *logger_open(const char *p > unsigned int rotations) > { > LOGGER_HANDLE new_log, *l_perm; > + //char *local_mysql_data_home; > +//#ifdef _WIN32 > + //local_mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); > +//#else > + //local_mysql_data_home= mysql_data_home; > +//#endif /*_WIN32*/ I think you can completely remove this and the #ifndef above. For your MySQL plugin you can simply #include this in the plugin body like extern char *mysql_data_home; #include "file_logger.c" and later in your plugin_init function: #ifdef _WIN32 mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); #endif This way you will keep file_logger.c completely free of the dynamic plugin hacks. > /* > I don't think we ever need more rotations, > @@ -168,6 +176,29 @@ int logger_vprintf(LOGGER_HANDLE *log, c > int logger_rotate(LOGGER_HANDLE *log) > { > int result; I believe you've added recently a test to disable rotation if number of rotations is 0? Regards, Sergei

2 1

[Maria-developers] first contribution
by AugustQ 06 Sep '13

06 Sep '13

Hi, after some struggling I got this message from Bazaar: Created new stacked branch referring to /+branch-id/633549. I hope everything worked fine. If not let me know so that I can correct it. I created a branch to 10.0.4 of MariaDB and named it: contrib_AQ_01. I've uploaded this some minutes before. For some time I played with the source of MySQL 5.5.8. I've taken some of my modifications to MariaDB 10.0.4 and want to give them to you for further discussions. To say it first: this is not production-ready-code, it's a proof of a concept. My modifications are made for the MyIsam-engine in the handling of a full table-scan with a table of dynamic sized records. The reason for uploading this is: during my tests I executed statements like SELECT * FROM table INTO OUTFILE 'x.x'; With my test-data this took about 20 seconds with the original code and 12 seconds with my modifications. I've tested my code with a single table which contains 10 Mio records, a total size of 1GB. As i've written before: it's a proof of concept. I've not implemented all the possible cases. I wanted to know if a full implementation is a complete waste of time or if it is a bit useful (in this case I will do it). Some more details: in mi_dynrec.c there is a funtion _mi_get_block_info() and i've added a funtion AQ_mi_get_block_info(), these 2 functions do the same thing but in a different way. In my version of this function I've implemented only those case that I needed for my tests. Because I started with examining the source of MySQL I modified the code in the MyIsam-engine and applied these to the MyIsam-engine in MariaDB. For staying compatible with MySQL I think it's better to use these modifications in the Maria-engine of MariaDB, but only if you agree with my approach. This is what I modified: in myisamdef.h I added: #define USE_CODE_OF_AQ converting this line in a comment allowed me to use the old code, a define with this identifier activates my code after a recompile. This allowed me to do some comparisons. You will find my modifications in myisamdef.h, mi_dynrec.c and mi_extra.c if you look for this identifier. Please look at my approach. If you agree I will continue with this and check all the cases which i didn't touch for now. thank you AugustQ

1 0

Re: [Maria-developers] MDEV-4978 - Server cursor is broken with blobs in the select list
by Sergei Golubchik 06 Sep '13

06 Sep '13

Hi, Sergey! On Sep 06, Sergey Vojtovich wrote: > revno: 3878 > revision-id: svoj(a)mariadb.org-20130906111656-oy2c3pwy2o1q0zqn > parent: wlad(a)montyprogram.com-20130903204512-hgwu77prle0lw7a3 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5-mdev4978 > timestamp: Fri 2013-09-06 15:16:56 +0400 > message: > MDEV-4978 - Server cursor is broken with blobs in the select list, > ORDER BY does not work > > Use "dynamic" row format (instead of "block") for MARIA internal > temporary tables created for cursors. > With "block" row format MARIA may shuffle rows, with "dynamic" row > format records are inserted sequentially (there are no gaps in data > file while we fill temporary tables). > This is needed to preserve row order when scanning materialized cursors. > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2013-09-03 14:41:07 +0000 > +++ b/sql/sql_select.cc 2013-09-06 11:16:56 +0000 > @@ -15373,7 +15374,8 @@ create_tmp_table(THD *thd, TMP_TABLE_PAR > if (thd->is_fatal_error) // If end of memory > goto err; /* purecov: inspected */ > share->db_record_offset= 1; > - table->used_for_duplicate_elimination= (param->sum_func_count == 0 && > + table->used_for_duplicate_elimination= keep_row_order || > + (param->sum_func_count == 0 && > (table->group || table->distinct)); That's very confusing. Why keep_row_order means that the table is "used for duplicate elimination"? I had to check the code to see that used_for_duplicate_elimination affects the row format in Aria. This is not obvious too. And it forces the row format to be DYNAMIC for a very different reason, not for preserving row order. Perhaps it'd be better to rename used_for_duplicate_elimination to "force_dynamic_Aria_row_format" ? A prettier solution would be to introduce another flag: table->keep_row_order. Regards, Sergei

2 2

[Maria-developers] Bazaar-problem
by AugustQ 06 Sep '13

06 Sep '13

Hi, I'm struggling with Bazaar. Maybe you can help me. I've made some modifications to the source-code and committed these to the local repository. Next I tried to upload them via push. This is what I receive: Cannot lock LockDir(chroot-70341520:///% 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: readonly transport As I'm using Eclipse I found this within Eclipse on the console: Error while executing push Cannot lock LockDir(chroot-70341520:///% 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: readonly transport ------ XmlRpcClient Executing: push "bzr+ssh://bazaar.launchpad.net/+branch/maria/" Can you please give me a tip. Thanks AugustQ

5 9

[Maria-developers] Incorrect format description event skipping in queue_event()
by Pavel Ivanov 04 Sep '13

04 Sep '13

Kristian, You have the following comment in the queue_event() in sql/slave.cc: /* Do not queue any format description event that we receive after a reconnect where we are skipping over a partial event group received before the reconnect. (If we queued such an event, and it was the first format_description event after master restart, the slave SQL thread would think that the partial event group before it in the relay log was from a previous master crash and should be rolled back). */ I don't understand which failure scenario you are talking about here and I claim that this bypassing of queuing into relay log is incorrect. My reasoning is: relay log can always be rotated in the middle of event group. During rotation when new relay log file is created format description event is always written at the beginning of the file. So in this case SQL thread will see format description event in the middle of event group and according to my testing it doesn't rollback the active transaction because of that. So when do you think it will rollback? Now, you may say that there's no problem in skipping the format description event because everything works. But it's only for now. When IO thread is reconnecting it rotates relay log and as I said it writes format description event at the beginning of the new file. But it writes an event that it created itself, i.e. not the one that master have sent. And as format description event from master is not written into relay log SQL thread from this point on starts to use format description generated by slave which may be different from the one generated by master. It may lead to a broken replication and SQL thread may not recognize events in the relay log if difference between master's and slave's format descriptions is significant. Another somewhat related question: Gtid_log_event::peek() (as well as Gtid_log_event constructor from const char* buf) is implemented with assumption that Format_description_log_event::common_header_len is always equal to LOG_EVENT_HEADER_LEN. While currently it's true I believe common_header_len can be increased in future MariaDB versions. And when it happens old slave won't be able to replicate from a new master. I'd think this is a bug. What do you think? Pavel

2 10

Re: [Maria-developers] Rev 3876: MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 04 Sep '13

04 Sep '13

Hi, Holyfoot! Mostly ok. There are few comments, see below: On Sep 03, holyfoot(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 3876 > revision-id: holyfoot(a)askmonty.org-20130903082521-xpia2wond6oggpsk > parent: igor(a)askmonty.org-20130831163309-vqtls4oubhvz9zcz > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 55-4472 > timestamp: Tue 2013-09-03 13:25:21 +0500 > message: > MDEV-4472 Audit-plugin. Server-related part of the task. > file_logger became the service. > Data like query_id now are sent to the audit plugin. > Fix for MDEV-4770 ported from 10.0. > Fix added for the read_maria_plugin_info(). > === modified file 'include/mysql/plugin_audit.h' > --- a/include/mysql/plugin_audit.h 2013-04-19 10:50:16 +0000 > +++ b/include/mysql/plugin_audit.h 2013-09-03 08:25:21 +0000 > @@ -25,7 +25,7 @@ > > #define MYSQL_AUDIT_CLASS_MASK_SIZE 1 > > -#define MYSQL_AUDIT_INTERFACE_VERSION 0x0301 > +#define MYSQL_AUDIT_INTERFACE_VERSION 0x0302 > > > /************************************************************************* > @@ -59,6 +59,10 @@ struct mysql_event_general > struct charset_info_st *general_charset; > unsigned long long general_time; > unsigned long long general_rows; > + /* Added for 0x302 */ A bit confusing. Better "Added in version 0x0302" > + long long query_id; should it rather be unsigned? > + const char *database; > + unsigned int database_length; > }; > > > @@ -140,6 +144,8 @@ struct mysql_event_table > unsigned int new_database_length; > const char *new_table; > unsigned int new_table_length; > + /* Added for 0x302 */ > + long long query_id; > }; > > /************************************************************************* > > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-03 08:25:21 +0000 > @@ -51,6 +53,12 @@ LOGGER_HANDLE *logger_open(const char *p > unsigned int rotations) > { > LOGGER_HANDLE new_log, *l_perm; > + char *local_mysql_data_home; > +#ifdef _WIN32 > + local_mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); > +#else > + local_mysql_data_home= mysql_data_home; > +#endif /*_WIN32*/ You shouldn't need it, file_logger.c is part of the server, not a dynamically loaded plugin. > > /* > I don't think we ever need more rotations, > === modified file 'sql/sql_audit.h' > --- a/sql/sql_audit.h 2013-04-19 10:50:16 +0000 > +++ b/sql/sql_audit.h 2013-09-03 08:25:21 +0000 > @@ -93,7 +93,8 @@ void mysql_audit_general_log(THD *thd, t > > mysql_audit_notify(thd, MYSQL_AUDIT_GENERAL_CLASS, MYSQL_AUDIT_GENERAL_LOG, > 0, time, user, userlen, cmd, cmdlen, > - query, querylen, clientcs, 0); > + query, querylen, clientcs, (ha_rows) 0, > + thd->db, thd->db_length); no query_id? > } > } > > @@ -139,7 +140,8 @@ void mysql_audit_general(THD *thd, uint > > mysql_audit_notify(thd, MYSQL_AUDIT_GENERAL_CLASS, event_subtype, > error_code, time, user, userlen, msg, msglen, > - query.str(), query.length(), query.charset(), rows); > + query.str(), query.length(), query.charset(), rows, > + thd->db, thd->db_length); no query_id? > } > } > > === modified file 'sql/sql_plugin.cc' > --- a/sql/sql_plugin.cc 2013-06-13 18:19:11 +0000 > +++ b/sql/sql_plugin.cc 2013-09-03 08:25:21 +0000 > @@ -2645,13 +2647,16 @@ static void update_func_longlong(THD *th > static void update_func_str(THD *thd, struct st_mysql_sys_var *var, > void *tgt, const void *save) > { > - char *old= *(char **) tgt; > - *(char **)tgt= *(char **) save; > + char *value= *(char**) save; > if (var->flags & PLUGIN_VAR_MEMALLOC) > { > - *(char **)tgt= my_strdup(*(char **) save, MYF(0)); > + char *old= *(char**) tgt; > + if (value) > + *(char**) tgt= my_strdup(value, MYF(0)); add else *(char**) tgt= 0; > my_free(old); > } > + else > + *(char**) tgt= value; > } > Regards, Sergei

2 3

[Maria-developers] RFC - CREATE PROCEDURE clauses (question about docs from mariadb-discuss)
by Roberto Spadim 02 Sep '13

02 Sep '13

Hi mariadb-devel guys, could anyone help here in a documentation problem? (maybe not a problem...) *Please answer the BOLD questions* We are talking about CREATE PROCEDURE , about DETERMINISTIC / NO SQL http://dev.mysql.com/doc/refman/5.6/en/create-procedure.html *characteristic*: COMMENT '*string*' | LANGUAGE SQL | *[NOT] DETERMINISTIC* | { CONTAINS SQL | *NO SQL* | READS SQL DATA | MODIFIES SQL DATA } | SQL SECURITY { DEFINER | INVOKER } *--------* *from mariadb-discuss:* *--------* Hi Federico, sorry i read some points and start writing, i will read and write after... i will answer with my experience as an user, not as a mariadb/mysql developer... 2013/9/2 Federico Razzoli <federico_raz(a)yahoo.it> > Hi Roberto > I'll try to be more clear, please read carefully before answering > ok I know what a deterministic expression is - but *what kinds of output must > be deterministic for a procedure to be defined as DETERMINISTIC?* > hummm i don't know, but for procedure, there isn't a deterministic output right? just for funcitons... maybe it's a parser problem, allow deterministic keyword for procedures (must check) i'm thinking about, *why we use deterministic procs, and non deterministic procs?* *this change something important in mysql server? replication, query cache? other performace considerations?* > There are many types of output for a proc: > * OUT params > * SELECTs > * writes on tables > * SET @var > hum.... from what i remember procedure ~= function without return, something like a "void FUNCTION( any parameters )" function ~= a procedure that return values, in other words it must be different from void... that's why i think it's a problem at parser allowing deterministic keyword at procedure, but must check... i don't know if this keyword change something in mysql/mariadb code > For a deterministic proc, do all of these need to be deterministic? > well, for me, procs are always non deterministics since it don't change query cache invalidation and replication (must be done in binary i think) > * I guess the answer isn't easy. > * *yeah, i see, we need a simple answer not a simplified answer* > -- > For "NO SQL" a complete list of SQL statements would be needed. If we use > this as a reference: > http://dev.mysql.com/doc/refman/5.6/en/sql-syntax.html > , even RETURN seems to be a SQL statement, yes, maybe the point isn't the SQL statements list, but other functions that mysql/mariadb server do? i don't know... NO SQL maybe is different from NOSQL (memcache,mongo,etc...) since nosql is newer than mysql procedures the point is *what's NO SQL in CREATE PROCEDURE ?* but of course it can't be true, because it's impossible to write a func > without RETURN. > yeah, maybe we have a problem at docs... let's see if anyone else know about this... maybe we need help from mariadb-devel list, i will include in this email... > > Greetings > Federico > bye =) thanks guys -- Roberto Spadim

1 0

Re: [Maria-developers] FWD: Status on MDEV-4506, parallel replication
by Kristian Nielsen 02 Sep '13

02 Sep '13

Michael Widenius <monty(a)askmonty.org> writes: >> On the master, I implemented --binlog-commit-wait-count=N and >> --binlog-commit-wait-usec=T. A transaction will wait at most T microseconds >> for at least N transactions to queue up and be ready for group commit. This >> allows to deliberately delay transactions on the master in order to get bigger >> group commits and thus better opportunity for parallel execution (and again it >> makes testing easier). > > Do you think the above helps in any real world case, except testing? > > Assuming we are have N=4 and T=2 > > - We have 3 threads ready to commit > - We wait 2 milliseconds and get 2 more threads. > - We commit now 5 threads. During this time you get 3 other threads > waiting to commit. > - We wait again... > > In a scenario without waiting: > > - We have 3 threads ready to commit > - We commit 3 threads. During this time we get 5 more threads waiting > to commit. > - We commit 5 threads. > > In other words, as the group commit will anyway take up 50 > microseconds on a hard disk, and thus automaticly group things > together for the next commit, why do we need ever to wait more? > > I agree that with --binlog-commit-wait-count you may get less sync > calls, but at an expense that a lot of threads took up to T > microseconds longer to execute. > > The worst case scenario is when you have only one user doing a lot of > inserts with auto-commit. In this case using wait will slow down the > server with T microseconds for every query. > > Have you been able to run any kind of benchmark where using > binglog-commit-wait-count will give more performance? > > I agree that the options are good for testing. The main question I > have is if we want to have the variables in the production server and > how we should document when and how a user should use the variables. I agree that with respect to group commit, the --binlog-commit-wait-count in many cases will not improve performance (that is why I did not implement it earlier). However, for parallel replication things are a bit different. Suppose that the application is doing C transaction commits per second, and that the disk system is capable of F binlog fsyncs per second. Now, if C is significantly bigger than F, then things are as you describe. Generally several transactions will queue up while the previous group commit runs, and there will be sufficient parallelism without using --bicommit_wait-*. This will be typical for eg. a simple harddisk-based system (F=40 commits/second perhaps) where all data is cached in the InnoDB buffer pool (eg. C>500 transactions/second). On the other hand, if C is smaller than F (or of similar magnitide), then usually only few or no new transactions will have time to queue up while the previous transaction is committing. So there will not be much parallelism for parallel replication to exploit without using --binlog-commit-wait-*. This will be typical for eg. a good-quality server with battery-backup RAID controller (F > 1000 commits/second) where data is too big to fit in buffer pool and every update requires disk access to complete (C < 500 transactions/second for example). And in fact, it is the second case, where random I/O is the bottleneck and multiple disk spindels are needed to improve I/O throughput, that single-threaded slave hurts the most, and where increasing --binlog-commit-wait-usec can be used with least penalty. I agree we need to document clearly the risk that --binlog-commit-wait-* will decrease performance on the master. Basically, the user can look at the ratio of Binlog_commits and Binlog_group_commits to check if enough transactions are part of each group commit for parallel replication to be effective. >> 1. The existing code is not thread-safe for class Relay_log_info. This class >> contains a bunch of stuff that is specific to executed transactions, not >> related to relay-log at all. This needs to be moved to the new struct >> rpl_group_info I introduced, and all code updated to pass around a pointer to >> that struct instead. There may also be a need to add additional locking on >> Relay_log_info, existing code needs review for this. > > I could take a look working on the above tomorrow and next week. Ok, great. Ping me when you can so we can coordinate, I may have some partial patches for this lying around. - Kristian.

1 0

[Maria-developers] Fwd: Bazaar-problem
by Nitish Upreti 31 Aug '13

31 Aug '13

I cant even clone the repository. Bazaar crashes with a bug midway. Can we have an active git mirror of the project for people just starting out? On Sat, Aug 31, 2013 at 2:13 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, AugustQ! > > On Aug 31, AugustQ wrote: > > I'm struggling with Bazaar. Maybe you can help me. > > > > I've made some modifications to the source-code and committed these to > > the local repository. Next I tried to upload them via push. This is what > > I receive: > > > > Cannot lock LockDir(chroot-70341520:///% > > 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: > > readonly transport > > You don't have write access to the mariadb source repository. > That's why you can only pull, but you cannot push. > > Regards, > Sergei > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-developers > Post to : maria-developers(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-developers > More help : https://help.launchpad.net/ListHelp >

2 1

[Maria-developers] introduction
by AugustQ 29 Aug '13

29 Aug '13

Hi, i've joined this list some days ago because I want to participate in the discussions here. The following links give you an overview of what i've done in the past: without registration: https://www.xing.com/profiles/August_Quint with registration: http://www.linkedin.com/profile/view?id=9080630 unfortunately the texts are in German, so in short: i've worked as a software-developer mainly in the backend area. I've made some small modifications on the source code but currently I'm struggling with Bazaar. This should be solved soon and I will then upload some code to the repository. I hope I can help you a bit. AugustQ PS: i found this: in MySQL I call cmake with an argument: -DWITH_DEBUG=1 in MariaDB this is: -DCMAKE_BUILD_TYPE=Debug is this correct?

2 1

[Maria-developers] alter online table documentation
by Roberto Spadim 26 Aug '13

26 Aug '13

Hi guys, i was searching alter online docs, at mariadb https://kb.askmonty.org/en/alter-table-statement/ but didn't found anything, could we add ? thanks! -- Roberto Spadim

4 5

Re: [Maria-developers] Calll for testing: MariaDB 5.5.32 packages
by Otto Kekäläinen 23 Aug '13

23 Aug '13

2013/8/23 Сергей Войтович <svojtovich(a)gmail.com>: > thanks for packaging MariaDB. I did a few basic tests with your packages and I confirm that they work well. > The only minor note I have is that I was asked for root password 3x during installation, which is a bit too much. Thanks! I haven't noticed any extra password asking during installation. It should ask for MariaDB root password in two screens, and of course if you run sudo apt-get it will ask once for your user password in the beginning. Could you clarify a little what happened in you installation?

2 3

[Maria-developers] Additional documentation for GTID
by Kristian Nielsen 23 Aug '13

23 Aug '13

Hi Daniel, Ian, I have implemented a new system variable for global transaction ID and pushed it into 10.0-base (so it should appear in 10.0.5 I suppose). I have written some text for documenting the new variable, appended below. Can one of you help me get this text into the Knowledgebase with proper formatting and cross-referencing across relevant pages? - Kristian. ----------------------------------------------------------------------- Variable: gtid_binlog_state Scope: global Dynamic: Yes Type: String The variable gtid_binlog_state holds the internal state of the binlog. The state consists of the last GTID ever logged to the binary log for every combination of domain_id and server_id. This information is used by the master to determine whether a given GTID has been logged to the binlog in the past, even if it has later been deleted due to binlog purge. For each domain_id, the last entry in @@gtid_binlog_state is the last GTID logged into binlog, ie. this is the value that appears in @@gtid_binlog_pos. Normally this internal state is not needed by users, as @@gtid_binlog_pos is more useful in most cases. The main usage of @@gtid_binlog_state is to restore the state of the binlog after RESET MASTER (or equivalently if the binlog files are lost). If the value of @@gtid_binlog_state is saved before RESET MASTER and restored afterwards, the master will retain information about past history, same as if PURGE BINARY LOGS had been used (of course the actual events in the binary logs are still deleted). Note that to set the value of @@gtid_binlog_state, the binary log must be empty, that is it must not contain any GTID events and the previous value of @@gtid_binlog_state must be the empty string. If not, then RESET MASTER must be used first to erase the binary log first. For completeness, note that setting @@gtid_binlog_state internally executes a RESET MASTER. This is normally not noticable as it can only be changed when the binlog is empty of GTID events. However, if executed eg. immediately after upgrading to MariaDB 10, it is possible that the binlog is non-empty but without any GTID events, in which case all such events will be deleted, just as if RESET MASTER had been run.

2 1

[Maria-developers] Error in 10.0.4 changelog, "Partition lock pruning"?
by Sergei Petrunia 23 Aug '13

23 Aug '13

Hello, Igor has pointed out that the changelog for 10.0.4 https://kb.askmonty.org/en/mariadb-1004-release-notes/ has this line: Partition lock pruning (MySQL Bug #37252, Bug #11748732) while the feature doesn't seem to be there. I did some investigation: - The line was in the changelog from the very first edit, it was put there by Daniel (who currently is on vacation) - I was unable to find any trace of MySQL Bug #37252 or Bug #11748732 in current 10.0 tree. - It seems, the whole "partition lock pruning" feature so far has not been ported to MariaDB 10.0. Does anybody know anything else about this? Should we remove the line from the changelog? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

3 3

Re: [Maria-developers] Fwd: [Commits] Rev 3674: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-base-MDEV-4645/
by Kristian Nielsen 23 Aug '13

23 Aug '13

Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Could you review this? > > (it is https://mariadb.atlassian.net/browse/MDEV-4645 ) Ok to push. - Kristian.

1 0

[Maria-developers] Buildbot failure in 10.0-base
by Sergei Petrunia 23 Aug '13

23 Aug '13

Hi Kristian, 10.0-base started to get compile failures on [nearly] all hosts after your yesterday push: > 3682. By Kristian Nielsen 22 hours ago > MDEV-4488: When master is on the list of ignore_server_ids, GTID position on > slave is not updated > > The ignored events are not written to the relay log, but instead a fake > Rotate event is generated to handle update of position. > > Extend this for Gtid so we similarly generate a fake Gtid_list event > to update the GTID position. > > Also fix an unrelated test issue that got triggered by the added test cases. Could you please look into this? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] MariaDB mutex contention (was: Re: MariaDB 10.0 benchmarks on upcoming IVB-EP processors ...)
by Axel Schwenke 22 Aug '13

22 Aug '13

Hi, Michael Widenius wrote: > 10.0.4 should be very soon ready for benchmarking (Tuesday or > Wednesday). We are in the final stage of fixing the failing test cases > after our last merge with MySQL 5.6 GA. > > I would just prefer to see Sergei Vojtovich push his code that changes > how the table cache works. > > This solves the same problem as the split table cache in 5.6, > but is a much simpler and better implementation. > > After this, I would expect that MariaDB 10.0 performance would be > closer to 5.6 performance for this benchmark. > Axel, could you at some point run the same benchmark with MariaDB > 10.0.4 and MySQL 5.6 with gprof to see where time is spent? I have now beefed up my knowledge of performance_schema and used it to look at Mutex/RW-lock contention for readonly OLTP benchmark. ----- TL;DR: 10.0.4 spends very long time in LOCK_PLUGIN, LOCK_OPEN is indeed much better than 10.0.3 but not(?) nearly as good as MySQL-5.6. ----- Contestants: - MySQL-5.6.13 (release) - MariaDB-10.0.3 (release) - MariaDB-10.0 r3799 (bzr trunk) all 3 compiled from source on benchmark machine using the same build script Environment: - many core machine (32 cores, 64 threads) - sysbench-0.5 OLTP readonly, 10M rows, single table - mysqld bound to 48 cores (sysbench to the other 8) - tested with 64 threads (this is the sweet spot) - using tcmalloc (preloaded) - mutexes and rw locks fully instrumented via performance_schema, see my.cnf Benchmark results (transactions per second): P_S on P_S off MySQL: ~15K ~18K 10.0.3: ~7K ~7.5K 10.0.4: ~6.5K ~9K After stabilizing I took 3 probes from P_S, sorting by sum_timer_wait desc. Times shown by P_S are picoseconds. If this were seconds, the decimal point would be in the column between "m" and "_" in "sum_timer_wait". So i.e. 10.0.4 waited 242 seconds for LOCK_PLUGIN during the 10 second measurement interval. Since we have 48 cores, there are total 480 seconds and 10.0.4 wasted about 50% of that. Questions: - what is protected by LOCK_PLUGIN? - how is LOCK_OPEN split in MySQL-5.6?If some of that is now covered by THR_LOCK::mutex, then it would explain why this is the busiest mutex in MySQL-5.6. Let me know if you need more data from me or if you have ideas what else I should test. XL Top 20 Mutexes/RW-locks MySQL-5.6.13 +---------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+ | wait/synch/mutex/mysys/THR_LOCK::mutex | 3397732 | 123367802416236 | | wait/synch/rwlock/innodb/btr_search_latch | 1632176 | 22207357475142 | | wait/synch/rwlock/sql/LOCK_grant | 1682983 | 1526912505972 | | wait/synch/mutex/sql/LOCK_table_cache | 3397786 | 1247140492941 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 9219327 | 1076510130276 | | wait/synch/rwlock/innodb/hash_table_locks | 4086413 | 1026755227155 | | wait/synch/mutex/innodb/trx_sys_mutex | 364146 | 639270297372 | | wait/synch/mutex/innodb/os_mutex | 149491 | 630758755641 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 1016436 | 344264365074 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 242778 | 183463650006 | | wait/synch/mutex/sql/MDL_map::mutex | 121342 | 69121460394 | | wait/synch/mutex/mysys/BITMAP::mutex | 242667 | 54426177477 | | wait/synch/mutex/innodb/fil_system_mutex | 179586 | 20229032103 | | wait/synch/mutex/sql/LOCK_plugin | 121352 | 15004585617 | | wait/synch/mutex/innodb/trx_mutex | 121475 | 13105527141 | | wait/synch/mutex/innodb/lock_mutex | 121374 | 12627692868 | | wait/synch/mutex/innodb/buf_pool_mutex | 2240 | 228234006 | | wait/synch/mutex/innodb/log_sys_mutex | 540 | 119752122 | | wait/synch/mutex/innodb/flush_list_mutex | 704 | 86802870 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 640 | 30728817 | +---------------------------------------------+------------+-----------------+ MariaDB-10.0.3 +------------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +------------------------------------------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_open | 3334966 | 251668176449112 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 2206782 | 2761461343332 | | wait/synch/mutex/sql/LOCK_plugin | 2540899 | 1471913471196 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 6485120 | 770989733856 | | wait/synch/rwlock/innodb/hash table locks | 3315788 | 674878255920 | | wait/synch/rwlock/innodb/btr_search_latch | 1358771 | 540950632524 | | wait/synch/mutex/innodb/trx_sys_mutex | 380210 | 245662486416 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 191698 | 125257499748 | | wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data | 1067704 | 95258676492 | | wait/synch/mutex/mysys/THR_LOCK_lock | 157621 | 52329482172 | | wait/synch/mutex/mysys/BITMAP::mutex | 154870 | 49415303736 | | wait/synch/mutex/sql/MDL_map::mutex | 79173 | 41936664096 | | wait/synch/mutex/innodb/fil_system_mutex | 129610 | 40258209024 | | wait/synch/mutex/innodb/lock_mutex | 93667 | 26848457928 | | wait/synch/mutex/innodb/trx_mutex | 94131 | 11398946760 | | wait/synch/mutex/innodb/buf_pool_mutex | 12672 | 571983480 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 9152 | 312365052 | | wait/synch/mutex/innodb/log_sys_mutex | 456 | 138632868 | | wait/synch/mutex/innodb/flush_list_mutex | 480 | 109226268 | | wait/synch/mutex/innodb/os_mutex | 350 | 71117100 | +------------------------------------------------+------------+-----------------+ MariaDB-10.0-r3799 +---------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_plugin | 4194896 | 242595666533160 | | wait/synch/mutex/sql/LOCK_open | 1835243 | 33790056812460 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 1835221 | 3136249420092 | | wait/synch/rwlock/sql/LOCK_tdc | 911928 | 1088553356028 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 5766813 | 695664363024 | | wait/synch/rwlock/innodb/hash table locks | 2144715 | 513517462020 | | wait/synch/rwlock/innodb/btr_search_latch | 915043 | 399309801912 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 131171 | 176610102480 | | wait/synch/mutex/innodb/trx_sys_mutex | 196751 | 154609451460 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 527543 | 137903039712 | | wait/synch/mutex/sql/MDL_map::mutex | 158268 | 95133455712 | | wait/synch/mutex/innodb/fil_system_mutex | 97493 | 27285605172 | | wait/synch/mutex/mysys/BITMAP::mutex | 131099 | 26589125196 | | wait/synch/mutex/mysys/THR_LOCK_lock | 131105 | 23952666708 | | wait/synch/mutex/innodb/lock_mutex | 65579 | 10355822676 | | wait/synch/mutex/innodb/trx_mutex | 65812 | 9790097724 | | wait/synch/mutex/innodb/buf_pool_mutex | 11550 | 644569236 | | wait/synch/rwlock/innodb/checkpoint_lock | 2 | 236298492 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 3880 | 156907368 | | wait/synch/mutex/innodb/flush_list_mutex | 692 | 124740900 | +---------------------------------------------+------------+-----------------+ Comparison ---------- MariaDB-10.0-r3799 MariaDB-10.0.3 MySQL-5.6.13 +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | count_star | sum_timer_wait | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_plugin | 4194896 | 242595666533160 | 2540899 | 1471913471196 | 121352 | 15004585617 | | wait/synch/mutex/sql/LOCK_open | 1835243 | 33790056812460 | 3334966 | 251668176449112 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 1835221 | 3136249420092 | 2206782 | 2761461343332 | 3397732 | 123367802416236 | | wait/synch/rwlock/sql/LOCK_tdc | 911928 | 1088553356028 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 5766813 | 695664363024 | 6485120 | 770989733856 | 9219327 | 1076510130276 | | wait/synch/rwlock/innodb/hash table locks | 2144715 | 513517462020 | 3315788 | 674878255920 | 4086413 | 1026755227155 | | wait/synch/rwlock/innodb/btr_search_latch | 915043 | 399309801912 | 1358771 | 540950632524 | 1632176 | 22207357475142 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 131171 | 176610102480 | 191698 | 125257499748 | 242778 | 183463650006 | | wait/synch/mutex/innodb/trx_sys_mutex | 196751 | 154609451460 | 380210 | 245662486416 | 364146 | 639270297372 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 527543 | 137903039712 | 1016436 | 344264365074 | | wait/synch/mutex/sql/MDL_map::mutex | 158268 | 95133455712 | 79173 | 41936664096 | 121342 | 69121460394 | | wait/synch/mutex/innodb/fil_system_mutex | 97493 | 27285605172 | 129610 | 40258209024 | 179586 | 20229032103 | | wait/synch/mutex/mysys/BITMAP::mutex | 131099 | 26589125196 | 154870 | 49415303736 | 242667 | 54426177477 | | wait/synch/mutex/mysys/THR_LOCK_lock | 131105 | 23952666708 | 157621 | 52329482172 | | wait/synch/mutex/innodb/lock_mutex | 65579 | 10355822676 | 93667 | 26848457928 | 121374 | 12627692868 | | wait/synch/mutex/innodb/trx_mutex | 65812 | 9790097724 | 94131 | 11398946760 | 121475 | 13105527141 | | wait/synch/mutex/innodb/buf_pool_mutex | 11550 | 644569236 | 12672 | 571983480 | 2240 | 228234006 | | wait/synch/rwlock/innodb/checkpoint_lock | 2 | 236298492 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 3880 | 156907368 | 9152 | 312365052 | 640 | 30728817 | | wait/synch/mutex/innodb/flush_list_mutex | 692 | 124740900 | 480 | 109226268 | 704 | 86802870 | +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+

7 18

[Maria-developers] Flashback Feature patch review
by Lixun Peng 22 Aug '13

22 Aug '13

Hi Monty, I patched my flashback feature to 10.0-base, and this is the diff file, please help me to review it. How to use this feature? I added several new arguments for mysqlbinlog: --flashback (-B), it will let mysqlbinlog working on FLASHBACK mode. --table (-T), it likes -d, but this argument is for table. --review, it will le mysqlbinlog print the SQL for reviewing. Reviewing feature will create a new "review" table to record the data that will be changed by FLASHBACK feature. If you aren't special the --review-dbname/tablename, "review" table will created on current DB, and the table name will be "__"+original_table_name. And for getting the original table struct, mysqlbinlog need to connect mysql, so --user/--host/--password is needed if necessary. --review-dbname, the DB that you want to store the review table. --review-tablename the TABLE that you want to store the changed data. Only if you set -T, this argument is useful. For example: I have a table "t" in database "test", CREATE TABLE `t` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `info` varchar(32) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=9043880 DEFAULT CHARSET=utf8; And I done this operation, root@localhost : (none) 14:52:50> reset master; root@localhost : (none) 14:53:02> use test root@localhost : test 14:53:08> delete from t limit 10; root@localhost : test 14:53:25> insert into t (info) select "你好"; root@localhost : test 14:54:01> update t set info ='hello你好' limit 5; We can compare the output with --flashback or not. #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -d test -T t --start-datetime="2013-03-27 14:54:00" > /tmp/1.sql #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -uroot -d test -T t --start-datetime="2013-03-27 14:54:00" -B --review > /tmp/2.sql How to implement Flashback: 1. mysqlbinlog tool can parse the binlog files and print them, so we can use the most of code in mysqlbinlog. 2. Changing the event type (INSERT -> DELETE, DELETE -> INSERT) 3. Exchanging the SET/WHERE part in UPDATE (UPDATE table SET x=1 WHERE y=2 -> UPDATE table SET y=2 WHERE x=1) 4. Invert the event output, from end to beginning (binlog: e1,e2,e3... -> flashback_log: en...e3,e2,e1) Then, importing the flashback output can flashback database/table to the special time. But binlog_format should be ROW. Thanks, Lixun -- Senior MySQL Developer @ Taobao.com Mobile Phone: +86 18658156856 (Hangzhou) Gtalk: penglixun(at)gmail.com Twitter: http://www.twitter.com/plinux Blog: http://www.penglixun.com

2 3

[Maria-developers] MariaDB config files order
by Honza Horak 20 Aug '13

20 Aug '13

Hi guys, maybe I just need some little kick or I should get a coffee, but I can't understand the following config file ordering difference. Reading [1] and code in mariadb-5.5.32/mysys/default.c:1226, it looks like expected order of config files in Linux should be: /etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf However, when asking for that using --help --verbose, I get a bit different output: $ /usr/libexec/mysqld --help --verbose 2>/dev/null | grep --after=1 '^Default options' | tail -n 2 Default options are read from the following files in the given order: /etc/mysql/my.cnf /etc/my.cnf ~/.my.cnf I don't understand why /etc/my.cnf is not on the first place in that output. Can somebody give me a hint? Am I just blind? Just FTR I'm trying that on Fedora 19, but there are no changes in Fedora packages that would do anything related, so I guess the same result would be in other distros as well. [1] https://kb.askmonty.org/en/mysqld-startup-options/ Cheers, Honza

2 3

Re: [Maria-developers] [Commits] Rev 3677: Fix bug MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL in file:///home/tsk/mprog/src/5.3/
by Alexander Barkov 20 Aug '13

20 Aug '13

Hi Timour, The patch looks Okey to push for me. Please find two small neat-peaking comments below though :) On 08/20/2013 04:36 PM, timour(a)askmonty.org wrote: > At file:///home/tsk/mprog/src/5.3/ > > ------------------------------------------------------------ > revno: 3677 > revision-id: timour(a)askmonty.org-20130820123610-cjtgvaybl1ip3128 > parent: igor(a)askmonty.org-20130815235920-io2h7tlypwlbunsp > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4895 > committer: timour(a)askmonty.org > branch nick: 5.3 > timestamp: Tue 2013-08-20 15:36:10 +0300 > message: > Fix bug MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > > Analysis: > The cause of the valgrind warning was an attempt to evaluate a Field that was not yet read. > The reason was that on one hand Item_func_isnotnull was marked as constant by > Item_func_isnotnull::update_used_tables, and this allowed eval_const_cond() to be called. > On the other hand Item_func_isnotnull::val_int() evaluated its argument as if it was not > constant. > > Solution: > The fix make sure that Item_func_isnotnull::val_int() doesn't evaluate its argument when > it is constant and cannot be NULL, because the result is known in this case. > > > > === modified file 'mysql-test/r/null.result' > --- a/mysql-test/r/null.result 2011-11-21 13:16:16 +0000 > +++ b/mysql-test/r/null.result 2013-08-20 12:36:10 +0000 > @@ -343,3 +343,33 @@ Field Type Null Key Default Extra > IFNULL(NULL, b) decimal(1,0) YES NULL > DROP TABLE t1, t2; > # End of 5.0 tests > +# > +# MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > +# > +CREATE TABLE t1 (dt DATETIME NOT NULL); > +INSERT INTO t1 VALUES (NOW()),(NOW()); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +dt > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +dt > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +dt > +drop table t1; Please fix "drop table" to upper case here, and in the other places. > > === modified file 'mysql-test/t/null.test' > --- a/mysql-test/t/null.test 2009-02-10 03:00:11 +0000 > +++ b/mysql-test/t/null.test 2013-08-20 12:36:10 +0000 > @@ -255,3 +255,31 @@ DESCRIBE t2; > DROP TABLE t1, t2; > > --echo # End of 5.0 tests > + > +--echo # > +--echo # MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > +--echo # > + > +CREATE TABLE t1 (dt DATETIME NOT NULL); > +INSERT INTO t1 VALUES (NOW()),(NOW()); > + > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > + > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > + > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > + > +EXPLAIN > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > + > +drop table t1; > > === modified file 'sql/item_cmpfunc.cc' > --- a/sql/item_cmpfunc.cc 2013-08-15 23:59:20 +0000 > +++ b/sql/item_cmpfunc.cc 2013-08-20 12:36:10 +0000 > @@ -4621,6 +4621,8 @@ Item *and_expressions(Item *a, Item *b, > longlong Item_func_isnull::val_int() > { > DBUG_ASSERT(fixed == 1); > + if (const_item() && !args[0]->maybe_null) > + return false; "return 0" should be better in a longlong function. > return args[0]->is_null() ? 1: 0; > } > > @@ -4628,6 +4630,8 @@ longlong Item_is_not_null_test::val_int( > { > DBUG_ASSERT(fixed == 1); > DBUG_ENTER("Item_is_not_null_test::val_int"); > + if (const_item() && !args[0]->maybe_null) > + DBUG_RETURN(1); > if (args[0]->is_null()) > { > DBUG_PRINT("info", ("null")); >

1 0

Re: [Maria-developers] [Commits] MDEV-4923 file://home/jan/mysql/mariadb-5.5-mdev4923
by Sergei Golubchik 20 Aug '13

20 Aug '13

Hi, Jan! On Aug 20, Jan Lindström wrote: > Serg, > > Could you please review fix for MDEC-4923, where these changes are > pushed lp:maria/5.5 or lp:~maria-captains/maria/5.5 ? Looks correct. This seems to be XtraDB bug. lp:maria/5.5 is an alias for lp:~maria-captains/maria/5.5 > === modified file 'storage/xtradb/os/os0file.c' > --- storage/xtradb/os/os0file.c 2013-07-29 14:03:41 +0000 > +++ storage/xtradb/os/os0file.c 2013-08-20 05:35:22 +0000 > @@ -1658,8 +1658,8 @@ > } > } > > - if (srv_use_atomic_writes && type == OS_DATA_FILE && > - os_file_set_atomic_writes(file, name)) { > + if (srv_use_atomic_writes && type == OS_DATA_FILE && > + !os_file_set_atomic_writes(file, name)) { > CloseHandle(file); > *success = FALSE; > file = INVALID_HANDLE_VALUE; > @@ -1786,7 +1786,7 @@ > #endif /* USE_FILE_LOCK */ > > if (srv_use_atomic_writes && type == OS_DATA_FILE > - && os_file_set_atomic_writes(name, file)) { > + && !os_file_set_atomic_writes(name, file)) { > > *success = FALSE; > close(file); > Regards, Sergei

1 0

[Maria-developers] [Otto Kekäläinen] Fwd: initial feedback on mariadb
by Kristian Nielsen 20 Aug '13

20 Aug '13

Hi, Otto asked for help with the copyright file for the Debian MariaDB packages. As I understand it, the .deb packages have a file /usr/share/doc/mariadb-server-5.5/copyright and there are some scripts in Debian that check the file against what is in the headers of source files and possibly other checks (error output attached below). Otto did something, but there is apparently more needed. Can some of you who understand the finer points of licensing among Oracle, Monty Program, MariaDB Services, MariaDB foundation, Percona, ... please look into this and help Otto fix up things? Debian are usually very careful with licensing, so this could easily become a blocker otherwise (I assume). - Kristian.

4 4

[Maria-developers] MDEV-4911 - KILL command with query id
by Roberto Spadim 20 Aug '13

20 Aug '13

Hi guys, i sent some patchs, could anyone check it? it have some things to do: query syntax isn't in the MDEV-4911, i did a QUERY_ID instead of a more comlpex syntax (since i never used bison before :P ) the query_id works, but i need a new error message ( "Unknown query id: %") well i didn't make the gcc happy, there's some warnings, and i don't know if it's all right i send this patch after MDEV-4916 patch, in other words the %expect value of sql_yacc.yy is wrong if you don't apply the MDEV-4916 patch the 4916 is a IF_IDLE keyword to KILL command too it only kill if there's no query running well :) thanks guys! <https://mariadb.atlassian.net/browse/MDEV-4916> -- Roberto Spadim

2 3

Re: [Maria-developers] More Secure Kill Command
by Sergei Golubchik 20 Aug '13

20 Aug '13

Hi, Roberto! On Aug 17, Roberto Spadim wrote: > Hi sergei, > What about give two informations, thread id and query id > If thread id and query id is equal, kill it There's no need to. Query id identifies the query and the connection unambiguously. Specifying also a thread id is redundant. Regards, Sergei

2 9

[Maria-developers] linking an external library
by rg 19 Aug '13

19 Aug '13

Folks, I want to link an external library for a language extension. After several hours of trying to tell cmake (^%$@*&) to pick it up for linking the library would still not be built with. CMakeLists.txt in trunk/sql is in the meantime full of pointers to the library without avail. I did the normal FIND_LIBRARY() followed by TARGET_LINK_LIBRARIES({$LIBNAME}) on various targets plus as nothing worked: SET(CMAKE_LIBRARY_PATH) on it, explicity LINK_DIRECTORIES(), SET(CMAKE_PREFIX_PATH), SET(CMAKE_LIBRARY_PATH) ... How do I link mariadb with an external library?? Thanks Rob

3 4

[Maria-developers] Audit plugin and qc_info plugin - MDEV-4682
by Roberto Spadim 18 Aug '13

18 Aug '13

Hi guys, i put some fields in qc_info plugin https://mariadb.atlassian.net/browse/MDEV-4581 the second part was add some others informations (stats): https://mariadb.atlassian.net/browse/MDEV-4682 i was reading the "response_time_distribution" of percona, patch at https://mariadb.atlassian.net/browse/MDEV-4568 could i use the audit plugin to implement MDEV-4682? what i need is: 1)know if the query was executed with/without query cache 2)know the time expend (lock time, and total time) to execute the query (first execution) 3)know the time expend with query cache hit 4) both (3) and (2) don't include time of network packages, just server side times... is it possible? any idea how could i implement it? i didn't write a audit plugin yet =( well.. time to learn =D thanks guys! -- Roberto Spadim

2 2

[Maria-developers] ALTER ONLINE TABLE syntax
by Sergei Golubchik 18 Aug '13

18 Aug '13

Hi, You've added this ALTER ONLINE TABLE syntax, where ONLINE de facto means "without copying the data". Now, after 5.6 merge we also have ALGORITHM=COPY|INPLACE and LOCK=NONE|SHARED|EXCLUSIVE. I've mapped ONLINE to ALGORITHM=INPLACE, because that's what it means. But it's confusing, one can have an "inplace" operation under an exclusive lock, if the engine wants it that way. And it won't be "online" by any stretch of imagination. One can also technically have a "copy" operation with a very permissive write-allow-write lock, and it'll be as online as one can get it. I mean, logically ONLINE should mean LOCK=NONE, not ALGORITHM=INPLACE. But that would change the existing behavior and the test results too. For example, *any* ALTER on the temporary table would always be "online", because, well, it wouldn't take any lock on that table. Alternatively, we can keep ONLINE as ALGORITHM=INPLACE, and deprecate it, issue a warning, suggesting to use the new 5.6 syntax. Opinions? Regards, Sergei

5 7

[Maria-developers] MDEV-4916
by Roberto Spadim 18 Aug '13

18 Aug '13

Patch uploaded, and test work (but i didn't add a unit test yet :/ sorry i don't know yet how to write it) -- Roberto Spadim

1 0

[Maria-developers] Joining the team
by Esteban Orduno 17 Aug '13

17 Aug '13

Hi, all: I would like to contribute to the maria-db team. Who can I talk iniatially to get some type of orientation about it? Esteban Orduno

2 1

[Maria-developers] More Secure Kill Command
by Roberto Spadim 17 Aug '13

17 Aug '13

Hi guys i was mailing at maria-discuss, but i think it a new feature, and want some view point from developers... check this problem: connection 1) big select... connection 2) show processlist connection 1) other query connection 2) kill connection 1 id (i take +- 1 second between show processlist and this command) connection 1) killed the problem? the kill connection 1 (KILL thread_id) killed the wrong query could we implement something like: KILL [CONNECTION | QUERY] thread_id [QUERY "some relative unique query id in this database"] the relative unique query id, could be: substr(sha1 or md5 of (thread_id + query start time),0,6) (6 hex numbers is good?) it's like a git small hash commit id, maybe we could make it more unique... but since i will not use it very often, i think a less intensive work is nice here... a global query id could add more one lock for every query... example: KILL 1 QUERY "abcdef" the "abcdef" = substr(MD5( thread id + query start time ),0,6) we could add client ip or others unique information about this query, but something that don't increase (ok just a little) the load of a show processlist / select * from information_schema thanks -- Roberto Spadim

4 7

[Maria-developers] MariaDB 10.0.4 Release Prep
by Daniel Bartholomew 16 Aug '13

16 Aug '13

All, I've begun prep work for the MariaDB 10.0.4 release. Draft Release Notes and Changelogs are here: - https://kb.askmonty.org/en/mariadb-1004-release-notes/ - https://kb.askmonty.org/en/mariadb-1004-changelog/ Please let me know ASAP if anything needs to be added/changed/etc... Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

3 4

[Maria-developers] RFC - query cache insert
by Roberto Spadim 16 Aug '13

16 Aug '13

Hi guys! I was reading (again) the query cache source code of 10.0.3 ... please tell me if i understood right when we insert at query cache... thread put a lock in query cache (no one can result from query cache) thread read all data and write to network (caching the result to query cache) after end of thread real the query cache is unlocked.. my doubts are... what happen if the query is too slow? the query cache is disabled while executing the query? what happen if two equal queries (same SELECT) execute? the first one is locked until the second one end the query and get the result from query cache? or the first lock query cache and the second runs without query cache? the last one there's any query cache changes in mind? i was reading the sql_cache.cc of old mysql servers and there's a todo list (very old i think) that didn't changed from old mysql to today mariadb 10.0.3, this todo list is inside mysql bug tracker? or mariadb jira? thanks guys! -- Roberto Spadim

2 11

[Maria-developers] doubt about workaround on MDEV-4903
by Roberto Spadim 16 Aug '13

16 Aug '13

hi guys, could anyone help me with a workaround to this MDEV? https://mariadb.atlassian.net/browse/MDEV-4903 basically the problem is a (more than one column primary key) IN ( (a primary key const value), (a primary key const value), (a primary key const value), (a primary key const value) ) it runs a table scan instead of using the primary key index using OR = OR = OR =, use the index but i want a easier way to write this query thanks -- Roberto Spadim

1 0

[Maria-developers] RFC - query cache using storage engine
by Roberto Spadim 15 Aug '13

15 Aug '13

Hi guys, i was thinking about ideas for query cache... there's an mdev about partitioning query cache ( https://mariadb.atlassian.net/browse/MDEV-4454) instead of this, why not use a HEAP storage engine (with partitioning), or maybe a myisam or aria or innodb as "backend", and write a "frontend" (query cache) that execute commands over this storage engine? i'm talking about, changing the query/table/result blocks of query cache, to query/table/result tables when start mysqld truncate these tables when add new query "insert" to these tables when a table prune occur, "delete" from these tables when flush, optimize these tables when reset query cache, truncate this tables these tables could be at mysql schema, if they don't exists create as ENGINE=MEMORY for example since we must have a MAX SIZE for query cache, we could add a MAX SIZE for HEAP engine and a LOW MEMORY algorithm ( https://mariadb.atlassian.net/browse/MDEV-4675) at high level, what you think about this idea? -- Roberto Spadim

2 3

Re: [Maria-developers] Fwd: [Commits] Rev 3674: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-base-MDEV-4645/
by Kristian Nielsen 15 Aug '13

15 Aug '13

Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Could you review this? > > (it is https://mariadb.atlassian.net/browse/MDEV-4645 ) Ok, but it will be at least a couple more days before I will have time to start. If urgent, you can also ask Sergei Golubchik, IIRC this is the patch from Jeremy Cole that he asked me about, and I already looked at once and accepted the general idea? - Kristian.

2 1

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 15 Aug '13

15 Aug '13

Vilho Raatikka <vilho.raatikka(a)skysql.com> writes: > Is this documented profoundly already? Should be because probably we'll see > at least some users as ignorant as I am asking the same questions soon. I wrote up some comprehensive documentation in the knowledgebase, and also wrote a series of articles about in on my blog: https://kb.askmonty.org/en/global-transaction-id/ http://kristiannielsen.livejournal.com/ But this is complex stuff, I am sure it can be improved. Any help improving the documentation would be greatly appreciated. I fear that I have spent too much time on the details of the design to clearly see how to explain the high-level picture in a way suitable for new users. - Kristian.

1 0

[Maria-developers] Value returned by my_getcputime
by Weldon Whipple 14 Aug '13

14 Aug '13

Hello! (This is my first post to this list; long-time follower, however.) I've been looking at the block comment at the top of the my_getcputime() function at the bottom of the file mysys/my_getsystime.c The block comment reads: Return cpu time in milliseconds * 10 I interpret milliseconds * 10 to mean centiseconds (seconds * 10 ^ -2) I'm pretty sure I've convinced myself that it REALLY returns: seconds * 10 ^ -7 I think it would be more accurate to have the block comment read: Return cpu time in microseconds / 10 Am I "up in the night"? (I think the code itself is fine, and the callers of the function all do it correctly. I was just initially confused when I looked at the comment then the code.) Thanks in advance! Weldon Whipple weldon(a)whipple.org (P.S. I think I like Maria's implementation of userstat_v2 better than the original by Google/Percona ...)

2 1

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 14 Aug '13

14 Aug '13

>> SHOW VARIABLES or SELECT @@GLOBAL.gtid_strict_mode will show the current >> value. But there is no way to check if it was enabled eg. two weeks ago, if >> that is what you are asking (the value is not stored in the binlog). > That is really pity. Is it really so useful to give the user opportunity to > switch gtid mode on-the-fly? That option sounds really expensive. Isn't > binlog event ordering lost - or it can't be trusted anymore? Binlog event order is never lost, strict mode or no strict mode. That is the whole point: even if the user messes up, MariaDB GTID is still able to correctly find the right position in the binlog to start at. There is nothing expensive about changing gtid strict mode, nor any loss of trust. Gtid strict mode is something to help the _user_. The GTID _code_ does not care, it works correctly whether strict mode is enabled or not. The only thing strict mode does is give an error for some uses that are considered undesirable by people like you and me. All non-error cases work exactly the same whether strict mode is enabled or not. It seems the more I try to explain, the more I confuse the issue :-( Maybe it is better if I just go fix the actual bugs ... - Kristian.

1 0

Re: [Maria-developers] Rev 3804: MDEV-4712 : Fix "shutdown" test
by Sergei Golubchik 13 Aug '13

13 Aug '13

Hi, Vladislav! On Jul 31, Vladislav Vaintroub wrote: > ------------------------------------------------------------ > revno: 3804 > revision-id: wlad(a)montyprogram.com-20130731130414-ukqxbb0kxpexjnzp > parent: svoj(a)mariadb.org-20130731110211-09p66bwf4jrmi03i > committer: Vladislav Vaintroub <wlad(a)montyprogram.com> > branch nick: 10.0-serg > timestamp: Wed 2013-07-31 15:04:14 +0200 > message: > MDEV-4712 : Fix "shutdown" test. > > There is a race condition during shutdown, that happens when "show status" runs during shutdown. > Functions show_slave_running(), show_slave_received_heartbeats(), show_heartbeat_period() would dereference NULL pointer master_info_index, after this variable is set to NULL in slave_end() during shutdown. > Fix introduces check for master_info_index == NULL in the affected functions. Was it related to the new shutdown command? Or it could be also observed without it by, say, running show status in a loop, while stopping the server with mysqladmin? Regards, Sergei

2 1

[Maria-developers] Lack of linkage from MariaDB JIRA to commits
by Jeremy Cole 13 Aug '13

13 Aug '13

All, I have noticed that the MariaDB JIRA tickets are rarely linked to any specific commit which purportedly fixes them. Is there any standard at the moment for what should be done (and then it isn't followed) or is there no standard (and perhaps it should be made)? I would really like a consistent way to get to view the commits into various branches related to each MDEV issue, in a consistent way. Regards, Jeremy

5 9

[Maria-developers] 10.x compatibility and separate namespace
by Honza Horak 13 Aug '13

13 Aug '13

Hi Guys, since 10.x won't be compatible with mysql in every aspect any more, I'm wondering if there happen to be any plans to move all compatibility left-overs (client binary names, client library name, daemon binary name, etc.). Are there any such ideas? I understand that we would lose the compatibility for any later (keeping original names was mentioned in [1] as a MUST for compatibility), but when keeping the original names, it could become tricky to say which tools are compatible and which not any more in mariadb 10+. How this information will be provided actually? How to get users informed properly, which tools are not compatible any more? And a bit another POV -- is the incompatibility in 10.x strictly limited to the server only (so client side will stay compatible for anytime later)? [1] https://lists.launchpad.net/maria-discuss/msg00658.html Regards, Honza

3 2

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 13 Aug '13

13 Aug '13

Vilho Raatikka <vilho.raatikka(a)skysql.com> writes: > What exactly does it require for the user to enable strict or 'relaxed' > (=disasterous imho) mode? Technically, one needs to SET GLOBAL gtid_strict_mode=1 with root privileges (config file option works as well of course). That's it. The practical issue is that old-style MariaDB/MySQL replication allows a number of "disasterous" uses. One example is a multi-master ring A->B->C->A, writing to all masters simultaneously. In such setup, it is fundamentally impossible to automatically eg. move a slave S from A->S to B->S - there is just not enough information about what events have been applied to S from A, B, and C. GTID introduces a way to configure this correctly (replication domains), but if we were to give an error by default then we would break upgrades from 5.5. So instead we provide gtid strict mode, so that once users understand what the "disasterous" uses are and how to avoid them, they can enable strict mode and get saner semantics. BTW, for the normal non-advanced user, I think silly stuff like running random manual transactions on slaves that get into the binlog is a far more common mistake. Gtid strict mode will cause errors on this, so it needs to be explictly enabled by users to not break/confuse upgrades. > Is it possible at any point in server's life cycle to check which mode it > is running? SHOW VARIABLES or SELECT @@GLOBAL.gtid_strict_mode will show the current value. But there is no way to check if it was enabled eg. two weeks ago, if that is what you are asking (the value is not stored in the binlog). > Can that setting be modified during runtime or only prior startup? It can be changed at any time, though I think you need to STOP SLAVE/START SLAVE to get the change to take effect on a slave. - Kristian.

1 1

[Maria-developers] RFC: Compile-time checking for unmatched DBUG_ENTER/LEAVE via unused variable
by Jeremy Cole 11 Aug '13

11 Aug '13

Maria developers/captains, Prior to posing it as a final patch contribution or opening a JIRA ticket, I wanted to propose this idea. The attached patch adds what amounts to compile-time checking for unmatched DBUG_ENTER/DBUG_LEAVE (DBUG_RETURN, DBUG_VOID_RETURN) by introducing a variable in DBUG_ENTER which is only used in DBUG_LEAVE. This allows any compiler which can robustly detect unused variables to detect the mismatch at compile-time. There is already a run-time check for this case, but it is somewhat limited as it requires _db_return_ to be called in order to detect the mismatch, and this is in practice not always the case. Particularly three cases allow this to escape detection: 1. Some instrumented functions are called only from non-instrumented functions. 2. Some instrumented functions are called only from non-returning functions, such as abort or die functions. 3. Some instrumented functions are called from functions who call my_end or my_thread_end (making cs unavailable) before returning. In any case please see the attached DRAFT patch for a full implementation of the idea. Note one bit of ugliness is that non-returning functions, after this patch, must be marked explicitly as DBUG_ENTER_NO_RETURN -- this actually is not really a bad thing, though in and of itself. Regards, Jeremy

4 4

Re: [Maria-developers] Rev 3671: MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'), TIME('00:00:00')))
by Sergei Golubchik 11 Aug '13

11 Aug '13

Hi, Alexander! On Aug 07, Alexander Barkov wrote: > At lp:~maria-captains/maria/5.3 > > ------------------------------------------------------------ > revno: 3671 > revision-id: bar(a)mariadb.org-20130807093420-khdah292yerbor8y > parent: sergii(a)pisem.net-20130715163225-6ch6x34lsufode3d > committer: Alexander Barkov <bar(a)mariadb.org> > branch nick: 5.3.b4652 > timestamp: Wed 2013-08-07 13:34:20 +0400 > message: > MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'),TIME('00:00:00'))) Ok to push. > === modified file 'sql/item.cc' > --- a/sql/item.cc 2013-07-03 07:46:20 +0000 > +++ b/sql/item.cc 2013-08-07 09:34:20 +0000 > @@ -252,7 +252,8 @@ String *Item::val_string_from_decimal(St > String *Item::val_string_from_date(String *str) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, 0) || > + if (get_date(&ltime, > + field_type() == MYSQL_TYPE_TIME ? TIME_TIME_ONLY : 0) || > str->alloc(MAX_DATE_STRING_REP_LENGTH)) > { > null_value= 1; > > === modified file 'sql/item_func.cc' > --- a/sql/item_func.cc 2013-07-03 07:46:20 +0000 > +++ b/sql/item_func.cc 2013-08-07 09:34:20 +0000 > @@ -2480,6 +2480,13 @@ bool Item_func_min_max::get_date(MYSQL_T > ltime->time_type= MYSQL_TIMESTAMP_DATE; > ltime->hour= ltime->minute= ltime->second= ltime->second_part= 0; > } > + else if (compare_as_dates->field_type() == MYSQL_TYPE_TIME) > + { > + ltime->time_type= MYSQL_TIMESTAMP_TIME; > + ltime->hour+= (ltime->month * 32 + ltime->day) * 24; > + ltime->month= ltime->day= 0; > + } > + > > return 0; > } Regards, Sergei

1 0

Re: [Maria-developers] [Commits] Rev 3552: Fix bug MDEV-4648 SQ pushdown: Wrong result (missing rows) with materialization+semijoin, IN and ALL subqueries, UNION in file:///home/tsk/mprog/src/10.0-md83/
by Oleksandr Byelkin 09 Aug '13

09 Aug '13

09.08.2013 14:30, timour(a)askmonty.org пишет: > At file:///home/tsk/mprog/src/10.0-md83/ > > ------------------------------------------------------------ > revno: 3552 > revision-id: timour(a)askmonty.org-20130809113013-ia2f7zzw3mv3in7u > parent: timour(a)askmonty.org-20130611081135-rge7p00h8e9nhypj > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4648 > committer: timour(a)askmonty.org > branch nick: 10.0-md83 > timestamp: Fri 2013-08-09 14:30:13 +0300 > message: > Fix bug MDEV-4648 SQ pushdown: Wrong result (missing rows) with materialization+semijoin, IN and ALL subqueries, UNION > (part of MDEV-83) > > Analysis: > > The cause of the wrong result was that the ALL subquery was incorrectly > attached to the last table in the join plan <t1, sj-mat, t2>. The ALL > subquery was transformed into a MIN expression, however it still depended > on OUTER_REF_TABLE_BIT as if it was a subquery. The reason for this > dependence was that the transformed MIN expression referred to the > Item_cache object of the left subquery operand created for the initial > Item_in_optimizer. > > Solution: > > When transforming an ALL subquery into a MIN/MAX expression, use the > actual left argument expression, instead of the Item_cache created > for the subquery. > OK to push. Thank you!

1 0

[Maria-developers] Is it a bug?
by Alexey Botchkov 09 Aug '13

09 Aug '13

Hi, SerG. The code in sql/sql_plugin.cc function read_maria_plugin_info() sym= dlsym(plugin_dl->handle, maria_plugin_declarations_sym); ... if (plugin_dl->mariaversion != MARIA_PLUGIN_INTERFACE_VERSION) { ... if ((sym= dlsym(plugin_dl->handle, maria_sizeof_st_plugin_sym))) sizeof_st_plugin= *(int *)sym; ... if (sizeof_st_plugin != sizeof(st_mysql_plugin)) { ... sym= cur; } } plugin_dl->plugins= (struct st_maria_plugin *)sym; So that when we enter the first 'if' but not the second (sizeof_st_plugin == sizeof(st_mysql_plugin) the sym still points at the maria_sizeof_st_plugin_sym which makes a bad st_maria_plugin pointer. I think it's a bug, isn't it? Best regards. HF

2 1

Re: [Maria-developers] GTID and removing binary logs
by Kristian Nielsen 08 Aug '13

08 Aug '13

Elena Stepanova <elenst(a)montyprogram.com> writes: > Reading recent Pavel Ivanov's reports (the last one is MDEV-4820, but > I think there were some others which you've already closed as fixed), > I started wondering if I conceptually misunderstood something > before. From our previous discussions I made a conclusion that > removing *all* binary logs before trying to replicate from the server > is very wrong and against the whole idea of GTID. Is it not so? If > not, it's definitely an area which didn't undergo any testing from me, > except for my very first clumsy attempts to get familiar with GTID, so > I need to fill the gap. Well, we _do_ want to support running a slave with --log-slave-updates=0, and then making that slave into a master. Indeed, you tested that early and it was horribly broken. However, my intention is that this should now be supported (and thus can be tested). There are MTR testcases for this. The reason I mention this is that the case of --log-slave-updates=0 is actually quite similar (from the server's point of view) to manually deleting the logs. So I do also intend to support removing binlogs as Pavel wants to do. The main problem with doing this (deleting binlogs or using --log-slave-updates=0) is that it makes it harder to detect certain user errors, because the information to know that it is an error is missing. I suppose the limitations should be pretty obvious: If user deletes binlogs on a server, then that server, acting as a master, can not replicate to slaves starting from an event that was deleted. So as long as the user does not try to do that, things should be fine. I think the main problems come if the user makes a mistake in this respect, and the server fails to detect the error. For example, suppose that server A is a slave, replicates up to position 0-2-10. Then it becomes a master, generates events up to 0-1-20. Then user deletes the binlogs on A, and tries to have server B replicate from A starting at position 0-2-10. In this case, server A will allow B to connect, effectively loosing the deleted events 0-1-11 .. 0-1-20. I do not see any way to avoid this. Server A has no way to know that those events ever existed, as the user deliberately deleted all information about them. (In fact the user might be doing this deliberately, in case the events were wrongly created and should be removed from the history). A workaround in this example is that the user sets @@gtid_slave_pos explicitly to 0-1-20 after deleting the logs, thus preserving the needed information. So in summary: Yes, we try to support deleting binlogs, however there is some responsibility on the part of the user to not delete information that they actually need. I think that is rather reasonable. It is how DELETE and SELECT work, as well :-) - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 3772: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4548/
by Jeremy Cole 08 Aug '13

08 Aug '13

Sanja, Re: https://mariadb.atlassian.net/browse/MDEV-4645 Re: http://lists.askmonty.org/pipermail/commits/2013-July/005093.html I saw your commit on the commits list (sorry, wasn't on the list before the mail went out, so I can't reply directly to that message). A couple of points related to the commit: - You seem to have an error in the get_header_len, as it's implemented only in the base class, and you've left functions with the correct signature for it but the wrong names in the two frozen classes (they were left named has_fixed_minimal_header). This would mean the change is completely broken. :) - The change to get_header_len I think is in general fine, although it does alter the point of the whole thing -- there is again no way to exactly identify which events have frozen headers, programmatically (i.e. you can't differentiate them from events who just "happen" to have header_len = LOG_EVENT_MINIMAL_HEADER_LEN). I think that doesn't matter for this patch, but does defeat the point of what I was trying to add. - I don't really understand why you've made the hex printing code so complex and completely unreadable (IMHO). This is not performance-sensitive code and in my opinion should favor readability over performance. All you seem to have saved in this change is to avoid printing some static formatting characters ("#", "|", spaces, etc.) inside the loop. However, I can hardly even figure out what it's supposed to be doing now. - Additionally, you've changed the output slightly -- normally the ASCII character section prints its ending "|" immediately after the last character; it seems now to be fixed position (leaving extra spaces before "|". This differs from the output of hexdump, and it somewhat ambiguous/confusing. - You've mixed tabs and spaces. :( Regards, Jeremy

2 8

[Maria-developers] RFC - temporary path and SSD/HDD
by Roberto Spadim 07 Aug '13

07 Aug '13

Hi guys I have some server running with bcache and some in test running flash cache and others linux ssd cache solution I was thinking about how to optimize the temporary tables of mysql/mariadb... could we implement something like: Temporary Slow/Big Path = /tmp/hdd Temporary Fast/Small Path = /tmp/ssd Temporary Fast Max Size = 10MB When we start using temporary tables (file sort for example) instead of write to a hdd temporary disk, use a ssd temporary disk, if table get bigger than 10MB, copy the file to hdd drive, and remove from ssd drive, and continue writing to hdd I think with this i could optimize the temporary table use Any other idea about this kind of optimization? -- Roberto Spadim

1 0

Re: [Maria-developers] [Commits] Rev 3671: MDEV-4811 Assertion `offset < 0x1f' fails in type_and_offset_store on COLUMN_ADD in file:///home/bell/maria/bzr/work-maria-5.3-MDEV-4811/
by Sergei Petrunia 07 Aug '13

07 Aug '13

Hi Sanja, Ok to push. On Thu, Aug 01, 2013 at 11:46:14AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.3-MDEV-4811/ > > ------------------------------------------------------------ > revno: 3671 > revision-id: sanja(a)montyprogram.com-20130801084611-cu3wpoazd197g41w > parent: sergii(a)pisem.net-20130715163225-6ch6x34lsufode3d > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.3-MDEV-4811 > timestamp: Thu 2013-08-01 11:46:11 +0300 > message: > MDEV-4811 Assertion `offset < 0x1f' fails in type_and_offset_store on COLUMN_ADD > MDEV-4812 Valgrind warnings (Invalid write) in dynamic_column_update_many on COLUMN_ADD > > Fixed problem of working on wrong data (do not allow offset to out of string length). > === modified file 'mysql-test/r/dyncol.result' > --- a/mysql-test/r/dyncol.result 2013-03-26 11:07:46 +0000 > +++ b/mysql-test/r/dyncol.result 2013-08-01 08:46:11 +0000 > @@ -1404,5 +1404,26 @@ v1 CREATE ALGORITHM=UNDEFINED DEFINER=`r > drop view v1; > drop table t1; > # > +# MDEV-4811: Assertion `offset < 0x1f' fails in type_and_offset_store > +# on COLUMN_ADD > +# > +CREATE TABLE t1 (dyn TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyn = COLUMN_CREATE( 40, REPEAT('a', 233), 4, REPEAT('b', 322) ); > +Warnings: > +Warning 1265 Data truncated for column 'dyn' at row 1 > +SELECT COLUMN_ADD( dyn, 6, REPEAT('x',80), 4, REPEAT('y',215) AS INTEGER ) FROM t1; > +ERROR HY000: Encountered illegal format of dynamic column string > +DROP table t1; > +# > +# MDEV-4812: Valgrind warnings (Invalid write) in > +# dynamic_column_update_many on COLUMN_ADD > +# > +CREATE TABLE t1 (dyncol TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyncol = COLUMN_CREATE( 7, REPEAT('k',487), 209, REPEAT('x',464) ); > +Warnings: > +Warning 1265 Data truncated for column 'dyncol' at row 1 > +SELECT COLUMN_ADD( dyncol, 7, '22:22:22', 8, REPEAT('x',270) AS CHAR ) FROM t1; > +DROP table t1; > +# > # end of 5.3 tests > # > > === modified file 'mysql-test/t/dyncol.test' > --- a/mysql-test/t/dyncol.test 2013-03-26 11:07:46 +0000 > +++ b/mysql-test/t/dyncol.test 2013-08-01 08:46:11 +0000 > @@ -600,6 +600,29 @@ drop view v1; > drop table t1; > > --echo # > +--echo # MDEV-4811: Assertion `offset < 0x1f' fails in type_and_offset_store > +--echo # on COLUMN_ADD > +--echo # > + > +CREATE TABLE t1 (dyn TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyn = COLUMN_CREATE( 40, REPEAT('a', 233), 4, REPEAT('b', 322) ); > +--error ER_DYN_COL_WRONG_FORMAT > +SELECT COLUMN_ADD( dyn, 6, REPEAT('x',80), 4, REPEAT('y',215) AS INTEGER ) FROM t1; > + > +DROP table t1; > + > +--echo # > +--echo # MDEV-4812: Valgrind warnings (Invalid write) in > +--echo # dynamic_column_update_many on COLUMN_ADD > +--echo # > +CREATE TABLE t1 (dyncol TINYBLOB) ENGINE=MyISAM; > + > +INSERT INTO t1 SET dyncol = COLUMN_CREATE( 7, REPEAT('k',487), 209, REPEAT('x',464) ); > +--error 0,ER_DYN_COL_WRONG_FORMAT > +SELECT COLUMN_ADD( dyncol, 7, '22:22:22', 8, REPEAT('x',270) AS CHAR ) FROM t1; > +DROP table t1; > + > +--echo # > --echo # end of 5.3 tests > --echo # > > > === modified file 'mysys/ma_dyncol.c' > --- a/mysys/ma_dyncol.c 2011-10-18 10:44:12 +0000 > +++ b/mysys/ma_dyncol.c 2013-08-01 08:46:11 +0000 > @@ -1228,13 +1228,14 @@ dynamic_column_create(DYNAMIC_COLUMN *st > @param header_end Pointer to the header end > @param offset_size Size of offset field in bytes > @param last_offset Size of the data segment > + @param error Set in case of error > > @return number of bytes > */ > > static size_t get_length_interval(uchar *entry, uchar *entry_next, > uchar *header_end, size_t offset_size, > - size_t last_offset) > + size_t last_offset, my_bool *error) > { > size_t offset, offset_next; > DYNAMIC_COLUMN_TYPE type, type_next; > @@ -1242,8 +1243,12 @@ static size_t get_length_interval(uchar > > type_and_offset_read(&type, &offset, entry, offset_size); > if (entry_next >= header_end) > + { > + *error= 0; > return (last_offset - offset); > + } > type_and_offset_read(&type_next, &offset_next, entry_next, offset_size); > + *error= (offset_next > last_offset); > return (offset_next - offset); > } > > @@ -1255,17 +1260,18 @@ static size_t get_length_interval(uchar > @param header_end Pointer to the header end > @param offset_size Size of offset field in bytes > @param last_offset Size of the data segment > + @param error Set in case of error > > @return number of bytes > */ > > static size_t get_length(uchar *entry, uchar *header_end, > size_t offset_size, > - size_t last_offset) > + size_t last_offset, my_bool *error) > { > return get_length_interval(entry, > entry + offset_size + COLUMN_NUMBER_SIZE, > - header_end, offset_size, last_offset); > + header_end, offset_size, last_offset, error); > } > > > @@ -1304,6 +1310,7 @@ find_column(DYNAMIC_COLUMN_TYPE *type, u > uchar *entry; > size_t offset, total_data, header_size, entry_size; > uchar key[2+4]; > + my_bool error; > > if (!entry_pos) > entry_pos= &entry; > @@ -1329,12 +1336,12 @@ find_column(DYNAMIC_COLUMN_TYPE *type, u > return 1; > *data= header + header_size + offset; > *length= get_length(entry, header + header_size, offset_size, > - total_data); > + total_data, &error); > /* > Check that the found data is withing the ranges. This can happen if > we get data with wrong offsets. > */ > - if ((long) *length < 0 || offset + *length > total_data) > + if (error || (long) *length < 0 || offset + *length > total_data) > return 1; > > *entry_pos= entry; > @@ -1837,12 +1844,13 @@ dynamic_column_update_many(DYNAMIC_COLUM > entry_size, column_count, &entry)) > { > size_t entry_data_size; > + my_bool error; > > /* Data existed; We have to replace or delete it */ > > entry_data_size= get_length(entry, header_end, > - offset_size, max_offset); > - if ((long) entry_data_size < 0) > + offset_size, max_offset, &error); > + if (error || (long) entry_data_size < 0) > { > rc= ER_DYNCOL_FORMAT; > goto end; > @@ -2038,12 +2046,13 @@ dynamic_column_update_many(DYNAMIC_COLUM > /* copy first the data that was not replaced in original packed data */ > if (start < end) > { > + my_bool error; > /* Add old data last in 'tmp' */ > size_t data_size= > get_length_interval(header_base + start * entry_size, > header_base + end * entry_size, > - header_end, offset_size, max_offset); > - if ((long) data_size < 0 || > + header_end, offset_size, max_offset, &error); > + if (error || (long) data_size < 0 || > data_size > max_offset - first_offset) > { > dynamic_column_column_free(&tmp); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/
by Kristian Nielsen 07 Aug '13

07 Aug '13

Sergey Petrunya <psergey(a)askmonty.org> writes: > revision-id: psergey(a)askmonty.org-20130802141209-4dqfvx2db8acxwbl > message: > MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg > Temorary fix for a number of replication tests (rpl.rpl_temp_table_mix_row > rpl.rpl_trunc_temp rpl.rpl_current_user rpl.rpl_gtid_master_promote): The patch is not ok to push. We should not introduce a special flag just for the table gtid_slave_pos. Instead of setting table->no_replicate in the first place in rpl_slave_state::truncate_state_table() and rpl_slave_state::record_gtid(), we can clear the OPTION_BIN_LOG option. I checked MySQL 5.6 code and this is what they do for their similar "crash-safe replication" tables. So remove the two statements "table->no_replicate= 1" in rpl_gtid.cc and replace with clear of OPTION_BIN_LOG in thd->variables.option_bits. Then table->no_replicate is not touched and the new logic in THD::decide_logging_format() does not trigger, so the problem should be solved. In rpl_slave_state::truncate_state_table() we can use tmp_disable_binlog() and reenable_binlog() I think. In record_gtid() it is probably better to combine the clearing of OPTION_BIN_LOG with the existing save/restore of variables.option_bits. > The question of what should be done when a user issues a statement that > explicitly modifies mysql.gtid_slave_pos table remains open. I do not think we need to do anything special for this case. User explicitly modifying the table should be treated no different than any other table. Hope this helps, - Kristian. > === modified file 'sql/rpl_gtid.cc' > --- a/sql/rpl_gtid.cc 2013-06-21 09:53:46 +0000 > +++ b/sql/rpl_gtid.cc 2013-08-02 14:12:09 +0000 > @@ -210,6 +210,7 @@ rpl_slave_state::truncate_state_table(TH > { > table= tlist.table; > table->no_replicate= 1; > + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE > err= table->file->ha_truncate(); > > if (err) > @@ -346,6 +347,7 @@ rpl_slave_state::record_gtid(THD *thd, c > goto end; > > table->no_replicate= 1; > + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE > if (!in_transaction) > thd->variables.option_bits&= > ~(ulonglong)(OPTION_NOT_AUTOCOMMIT|OPTION_BEGIN); > > === modified file 'sql/sql_class.cc' > --- a/sql/sql_class.cc 2013-07-24 10:43:57 +0000 > +++ b/sql/sql_class.cc 2013-08-02 14:12:09 +0000 > @@ -5001,7 +5001,7 @@ int THD::decide_logging_format(TABLE_LIS > DBUG_PRINT("info", ("table: %s; ha_table_flags: 0x%llx", > table->table_name, flags)); > > - if (table->table->no_replicate) > + if (table->table->no_replicate && !table->table->s->is_gtid_slave_pos) > { > /* > The statement uses a table that is not replicated. > > === modified file 'sql/table.cc' > --- a/sql/table.cc 2013-07-21 14:43:42 +0000 > +++ b/sql/table.cc 2013-08-02 14:12:09 +0000 > @@ -315,6 +315,8 @@ TABLE_SHARE *alloc_table_share(const cha > strmov(share->path.str, path); > share->normalized_path.str= share->path.str; > share->normalized_path.length= path_length; > + /* TEMPORARY FIX: if true, this means this is mysql.gtid_slave_pos table */ > + share->is_gtid_slave_pos= FALSE; > share->table_category= get_table_category(& share->db, & share->table_name); > share->set_refresh_version(); > share->open_errno= ENOENT; > > === modified file 'sql/table.h' > --- a/sql/table.h 2013-07-21 14:39:19 +0000 > +++ b/sql/table.h 2013-08-02 14:12:09 +0000 > @@ -644,6 +644,8 @@ struct TABLE_SHARE > LEX_STRING normalized_path; /* unpack_filename(path) */ > LEX_STRING connect_string; > > + bool is_gtid_slave_pos; > + > /* > Set of keys in use, implemented as a Bitmap. > Excludes keys disabled by ALTER TABLE ... DISABLE KEYS. > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] COT() function
by Ian Gilfillan 03 Aug '13

03 Aug '13

Hi SELECT COT(0) returns: ERROR 1690 (22003): DOUBLE value is out of range in 'cot(0)' in both MariaDB 10 and 5.5, but in MySQL 5.1 it returns NULL. I can't find a changelog reference to this, but I see it's still documented as returning NULL in MySQL 5.5 (I don't have this to test on). Is this a MariaDB-specific change? When did this change in behaviour happen? thanks, ian

3 5

[Maria-developers] Please review: [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/
by Sergei Petrunia 02 Aug '13

02 Aug '13

Hi Kristian, In order to get rid of test failures of 10.0-serg, I have pushed the below patch. It seems to have the right effect, but I have no confidence about this patch being achitecturally correct, etc. Could you please review this and/or make another fix for the problem? ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: commits(a)mariadb.org X-Mailer: mail (GNU Mailutils 2.2) Date: Fri, 2 Aug 2013 18:12:11 +0400 (MSK) Subject: [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/ At file:///data0/psergey/dev2/10.0-serg/ ------------------------------------------------------------ revno: 3797 revision-id: psergey(a)askmonty.org-20130802141209-4dqfvx2db8acxwbl parent: svoj(a)mariadb.org-20130729140849-dciz5ffq25ciip6q committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: 10.0-serg timestamp: Fri 2013-08-02 18:12:09 +0400 message: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg Temorary fix for a number of replication tests (rpl.rpl_temp_table_mix_row rpl.rpl_trunc_temp rpl.rpl_current_user rpl.rpl_gtid_master_promote): - THD::decide_logging_format() should not assume that mysql.gtid_slave_pos is a non-replicated table. This used to cause unintended behavior for COMMIT statement: replication would switch to row-based, etc. The question of what should be done when a user issues a statement that explicitly modifies mysql.gtid_slave_pos table remains open. === modified file 'sql/rpl_gtid.cc' --- a/sql/rpl_gtid.cc 2013-06-21 09:53:46 +0000 +++ b/sql/rpl_gtid.cc 2013-08-02 14:12:09 +0000 @@ -210,6 +210,7 @@ rpl_slave_state::truncate_state_table(TH { table= tlist.table; table->no_replicate= 1; + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE err= table->file->ha_truncate(); if (err) @@ -346,6 +347,7 @@ rpl_slave_state::record_gtid(THD *thd, c goto end; table->no_replicate= 1; + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE if (!in_transaction) thd->variables.option_bits&= ~(ulonglong)(OPTION_NOT_AUTOCOMMIT|OPTION_BEGIN); === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2013-07-24 10:43:57 +0000 +++ b/sql/sql_class.cc 2013-08-02 14:12:09 +0000 @@ -5001,7 +5001,7 @@ int THD::decide_logging_format(TABLE_LIS DBUG_PRINT("info", ("table: %s; ha_table_flags: 0x%llx", table->table_name, flags)); - if (table->table->no_replicate) + if (table->table->no_replicate && !table->table->s->is_gtid_slave_pos) { /* The statement uses a table that is not replicated. === modified file 'sql/table.cc' --- a/sql/table.cc 2013-07-21 14:43:42 +0000 +++ b/sql/table.cc 2013-08-02 14:12:09 +0000 @@ -315,6 +315,8 @@ TABLE_SHARE *alloc_table_share(const cha strmov(share->path.str, path); share->normalized_path.str= share->path.str; share->normalized_path.length= path_length; + /* TEMPORARY FIX: if true, this means this is mysql.gtid_slave_pos table */ + share->is_gtid_slave_pos= FALSE; share->table_category= get_table_category(& share->db, & share->table_name); share->set_refresh_version(); share->open_errno= ENOENT; === modified file 'sql/table.h' --- a/sql/table.h 2013-07-21 14:39:19 +0000 +++ b/sql/table.h 2013-08-02 14:12:09 +0000 @@ -644,6 +644,8 @@ struct TABLE_SHARE LEX_STRING normalized_path; /* unpack_filename(path) */ LEX_STRING connect_string; + bool is_gtid_slave_pos; + /* Set of keys in use, implemented as a Bitmap. Excludes keys disabled by ALTER TABLE ... DISABLE KEYS. _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message ----- -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3215: MDEV-4823: Server crashes in Item_func_not::fix_fields on creating a table with a virtual column using NOT in file:///home/bell/maria/bzr/work-maria-5.2-MDEV-4823/
by Sergei Petrunia 31 Jul '13

31 Jul '13

Hi Sanja, On Tue, Jul 30, 2013 at 11:00:59AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.2-MDEV-4823/ > > ------------------------------------------------------------ > revno: 3215 > revision-id: sanja(a)montyprogram.com-20130730080057-ylgl6h11l2b3yflc > parent: wlad(a)montyprogram.com-20130709202457-6n9syq59jdd2sy3b > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.2-MDEV-4823 > timestamp: Tue 2013-07-30 11:00:57 +0300 > message: > MDEV-4823: Server crashes in Item_func_not::fix_fields on creating a table with a virtual column using NOT > > fix_field() call protocol was brocken (zero pointer passed as link to item which is possible only if you are sure that there can not be Items which transforms). ... > === modified file 'sql/table.cc' > --- a/sql/table.cc 2013-01-16 00:46:27 +0000 > +++ b/sql/table.cc 2013-07-30 08:00:57 +0000 > @@ -1812,7 +1812,7 @@ bool fix_vcol_expr(THD *thd, > bool result= TRUE; > TABLE_LIST tables; > TABLE_LIST *save_table_list, *save_first_table, *save_last_table; > - int error; > + int error= 0; > Name_resolution_context *context; > const char *save_where; > char* db_name; > @@ -1860,7 +1860,13 @@ bool fix_vcol_expr(THD *thd, > save_use_only_table_context= thd->lex->use_only_table_context; > thd->lex->use_only_table_context= TRUE; > /* Fix fields referenced to by the virtual column function */ > - error= func_expr->fix_fields(thd, (Item**)0); > + if (!func_expr->fixed) > + error= func_expr->fix_fields(thd, (Item**)&vcol_info->expr_item); There is no need to use (Item**) cast, please remove it. Ok to push after that. > + > + /* fix_fields could change the expression */ > + func_expr= vcol_info->expr_item; > + /* Number of columns will be checked later */ > + > /* Restore the original context*/ > thd->lex->use_only_table_context= save_use_only_table_context; > context->table_list= save_table_list; > BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Plugin security
by Leif Walsh 31 Jul '13

31 Jul '13

Hi, I'm working on adding a plugin API (not exactly a storage engine API, rather an extension one) to TokuMX[1] and I'm looking for advice on security. The basic idea is fairly simple: a plugin is a shared library (we're only targeting Linux right now, by the way) that defines a symbol to bootstrap the loading of additional functionality. We have an admin-only command that calls dlopen and dlsym to load the plugin. I haven't read any of the MariaDB plugin code yet, so pointers to where to start reading would be appreciated. Apart from that, I'm just looking for any advice from you all about what to watch out for. My concerns break down into two main areas: 1. When loading a single plugin, what should I watch out for? The command itself is admin-only so calling it is protected, but what about an attacker that drops a malicious plugin in a directory earlier on the load path? What should I check about the permissions/owner of the library file and the directory in which it lives? Is erroring out on world-writable libraries enough? If my process is setuid, should I be more careful? What does Maria do here? 2. About plugin persistence/autoloading: I understand MariaDB has a system table that tracks which plugins are installed and automatically loads those on startup. I believe I can do the same thing in TokuMX and I can restrict access to that collection to the admin user. What does MariaDB store in that table and how does it use it? I could imagine storing the checksum of the installed plugin in the system table and verifying it before loading the plugin, but this seems rather draconian and it might make plugin upgrades too difficult. Does MariaDB verify the permissions of the data files that store this system table? I could imagine an attack where you would replace this system table with one that includes instructions to load your malicious plugin, while the server is offline. Again, how does MariaDB solve these problems? Another option is to autoload all libraries in some directory, and to just be very careful about the permissions of that directory, but this seems very hard to get totally right so I'm leaning away from it. [1]: http://www.tokutek.com/products/tokumx-for-mongodb -- Cheers, Leif

3 5

[Maria-developers] Master slave
by Richard Couture 30 Jul '13

30 Jul '13

Is it possible to set up replication in such a manner as to permit writing records to either a master or a slave replica? We want to set up a fail safe replica that can easily take over when our connection to the main replica fails, which, unfortunately, is fairly often. I read a document, some time ago which mentioned several types of replication, i.e.: by record, by table ... and the pitfalls, like IDs not necessarily matching... which confused me more than anything else... Can someone point me to some good documentation to enable me to pick the correct type of replication that I need to accomplish my task?? Thanks Richard -- LinuxCabal Asociación Civil Ing. Richard Couture Novell CNE, ECNE, MCNE HP/Compaq ASE Tel.: (+52) (333) 145-2638 Cel.: (+52) (044) 333 377-7505 Cel.: (+52) (044) 333 377-7506 Web: http://www.LinuxCabal.org E-Mail: rrc(a)linuxcabal.org Hosted en la nube Cloud Sigma - www.CloudSigma.com AVISO DE CONFIDENCIALIDAD: Este correo electrónico, incluyendo en su caso, los archivos adjuntos al mismo, pueden contener información de carácter confidencial y/o privilegiada, y se envían a la atención única y exclusivamente de la persona y/o entidad a quien va dirigido. La copia, revisión, uso, revelación y/o distribución de dicha información confidencial sin la autorización por escrito de LinuxCabal está prohibida. Si usted no es el destinatario a quien se dirige el presente correo, favor de contactar al remitente respondiendo al presente correo y eliminar el correo original incluyendo sus archivos, así como cualesquiera copia del mismo. Mediante la recepción del presente correo usted reconoce y acepta que en caso de incumplimiento de su parte y/o de sus representantes a los términos antes mencionados, LinuxCabal tendrá derecho a los daños y perjuicios que esto le cause.

1 0

Re: [Maria-developers] Implementing implicit primary key in mysql server
by Lixun Peng 30 Jul '13

30 Jul '13

Hi Sergei, You are right, let users add primary key is best. But I can't let users who don't want to create primary key can't use our MySQL service. Amazon RDS also allow users to create the tables without primary key, just change binlog_format to MIXED to solve replication problem. I think this feature is very useful for any MySQL cloud service providers, all of them will face this problem in one day. I will try to do some analysis/research in this feature implement first, I will need your help :-) I will notice any new updates in this email. Thanks, Lixun On Wed, Jun 19, 2013 at 5:14 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Lixun! > > On Jun 18, Lixun Peng wrote: > > Hi, > > > > As we know, InnoDB has implicit primary key if a table hasn't defined > > a primary key. However MySQL server doesn't know this primary key, so > > this primary key will not apear in binlog. > > > > When we are using ROW format binlog for replication, if a table has no > any > > indexes, that's a disaster. If a table without indexes do a DML > > (UPDATE/DELETE), of course it will run a long time in master, but in > slave, > > it still need a long time. It will cause serious slave replication delay. > ... > > I think mysql server can do the same thing as InnoDB do, if user doesn't > > define the primary key, mysql can add the primary key automatically. > > > > How do you think? > > Well, that's doable. A much easier solution would be to require a user > to create a primary key. It's a one-line change: > > - Table_flags ha_table_flags() const { return cached_table_flags; } > + Table_flags ha_table_flags() const { return cached_table_flags | > HA_REQUIRE_PRIMARY_KEY; } > > But what you suggest is possible too, I believe. > > Regards, > Sergei > -- Senior Database Engineer @ Taobao.com Mobile Phone: +86 18658156856 (Hangzhou) Gtalk: penglixun(at)gmail.com Twitter: http://www.twitter.com/plinux Blog: http://www.penglixun.com

4 8

Re: [Maria-developers] [Commits] Rev 3792: main.gis test fixed. in file:///home/hf/wmar/10.0-serg/
by Sergey Vojtovich 27 Jul '13

27 Jul '13

Hi Alexey, I just checked 5.6 and 10.0-base, this test is passing as is. I'm a bit worried that something functional was broken during the merge. Regards, Sergey 26.07.2013, в 16:13, <holyfoot(a)askmonty.org> написал(а): > At file:///home/hf/wmar/10.0-serg/ > > ------------------------------------------------------------ > revno: 3792 > revision-id: holyfoot(a)askmonty.org-20130726110356-9s17c1kw558ls14a > parent: svoj(a)mariadb.org-20130726091143-o2v0tt08n81n923m > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 10.0-serg > timestamp: Fri 2013-07-26 16:03:56 +0500 > message: > main.gis test fixed. > === modified file 'mysql-test/r/gis.result' > --- a/mysql-test/r/gis.result 2013-03-27 09:03:28 +0000 > +++ b/mysql-test/r/gis.result 2013-07-26 11:03:56 +0000 > @@ -918,25 +918,25 @@ SELECT Overlaps(@horiz1, @point2) FROM D > Overlaps(@horiz1, @point2) > 0 > DROP TABLE t1; > -create table t1(f1 geometry, f2 point, f3 linestring); > +create table t1(f1 geometry, f2 linestring, f3 linestring); > select f1 from t1 union select f1 from t1; > f1 > -insert into t1 (f2,f3) values (GeomFromText('POINT(1 1)'), > +insert into t1 (f2,f3) values (GeomFromText('LINESTRING(1 1, 2 2)'), > GeomFromText('LINESTRING(0 0,1 1,2 2)')); > select AsText(f2),AsText(f3) from t1; > AsText(f2) AsText(f3) > -POINT(1 1) LINESTRING(0 0,1 1,2 2) > +LINESTRING(1 1,2 2) LINESTRING(0 0,1 1,2 2) > select AsText(a) from (select f2 as a from t1 union select f3 from t1) t; > AsText(a) > -POINT(1 1) > +LINESTRING(1 1,2 2) > LINESTRING(0 0,1 1,2 2) > create table t2 as select f2 as a from t1 union select f3 from t1; > desc t2; > Field Type Null Key Default Extra > -a point YES NULL > +a linestring YES NULL > select AsText(a) from t2; > AsText(a) > -POINT(1 1) > +LINESTRING(1 1,2 2) > LINESTRING(0 0,1 1,2 2) > drop table t1, t2; > SELECT 1; > > === modified file 'mysql-test/t/gis.test' > --- a/mysql-test/t/gis.test 2013-03-27 09:03:28 +0000 > +++ b/mysql-test/t/gis.test 2013-07-26 11:03:56 +0000 > @@ -609,9 +609,9 @@ DROP TABLE t1; > # > # Bug#28763: Selecting geometry fields in UNION caused server crash. > # > -create table t1(f1 geometry, f2 point, f3 linestring); > +create table t1(f1 geometry, f2 linestring, f3 linestring); > select f1 from t1 union select f1 from t1; > -insert into t1 (f2,f3) values (GeomFromText('POINT(1 1)'), > +insert into t1 (f2,f3) values (GeomFromText('LINESTRING(1 1, 2 2)'), > GeomFromText('LINESTRING(0 0,1 1,2 2)')); > select AsText(f2),AsText(f3) from t1; > select AsText(a) from (select f2 as a from t1 union select f3 from t1) t; > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] we have seen sporadic crashes in the Protocol::net_store_data function
by Rich Prohaska 26 Jul '13

26 Jul '13

Crash occurs when using the TokuDB storage engine in MySQL 5.5.30. Any insight into what may be causing this problem would be really nice. Sorry, but this is all the info that i have. Here is a stack trace from the mysql error log: #1 0x00000000006b760f in handle_fatal_signal (sig=11) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/signal_handler.cc:247 #2 <signal handler called> #3 0x00007ff213a3b447 in __strncasecmp_l_sse2 () from /lib64/libc.so.6 #4 0x00000000005651a7 in Protocol::net_store_data (this=0x0, this@entry=0x7ff18f01e4b8, from=0x0, length=1936615494) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:59 #5 0x00000000005662e4 in Protocol::store_string_aux (this=this@entry=0x7ff18f01e4b8, from=<optimized out>, length=<optimized out>, fromcs=<optimized out>, tocs=tocs@entry=0x1022920 <my_charset_latin1>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:960 #6 0x00000000005663a5 in Protocol_text::store (this=0x7ff18f01e4b8, field=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:1119 #7 0x0000000000565ea0 in Protocol::send_result_set_row (this=0x7ff18f01e4b8, row_items=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:847 #8 0x000000000059fc37 in select_send::send_data (this=0x7fe338243b60, items=...) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_class.cc:2184 #9 0x00000000005f920c in end_send (join=0x7fe77f4a4010, join_tab=<optimized out>, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:12762 #10 end_send (join=0x7fe77f4a4010, join_tab=<optimized out>, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:12739 #11 0x00000000005e8a32 in evaluate_join_record (join=join@entry=0x7fe77f4a4010, join_tab=join_tab@entry=0x7fe77f470a60, error=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11925 #12 0x00000000005e8c36 in sub_select (join=0x7fe77f4a4010, join_tab=0x7fe77f470a60, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11778 #13 0x00000000005f681d in do_select (join=join@entry=0x7fe77f4a4010, fields=fields@entry=0x7ff18f020130, table=table@entry=0x0, procedure=0x0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11543 #14 0x000000000060655e in JOIN::exec (this=0x7fe77f4a4010) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:2385 #15 0x0000000000602095 in mysql_select (thd=thd@entry=0x7ff18f01e000, rref_pointer_array=rref_pointer_array@entry=0x7ff18f0201f8, tables=<optimized out>, wild_num=<optimized out>, fields=..., conds=<optimized out>, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=result@entry=0x7fe338243b60, unit=0x7ff18f01fa00, select_lex=select_lex@entry=0x7ff18f020020) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:2604 #16 0x0000000000608235 in handle_select (thd=0x7ff18f01e000, lex=0x7ff18f01f950, result=0x7fe338243b60, setup_tables_done_option=0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:297 #17 0x00000000005c23d1 in execute_sqlcom_select (thd=thd@entry=0x7ff18f01e000, all_tables=0x7fe338243270) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:4604 #18 0x00000000005ca700 in mysql_execute_command (thd=thd@entry =0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:2154 #19 0x00000000005cd7c2 in mysql_parse (parser_state=0x7ff21213a7e0, thd=0x7ff18f01e000, rawbuf=<optimized out>, length=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:5641 #20 mysql_parse (thd=0x7ff18f01e000, rawbuf=<optimized out>, length=65, parser_state=0x7ff21213a7e0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:5565 #21 0x00000000005cebcc in dispatch_command (command=COM_QUERY, thd=0x7ff18f01e000, packet=<optimized out>, packet_length=65) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:1041 #22 0x000000000066495f in do_handle_one_connection (thd_arg=thd_arg@entry =0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_connect.cc:853 #23 0x0000000000664a11 in handle_one_connection (arg=0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_connect.cc:772 #24 0x00007ff214711851 in start_thread () from /lib64/libpthread.so.0

1 0

[Maria-developers] help with mdev-4687 - aria engina problem with <= operator
by Roberto Spadim 26 Jul '13

26 Jul '13

hi guys i have a problem with aria https://mariadb.atlassian.net/browse/MDEV-4687 some queries that depend on <= operator don't return right rows (return different from myisam or innodb) i'm getting back to myisam, or innodb these are read intensive tables, and innodb runs slower, and myisam have a crash safe problem (but runs...) please any help is very wellcome thanks guys -- Roberto Spadim

2 4

[Maria-developers] innodb key comparisons
by Rich Prohaska 25 Jul '13

25 Jul '13

i am interested in the performance of in memory join queries on tokudb and innodb and, for innodb, i would like to know how many key comparisons were needed when running the query. i counted the number of calls to a function called "cmp_dtuple_rec_with_match". does this include ALL of the key comparisons? thanks rich prohaska

1 0

[Maria-developers] Documentation for old regex library calls
by Sudheera Palihakkara 25 Jul '13

25 Jul '13

Hi all, I'm replacing the current regular expression library with modern library(pcre) for mariadb. I need some information about the ways that code uses the old regex library. The relevant codes are in sql/item_cmpfunc.cc directory. For exapmle in the function : int Item_func_regex::regcomp(bool send_error) I can't understand most of the lines, and the code is not well-commented. In the new library(pcre) uses different way to compile a regEx so I have to understand this and code for pcre preserving the functionalists of the class. Is there any documentation for this file or the way they used to work with old regex library ? thanks. -- *Sudheera Palihakkara.* Undergraduate Department of *Computer Science and Engineering, *Faculty of Engineering, *University of Moratuwa*, Sri Lanka.

2 2

[Maria-developers] some bugs in dingqing parallel replication
by nanyi607rao 22 Jul '13

22 Jul '13

Hi,guys I have worked on this branch https://code.launchpad.net/~knielsen/maria/dingqi-parallel-replication for some days, and found bugs listed below.May this would be helpful to you. 1, when slave switch on table filter,this bug could lead server crash. how to reappear: on slave set replicate-wild-ignore-table = test.t5 in config file on master do these operations CREATE TABLE test.t3 (a INT AUTO_INCREMENT PRIMARY KEY, b DECIMAL(20,20), c INT); SET INSERT_ID=1; SET @c=2; SET @@rand_seed1=10000000, @@rand_seed2=1000000; INSERT INTO t3 VALUES (NULL, RAND(), @c); codes lead this bug: In execute_single_transaction() case RAND_EVENT: need_remove_from_trans= true; if(!rli->is_deferred_event(ev)) delete ev; break; reason: Rand Event object is deleted in execute_single_transaction(), but it's pointer would be used is slave_execute_deferred_events() later. 2, SQL thread could read and apply some log events repeated. how to reappear: it's a little hard to reappear. if you set max_relay_log_size=100M and keep SQL thread closed to IO thread, this bug may reappear. codes lead this bug: In reopen_relay_log() rli->event_relay_log_pos= max(rli->event_relay_log_pos, BIN_LOG_HEADER_SIZE); my_b_seek(cur_log,rli->event_relay_log_pos); reason: when SQL thread use a hot log,but the hot log was closed by IO thread just recently, SQL thread need to reopen this log and set read offset to rli->event_relay_log_pos, while rli->event_relay_log_pos could be set new value in other thread for there are many threads apply log events.so rli->event_relay_log_pos could be less then rli->future_event_relay_log_pos. 3, SQL thread do not report error information in result of "show slave status"and replication do not stop, when the slave insert duplicate record into a table with primary key. how to reappear: Just need to change master_log_pos to read duplicate records from master. codes lead this bug: In execute_single_transaction() retry_transaction: ev= trans->event_list_head; ... ... if (ret && rli->trans_retries < slave_trans_retries) { ... goto retry_transaction; } reason: as I have sayed in other email: Rows_log_event::do_apply_event() do twice but return different results for m_curr_row==m_rows_end in the second time. 4, when do oparetions such as "show slave status" and "stop slave", it could be blocked for a long time. how to reappear: just do "show slave status" again and again. codes lead this bug: In the queue_event() case FORMAT_DESCRIPTION_EVENT: ... wait_for_all_dml_done(&mi->rli, true); and in process_io_rotate() wait_for_all_dml_done(&mi->rli, true); reason: IO thread could wait in wait_for_all_dml_done() while holding the rpl_mi->data_lock, so operations like "show slave status" could be blocked for waiting rpl_mi->data_lock. 5, "START SLAVE UNTIL" make replication stop in different place. how to reappear: suppose log events in relay log like: BEGIN; ------->pos1 LOG_EVENT1; LOG_EVENT2; COMMIT; ------>pos2 BEGIN; ------>pos3 LOG_EVENT3; --->stop_pos LOG_EVENT4; COMMIT; ------>pos4 If we do START SLAVE UNTIL relay_log_pos=stop_pos; The replication should stop at pos4 but it stop pos2. 6, log_event->thd is wrong. suppose log_event was read is thread_1 so the log_event->thd==thread_1, but this log_event may be dispatch to other thread (suppose thread_2).the log_event is applyed in thread_2 but the log_event->thd==thread_1.this problem can make log event apply failed in MySQL, but in mariaDB it seems ok. 2013-07-19 nanyi607rao

2 1

[Maria-developers] RFC getting information from image (blob)
by Roberto Spadim 21 Jul '13

21 Jul '13

Hi guys, i will implement a gis system, but my problem isn't gis, my problem is raster (bitmap) images I will save images at mariadb using longblobs And i will create a table with, datetime and image I will need read each r,g,b value from each pixel, example.. Select rvalue(1,2,loadimage(blobfield)) >From table Order by datetime Rvalue= red value of x1 and y2 What's the faster or at least, how to use less cpu and memory I didn't search about images udf functions for mysql Any help is wellcome and ideas too :) Well i will use others sum, max, min functions too I don't want to read all images at client side, i want that server do this job I don't know the image size (it can be any pratical size, max near to 16mega pixels), image format could be png, jpg, or another one but i can use client side to rewrite image file to right format I'm considering implement some imagemagick functions inside mariadb, what you think? Thanks guys

2 2

[Maria-developers] Make error - cannot find -lregex
by Sudheera Palihakkara 21 Jul '13

21 Jul '13

Hi all, I have done some editing to sql/item_cmpfunc.cc file and CMakeList filesand then when I run the make command( after running cmake) it halts with this error in the middle. ([68%]) Linking CXX executable explain_filename-t /usr/bin/ld.bfd.real: cannot find -lregex collect2: error: ld returned 1 exit status make[2]: *** [unittest/mysys/explain_filename-t] Error 1 make[1]: *** [unittest/mysys/CMakeFiles/explain_filename-t.dir/all] Error 2 make: *** [all] Error 2 what might have gone wrong. :( -- *Sudheera Palihakkara.* Undergraduate Department of *Computer Science and Engineering, *Faculty of Engineering, *University of Moratuwa*, Sri Lanka.

2 3

[Maria-developers] Structure of MariaDB 10.0 binary logs
by Giuseppe Maxia 20 Jul '13

20 Jul '13

Hi, Where can I find a detailed structure of the binary logs supported by MariaDB 10.0.x? Specifically, I would like to know * what are the differences between binary logs in MariaDB 5.5 and 10.0 * what are the differences between binary logs in MariaDB 10.0 and MySQL 5.6. The purpose of this request id to parse the binary logs for usage in external applications. Thanks Giuseppe

2 2

[Maria-developers] Help about patchs in mariadb - jira and lauchpad url
by Roberto Spadim 18 Jul '13

18 Jul '13

Hi guys, where patchs are commited in mariadb? for example this one: https://mariadb.atlassian.net/browse/MDEV-617 it don't have a link to any patch, there's a link to any lauchpad code? -- Roberto Spadim

2 5

[Maria-developers] MariaDB 5.5.32 Release Prep
by Daniel Bartholomew 18 Jul '13

18 Jul '13

All, I've begun the prep work for the MariaDB 5.5.32 release. The packages are uploaded and the mirrors should start picking them up soon. The draft changelog and release notes are here: - https://kb.askmonty.org/en/mariadb-5532-changelog/ - https://kb.askmonty.org/en/mariadb-5532-release-notes/ Please let me know ASAP if there's anything that needs to be mentioned/highlighted/removed/etc... Assuming all goes well, the plan is to activate the release at around 12:00 US Eastern Time tomorrow (18 Jul 2013). Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

2 4

Re: [Maria-developers] [Commits] Rev 3839: Fix of using uninitialized variadle. in file:///home/bell/maria/bzr/work-maria-5.5-valgrind/
by Sergei Petrunia 18 Jul '13

18 Jul '13

Hi Sanja, On Thu, Jul 18, 2013 at 11:16:21AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-valgrind/ > > ------------------------------------------------------------ > revno: 3839 > revision-id: sanja(a)montyprogram.com-20130718081618-6ax63urznfnqmzgt > parent: sanja(a)montyprogram.com-20130717140359-123bw8dzelj6wddn > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.5-valgrind > timestamp: Thu 2013-07-18 11:16:18 +0300 > message: > Fix of using uninitialized variadle. > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2013-07-17 13:42:13 +0000 > +++ b/sql/sql_select.cc 2013-07-18 08:16:18 +0000 > @@ -15674,6 +15674,8 @@ create_internal_tmp_table_from_heap2(THD > const char *save_proc_info; > int write_err= 0; > DBUG_ENTER("create_internal_tmp_table_from_heap2"); > + if (is_duplicate) > + *is_duplicate= FALSE; > > if (table->s->db_type() != heap_hton || > error != HA_ERR_RECORD_FILE_FULL) > > === modified file 'sql/sql_union.cc' > --- a/sql/sql_union.cc 2013-07-16 05:22:17 +0000 > +++ b/sql/sql_union.cc 2013-07-18 08:16:18 +0000 > @@ -83,7 +83,7 @@ int select_union::send_data(List<Item> & > */ > return -1; > } > - bool is_duplicate; > + bool is_duplicate= FALSE; If you do the part in create_internal_tmp_table_from_heap2(), is this change really necessary? > /* create_internal_tmp_table_from_heap will generate error if needed */ > if (table->file->is_fatal_error(write_err, HA_CHECK_DUP) && > create_internal_tmp_table_from_heap(thd, table, > Ok to push either way. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] LDAP authentication plugin
by Sergei 18 Jul '13

18 Jul '13

Hi, Ilija! On Jul 11, Ilija Lazarevic wrote: > Hi Sergei, > > The reason I'm writing you is because I'm interested > in details about this gsoc2013 proposal, and I saw you were > supposed to be the mentor. > > Is it open for contributing apart from gsoc? Yes, it is. But please, write to maria-developers(a)lists.launchpad.net next time, not to me personally. Regards, Sergei

3 2

[Maria-developers] help creating error messages
by Roberto Spadim 16 Jul '13

16 Jul '13

hi guys, how could i create a new error / warning message in mariadb? 1) how to add the message text to error list (there's a list, right?) 2) how to return (send) a error to client 3) how to add a message to 'warning list' and after get it with SHOW WARNINGS maybe some example or point lines / files that i should read thanks guys :) -- Roberto Spadim

2 5

[Maria-developers] More enum values in enum fields
by Roberto Spadim 16 Jul '13

16 Jul '13

Hi guys, i have a problem, that i don't know how to easly solve (ok i know but i want the user easy way) i have a ENUM column with some values (today 248 distinct values) i see that in a near future i will have more than 65535 different values inside this enum column, the first solution is change it to varchar, but my problem will be data size the enum is very optimized in size, a char is 'converted' to a smallint value my second solution is a int column, and a external table with int/text values, but here i need to change apps, i don't want the second one, but i don't want a very big database (char column) how could i change the limit of enum size? maybe a new enum2 type? with a limit of bigint value (64bits), ok maybe less (32bits) today (reading code) i see that big enum is a problem for .frm file that don't allow this size of enums and i need to get back to a second table, but what should be done to allow a very big enum field? there's a "easy" way to change mariadb code to allow a bigger enum size? thanks guys -- Roberto Spadim

4 19

[Maria-developers] The funcation of retry_transaction in parallen replication
by nanyi607rao 16 Jul '13

16 Jul '13

hi, In order to test "retry_transaction" in function execute_single_transaction(), I change Relay_log_pos to make SQL thread insert Duplicate entry, but I find it successed and do not report error! so I debug the function execute_single_transaction() and find the reason : at first time, the insert Rows_log_event::do_apply_event() return 121, that means insert fails. But when retry the insert Rows_log_event::do_apply_event(),it return 0, that means insert success! it seems the Rows_log_event::do_apply_event() do twice, but return different results. how this happened? is it a bug of Rows_log_event::do_apply_event() ? By the way, I do this test in mysql-5.5.30, do mariaDB has the some problem? 2013-07-12 nanyi607rao

3 3

[Maria-developers] What issue tracker should we use?
by Roberto Spadim 16 Jul '13

16 Jul '13

hi guys, i see that mariadb have three issue tracker, i'm right? i saw the https://askmonty.org/worklog/Server-RawIdeaBin/?tid=145 (ask monty), jira, and lauchpad what issue tracker should be used? there's any idea porting the others issue to the main issue tracker? -- Roberto Spadim SPAEmpresarial

3 4

[Maria-developers] Reserved words
by Ian Gilfillan 14 Jul '13

14 Jul '13

How can one find a definitive list of reserved words for a particular version? The comments in sql/lex.h say that that's a list of all reserved words and functions, but it appears to contain some words that are neither (such as ACTION). SELECT * FROM mysql.help_keyword also lists ACTION ACTION is listed in the MYSQL docs as an example of one of the keywords that MySQL permits "because many people previously used them"

2 1

[Maria-developers] Pluggable authentication, custom error messages
by Vladislav Vaintroub 14 Jul '13

14 Jul '13

Hi Serg, is there any possibility to return own error messages and codes , if pluggable authentication failed on client on some reason ? Case in point here is Kerberos (GSSAPI or SSPI) - it may fail on client or on server, with some GSSAPI/SSPI specific code. We would like user to know what exactly happened , rather than return a generic CR_ERROR. Perhaps user can fix the situation - maybe he just needs to obtain Kerberos ticket with kinit, or login as Windows domain user, or something like that. If error happens on the server side, at least there it is possible to write to error log. But on the client side, there is no error log, and writing to stdout/err is not a good idea. Is there a trick that allows custom messages on the client? Wlad

2 2

[Maria-developers] MariaDB 10/trunk (re)install destructively overwrites existing my.cnf* @ spec'd SYSCONFDIR
by darx＠sent.com 14 Jul '13

14 Jul '13

Hi, I've a custom my.cnf ls -al /usr/local/etc/mariadb/my.cnf -rw-r--r--+ 1 root root 7.9K Jul 13 13:03 /usr/local/etc/mariadb/my.cnf If I (re)install mariadb 10/trunk from a src build with SYSCONFDIR spec'd for that dir, as done similarly to a MySQL build, cd mariadb/bld cmake .. \ ... -DINSTALL_SYSCONFDIR=/usr/local/etc/mariadb \ ... make make install The INSTALL process *destructively* overwrites the existing my.cnf ... -- Up-to-date: /usr/local/etc/mariadb/logrotate.d/mysql -- Up-to-date: /usr/local/etc/mariadb/init.d/mysql -- Installing: /usr/local/etc/mariadb/my.cnf <================================ !! -- Up-to-date: /usr/local/etc/mariadb/my.cnf.d/client.cnf -- Up-to-date: /usr/local/etc/mariadb/my.cnf.d/mysql-clients.cnf -- Up-to-date: /usr/local/etc/mariadb/my.cnf.d/server.cnf ... with an older default cnf, ls -al /usr/local/etc/mariadb/my.cnf -rw-r--r--+ 1 root root 202 Jul 12 11:31 /usr/local/etc/mariadb/my.cnf with no backup made of the existing cnf, find /usr/local/etc/mariadb -type f -name 'my.cnf*' /usr/local/etc/mariadb/my.cnf This is critically different behavior that MySQL. If that's intended, how are we to exec a mariadb install without overwriting our config files/dirs? If it's not, I suspect this should be fixed. Thanks.

2 2

[Maria-developers] building MariaDB 10/trunk with external SSL generates errors/warnings. Docs are inconsistent. What's the right procedure?
by darx＠sent.com 13 Jul '13

13 Jul '13

hi, building cd /usr/local/src/mariadb bzr log | head -n 5 ------------------------------------------------------------ revno: 3766 [merge] committer: Alexander Barkov <bar(a)mariadb.org> branch nick: maria-10.0.ms timestamp: Wed 2013-07-10 18:46:33 +0400 reading ./BUILD/SETUP.sh ... 120 # SSL library to use.--with-ssl will select our bundled yaSSL # implementation of SSL. To use OpenSSL you will need to specify # the location of OpenSSL headers and libs on your system. # Ex --with-ssl=/usr <================================= OK, do this SSL_LIBRARY=--with-ssl ... configuring @ `cmake` cd /usr/local/src/mariadb rm -rf bld mkdir bld cd bld cmake .. \ -DWITH_SSL=/usr/local/ssl \ -DOPENSSL_INCLUDE_DIR=/usr/local/ssl/include \ -DOPENSSL_SSL_LIBRARY=/usr/local/ssl/lib64/libssl.so \ -DOPENSSL_CRYPTO_LIBRARY=/usr/local/ssl/lib64/libcrypto.so fails, returning, ... CMake Error at cmake/ssl.cmake:90 (MESSAGE): Wrong option for WITH_SSL. Valid values are : yes, no, bundled Call Stack (most recent call first): CMakeLists.txt:260 (MYSQL_CHECK_SSL) ... -- Configuring incomplete, errors occurred! vi ../CmakeLists.txt ... # Add bundled or system zlib. MYSQL_CHECK_ZLIB_WITH_COMPRESS() # Optionally add bundled yassl/taocrypt or system openssl. 260 MYSQL_CHECK_SSL() # Add readline or libedit. MYSQL_CHECK_READLINE() checking vi cmake/ssl.cmake ... # MYSQL_CHECK_SSL # # Provides the following configure options: # WITH_SSL=[yes|no|bundled] MACRO (MYSQL_CHECK_SSL) IF(NOT WITH_SSL) IF(WIN32) CHANGE_SSL_SETTINGS("bundled") ELSE() CHANGE_SSL_SETTINGS("no") ENDIF() ENDIF() IF(WITH_SSL STREQUAL "bundled") MYSQL_USE_BUNDLED_SSL() ELSEIF(WITH_SSL STREQUAL "system" OR WITH_SSL STREQUAL "yes") # Check for system library SET(OPENSSL_FIND_QUIETLY TRUE) INCLUDE(FindOpenSSL) FIND_LIBRARY(CRYPTO_LIBRARY crypto) MARK_AS_ADVANCED(CRYPTO_LIBRARY) INCLUDE(CheckSymbolExists) SET(CMAKE_REQUIRED_INCLUDES ${OPENSSL_INCLUDE_DIR}) CHECK_SYMBOL_EXISTS(SHA512_DIGEST_LENGTH "openssl/sha.h" HAVE_SHA512_DIGEST_LENGTH) SET(CMAKE_REQUIRED_INCLUDES) IF(OPENSSL_FOUND AND CRYPTO_LIBRARY AND HAVE_SHA512_DIGEST_LENGTH) SET(SSL_SOURCES "") SET(SSL_LIBRARIES ${OPENSSL_LIBRARIES} ${CRYPTO_LIBRARY}) SET(SSL_INCLUDE_DIRS ${OPENSSL_INCLUDE_DIR}) SET(SSL_INTERNAL_INCLUDE_DIRS "") SET(SSL_DEFINES "-DHAVE_OPENSSL") CHANGE_SSL_SETTINGS("system") ELSE() IF(WITH_SSL STREQUAL "system") MESSAGE(SEND_ERROR "Cannot find appropriate system libraries for SSL. Use WITH_SSL=bundled to enable SSL su ENDIF() MYSQL_USE_BUNDLED_SSL() ENDIF() ELSEIF(NOT WITH_SSL STREQUAL "no") MESSAGE(SEND_ERROR "Wrong option for WITH_SSL. Valid values are : yes, no, bundled") ENDIF() ENDMACRO() there's no accommodation for "--with-ssl=/path/to/ssl/install". If I instead cmake .. \ -DWITH_SSL=yes \ -DOPENSSL_INCLUDE_DIR=/usr/local/ssl/include \ -DOPENSSL_SSL_LIBRARY=/usr/local/ssl/lib64/libssl.so \ -DOPENSSL_CRYPTO_LIBRARY=/usr/local/ssl/lib64/libcrypto.so it returns, ... -- Running cmake version 2.8.11.1 -- MariaDB 10.0.3 -- Packaging as: mariadb-10.0.3-Linux-x86_64 -- Found OpenSSL: /usr/local/ssl/lib64/libssl.so;/usr/local/ssl/lib64/libcrypto.so (found version "1.0.1e") ... -- Configuring done then a bunch of these messages: ... CMake Warning at cmake/plugin.cmake:184 (ADD_LIBRARY): Cannot generate a safe runtime search path for target feedback because files in some directories may conflict with libraries in implicit directories: runtime library [libcrypto.so.1.0.0] in /usr/lib64 may be hidden by files in: /usr/local/ssl/lib64 Some of these libraries may not be found correctly. Call Stack (most recent call first): plugin/feedback/CMakeLists.txt:20 (MYSQL_ADD_PLUGIN) CMake Warning at cmake/libutils.cmake:259 (ADD_LIBRARY): Cannot generate a safe runtime search path for target libmysql because files in some directories may conflict with libraries in implicit directories: runtime library [libcrypto.so.1.0.0] in /usr/lib64 may be hidden by files in: /usr/local/ssl/lib64 Some of these libraries may not be found correctly. Call Stack (most recent call first): libmysql/CMakeLists.txt:374 (MERGE_LIBRARIES) CMake Warning at client/CMakeLists.txt:78 (ADD_EXECUTABLE): Cannot generate a safe runtime search path for target async_example because files in some directories may conflict with libraries in implicit directories: runtime library [libcrypto.so.1.0.0] in /usr/lib64 may be hidden by files in: /usr/local/ssl/lib64 Some of these libraries may not be found correctly. ... What's the right method to find/use external SSL libs/headers in a MariaDB build without errors or warnings? --darx

2 1

[Maria-developers] Doubts about index and optimization
by Roberto Spadim 13 Jul '13

13 Jul '13

Hi guys i have two doubts... First the r-tree index Could i index two integer or float or decimal columns using rtree And queries like X>=1 and x<=2 and y>=1 and y<=2 Be optimized using rtree? Another doubt... i dont have a mariadb now to test, i'm in cellphone... this kind of query is optimized if i have a x,y btree index? Btree will use x only, or x and y? Second doubt I'm thinking about enum rewrite optimzation and i didn't write anything yet, just studying mariadb code... When using queries like this Where x<=5 Or where x>=5 Or without = part Does mysql know what's the max and min value of x, and rewrite it to something X<=5. To. X in (0,1,2,3,4,5) when x is a integer for example and min(x)=0 I'm thinking about this, in conditions like the rtree optimization, example... we could optimize x and y with something like X>1 and x<5 and y>1 and y<5 to X in (2,3,4) and y in (2,3,4) It's something like enum optimization but using integers and max min informations Well i don't know the full code but i didn't found this information (i don't remember at least) Anyone know something about this in source? If not i will consider this optimization when doing the enum optimization (i don't remember the mdev number now, but anyone interested in this optimization can search in jira and find it, i reported and done any job yet just the raw and high implementation idea) Thanks guys

1 0

[Maria-developers] OLAP
by Roberto Spadim 12 Jul '13

12 Jul '13

Hi guys, there's any good OLAP tool for mariadb? maybe one that we could package with mariadb as default? -- Roberto Spadim

2 5

[Maria-developers] Status on MDEV-4506, parallel replication
by Kristian Nielsen 12 Jul '13

12 Jul '13

So I have been working for some weeks now on implementing MDEV-4506. This task is about making the slave apply (some) events in parallel threads, to speed up replication and reduce the risk of slave not being able to keep up with a busy master. Events are applied in parallel on the slave if they were group-committed together on the master. This is an easy way to detect transactions that are independent. Note that this is transparent to applications; while transactions are executed in parallel on the slave, they are still committed in the same order as on the master. I also added parallel execution of events with different GTID domain id. This makes testing a lot easier (no need to carefully arrange timing to get a specific group commit on the master), and also really is the whole point of much of my hard work on GTID. So if we have multi-source M1->S1, M2->S1, and S1->S2, then S2 will be able to execute events from M1 in parallel with those from M2, just like S1 can. And the user can explicitly set the domain_id different for eg. a long-running ALTER or UPDATE, and this way get it to run in parallel and not cause a huge replication delay. ---- On the master, I added to each GTID event a commit_id. If two transactions group-commit together, they are binlogged with the same commit_id; if not, they get different commit_ids. Thus, the slave can detect the possibility of executing two transactions in parallel by checking if the commit_ids are equal. On the master, I implemented --binlog-commit-wait-count=N and --binlog-commit-wait-usec=T. A transaction will wait at most T microseconds for at least N transactions to queue up and be ready for group commit. This allows to deliberately delay transactions on the master in order to get bigger group commits and thus better opportunity for parallel execution (and again it makes testing easier). On the slave, I implemented --slave-parallel-threads=N. If N>0, that many threads will be spawned, and events will be executed in parallel (if possible) by those threads. ---- The current code is pushed here (it is based on 10.0-base): lp:~maria-captains/maria/10.0-knielsen It is still far from finished, but it now works sufficiently that I could do some basic benchmarking (just on my laptop, completely unscientifically). First, I prepared a binlog on the master with plenty of opportunity for parallel replication. I started the master with --binlog-commit-wait-count=20 --binlog-commit-wait-usec=1000000. I then ran this Gypsy script: ----------------------------------------------------------------------- i|1|DROP TABLE IF EXISTS t1 i|1|CREATE TABLE t1 (a INT PRIMARY KEY, b VARCHAR(100)) ENGINE=InnoDB p|1|REPLACE INTO t1 (a,b) VALUES (? MOD 10000, ?)|int,varchar /home/knielsen/my/gypsy/words ----------------------------------------------------------------------- gypsy --queryfile=simple_replace_load.gypsy --duration=20 --threads=40 This results in a binlog with about 65k updates to the table, group-committed in batches of 20 transactions. I then started a fresh slave and let it replicate everything with START SLAVE UNTIL; the time to replicate all the events is then easy to see in the slave error log. The time to replicate everything with unmodified 10.0-base was 99 seconds. With --slave-parallel-threads=25, it was just 22 seconds. So that is a 4.5 times speedup, which is quite promising. Also note that at 22 seconds, the slave is within 10% of the speed of the master. ---- But as I said, there is still significant work left to do. I put a ToDo list at the top of sql/rpl_parallel.cc. Some of the big remaining issues: 1. The existing code is not thread-safe for class Relay_log_info. This class contains a bunch of stuff that is specific to executed transactions, not related to relay-log at all. This needs to be moved to the new struct rpl_group_info I introduced, and all code updated to pass around a pointer to that struct instead. There may also be a need to add additional locking on Relay_log_info, existing code needs review for this. 2. Error handling needs to be implemented, it is rather more complex in the parallel case. If one transaction fails (and retry also fails), then we need to somehow get hold of all later transactions that are in the process of parallel execution, and abort them + roll them back. Otherwise we get inconsistent binlog position for the next slave restart. 3. In the old code, when the SQL thread is stopped, it has logic to let the current event group (=transaction) replicate to completeness first, with a timeout to force a stop in the middle of the event group if eg. the master has disappeared. This logic needs to be re-implemented to work when having any number of event groups executing in parallel. (It is important to let the groups complete execution when doing non-transactional stuff that cannot be rolled back, otherwise again the slave position becomes inconsistent for next slave restart). So as you see, there is quite a bit of work left on this (as well as on GTID). So I would very much welcome any help on this to avoid causing delays for 10.0-GA ... - Kristian.

1 0

[Maria-developers] a bug of ding-qing parallel replication
by nanyi607rao 11 Jul '13

11 Jul '13

hi guys, I find a bug of ding-qing parallel replication. let's consider a table like this: create table t1( a int(11) NOT NULL DEFAULT '0', b varchar(10), PRIMARY KEY (a) )ENGINE=InnoDB if we do transactions on master(binlog_format=ROW) like this: trans 1: begin; insert into t1 values(1, "tt"); update t1 set a=2 where a=1; commit; trans 2; begin; update t1 set b="kk" where a=2; commit; then this two transactions binlog transmit to the slave, because trans 1 and trnas 2 has different primary keys, it would disaptch in different threads like this: thread 1 do trans 1 thread 2 do trans 2 thread 2 may update t1 before thread 1, then it will fail. maybe thread 2 can retry sever times ,but it don't solve this problem compeletly.. 2013-07-11 nanyi607rao

3 2

[Maria-developers] deferred_event
by nanyi607rao 10 Jul '13

10 Jul '13

Hi guys, as I know, when replication filter used, the defered events is a container hold on events(such as Intvar-, Rand-, User_var- log_event)that be applied delay. why those events need be applied delay? and what is parent Query mentioned in codes? Thanks! 2013-07-10 nanyi607rao

2 1

[Maria-developers] buildbot config
by Leif Walsh 09 Jul '13

09 Jul '13

Hi, I'm learning buildbot and I was wondering if I could take a look at the master.cfg for buildbot.askmonty.org as a reference for learning common practices and things like that. Is it available somewhere? -- Cheers, Leif

2 2

[Maria-developers] merge for MySQL56 FSP data types
by Alexander Barkov 09 Jul '13

09 Jul '13

Hi Serg, Please review the patch merging MySQL-5.6 fractional second precision enabled data types. Thanks.

3 7

[Maria-developers] Connect: compilation warnings
by Alexander Barkov 08 Jul '13

08 Jul '13

Hi Oliver, I fixed almost all compilation warnings that happened in Connect on my Fedora 17 laptop with gcc-4.7.2. Most of them were "variable is initialized but not used" and "no previous declaration". There were not serious, but very unpleasant for developers. (Note, we try to keep the tree in a shape that it compiles without warnings. Connect must follow this rule as well!!!) I think there was a bug in one place: how SQLSetStmtAttr() was called. See "bzr diff -c 3827" for details. The value of m_RowsetSize was erroneously passed, but SQLSetStmtAttr() expects the address of a SQLUINT variable (not a value!). See the ODBC docs for the details: http://msdn.microsoft.com/en-us/library/windows/desktop/ms712631%28v=vs.85%… One compilation warning still remains: maria-10.0-connect/storage/connect/tabsys.cpp:279:26: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] It looks like a bug. Cast from 4-byte integer to 8-byte pointer is obviously wrong. Please address the problem. Thanks.

1 0

[Maria-developers] Warning in dynamic columns code
by Sergei Petrunia 08 Jul '13

08 Jul '13

Hi Sanja, I have been seeing this warning on a number of platforms: 10.0-monty/sql/item_strfunc.cc: In member function ‘virtual bool Item_func_dyncol_create::fix_fields(THD*, Item**)’: 10.0-monty/sql/item_strfunc.cc:3887: warning: suggest a space before ‘;’ or explicit braces around empty body in ‘for’ statement I assume it is harmless, but just in case, passing it over to you. Could you please check it and get rid of it? Thanks Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] SO_REUSEPORT and mariadb
by Roberto Spadim 08 Jul '13

08 Jul '13

hi guys, i was reading the SO_REUSEPORT of kernel 3.9, and i didn't found it in mariadb maybe we could use it? here are the SO_REUSEPORT topics: https://lwn.net/Articles/542629/ http://linux.dell.com/files/presentations/Linux_Plumbers_Conf_2010/Scaling_… here the important parts: ------------------ 1)The first of the traditional approaches is to have a single listener thread that accepts all incoming connections and then passes these off to other threads for processing. The problem with this approach is that the listening thread can become a bottleneck in extreme cases. In early discussions<http://thread.gmane.org/gmane.linux.network/102140/focus=102150> on SO_REUSEPORT, Tom noted that he was dealing with applications that accepted 40,000 connections per second. Given that sort of number, it's unsurprising to learn that Tom works at Google. 2)The second of the traditional approaches used by multithreaded servers operating on a single port is to have all of the threads (or processes) perform an accept() call on a single listening socket in a simple event loop of the form: while (1) { new_fd = accept(...); process_connection(new_fd); } ------------------ i didn -- Roberto Spadim

1 1

[Maria-developers] Replication
by Roberto Spadim 05 Jul '13

05 Jul '13

hi guys, i don't know if it's the better place to ask... maybe in user list is the right one, please tell me what's better... i'm using replication, and i need to mark all tables that don't replicate, it's very sad boring time consuming, etc etc... there's a easier way to mark tables? maybe alter table xxxx REPLICATION=ON/OFF if not, could we add this feature? thanks guys -- Roberto Spadim

1 0

[Maria-developers] Developer course
by Roberto Spadim 05 Jul '13

05 Jul '13

hi guys, there's a internal mysql/mariadb developer course? i'm studying source, but it's a bit difficult understand what's the best class/methods to use, maybe a course could help me developing and understanding better the source any one know about a course like it? i will focus on query rewrite (select.cc), optimizer (opt_range.cc/opt_*) and partition engine, others engine (myisam,heap,innodb,aria) is not my priority now -- Roberto Spadim

2 3

[Maria-developers] Review request : MDEV-573 (mysqlreport update)
by Jean Weisbuch 04 Jul '13

04 Jul '13

Hi, I have been doing modifications to the MariaDB mysqlreport version (which is shipped on the Debian/Ubuntu packages and can be found on the debian/additions/ directory), especially to add stats/informations about MariaDB specific functions but also to fix some bugs and limitations of the tool. As of today, the script works without throwing a warning and seems to output correct informations on MariaDB 5.5 (either with or without threadpool and binary log activated), MariaDB 5.5 + TokuDB, MySQL 5.1, MySQL 5.5 and it solves some small bugs and issues that could lead warning messages, invalid values returned or truncated values. The main additions are the AriaDB pagecache stats, binlog cache stats, subquery cache stats, TokuDB stats and Threadpool stats. There are also some "cosmetic" changes/fixes : the addition of values such as the % of disk tables used or the InnoDB read "From file" modified to "From disk" as of the documentation of the status variables states ; values such as the threads cache size or the table cache size that were limited to 3 chars are now limited to 5 chars ; the per seconds stats was truncated to 5 chars max on Questions and Query Cache sections (for example 153.6k/s was truncated to 153.6/s). The JIRA task : https://mariadb.atlassian.net/browse/MDEV-573 The GIT repository where i put the latest revisions (i only have been using it lately but i commented all the changes and bugfixes along the time on the JIRA task and could provide older revisions/diff if required) : https://github.com/jb-boin/mariadb/blob/5.5.30/debian/additions/mysqlreport I would like to be sure that the values/stats i added are really useful to monitor (and the calculations of these correct) and that the way informations are shown, named and ordered is clear enough ; for example, the way the test that determines if its a MariaDB or a MySQL server might not be the best possible (it checks if the Aria engine is available). There is still an upstream bug on the "Questions" and the "SELECT and Sort" section, the %Total values are not correctly calculated and lead to values > 100%, i havent got time to give it a look yet and i included in the example output below values hitting that bug. Here is an example of the output of the latest revision on a server having about all the possible functions enabled (values are mixed from different servers) : MariaDB 5.5.30-tokudb-7.0.1 uptime 70 9:4:47 Thu Jul 4 01:48:24 2013 __ Key _________________________________________________________________ Buffer used 651.41M of 1.00G %Used: 63.61 Current 675.43M %Usage: 65.96 Write hit 79.79% Read hit 99.99% __ Questions ___________________________________________________________ Total 2.57M 0.4/s -Unknown 39.92M 6.6/s %Total: 1550.4 DMS 39.78M 6.5/s 1545.3 Com_ 2.71M 0.4/s 105.09 COM_QUIT 444 0.0/s 0.02 Slow 4 s 285 0.0/s 0.01 %DMS: 0.00 Log: OFF DMS 39.78M 6.5/s 1545.3 UPDATE 39.76M 6.5/s 1544.5 99.95 SELECT 21.11k 0.0/s 0.82 0.05 REPLACE 0 0/s 0.00 0.00 DELETE 0 0/s 0.00 0.00 INSERT 0 0/s 0.00 0.00 Com_ 2.71M 0.4/s 105.09 show_proces 2.51M 0.4/s 97.33 admin_comma 152.51k 0.0/s 5.92 show_status 9.23k 0.0/s 0.36 __ Rows ________________________________________________________________ Rows 186.00G 240.3k/s Using idx 168.06G 217.1k/s %Index: 90.35 Rows/question 158.90 __ SELECT and Sort _____________________________________________________ Scan 42.33k 0.0/s %SELECT: 200.53 Range 0 0/s 0.00 Full join 19.27k 0.0/s 91.29 Range check 0 0/s 0.00 Full rng join 0 0/s 0.00 Sort scan 0 0/s Sort range 0 0/s Sort mrg pass 0 0/s __ Table Locks _________________________________________________________ Waited 8.86M 11.4/s %Total: 0.46 Immediate 1.93G 2.5k/s __ Tables ______________________________________________________________ Open 6843 of 8192 %Cache: 83.53 Opened 14.14k 0.0/s __ Connections _________________________________________________________ Max used 5403 of 22000 %Max: 24.56 Total 176.48M 228.0/s __ Created Temp ________________________________________________________ Disk table 7.76k 0.0/s %Disk: 14.55 Table 53.38k 0.0/s Size: 64.0M File 12 0.0/s __ Threads _____________________________________________________________ Running 274 of 2788 Created 181.68k 0.2/s Slow 0 0/s Threadpool 892 of 1200 %Used: 74.33 Running 458 of 1200 %Running: 38.17 Idle 434 of 1200 %Idle: 36.17 __ Aborted _____________________________________________________________ Clients 5.03k 0.0/s Connects 3.65k 0.0/s __ Bytes _______________________________________________________________ Sent 1.51T 1.9M/s Received 445.91G 576.0k/s __ InnoDB Buffer Pool __________________________________________________ Usage 25.97G of 26.00G %Usage: 99.89 Read hit 100.00% Pages Free 1.83k %Total: 0.11 Data 1.67M 98.23 %Drty: 0.07 Misc 28381 1.67 Latched 0 0.00 Reads 527.54G 681.5k/s From disk 100.00k 0.1/s %Disk: 0.00 Ahead Rnd 0 0/s Writes 5.21G 6.7k/s Flushes 17.35M 22.4/s Wait Free 9 0.0/s %Wait: 0.00 __ InnoDB Lock _________________________________________________________ Waits 36660726 47.4/s Current 0 Time acquiring Total 627441780 ms Average 171 ms Max 1537520 ms __ InnoDB Data, Pages, Rows ____________________________________________ Data Reads 512.26k 0.7/s Writes 181.92M 235.0/s fsync 2.07M 2.7/s Pending Reads 0 Writes 0 fsync 0 Pages Created 3.72M 4.8/s Read 512.18k 0.7/s Written 17.35M 22.4/s Rows Deleted 175.85M 227.2/s Inserted 200.46M 259.0/s Read 174.52G 225.4k/s Updated 224.40M 289.9/s __ Aria Pagecache ______________________________________________________ Buffer used 3.43M of 512.00M %Used: 0.67 Current 20.26M %Usage: 3.96 Write hit 47.09% Read hit 99.96% __ Subquery Cache ______________________________________________________ Hit ratio 99.91% Hits 38.63G 49.9k/s Miss 35.97M 46.5/s __ Binary Log Cache _____________________________________________________ Disk use Transactional 0.43% Non transactional 0.50% __ TokuDB ______________________________________________________________ Cachetable 4.00G of 4.00G %Usage: 99.99 Miss 1.83M 0.3/s Evictions 1.95M 0.3/s Thanks in advance, Jean Weisbuch

1 0

Re: [Maria-developers] Rev 3824: MDEV-4472 Audit plugin
by Sergei Golubchik 03 Jul '13

03 Jul '13

Hi, Holyfoot! Generally, it looks pretty good. One comment, see below about MARIADB_ONLY. On Jul 02, holyfoot(a)askmonty.org wrote: > revno: 3824 > revision-id: holyfoot(a)askmonty.org-20130702213841-xwmzadaotzxqhz2k > parent: sergii(a)pisem.net-20130701155424-lx0dvaocejx0cn8l > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-4472 > timestamp: Wed 2013-07-03 02:38:41 +0500 > message: > MDEV-4472 Audit plugin. > Plugin implemented in plugin/server_audit/server_audit.c. > Rotating log file support modified to be mysql-service. > === added directory 'plugin/server_audit' > === added file 'plugin/server_audit/CMakeLists.txt' > --- a/plugin/server_audit/CMakeLists.txt 1970-01-01 00:00:00 +0000 > +++ b/plugin/server_audit/CMakeLists.txt 2013-07-02 21:38:41 +0000 > @@ -0,0 +1,16 @@ > +# Copyright (C) 2012 Monty Program Ab. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; version 2 of the License. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write to the Free Software > +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > + > +MYSQL_ADD_PLUGIN(server_audit server_audit.c MODULE_ONLY) I'd remove MODULE_ONLY, if the plugin can be complied as a static plugin. No need to add artificial restrictions. > === added file 'plugin/server_audit/server_audit.c' > --- a/plugin/server_audit/server_audit.c 1970-01-01 00:00:00 +0000 > +++ b/plugin/server_audit/server_audit.c 2013-07-02 21:38:41 +0000 ... > +#ifndef MARIADB_ONLY > +#define MYSQL_SERVICE_LOGGER_INCLUDED > +#endif /*MARIADB_ONLY*/ Is MARIADB_ONLY normally defined? As far as I can see, it's not. So in the packages this plugin won't have MARIADB_ONLY. That's ok in your tree, but let's think about how to solve this before pushing into the main tree. Regards, Sergei

2 1

[Maria-developers] merge for MySQL56 temporal literals
by Alexander Barkov 03 Jul '13

03 Jul '13

Hi Serg, Please review the patch merging MySQL-5.6 temporal literals. Thanks.

3 4