- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] 685c70d: MDEV-4829 BEFORE INSERT triggers dont issue 1406 error.
by Sergei Golubchik 17 Nov '15

17 Nov '15

Hi, Alexey! On Nov 17, Alexey Botchkov wrote: > revision-id: 685c70d39ac6599b448e86a3896a8c6836079b78 (mariadb-10.1.8-54-g685c70d) > parent(s): 0dfa0eef596ee677b55976793d632dc9b36928c7 > committer: Alexey Botchkov > timestamp: 2015-11-16 17:20:35 +0400 > message: > > MDEV-4829 BEFORE INSERT triggers dont issue 1406 error. > > The STRICT mode was reimplemented using Strict_handler instead of the > THD::abort_on_warning flag. I like the approach, in general. But 1. Let's do it in 10.2 2. Let's first wait for Bar to do his changes where only "true" fields will respect strict mode. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 5803178: MDEV-4829 BEFORE INSERT triggers dont issue 1406 error.
by Sergei Golubchik 17 Nov '15

17 Nov '15

Hi, Alexey! On Nov 17, Alexey Botchkov wrote: > revision-id: 580317835f9758a9536150e02031ff520e747089 (mariadb-10.1.8-54-g5803178) > parent(s): 0dfa0eef596ee677b55976793d632dc9b36928c7 > committer: Alexey Botchkov > timestamp: 2015-11-17 00:42:18 +0400 > message: > > MDEV-4829 BEFORE INSERT triggers dont issue 1406 error. > Turn the 'abort_on_warning' on for assigning value to fields. ok to push Regards, Sergei

1 0

Re: [Maria-developers] [Commits] bff67808: MDEV-7806 - thread_pool_size is not auto-sized
by Sergei Golubchik 17 Nov '15

17 Nov '15

Hi, Sergey! On Nov 17, Sergey Vojtovich wrote: > revision-id: bff6780806f4afed8faaf4c79b1593964564097e (mariadb-10.1.8-66-gbff67808) > parent(s): c0216f1d02e63686f986fa8f352fdf6de61249a7 > committer: Sergey Vojtovich > timestamp: 2015-11-17 19:33:01 +0400 > message: > > MDEV-7806 - thread_pool_size is not auto-sized > > thread_pool_size is auto-sized before my_getopt(). But my_getopt starts from > resetting all options to their default values. So the auto-sized value is lost. > > Fixed by moving thread_pool_size auto-sizing after my_getopt(). Ok to push! Regards, Sergei

1 0

Re: [Maria-developers] [Commits] f21705d: MDEV-7806 - thread_pool_size is not auto-sized
by Sergei Golubchik 17 Nov '15

17 Nov '15

Hi, Sergey! On Nov 17, Sergey Vojtovich wrote: > revision-id: f21705dfbfcd87e46937222eb860727beb21a89b (mariadb-10.1.8-66-gf21705d) > parent(s): c0216f1d02e63686f986fa8f352fdf6de61249a7 > committer: Sergey Vojtovich > timestamp: 2015-11-17 13:13:47 +0400 > message: > > MDEV-7806 - thread_pool_size is not auto-sized > > thread_pool_size is auto-sized before my_getopt(). But my_getopt starts from > resetting all options to their default values. So the auto-sized value is lost. > > Fixed by autosizing default value directly. > > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 682d803..e65a939 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -3270,7 +3270,7 @@ static Sys_var_uint Sys_threadpool_size( > "This parameter is roughly equivalent to maximum number of concurrently " > "executing threads (threads in a waiting state do not count as executing).", > GLOBAL_VAR(threadpool_size), CMD_LINE(REQUIRED_ARG), > - VALID_RANGE(1, MAX_THREAD_GROUPS), DEFAULT(8), BLOCK_SIZE(1), > + VALID_RANGE(1, MAX_THREAD_GROUPS), DEFAULT(my_getncpus()), BLOCK_SIZE(1), > NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(check_threadpool_size), > ON_UPDATE(fix_threadpool_size) > ); Strange solution. Why wouldn't you move autosizing after config variables are read? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 4

Re: [Maria-developers] [Commits] 096fd8f: MDEV-9033 : Incorrect statements binlogged on slave with do_domain_ids=(...)
by Nirbhay Choubey 17 Nov '15

17 Nov '15

Hi Kristian, Can you please review this patch? - Nirbhay On Fri, Nov 13, 2015 at 5:23 PM, Nirbhay Choubey <nirbhay(a)mariadb.com> wrote: > revision-id: 096fd8f57664d0b3061f0e781a31bfa3cc6bf6cd > parent(s): a574407444fc3ea93ca88afa93dc18154251bf74 > committer: Nirbhay Choubey > branch nick: 10.1.b9033 > timestamp: 2015-11-13 17:23:49 -0500 > message: > > MDEV-9033 : Incorrect statements binlogged on slave with > do_domain_ids=(...) > > In domain ID based filtering, a flag is used to filter-out > the events that belong to a particular domain. This flag gets > set when IO thread receives a GTID_EVENT for the domain on > filter list and its reset at the last event in the GTID group. > The resetting, however, was wrongly done before the decision to > write/filter the event from relay log is made. As a result, the > last event in the group will always pass through the filter. > Fixed by deferring the reset logic. Also added a test case. > > --- > mysql-test/suite/rpl/r/rpl_gtid_mdev9033.result | 73 ++++++++++++++++++++ > mysql-test/suite/rpl/t/rpl_gtid_mdev9033.cnf | 20 ++++++ > mysql-test/suite/rpl/t/rpl_gtid_mdev9033.test | 92 > +++++++++++++++++++++++++ > sql/slave.cc | 19 +---- > 4 files changed, 188 insertions(+), 16 deletions(-) > > diff --git a/mysql-test/suite/rpl/r/rpl_gtid_mdev9033.result > b/mysql-test/suite/rpl/r/rpl_gtid_mdev9033.result > new file mode 100644 > index 0000000..fdee91f > --- /dev/null > +++ b/mysql-test/suite/rpl/r/rpl_gtid_mdev9033.result > @@ -0,0 +1,73 @@ > +include/rpl_init.inc [topology=1->2->3] > +include/stop_slave.inc > +CHANGE MASTER TO > +MASTER_USE_GTID = SLAVE_POS, IGNORE_SERVER_IDS = (2,3); > +CHANGE MASTER "M_3" TO > +MASTER_HOST = "127.0.0.1", MASTER_PORT = SERVER_MYPORT_3, > +MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > +IGNORE_SERVER_IDS = (1,2); > +START ALL SLAVES; > +Warnings: > +Note 1937 SLAVE 'M_3' started > +Note 1937 SLAVE '' started > +include/stop_slave.inc > +CHANGE MASTER "M_1" TO > +MASTER_HOST = "127.0.0.1", MASTER_PORT = MASTER_MYPORT, > +MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > +IGNORE_SERVER_IDS = (2,3); > +CHANGE MASTER TO > +MASTER_USE_GTID = SLAVE_POS, IGNORE_SERVER_IDS = (1,3); > +START ALL SLAVES; > +Warnings: > +Note 1937 SLAVE 'M_1' started > +Note 1937 SLAVE '' started > +CHANGE MASTER "M_2" TO > +MASTER_HOST = "127.0.0.1", MASTER_PORT = SLAVE_MYPORT, > +MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > +IGNORE_SERVER_IDS = (1,3); > +CHANGE MASTER "M_3" TO > +MASTER_HOST = "127.0.0.1", MASTER_PORT = SERVER_MYPORT_3, > +MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > +IGNORE_SERVER_IDS = (1,2); > +START ALL SLAVES; > +Warnings: > +Note 1937 SLAVE 'M_3' started > +Note 1937 SLAVE 'M_2' started > +CREATE TABLE t1 (a INT PRIMARY KEY, b VARCHAR(10)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES (1, "m1"); > +INSERT INTO t1 VALUES (2, "m2"), (3, "m3"), (4, "m4"); > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +SELECT * FROM t1 ORDER BY a; > +a b > +1 m1 > +2 m2 > +3 m3 > +4 m4 > +include/sync_with_master_gtid.inc > +SELECT * FROM t1 ORDER BY a; > +a b > +1 m1 > +2 m2 > +3 m3 > +4 m4 > +# Cleanup > +DROP TABLE t1; > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +include/sync_with_master_gtid.inc > +STOP SLAVE "M_3"; > +RESET SLAVE "M_3" ALL; > +STOP SLAVE "M_1"; > +RESET SLAVE "M_1" ALL; > +STOP SLAVE "M_2"; > +RESET SLAVE "M_2" ALL; > +STOP SLAVE "M_3"; > +RESET SLAVE "M_3" ALL; > +include/sync_with_master_gtid.inc > +STOP SLAVE; > +CHANGE MASTER TO MASTER_USE_GTID = NO, IGNORE_SERVER_IDS = (); > +include/sync_with_master_gtid.inc > +STOP SLAVE; > +CHANGE MASTER TO MASTER_USE_GTID = NO, IGNORE_SERVER_IDS = (); > +# End of test. > diff --git a/mysql-test/suite/rpl/t/rpl_gtid_mdev9033.cnf > b/mysql-test/suite/rpl/t/rpl_gtid_mdev9033.cnf > new file mode 100644 > index 0000000..80b6649 > --- /dev/null > +++ b/mysql-test/suite/rpl/t/rpl_gtid_mdev9033.cnf > @@ -0,0 +1,20 @@ > +!include ../my.cnf > + > +[mysqld.1] > +log-slave-updates > +loose-innodb > +gtid_domain_id=1 > + > +[mysqld.2] > +log-slave-updates > +loose-innodb > +gtid_domain_id=2 > + > +[mysqld.3] > +log-slave-updates > +loose-innodb > +gtid_domain_id=3 > + > +[ENV] > +SERVER_MYPORT_3= @mysqld.3.port > +SERVER_MYSOCK_3= @mysqld.3.socket > diff --git a/mysql-test/suite/rpl/t/rpl_gtid_mdev9033.test > b/mysql-test/suite/rpl/t/rpl_gtid_mdev9033.test > new file mode 100644 > index 0000000..8b51d96 > --- /dev/null > +++ b/mysql-test/suite/rpl/t/rpl_gtid_mdev9033.test > @@ -0,0 +1,92 @@ > +--source include/have_innodb.inc > +--let $rpl_topology=1->2->3 > +--source include/rpl_init.inc > + > +--connection server_2 > +--source include/stop_slave.inc > +eval CHANGE MASTER TO > + MASTER_USE_GTID = SLAVE_POS, IGNORE_SERVER_IDS = (2,3); > +--replace_result $SERVER_MYPORT_3 SERVER_MYPORT_3 > +eval CHANGE MASTER "M_3" TO > + MASTER_HOST = "127.0.0.1", MASTER_PORT = $SERVER_MYPORT_3, > + MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > + IGNORE_SERVER_IDS = (1,2); > +START ALL SLAVES; > + > +--connection server_3 > +--source include/stop_slave.inc > +--replace_result $MASTER_MYPORT MASTER_MYPORT > +eval CHANGE MASTER "M_1" TO > + MASTER_HOST = "127.0.0.1", MASTER_PORT = $MASTER_MYPORT, > + MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > + IGNORE_SERVER_IDS = (2,3); > +eval CHANGE MASTER TO > + MASTER_USE_GTID = SLAVE_POS, IGNORE_SERVER_IDS = (1,3); > +START ALL SLAVES; > + > +--connection server_1 > +--replace_result $SLAVE_MYPORT SLAVE_MYPORT > +eval CHANGE MASTER "M_2" TO > + MASTER_HOST = "127.0.0.1", MASTER_PORT = $SLAVE_MYPORT, > + MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > + IGNORE_SERVER_IDS = (1,3); > +--replace_result $SERVER_MYPORT_3 SERVER_MYPORT_3 > +eval CHANGE MASTER "M_3" TO > + MASTER_HOST = "127.0.0.1", MASTER_PORT = $SERVER_MYPORT_3, > + MASTER_USER = "root", MASTER_USE_GTID = SLAVE_POS, > + IGNORE_SERVER_IDS = (1,2); > +START ALL SLAVES; > + > +connection server_1; > + > +CREATE TABLE t1 (a INT PRIMARY KEY, b VARCHAR(10)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES (1, "m1"); > +INSERT INTO t1 VALUES (2, "m2"), (3, "m3"), (4, "m4"); > + > +--source include/save_master_gtid.inc > + > +connection server_2; > +--source include/sync_with_master_gtid.inc > +let $wait_condition=SELECT COUNT(*)=4 FROM t1; > +--source include/wait_condition.inc > +SELECT * FROM t1 ORDER BY a; > + > +connection server_3; > +--source include/sync_with_master_gtid.inc > +--source include/wait_condition.inc > +SELECT * FROM t1 ORDER BY a; > + > +--echo # Cleanup > +--connection server_1 > +DROP TABLE t1; > +--source include/save_master_gtid.inc > +--connection server_2 > +--source include/sync_with_master_gtid.inc > +--connection server_3 > +--source include/sync_with_master_gtid.inc > + > +--connection server_2 > +STOP SLAVE "M_3"; > +RESET SLAVE "M_3" ALL; > + > +--connection server_3 > +STOP SLAVE "M_1"; > +RESET SLAVE "M_1" ALL; > + > +--connection server_1 > +STOP SLAVE "M_2"; > +RESET SLAVE "M_2" ALL; > +STOP SLAVE "M_3"; > +RESET SLAVE "M_3" ALL; > + > +--connection server_2 > +--source include/sync_with_master_gtid.inc > +STOP SLAVE; > +CHANGE MASTER TO MASTER_USE_GTID = NO, IGNORE_SERVER_IDS = (); > + > +--connection server_3 > +--source include/sync_with_master_gtid.inc > +STOP SLAVE; > +CHANGE MASTER TO MASTER_USE_GTID = NO, IGNORE_SERVER_IDS = (); > + > +--echo # End of test. > diff --git a/sql/slave.cc b/sql/slave.cc > index fed171b..b93fb3d 100644 > --- a/sql/slave.cc > +++ b/sql/slave.cc > @@ -5870,22 +5870,6 @@ static int queue_event(Master_info* mi,const char* > buf, ulong event_len) > } > };); > > - if (mi->using_gtid != Master_info::USE_GTID_NO && > - mi->domain_id_filter.is_group_filtered() && > - mi->events_queued_since_last_gtid > 0 && > - ((mi->last_queued_gtid_standalone && > - !Log_event::is_part_of_group((Log_event_type)(uchar) > - buf[EVENT_TYPE_OFFSET])) || > - (!mi->last_queued_gtid_standalone && > - ((uchar)buf[EVENT_TYPE_OFFSET] == XID_EVENT || > - ((uchar)buf[EVENT_TYPE_OFFSET] == QUERY_EVENT && > - Query_log_event::peek_is_commit_rollback(buf, event_len, > - checksum_alg)))))) > - { > - /* Reset the domain_id_filter flag. */ > - mi->domain_id_filter.reset_filter(); > - } > - > if (mi->using_gtid != Master_info::USE_GTID_NO && mi->gtid_event_seen) > { > if (unlikely(mi->gtid_reconnect_event_skip_count)) > @@ -6071,6 +6055,9 @@ static int queue_event(Master_info* mi,const char* > buf, ulong event_len) > */ > mi->gtid_current_pos.update(&mi->last_queued_gtid); > mi->events_queued_since_last_gtid= 0; > + > + /* Reset the domain_id_filter flag. */ > + mi->domain_id_filter.reset_filter(); > } > > skip_relay_logging: > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

[Maria-developers] On the issue of Seconds_behind_master and Parallel Replication
by Kristian Nielsen 16 Nov '15

16 Nov '15

It was brought to my attention an issue with parallel replication and the Seconds_Behind_Master field of SHOW SLAVE STATUS. I have a possible patch for this, but I wanted to discuss it on the list, as it changes semantics compared to the non-parallel case. Each binlog event contains a timestamp (**) of when the event was created on the master. Whenever the slave SQL thread reads an event from the relay log, it updates the value of Seconds_Behind_Master to the difference between the slave's current time and the event's timestamp. Now in parallel replication, the SQL thread can read a large number of events from the relay log and queue them in-memory for the worker threads. So a small value of Seconds_Behind_Master means only that recent events have been queued - it might still be a long time before the worker threads have had time to actually execute all the queued events. Apparently the problem is (justified) user confusion about this queuing delay not being reflected in Seconds_Behind_Master. The same problem actually exists in the non-parallel case. In case of a large transaction, the Seconds_Behind_Master can be small even though there is still a large amount of execution time remaining for the transaction to complete on the slave. However, in the non-parallel case, at most one transaction can be involved. In the parallel case, the problem is amplified by the potential of thousands of queued transactions awaiting execution. So how to solve it? Attached is a patch that implements one possible solution: the Seconds_Behind_Master is only updated after a transaction commits, with the timestamp of the commit events. This seems more intuitive anyway. But it does introduce a semantic difference between the non-parallel and parallel behaviour for Seconds_Behind_Master. The value will in general be larger on a parallel slave than on a non-parallel slave, for the same actual slave lag. Monty suggested changing the behaviour also for non-parallel mode - letting Seconds_Behind_Master reflect only events actually committed, not just read from the relay log. This would introduce an incompatible behaviour for Seconds_Behind_Master, but could perhaps be done for 10.1, if desired. Doing it in stable 10.0 would be more drastic. So any opinions on this? - Should Seconds_Behind_Master be changed as per above in parallel replication (from 10.0 on)? - If not, any suggestion for another semantics for Seconds_Behind_Master in parallel replication? - If so, should the change to Seconds_Behind_Master also be done in the non-parallel case in 10.1? What about 10.0? - Any comments on the patch? Here is a test case that shows the problem. It simulates a slave catching up with a bunch of INSERT(SLEEP(1)). Without my patch, Seconds_Behind_Master=2 just after starting the parallel slave, while the non-parallel slave shows Seconds_Behind_Master=9. With my patch, the parallel slave shows Seconds_Behind_Master=11. ----------------------------------------------------------------------- --source include/have_innodb.inc --source include/have_binlog_format_statement.inc --let $rpl_topology=1->2 --source include/rpl_init.inc # Test various aspects of parallel replication. --connection server_1 ALTER TABLE mysql.gtid_slave_pos ENGINE=InnoDB; CALL mtr.add_suppression("Unsafe statement written to the binary log using statement format"); CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; --save_master_pos --connection server_2 --sync_with_master --source include/stop_slave.inc SET GLOBAL slave_parallel_threads=1; --connection server_1 --disable_warnings INSERT INTO t1 VALUES (1, SLEEP(1)); INSERT INTO t1 VALUES (2, SLEEP(1)); INSERT INTO t1 VALUES (3, SLEEP(1)); INSERT INTO t1 VALUES (4, SLEEP(1)); INSERT INTO t1 VALUES (5, SLEEP(1)); INSERT INTO t1 VALUES (6, SLEEP(1)); INSERT INTO t1 VALUES (7, SLEEP(1)); INSERT INTO t1 VALUES (8, SLEEP(1)); INSERT INTO t1 VALUES (9, SLEEP(1)); INSERT INTO t1 VALUES (10, SLEEP(1)); --disable_warnings --save_master_pos --connection server_2 --source include/start_slave.inc SELECT SLEEP(1); --let $status_items= Seconds_Behind_Master --source include/show_slave_status.inc --sync_with_master --let $status_items= Seconds_Behind_Master --source include/show_slave_status.inc # Clean up. --connection server_2 --source include/stop_slave.inc SET GLOBAL slave_parallel_threads=DEFAULT; --source include/start_slave.inc --connection server_1 DROP TABLE t1; --source include/rpl_end.inc ----------------------------------------------------------------------- Thanks, - Kristian. ** The full detail is a bit more complex. Some events actually contains the time when the corresponding query _started_ executing, and another value which is the duration of execution. These are added by the slave to get the timestamp for the creation of the event (at the end of query exection). Also, the slave adjusts the timestamp with any time difference between the master and slave clock (eg. time zone for example).

6 10

[Maria-developers] MDEV-8109 unexpected CAST result
by Alexander Barkov 11 Nov '15

11 Nov '15

Hi Serg, Please review my patch for MDEV-8109. Thanks!

2 1

[Maria-developers] Please review MDEV-8140 patch
by chengxiaoz 10 Nov '15

10 Nov '15

Hi kentoku and Sergey, Please review mypatch for MDEV-8140. https://github.com/MariaDB/server/pull/61 Thanks.

3 3

[Maria-developers] Please review MDEV-8140 patch
by chengxiaoz＠189.cn 08 Nov '15

08 Nov '15

1 0

Re: [Maria-developers] [Commits] 7931ca4: MDEV-9024 Build fails with VS2015
by Sergei Golubchik 05 Nov '15

05 Nov '15

Hi, Wlad! Mostly ok, please see below On Nov 03, wlad(a)mariadb.com wrote: > revision-id: 7931ca4f89ad5cd38ea3dc54f1363b7de2fd3414 (mariadb-10.0.22-5-g7931ca4) > parent(s): fa1438cbf4307731a54ea4137d5f7d4b744cdfbc > committer: Vladislav Vaintroub > timestamp: 2015-11-03 11:32:00 +0100 > message: > > MDEV-9024 Build fails with VS2015 > > Fix build failures caused by new C runtime library > - isnan, snprintf, struct timespec are now defined, attempt to redefine them leads > - P_tmpdir, tzname are no more defined > - lfind() and lsearch() in lf_hash.c had to be renamed, declaration > conflicts with some C runtime functions with the same name declared > in a header included by stdlib.h > > Also fix couple of annoying warnings : > - remove #define NOMINMAX from config.h to avoid "redefined" compiler > warnings(NOMINMAX is already in compile flags) > > - disable incremental linker in Debug as well (feature not used much > and compiler crashes often) > > > Also simplify package building with Wix, require Wix 3.9 or later > (VS2015 is not compatible with old Wix 3.5/3.6) > > diff --git a/cmake/os/Windows.cmake b/cmake/os/Windows.cmake > index 4c01bab..fd6e739 100644 > --- a/cmake/os/Windows.cmake > +++ b/cmake/os/Windows.cmake > @@ -181,14 +184,13 @@ CHECK_SYMBOL_REPLACEMENT(S_IROTH _S_IREAD sys/stat.h) > CHECK_SYMBOL_REPLACEMENT(S_IFIFO _S_IFIFO sys/stat.h) > CHECK_SYMBOL_REPLACEMENT(SIGQUIT SIGTERM signal.h) > CHECK_SYMBOL_REPLACEMENT(SIGPIPE SIGINT signal.h) > -CHECK_SYMBOL_REPLACEMENT(isnan _isnan float.h) How can this be wrong? It checks whetherone should use _isnan instead of isnan. That seems like the kind of check that works both with old and new VS. > CHECK_SYMBOL_REPLACEMENT(finite _finite float.h) > diff --git a/config.h.cmake b/config.h.cmake > index 46eed79..a1d505f 100644 > --- a/config.h.cmake > +++ b/config.h.cmake > > -/* We don't want the min/max macros */ > -#ifdef __WIN__ > -#define NOMINMAX > -#endif I'd generally prefer to define symbols in headers, instead of putting everything into ever-growing compiler command line Besides, the minimal fix for the redefinition warning is -#define NOMINMAX +#define NOMINMAX 1 because -DNOMINMAX defines NOMINMAX to be 1, not empty. > /* > MySQL features > */ > diff --git a/mysys/lf_hash.c b/mysys/lf_hash.c > index aa96ca9..fa92971 100644 > --- a/mysys/lf_hash.c > +++ b/mysys/lf_hash.c > @@ -43,7 +43,7 @@ const int LF_HASH_OVERHEAD= sizeof(LF_SLIST); > > /* > a structure to pass the context (pointers two the three successive elements > - in a list) from lfind to linsert/ldelete > + in a list) from my_lfind to my_linsert/my_ldelete > */ > typedef struct { > intptr volatile *prev; > @@ -70,7 +70,7 @@ typedef struct { > cursor is positioned in either case > pins[0..2] are used, they are NOT removed on return > */ > -static int lfind(LF_SLIST * volatile *head, CHARSET_INFO *cs, uint32 hashnr, > +static int my_lfind(LF_SLIST * volatile *head, CHARSET_INFO *cs, uint32 hashnr, er... I don't like that. lfind/lsearch are internal static functions in lf_hash.c, they shouldn't need or use my_ prefix. Two alternatives: 1. #define lfind remap_lfind_for_win #include <my_global.h> #include <my_sys.h> #undef lfind 2. -static int lfind(LF_SLIST * volatile *head, CHARSET_INFO *cs, uint32 hashnr, +static int l_find(LF_SLIST * volatile *head, CHARSET_INFO *cs, uint32 hashnr, > const uchar *key, uint keylen, CURSOR *cursor, LF_PINS *pins) > { > uint32 cur_hashnr; > diff --git a/win/packaging/CMakeLists.txt b/win/packaging/CMakeLists.txt > index 3b9b72f..0535a48 100644 > --- a/win/packaging/CMakeLists.txt > +++ b/win/packaging/CMakeLists.txt > SET(CPACK_WIX_UPGRADE_CODE "49EB7A6A-1CEF-4A1E-9E89-B9A4993963E3") > - SET(CPACK_WIX_PACKAGE_NAME "MariaDB @MAJOR_VERSION@.@MINOR_VERSION@") > + SET(CPACK_WIX_PACKAGE_NAME "MariaDB ${MAJOR_VERSION}.${MINOR_VERSION}") > ELSE() > SET(CPACK_WIX_UPGRADE_CODE "2331E7BD-EE58-431B-9E18-B2B918BCEB1B") > - SET(CPACK_WIX_PACKAGE_NAME "MariaDB @MAJOR_VERSION@.@MINOR_VERSION@ (x64)") > + SET(CPACK_WIX_PACKAGE_NAME "MariaDB ${MAJOR_VERSION}.${MINOR_VERSION} (x64)") Weird. I was sure I've fixed this months ago :-[] Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 01bd373: MDEV-8437 - plugin variables conflict with bootstrap
by Sergei Golubchik 05 Nov '15

05 Nov '15

Hi, Sergey! On Nov 03, Sergey Vojtovich wrote: > revision-id: 01bd3737777f8f0cf2e1737d01a48033a7615b2a (mariadb-10.1.8-23-g01bd373) > parent(s): dc40d737d4af3cb0e272c58507ad54e4eb24be28 > committer: Sergey Vojtovich > timestamp: 2015-11-03 14:02:49 +0400 > message: > > MDEV-8437 - plugin variables conflict with bootstrap ok to push > There is no strong need to change password column: the only side effect is > that 4.0 -> 10.1 upgrades may get root/debian-sys-maint passwords stored in old > format. This should be perfectly acceptable, since all password at this point > are stored in old format. Regards, Sergei

2 3

Re: [Maria-developers] [Commits] 178cddd: MDEV-9082 - Debian: mysql_install_db is called on upgrade
by Sergei Golubchik 05 Nov '15

05 Nov '15

Hi, Sergey! On Nov 05, Sergey Vojtovich wrote: > revision-id: 178cddd664cb4ce2c6ccbe76c33e4fd2c1fc1b3d (mariadb-10.1.8-25-g178cddd) > parent(s): 2ef562d2c74297b0b92c4322697dad98767cc873 > committer: Sergey Vojtovich > timestamp: 2015-11-05 13:57:24 +0400 > message: > > MDEV-9082 - Debian: mysql_install_db is called on upgrade looks ok to me Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 2ef562d: MDEV-9080 - Debian: incorrect empty password check in postinst
by Sergei Golubchik 05 Nov '15

05 Nov '15

Hi, Sergey! ok to push On Nov 04, Sergey Vojtovich wrote: > revision-id: 2ef562d2c74297b0b92c4322697dad98767cc873 (mariadb-10.1.8-24-g2ef562d) > parent(s): 01bd3737777f8f0cf2e1737d01a48033a7615b2a > committer: Sergey Vojtovich > timestamp: 2015-11-04 20:37:16 +0400 > message: > > MDEV-9080 - Debian: incorrect empty password check in postinst > > There was code that was supposed to "catch upgrades from previous versions where > the root password wasn't set". But it is wrong in many regards: > - it is supposed to be executed against running server, but at this point server > should be down, which makes this code no-op > - if the above is fixed, root password will be requested twice (initial root > password request + this one) > - it asks for a password only once, while "initial root password request" asks > twice (password + password verification) > - it may give false positive if unix socket based authentication is in effect > > Removed this code since it didn't work for quite a while (at least since > mysql-5.1) and nobody cared about it. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] dc40d73: MDEV-8437 - plugin variables conflict with bootstrap
by Sergei Golubchik 05 Nov '15

05 Nov '15

Hi, Sergey! On Nov 03, Sergey Vojtovich wrote: > revision-id: dc40d737d4af3cb0e272c58507ad54e4eb24be28 (mariadb-10.1.8-22-gdc40d73) > parent(s): a574407444fc3ea93ca88afa93dc18154251bf74 > committer: Sergey Vojtovich > timestamp: 2015-11-03 12:08:23 +0400 > message: > > MDEV-8437 - plugin variables conflict with bootstrap ok to push > Removed redundant attempt to create mysql.plugin table: > - original code was supposed to INSTALL some plugins: > INSERT INTO plugin VALUES ('innodb', 'ha_innodb.so'), > ('federated', 'ha_federated.so'), ('blackhole', 'ha_blackhole.so'), > ('archive', 'ha_archive.so'); I didn't see any INSERT's in the postinst script. > - original code was supposed to fail if mysql.plugin exists: > The query sequence is supposed to be aborted if the CREATE TABLE fails due > to an already existent table in which case the admin might already have > chosen to remove one or more plugins. > - mysql.plugin must've been created by preceeding mysql_install_db anyway > Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 4685c90: mariadb@.service.in should be configured for all systemd platforms.
by Sergey Vojtovich 05 Nov '15

05 Nov '15

Hi Nirbhay, taking into account that CPackRPM won't be able to install it, what's the purpose of having this configured for all systemd platforms? Regards, Sergey On Tue, Nov 03, 2015 at 05:38:12PM -0500, Nirbhay Choubey wrote: > revision-id: 4685c903bc822596bcf12cbfdb5bcc4460ed0105 > parent(s): 95289e5b665d6a3e37f938d0db978d0d0166e493 > committer: Nirbhay Choubey > branch nick: 10.1 > timestamp: 2015-11-03 17:38:11 -0500 > message: > > mariadb@.service.in should be configured for all systemd platforms. > > --- > support-files/CMakeLists.txt | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/support-files/CMakeLists.txt b/support-files/CMakeLists.txt > index 2ab4350..2929db1 100644 > --- a/support-files/CMakeLists.txt > +++ b/support-files/CMakeLists.txt > @@ -79,9 +79,9 @@ IF(UNIX) > IF(HAVE_SYSTEMD) > CONFIGURE_FILE(mariadb.service.in > ${CMAKE_CURRENT_BINARY_DIR}/mariadb.service @ONLY) > + CONFIGURE_FILE(mariadb@.service.in > + ${CMAKE_CURRENT_BINARY_DIR}/mariadb@.service @ONLY) > IF(NOT RPM) > - CONFIGURE_FILE(mariadb@.service.in > - ${CMAKE_CURRENT_BINARY_DIR}/mariadb@.service @ONLY) > INSTALL(FILES use_galera_new_cluster.conf > ${CMAKE_CURRENT_BINARY_DIR}/mariadb.service > ${CMAKE_CURRENT_BINARY_DIR}/mariadb@.service > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 2

[Maria-developers] write_set.diff review
by Sergei Golubchik 04 Nov '15

04 Nov '15

Hi, Summary: 1. Three questions (are you sure, what guarantee, why) 2. A couple of suggestions (spilt a commit, move function call out of DBUG_RETURN) 3. Two suggestions for unrelated optimizations (don't allocate unneded bitmaps, don't modify read_set) > commit 2734eca05d499bcbcaf428d04d4f1acbb298589f > Author: Monty <monty(a)mariadb.org> > Date: Mon Nov 2 11:52:06 2015 +0200 > > diff --git a/sql/table.h b/sql/table.h > index b150e4f..6ecfdef 100644 > --- a/sql/table.h > +++ b/sql/table.h > @@ -1066,9 +1066,11 @@ struct TABLE > String alias; /* alias or table name */ > uchar *null_flags; > MY_BITMAP def_read_set, def_write_set, def_vcol_set, tmp_set; > + MY_BITMAP def_rpl_write_set; I would rather not allocate more bitmaps directly in the TABLE. They are not always needed, and only increase the TABLE size. The same is true for def_vcol_set, in fact, it's *even more true* for vcols. Instead I'd allocate them in the TABLE's memroot as needed. > MY_BITMAP eq_join_set; /* used to mark equi-joined fields */ > MY_BITMAP cond_set; /* used to mark fields from sargable conditions*/ > - MY_BITMAP *read_set, *write_set, *vcol_set; /* Active column sets */ > + /* Active column sets */ > + MY_BITMAP *read_set, *write_set, *vcol_set, *rpl_write_set; > /* > The ID of the query that opened and is using this table. Has different > meanings depending on the table type. > diff --git a/sql/table.cc b/sql/table.cc > index 76cb4b8..8816ee0 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -2884,20 +2884,28 @@ enum open_frm_error open_table_from_share(THD *thd, TABLE_SHARE *share, > /* Allocate bitmaps */ > > bitmap_size= share->column_bitmap_size; > - if (!(bitmaps= (uchar*) alloc_root(&outparam->mem_root, bitmap_size*6))) > + if (!(bitmaps= (uchar*) alloc_root(&outparam->mem_root, bitmap_size*7))) > goto err; > my_bitmap_init(&outparam->def_read_set, > (my_bitmap_map*) bitmaps, share->fields, FALSE); > my_bitmap_init(&outparam->def_write_set, > - (my_bitmap_map*) (bitmaps+bitmap_size), share->fields, FALSE); > + (my_bitmap_map*) (bitmaps+bitmap_size), share->fields, > + FALSE); > my_bitmap_init(&outparam->def_vcol_set, > - (my_bitmap_map*) (bitmaps+bitmap_size*2), share->fields, FALSE); > + (my_bitmap_map*) (bitmaps+bitmap_size*2), share->fields, > + FALSE); This is totally wasted memory if the table has no virtual columns. (that's what I meant by *even more true* - rpl bitmap and eq_join_set, cond_set bitmaps may or may not be used for this table, one never knows. But for vcols it's always known in advance whether the table has them or not). Still, I'd allocate other bitmaps on demand too. > my_bitmap_init(&outparam->tmp_set, > - (my_bitmap_map*) (bitmaps+bitmap_size*3), share->fields, FALSE); > + (my_bitmap_map*) (bitmaps+bitmap_size*3), share->fields, > + FALSE); > my_bitmap_init(&outparam->eq_join_set, > - (my_bitmap_map*) (bitmaps+bitmap_size*4), share->fields, FALSE); > + (my_bitmap_map*) (bitmaps+bitmap_size*4), share->fields, > + FALSE); > my_bitmap_init(&outparam->cond_set, > - (my_bitmap_map*) (bitmaps+bitmap_size*5), share->fields, FALSE); > + (my_bitmap_map*) (bitmaps+bitmap_size*5), share->fields, > + FALSE); > + my_bitmap_init(&outparam->def_rpl_write_set, > + (my_bitmap_map*) (bitmaps+bitmap_size*6), share->fields, > + FALSE); > outparam->default_column_bitmaps(); > > outparam->cond_selectivity= 1.0; > @@ -5986,36 +5997,48 @@ void TABLE::mark_columns_needed_for_insert() > we only want to log a PK and we needed other fields for > execution). > */ > + > void TABLE::mark_columns_per_binlog_row_image() > { > + THD *thd= in_use; > DBUG_ENTER("mark_columns_per_binlog_row_image"); > DBUG_ASSERT(read_set->bitmap); > DBUG_ASSERT(write_set->bitmap); > > - THD *thd= current_thd; Hmm. Are you sure that TABLE::in_use is *always* set at this point? You've basically replaced - THD *thd=current_thd; - if (in_use && in_use->is_current_stmt_binlog_format_row) + THD *thd=in_use; + if (thd->is_current_stmt_binlog_format_row) And, btw, where's your MDEV for caching rbr checks per statement? > + /* If not using row format */ > + rpl_write_set= write_set; > > /** > If in RBR we may need to mark some extra columns, > depending on the binlog-row-image command line argument. > */ > if ((WSREP_EMULATE_BINLOG(thd) || mysql_bin_log.is_open()) && > - in_use && > - in_use->is_current_stmt_binlog_format_row() && > + thd->is_current_stmt_binlog_format_row() && > !ha_check_storage_engine_flag(s->db_type(), HTON_NO_BINLOG_ROW_OPT)) > { > /* if there is no PK, then mark all columns for the BI. */ > if (s->primary_key >= MAX_KEY) > + { > bitmap_set_all(read_set); > + rpl_write_set= read_set; > + } > + else > + { > switch (thd->variables.binlog_row_image) { > case BINLOG_ROW_IMAGE_FULL: > - if (s->primary_key < MAX_KEY) > bitmap_set_all(read_set); > - bitmap_set_all(write_set); > + /* Set of columns that should be written (all) */ > + rpl_write_set= read_set; > break; > case BINLOG_ROW_IMAGE_NOBLOB: > - /* for every field that is not set, mark it unless it is a blob */ > + /* Only write changed columns + not blobs */ > + rpl_write_set= &def_rpl_write_set; > + bitmap_copy(rpl_write_set, write_set); > + > + /* > + for every field that is not set, mark it unless it is a blob or > + part of a primary key > + */ > for (Field **ptr=field ; *ptr ; ptr++) > { > Field *my_field= *ptr; > @@ -6027,23 +6050,24 @@ void TABLE::mark_columns_per_binlog_row_image() > If set in the AI, then the blob is really needed, there is > nothing we can do about it. > */ > - if ((s->primary_key < MAX_KEY) && > - ((my_field->flags & PRI_KEY_FLAG) || > - (my_field->type() != MYSQL_TYPE_BLOB))) > + if ((my_field->flags & PRI_KEY_FLAG) || > + (my_field->type() != MYSQL_TYPE_BLOB)) > + { > bitmap_set_bit(read_set, my_field->field_index); > - > - if (my_field->type() != MYSQL_TYPE_BLOB) > - bitmap_set_bit(write_set, my_field->field_index); > + bitmap_set_bit(rpl_write_set, my_field->field_index); > } > + } > break; > case BINLOG_ROW_IMAGE_MINIMAL: > - /* mark the primary key if available in the read_set */ > - if (s->primary_key < MAX_KEY) > + /* mark the primary key */ > mark_columns_used_by_index_no_reset(s->primary_key, read_set); > + /* Only write columns that have changed */ > + rpl_write_set= write_set; Where's the guarantee that the primary key is in the write set? > break; > > default: > DBUG_ASSERT(FALSE); > + } > } > file->column_bitmaps_signal(); > } > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 860426c..6201065 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -11140,26 +11138,21 @@ int QUICK_RANGE_SELECT::reset() > int QUICK_RANGE_SELECT::get_next() > { > range_id_t dummy; > + int result; > + DBUG_ENTER("QUICK_RANGE_SELECT::get_next"); > + > + if (!in_ror_merged_scan) > + DBUG_RETURN(file->multi_range_read_next(&dummy)); please don't put function calls inside DBUG_RETURN, always use a variable to store the return value first > + > MY_BITMAP * const save_read_set= head->read_set; > MY_BITMAP * const save_write_set= head->write_set; > - > - DBUG_ENTER("QUICK_RANGE_SELECT::get_next"); > - if (in_ror_merged_scan) > - { > /* > We don't need to signal the bitmap change as the bitmap is always the > same for this head->file > */ > head->column_bitmaps_set_no_signal(&column_bitmap, &column_bitmap); > - } > - > - int result= file->multi_range_read_next(&dummy); > + result= file->multi_range_read_next(&dummy); > - > - if (in_ror_merged_scan) > - { > - /* Restore bitmaps set on entry */ > head->column_bitmaps_set_no_signal(save_read_set, save_write_set); > - } > DBUG_RETURN(result); > } Ok, all changes in opt_range.cc are unrelated to your rpl_write_set patch, please commit them separately > diff --git a/sql/rpl_record.cc b/sql/rpl_record.cc > index ee222f0..208b2b6 100644 > --- a/sql/rpl_record.cc > +++ b/sql/rpl_record.cc > @@ -434,7 +433,7 @@ unpack_row(rpl_group_info *rgi, > *master_reclength = table->s->reclength; > } > > - DBUG_RETURN(error); > + DBUG_RETURN(0); > } > > /** also unrelated to rpl_write_set. If you'd like you can put it in the same commit with opt_range.cc changes, with the sybject "cleanups" or something > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 5054357..b238e89 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -6259,11 +6248,11 @@ int THD::binlog_delete_row(TABLE* table, bool is_trans, > DBUG_ASSERT(is_current_stmt_binlog_format_row() && > ((WSREP(this) && wsrep_emulate_bin_log) || mysql_bin_log.is_open())); > /** > - Save a reference to the original read and write set bitmaps. > - We will need this to restore the bitmaps at the end. > + Save a reference to the original read bitmaps > + We will need this to restore the bitmaps at the end as > + binlog_prepare_row_images() may changetable->read_set. > */ This doesn't make much sense. Here, the code is like that THD::binlog_delete_row: save table->read_set THD::binlog_prepare_row_images calculate table->tmp_set table->read_set=table->tmp_set pack_row(table->read_set) for every set bit in the set_arg call field->pack() restore table->read_set So, two thoughts here: 1. tmp_set must be a strict subset of read_set! other fields may not be present in the record[0] and there're no row reads between binlog_prepare_row_images and field->pack 2. One can pass table->tmp_set directly into pack_row() as an argument which means there's no need for binlog_prepare_row_images to modify table->read_set at all > MY_BITMAP *old_read_set= table->read_set; > - MY_BITMAP *old_write_set= table->write_set; > > /** > This will remove spurious fields required during execution but > diff --git a/sql/sql_insert.cc b/sql/sql_insert.cc > index 488826a..0661e70 100644 > --- a/sql/sql_insert.cc > +++ b/sql/sql_insert.cc > @@ -2867,5 +2867,7 @@ pthread_handler_t handle_delayed_insert(void *arg) > /* Tell client that the thread is initialized */ > mysql_cond_signal(&di->cond_client); > > + di->table->mark_columns_needed_for_insert(); why? > + > /* Now wait until we get an insert or lock to handle */ > /* We will not abort as long as a client thread uses this thread */ Regards, Sergei

1 0

[Maria-developers] Python Storage Engine
by Kim Ebert 02 Nov '15

02 Nov '15

Greetings All, I wanted to share the beginnings of a project. This is a Python Storage Engine. The idea is to plumb in the necessary C to Python calls to allow a storage engine to be written in Python instead of C or C++. This may be valuable to those who want to prototype a database engine in Python, or if the performance is sufficient for the use case, for production. I haven't taken things to far yet, but I think it gives a rough idea of how to move forward with adding the other calls. In my particular case I'm using it to prototype connecting to a search API, like Solr, to provide an SQL like interface. I did have a quick question. I want to license it as a more permissive license, but since I'm using the example engine, does that mean that I need to use the GPL license? Or can I go with the more permissive MIT-new license? https://github.com/perfectsearch/mariadb-plugin-python-engine Thoughts and/or feedback are/is appreciated. Thanks, Kim

3 2

Re: [Maria-developers] MDEV-8692 prefschema test failures on ARM (on Debian build system)
by Sergei Golubchik 28 Oct '15

28 Oct '15

Hi, Alexander! On Oct 23, Alexander Barkov wrote: > Hi, > > - the variant with MY_PERF_CYCLES generally works, but does not fix the > test failure. What do you mean by that? How does it "work" if it does not fix the test failure? > - the variant with MY_CLOCK_REALTIME_FOR_CYCLES works fine > and fixed the problem. > > But it's not really "CYCLES". Perhaps this is not a problem, > because on some other platforms we also fallback cycles to nanoseconds, That commit 49667f0 falls back from cycles to nanoseconds explicitly. So it's ok not to support cycles, as far as I understand, and there's no need to do nanoseconds in place of cycles like your MY_CLOCK_REALTIME_FOR_CYCLES. By the way, why 49667f0 didn't fix the problem? What happens on ARMv7 - no 'cycles' timer at all in perfschema, or there is one and it works incorrectly? > diff --git a/mysql-test/suite/perfschema/t/func_file_io.test b/mysql-test/suite/perfschema/t/func_file_io.test > index 2055fa9..5894cb0 100644 > --- a/mysql-test/suite/perfschema/t/func_file_io.test > +++ b/mysql-test/suite/perfschema/t/func_file_io.test > @@ -43,6 +43,11 @@ TRUNCATE TABLE performance_schema.events_waits_current; > > SELECT * FROM t1 WHERE id = 1; > > +--vertical_results > +SELECT * FROM performance_schema.events_waits_history_long > +WHERE EVENT_NAME = 'wait/io/file/myisam/dfile'; > +--horizontal_results forgotten debug output? > + > SET @before_count = (SELECT SUM(TIMER_WAIT) > FROM performance_schema.events_waits_history_long > WHERE (EVENT_NAME = 'wait/io/file/myisam/dfile') > diff --git a/mysys/my_rdtsc.c b/mysys/my_rdtsc.c > index 028c7f8..a5ed808 100644 > --- a/mysys/my_rdtsc.c > +++ b/mysys/my_rdtsc.c > @@ -533,6 +590,10 @@ void my_timer_init(MY_TIMER_INFO *mti) > mti->cycles.routine= MY_TIMER_ROUTINE_ASM_GCC_SPARC32; > #elif defined(__sgi) && defined(HAVE_CLOCK_GETTIME) && defined(CLOCK_SGI_CYCLE) > mti->cycles.routine= MY_TIMER_ROUTINE_SGI_CYCLE; > +#elif defined(MY_CLOCK_REALTIME_FOR_CYCLES) > + mti->cycles.routine= MY_TIMER_ROUTINE_CLOCK_GETTIME; > +#elif defined(MY_PERF_CYCLES) > + mti->cycles.routine= MY_TIMER_ROUTINE_SGI_CYCLE; I'd say that if we'll have it pushed, you'll need to add a dedicated MY_TIMER_ROUTINE_xxx for this, not reuse MY_TIMER_ROUTINE_SGI_CYCLE. > #elif defined(HAVE_SYS_TIMES_H) && defined(HAVE_GETHRTIME) > mti->cycles.routine= MY_TIMER_ROUTINE_GETHRTIME; > #else Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 72bb358: MDEV-7930 Assertion `table_share->tmp_table != NO_TMP_TABLE || m_lock_type != 2' failed in handler::ha_index_read_map
by Sergey Petrunia 27 Oct '15

27 Oct '15

Hi Sanja, Please find some cosmetic feedback below. Ok to push after it has been addressed. On Mon, Oct 26, 2015 at 05:00:34PM +0100, sanja(a)mariadb.com wrote: > revision-id: 72bb358c2ba98e6e0d4a15b92af514a45751fe8d (mariadb-10.0.21-44-g72bb358) > parent(s): 18f7dfed179204dcfc02a27790e22bb9cc4e2e32 > committer: Oleksandr Byelkin > timestamp: 2015-10-26 16:59:40 +0100 > message: > > MDEV-7930 Assertion `table_share->tmp_table != NO_TMP_TABLE || m_lock_type != 2' failed in handler::ha_index_read_map > > In optimizing aggregate function do not try to touch tables from outer > query. > ... > diff --git a/sql/opt_sum.cc b/sql/opt_sum.cc > index fc3ce09..ad29229 100644 > --- a/sql/opt_sum.cc > +++ b/sql/opt_sum.cc > @@ -341,7 +341,8 @@ int opt_sum_query(THD *thd, > there are no outer joins. > */ > if (!conds && !((Item_sum_count*) item)->get_arg(0)->maybe_null && > - !outer_tables && maybe_exact_count) > + !outer_tables && maybe_exact_count && > + ((item->used_tables() & OUTER_REF_TABLE_BIT) == 0)) > { > if (!is_exact_count) > { > @@ -369,7 +370,8 @@ int opt_sum_query(THD *thd, > indexes to find the key. > */ > Item *expr=item_sum->get_arg(0); > - if (expr->real_item()->type() == Item::FIELD_ITEM) > + if (((expr->used_tables() & OUTER_REF_TABLE_BIT) == 0) && > + expr->real_item()->type() == Item::FIELD_ITEM) > { > uchar key_buff[MAX_KEY_LENGTH]; > TABLE_REF ref; > @@ -394,16 +396,20 @@ int opt_sum_query(THD *thd, > const_result= 0; > break; > } > + /* > + Constant table is already read and the only row is the max > + and the min > + */ I think this comment is not relevant anymore? It's part of the first variant of the fix, which is no longer in the patch. Please remove it. > if (!(error= table->file->ha_index_init((uint) ref.key, 1))) > - error= (is_max ? > + error= (is_max ? > get_index_max_value(table, &ref, range_fl) : > get_index_min_value(table, &ref, item_field, range_fl, > prefix_len)); > > /* Verify that the read tuple indeed matches the search key */ > - if (!error && reckey_in_range(is_max, &ref, item_field->field, > - conds, range_fl, prefix_len)) > - error= HA_ERR_KEY_NOT_FOUND; > + if (!error && reckey_in_range(is_max, &ref, item_field->field, > + conds, range_fl, prefix_len)) > + error= HA_ERR_KEY_NOT_FOUND; These are just endspace changes. Can you remove them? > table->disable_keyread(); > table->file->ha_index_end(); > if (error) > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Several CVE's in Oracle MySQL, is MariaDB vulnerable?
by Christian Rebischke 26 Oct '15

26 Oct '15

Hello everyone, Sorry when I am on the wrong mailinglist. I wanted to submit a bugreport or issue for this but I can't signup for your JIRA because of license reasons. I am from the Archlinux-Security Team and want to ask if mariadb in the actual version is vulnerable to the following CVEs: CVE-2015-4913 CVE-2015-4910 CVE-2015-4905 CVE-2015-4904 CVE-2015-4895 CVE-2015-4890 CVE-2015-4879 CVE-2015-4870 CVE-2015-4862 CVE-2015-4864 CVE-2015-4861 CVE-2015-4858 CVE-2015-4836 CVE-2015-4833 CVE-2015-4830 CVE-2015-4826 CVE-2015-4819 CVE-2015-4815 CVE-2015-4807 CVE-2015-4802 CVE-2015-4800 CVE-2015-4792 CVE-2015-4791 CVE-2015-4766 I hope you can help me. Best regards -------------------------------------------------------------- Christian Rebischke Member of Archlinux CVE-Monitoring Team Website : www.nullday.de Twitter : @sh1bumi Jabber : shibumi(a)jabber.ccc.de PGP : 0xDFE2060D Fingerprint: 6DAF 7B80 8F9D F251 3962 0000 D214 61E3 DFE2 060D --------------------------------------------------------------

4 13

[Maria-developers] crosscompiling in buildroot: cannot find -laio and other link error
by Sylvain Raybaud 23 Oct '15

23 Oct '15

Hi I'm trying to get the latest version of mariadb-galera (10.0.21) into buildroot. I'm running into two errors which I have a feeling are related. But I may be wrong. These problems arise during the build of the host variant of the package. I get the two following errors (build does not stop at the first one because they happen in parallel jobs): 1: Linking CXX shared module ha_connect.so /home/sylvain/git/br/output/host/usr/arm-buildroot-linux-uclibcgnueabi/sysroot/usr/lib/libxml2.so: error adding symbols: File in wrong format --> for host variant ha_connect.so should be linked against /home/sylvain/git/br/output/host/usr/lib/libxml2.so 2: Linking CXX shared module ha_innodb.so /usr/bin/ld: cannot find -laio The thing is, libaio and libxml2 are both present in /home/sylvain/git/br/output/host/usr/lib/. The link command present in storage/connect/CMakeFiles/connect.dir/link.txt (http://pastebin.com/Qn1ZCw2X) contains: /home/sylvain/git/br/output/host/usr/arm-buildroot-linux-uclibcgnueabi/sysroot/usr/lib/libxml2.so which is incorrect, should be: /home/sylvain/git/br/output/host/usr/lib/libxml2.so The link command present in storage/innobase/CMakeFiles/innobase.dir/link.txt (http://pastebin.com/it6W9WVs) does not contain -L/home/sylvain/git/br/output/host/usr/lib/. I think it should. At least, when I manually run this command adding -L/home/sylvain/git/br/output/host/usr/lib, it works. I'm really not sure how these link.txt are generated. I checked connect/CMakeLists.txt and innobase/CMakeLists.txt and could not find anything obvious problem. Beside, toplevel CMakeCache.txt does contain: CMAKE_EXE_LINKER_FLAGS:STRING=-L/home/sylvain/git/br/output/host/lib -L/home/sylvain/git/br/output/host/usr/lib -Wl,-rpath,/home/sylvain/git/br/output/host/usr/lib which seems correct. I don't know why this setting does not reach storage/innobase and I'm also not sure why ha_connect.so tries to link against target variant of libxml2. It basically comes down to have cmake look for shared libraries in the right place, but I don't seem to be able to determine which lever to pull. Any hint as to how to achieve that is therefore welcome. Cheers, -- Sylvain Raybaud www.green-communications.fr

2 7

[Maria-developers] innodb holepunch compression vs the filesystem
by MARK CALLAGHAN 21 Oct '15

21 Oct '15

InnoDB punch-hole compression is very bad for a filesystem when done on every (or most) page writes. That feature was not designed to be used in that manner and generates far too much in-memory metadata in the filesystem. If nothing else it leads to multi-minute unlink() system calls. Yet both MariaDB and MySQL proudly exclaim this as a feature. Are you going to use the same kludge as MySQL (do unlink() in a background thread) to avoid locking up InnoDB for many minutes during drop table, maybe leading to InnoDB killing mysqld process? Why didn't you catch this problem via testing or reading my blog posts? https://mariadb.com/kb/en/mariadb/compression/ https://blog.mariadb.org/mariadb-10-1-is-stable-ga/ https://bugs.mysql.com/bug.php?id=78277 http://smalldatum.blogspot.com/2015/08/first-day-with-innodb-transparent-pa… http://smalldatum.blogspot.com/2015/09/second-day-with-innodb-transparent-p… http://smalldatum.blogspot.com/2015/09/third-day-with-innodb-transparent-pa… http://smalldatum.blogspot.com/2015/10/wanted-file-system-on-which-innodb.h… -- Mark Callaghan mdcallag(a)gmail.com

2 1

Re: [Maria-developers] Deadlock in Parallel Replication with FLUSH TABLE WITH READ LOCK.
by Kristian Nielsen 20 Oct '15

20 Oct '15

> I enabled // replication on the slave that has MyISAM tables. > The first results are good, but I ended up with what looks like a deadlock. > I opened support # 10289 for "Deadlock in Parallel Replication with > FLUSH TABLE WITH READ LOCK". There is a patch for this: https://lists.launchpad.net/maria-developers/msg08675.html But no reply on the review request so far. - Kristian.

1 0

[Maria-developers] Code review process
by Otto Kekäläinen 19 Oct '15

19 Oct '15

Hello! During the developer meetup there was a discussion about Gerrit. OpenStack has quite a nice documentation how to contribute, including the git review command and gerrit workflow: http://docs.openstack.org/infra/manual/developers.html -- Otto Kekäläinen MariaDB Foundation

1 0

Re: [Maria-developers] [Maria-discuss] Moving from Launchpad lists to something else... ?
by Tom Worster 19 Oct '15

19 Oct '15

On 10/15/15, 5:03 AM, "Maria-discuss on behalf of Colin Charles" <maria-discuss-bounces+fsb=thefsb.org(a)lists.launchpad.net on behalf of colin(a)mariadb.org> wrote: >> On 15 Oct 2015, at 05:35, Jean Weisbuch <jean(a)phpnet.org> wrote: >> >> To be able to give the best answer we must first know what is >>motivating the move from LP. > >So the reason we were thinking of ditching launchpad for mailing lists: >1. When there was a MariaDB-Manager google group, that group got a lot of >user posts about MariaDB Server. That group now no longer exists, but it >gave us a hint that people are interested in a Google group > >2. We moved everything from Launchpad (bugs to Jira, code to Github) and >the only thing remaining is mailing lists > >3. The MariaDB Knowledgebase: https://mariadb.com/kb/en/ used to have an >OpenID login that would be useful if you had a LP account, but OpenID is >apparently going away (or gone), so there’s less use for LP > >4. LaunchPad itself seems to be “hard” for people to signup (see #1). Good points. Making it easy for people to get involved is important, but if that's the idea then moving to Google Groups isn't a great solution. Apart from it being old and clunky owing to Google's neglect, the insurmountable problem is that it's still an email list at core. Personally, I like email lists best but I'm very old and have been in computing since the 70s, and I accept that most of the rest of the world has moved on. Another problem the Maria community might consider together with the discuss email list is the low quality and participation in the web Q&A of the Knowledge Base. That sometimes makes me sad. There is now a decent open-source system that can function as email list replacement and provide web and mobile Q&A, as well as other stuff: https://www.discourse.org I've used it at New Relic and it seems pretty good. It's got a lot of good stuff and I want to highlight just two. 1. "Mailing List Support - Opt into a special mode where all messages are sent to you via email, exactly like a mailing list. Start new topics via email." 2. You can host it yourself or pay someone you trust to do so. It's not a "free" cloud service where you pay by feeding the internet advertising/surveillance economy. I don't know if Discourse is right for Maria but I wouldn't mention it if I didn't think it might help build a bigger and more active support community, which I long for. Tom

2 1

[Maria-developers] JSON in MariaDB 10.2
by Colin Charles 14 Oct '15

14 Oct '15

Hi all, We just had a session here at the MariaDB Server developer’s meeting in Amsterdam on JSON in MariaDB 10.2. Before forgetting while things are still fresh in the mind, here are the notes from it (in Markdown format). cheers, -colin # JSON in MariaDB * Virtual columns with functional indexes (i.e. generated columns like in 5.7) can be implemented in 2-3 weeks (Sergei) * Eric: Concern about using non-standard function names (functions in 5.7 are `JSN_INSERT` whereas the standard uses `JSON_QUERY`) * Suggestion: Use standard names when available, create UDF to match MySQL names that can be loaded as desired. (or vice versa -- do what MySQL 5.7 does and have UDFs to follow the standard) * Come up with Usable JSON Test suite * Employee DB in JSON? * Good tests exist in DocStore (from the 5.6 facebook tree) * Monty believes we should have more strictly typed (DocStore for example says `unique key doc_c(doc.address.zipcode as int, c)`). Monty thinks it should be optional to keep compatibility with 5.7. This is also similar to the JSON proposed standard (where you have to choose between lax or strict mode) * Colin: Might want to convert JSON document to a relational table (This is part of the standard as `JSON_TABLE`) * Colin: Do we support something similar to what FB has or what oracle has? * SergeyP: We need to follow oracle to maintain compatibility. * Colin: We need to do this soon (i.e. since 5.7 is coming and its going to be a problem for 10.1 not having it; so 10.2 needs it as soon as possible). Everyone generally agrees this is a good idea. # References * [SQL SUPPORT FOR JSON](http://jtc1bigdatasg.nist.gov/_workshop/08_SQL_Support_for_JSON_abstr… * [WL#8132: JSON datatype and binary storage format](https://dev.mysql.com/worklog/task/?id=8132) * [WL#7909: Server side JSON functions](https://dev.mysql.com/worklog/task/?id=7909) * [WL#8249: JSON comparator](https://dev.mysql.com/worklog/task/?id=8249) * [WL#8170: Expression analyzer for GC](https://dev.mysql.com/worklog/task/?id=8170) * [WL#8149: B-tree Index Support on non-materialized virtual columns](https://dev.mysql.com/worklog/task/?id=8149) * [WL#8227: Support SEs to create index on virtual generated columns](https://dev.mysql.com/worklog/task/?id=8227) * [WL#8114: Don't store virtual generated columns in database](https://dev.mysql.com/worklog/task/?id=8114) * [JSON Labs Release: Native JSON Data Type and Binary Format](http://mysqlserverteam.com/json-labs-release-native-json-data-type-… * [JSON Labs Release: JSON Functions, Part 1 — Manipulation JSON Data](http://mysqlserverteam.com/json-labs-release-json-functions-part-1-ma… * [JSON Labs Release: JSON Functions, Part 2 — Querying JSON Data](http://mysqlserverteam.com/mysql-5-7-lab-release-json-functions-part-… * [JSON Labs Release: Effective Functional Indexes in InnoDB](http://mysqlserverteam.com/json-labs-release-effective-functional-i… * [11.6 The JSON Data Type](https://dev.mysql.com/doc/refman/5.7/en/json.html) * [DocStore: Document Database for MySQL at Facebook](https://www.percona.com/live/mysql-conference-2015/sites/default/… * [JSON Support in MySQL](https://www.percona.com/live/mysql-conference-2015/sites/default/fil… # Thanks Thanks to Bryan Alsdorf for writing some of the notes during the session -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/ | t: +1-347-903-3201 | Skype: colincharles | Twitter: @bytebot

1 0

Re: [Maria-developers] [Commits] 5f461a9: MDEV-7640: CHANGE MASTER TO doesn't work with prepared statements
by Oleksandr Byelkin 11 Oct '15

11 Oct '15

Hi! On 17.09.15 00:10, Nirbhay Choubey wrote: > revision-id: 5f461a94a5aa2e1a20fa559ca612d96b53201e4e > parent(s): 29ac245dd04c5416d57a90b0ab3c4d08cbc4d723 > committer: Nirbhay Choubey > branch nick: 5.5 > timestamp: 2015-09-16 18:10:06 -0400 > message: > > MDEV-7640: CHANGE MASTER TO doesn't work with prepared statements > > When CHANGE MASTER was executed as a PS, its attributes were wrongly > getting reset toward the end of PREPARE. As a result, the subsequent > executions had no effect. Fixed by making sure that the CHANGE MASTER > attributes are preserved during the lifetime of the PS. It is OK to push but after expanding the comment about what exactly reset in the persistent statement memory (as we agreed) [skip]

1 0

Re: [Maria-developers] [Commits] 06e3c05: MDEV-8842 add group support to pam_user_map module.
by Sergei Golubchik 07 Oct '15

07 Oct '15

Hi, Holyfoot! On Oct 07, holyfoot(a)askmonty.org wrote: > revision-id: 06e3c05f1ad3156b5a3e0a775b07fb13b9ebb1a8 (mariadb-10.1.7-68-g06e3c05) > parent(s): 8afe96f011eb8037a92b4b3aab16118b0771ad50 > committer: Alexey Botchkov > timestamp: 2015-10-07 15:52:26 +0500 > message: > > MDEV-8842 add group support to pam_user_map module. > Added to the pam_user_map module. Great, thanks! Ok to push. I assume you've tested that it still works after all the changes. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 50dc8c0: MDEV-8842 add group support to pam_user_map module.
by Sergei Golubchik 07 Oct '15

07 Oct '15

Hi, Holyfoot! On Oct 07, holyfoot(a)askmonty.org wrote: > revision-id: 50dc8c0e8a27d18a3d75ff43a87975fcd5e0e7f6 (mariadb-10.1.7-67-g50dc8c0) > parent(s): bed4e847950eef50930b44632eea43416e7b37d1 > committer: Alexey Botchkov > timestamp: 2015-10-07 00:51:33 +0500 > message: > > MDEV-8842 add group support to pam_user_map module. > Added to the pam_user_map module. Looks pretty much ok. Just one suggestion: I'd keep groups in the pam_sm_authenticate and passed the pointer down to user_in_group. Indeed, there may many @group entries in the mapping file, seems like a waste do repopulate group array every time. > --- > plugin/auth_pam/mapper/pam_user_map.c | 59 +++++++++++++++++++++++++++++++++-- > 1 file changed, 57 insertions(+), 2 deletions(-) > > diff --git a/plugin/auth_pam/mapper/pam_user_map.c b/plugin/auth_pam/mapper/pam_user_map.c > index e73ab6d..a3008bd 100644 > --- a/plugin/auth_pam/mapper/pam_user_map.c > +++ b/plugin/auth_pam/mapper/pam_user_map.c > @@ -13,22 +13,71 @@ auth required pam_user_map.so > > And create /etc/security/user_map.conf with the desired mapping > in the format: orig_user_name: mapped_user_name > + @user's_group_name: mapped_user_name > ========================================================= > -#comments and emty lines are ignored > +#comments and emtpy lines are ignored > john: jack > bob: admin > top: accounting > +@group_ro: readonly > ========================================================= > > */ > > +#include <stdlib.h> > #include <stdio.h> > #include <syslog.h> > +#include <grp.h> > +#include <pwd.h> > +#include <limits.h> > #include <security/pam_modules.h> > > #define FILENAME "/etc/security/user_map.conf" > #define skip(what) while (*s && (what)) s++ > > +#define GROUP_BUFFER_SIZE 100 > +int user_in_group(const char *user, const char *group) > +{ > + gid_t group_buffer[GROUP_BUFFER_SIZE]; > + gid_t *groups= group_buffer; > + gid_t group_id; > + gid_t user_group_id; > + int ng, i; > + > + { > + struct passwd *pw= getpwnam(user); > + struct group *g= getgrnam(group); > + if (pw == NULL || g == NULL) > + return 0; > + user_group_id= pw->pw_gid; > + group_id= g->gr_gid; > + } > + > + ng= GROUP_BUFFER_SIZE; > + if (getgrouplist(user, user_group_id, groups, &ng) < 0) > + { > + /* The rare case when the user is present in more than */ > + /* GROUP_BUFFER_SIZE groups. */ > + groups= (gid_t *) malloc(ng * sizeof (gid_t)); > + if (!groups) > + return 0; > + > + (void) getgrouplist(user, user_group_id, groups, &ng); > + } > + > + for (i= 0; i < ng; i++) > + { > + if (groups[i] == group_id) > + break; > + } > + > + if (groups != group_buffer) > + free(groups); > + > + return i < ng; > +} > + > + > int pam_sm_authenticate(pam_handle_t *pamh, int flags, > int argc, const char *argv[]) > { > @@ -51,10 +100,14 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > while (fgets(buf, sizeof(buf), f) != NULL) > { > char *s= buf, *from, *to, *end_from, *end_to; > + int check_group; > + > line++; > > skip(isspace(*s)); > if (*s == '#' || *s == 0) continue; > + if ((check_group= *s == '@')) > + s++; > from= s; > skip(isalnum(*s) || (*s == '_')); > end_from= s; > @@ -67,7 +120,9 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > if (end_to == to) goto syntax_error; > > *end_from= *end_to= 0; > - if (strcmp(username, from) == 0) > + if (check_group ? > + user_in_group(username, from) : > + (strcmp(username, from) == 0)) > { > pam_err= pam_set_item(pamh, PAM_USER, to); > goto ret; Regards, Sergei

2 1

Re: [Maria-developers] [Commits] c934b5e: MDEV-8842 add group support to pam_user_map module.
by Sergei Golubchik 07 Oct '15

07 Oct '15

Hi, Holyfoot! On Oct 06, holyfoot(a)askmonty.org wrote: > revision-id: c934b5e661b19b7934d1a9af40afc70b727c2273 (mariadb-10.1.7-64-gc934b5e) > parent(s): 90f2c822469a4f88f882ba8974e790f0fb0b2702 > committer: Alexey Botchkov > timestamp: 2015-10-06 14:02:29 +0500 > message: > > MDEV-8842 add group support to pam_user_map module. > Added to the pam_user_map module. Right, thanks. A couple of comments, see below. > diff --git a/plugin/auth_pam/mapper/pam_user_map.c b/plugin/auth_pam/mapper/pam_user_map.c > index e73ab6d..0787b1c 100644 > --- a/plugin/auth_pam/mapper/pam_user_map.c > +++ b/plugin/auth_pam/mapper/pam_user_map.c ... > + /* > + So below we get the buffer of the NGROUPS_MAX size > + which can take 256K on some systems. > + It supposed to be working differently - we'd get > + the 'ng' from the getgrouplist() and alloc the respective > + size, but the problem is a bug in getgrouplist(). > + The glibc 2.3.2 implementation of this function is broken: > + it overwrites memory when the actual number of groups is larger > + than *ngroups. > + */ Forget 2.3.2, use it as it was supposed to be used. Our oldest builder uses glibc-2.5 > + (void) getgrouplist(user, user_group_id, groups, &ng); > + for (i= 0; i < ng; i++) > + { > + struct group *g; > + if ((g= getgrgid(groups[i])) == NULL) > + return 0; > + if (g->gr_gid == group_id) > + return 1; Eh? Why don't you compare group_id with groups[i] directly ? Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 7ea735b: MDEV-8431 Feedback plugin needs an option for http proxy.
by Sergei Golubchik 06 Oct '15

06 Oct '15

Hi, Holyfoot! On Oct 05, holyfoot(a)askmonty.org wrote: > revision-id: 7ea735baa2df4f4577cf951bc69ccf87fd91f436 (mariadb-10.1.7-40-g7ea735b) > parent(s): ba0b6685510f4a271da8d386ff75bfd83d3156c2 > committer: Alexey Botchkov > timestamp: 2015-10-05 14:46:12 +0500 > message: > > MDEV-8431 Feedback plugin needs an option for http proxy. > 'feedback_http_proxy' system variable added to specify the > proxy server as host:port. Not a dynamic one. That's a bit more complex that I thought it'll be, but ok. Still - did you test it? How? Regards, Sergei

2 2

Re: [Maria-developers] [Commits] ee656a4: MDEV-7846: postreview fix
by Sergei Golubchik 06 Oct '15

06 Oct '15

Hi, Sanja! I have no idea how this postreview fix relates to any of my review comments. Let me repeat them again: 1. Add comments 2. Add a test case with row items 3. In your test case - if you remove GROUP BY you get a crash again. This might be a different bug, though. On Aug 05, sanja(a)mariadb.com wrote: > revision-id: ee656a4aabb5ebbe973b7e4c99be83eb5dc8c0ce > parent(s): 9f8a458fb2d07298810bb5d9824ce728017bfb32 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-08-05 11:57:35 +0200 > message: > > MDEV-7846: postreview fix > > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 29fff8e..e96d1bc 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -1455,12 +1455,13 @@ bool Item_in_optimizer::fix_left(THD *thd, Item **ref) > next execution we need to copy args[1]->left_expr again. > */ > ref0= &(((Item_in_subselect *)args[1])->left_expr); > + args[0]= ((Item_in_subselect *)args[1])->left_expr; > } > if ((!(*ref0)->fixed && (*ref0)->fix_fields(thd, ref0)) || > (!cache && !(cache= Item_cache::get_cache(*ref0)))) > DBUG_RETURN(1); > if (args[0] != (*ref0)) > - current_thd->change_item_tree(args, (*ref0)); > + args[0]= (*ref0); > DBUG_PRINT("info", ("actual fix fields")); Regards, Sergei

2 1

[Maria-developers] Problem in compiling Maria-db
by indiavaibhav vaibhavindia 02 Oct '15

02 Oct '15

Hello everyone, My name is Vaibhav.I am new to programming for open source. I am facing problems in trying to compile Maria-Db.As instructed on the website, I cloned the maria db source and installed all the files required for compilation. However on using cmake,I am getting an error,screenshot of which is attached is with this mail. Can someone please help me out? Thanks, Vaibhav Choudhary

2 3

Re: [Maria-developers] crosscompiling in buildroot: cannot find -laio and other link error
by Sylvain Raybaud 02 Oct '15

02 Oct '15

Ouch. I'm really sorry, I don't know what happend with my mua and why all these mails were sent to the list. Only the last one counts. Sincere apologies :'( On 02/10/15 17:10, Sylvain Raybaud wrote: > Hi Sergei! > > Thanks for your help. > > On 01/10/15 16:16, Sergei Golubchik wrote: >> Why would you build host variant for ha_connect.so with the library >> paths for the target architecture? > I don't actually, I don't know why this happened. > >> How do you invoke cmake? > The Buildroot system takes care of most of the maging. But... > >> Did you check >> https://mariadb.com/kb/en/mariadb/cross-compiling-mariadb/ ? Regards, >> Sergei > Ooops. For some reason I completely missed this page. Building only the > host variant of import_executables target eases cross-compilation a lot > indeed... Appart from that I had figured most of this out, or buildroot > was taking care of it. I still have a remark: I don't think I can > hardcode the value of > > HAVE_IB_GCC_ATOMIC_BUILTINS > > as suggested in https://lists.launchpad.net/maria-discuss/msg02911.html > because if I want this package accepted in buildroot it must > -- Sylvain Raybaud www.green-communications.fr

1 0

[Maria-developers] MariaDB Developer Meeting in Amsterdam Oct 13-15
by Rasmus Johansson 02 Oct '15

02 Oct '15

Hi maria-developers, There will be a MariaDB Developer Meeting taking place in Amsterdam on October 13-15. It would be great to see many of you there. The main focus of the event is as the title implies the development aspect of MariaDB, but also aspects like documentation, infrastructure and other community related topics will have their place in the event. Also MaxScale developers will be present. If you are interested please sign up by joining and rsvping the event's meetup page here: http://www.meetup.com/MariaDB-Developers/events/224068714/ The session backlog and the schedule for the three days are found here: https://docs.google.com/a/skysql.com/spreadsheets/d/17J_Jloc8I33nsEIE1KCZct… You'll notice that the schedule is not fixed and the idea is the following: - Everyone attending the meeting can affect the agenda by adding sessions to the session backlog - Everyone attending can/should vote on the session of interest in the backlog by raising the count in the Votes column Please feel free to suggest new sessions and remember to vote on the ones you think are most interesting (no limitation on how many you vote for). Best regards, Rasmus -- Rasmus Johansson, VP Engineering MariaDB | t: +358 50 499 9589 | Skype: ratzpo

2 2

Re: [Maria-developers] [Commits] 09eb231: MDEV-8762: file_key_management plugin should be disabled by default when linked statically
by Sergei Golubchik 01 Oct '15

01 Oct '15

Hi, Jan! That's not a good idea. It'd mean that installing a plugin is not enough, it must be explicitly enabled. I don't see any reason why one should be forced to do it. I'll comment in the bugreport. On Oct 01, Jan Lindström wrote: > revision-id: 09eb23148830c53e594cfb88dcb31f1b9e42ae5e (mariadb-10.1.7-33-g09eb231) > parent(s): 3a7cee05d6bac27c3c2f51afb8986c1581649bbc > committer: Jan Lindström > timestamp: 2015-10-01 14:27:10 +0300 > message: > > MDEV-8762: file_key_management plugin should be disabled by default when linked statically > > --- > sql/sql_plugin.cc | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 6d8268e..1d2e2d8 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -211,6 +211,7 @@ static struct > > /* we disable few other plugins by default */ > ,{ "feedback", PLUGIN_OFF } > + ,{ "file_key_management", PLUGIN_OFF } > }; Regards, Sergei

2 2

[Maria-developers] MDEV-8831: review request
by Nirbhay Choubey 01 Oct '15

01 Oct '15

Hi Jan, Could you please review the attached patch? Thanks, Nirbhay

1 0

[Maria-developers] Groups in PAM plugin.
by Alexey Botchkov 28 Sep '15

28 Sep '15

Hi. I came up with a small program that checks if an Unix user belongs to a group 'wheel'. That can be attached to the 'plugin/auth_pam/mapper/pam_user_map.c' in no time if that's what we want. http://myoffice.izhnet.ru/~af/gr.cc An it must be LDAP-compliant AFAIK. Best regards. HF

1 0

[Maria-developers] New Programmer(Maria-DB)
by indiavaibhav vaibhavindia 28 Sep '15

28 Sep '15

Hello everyone, My name is Vaibhav Choudhary.I am in my second year undergraduation and am new to open source. I would like to contribute to your project.Can anyone please guide me as to how should i work on a patch after i have identified the problem on JIRA? I have a good knowledge of C++ and am studying MySQL.But I am unable to figure out as to how should I go about writing the code for the issues on JIRA. Thanks Vaibhav

2 4

Re: [Maria-developers] [Commits] 1d92131: MDEV-4829 BEFORE INSERT triggers dont issue 1406 error.
by Sergei Golubchik 27 Sep '15

27 Sep '15

Hi, Holyfoot! On Sep 24, holyfoot(a)askmonty.org wrote: > revision-id: 1d92131613770b505fa462b33c1091e00d9475be (mariadb-10.1.7-61-g1d92131) > parent(s): e5418942609833edb681d16c4e2705f8c338bfee > committer: Alexey Botchkov > timestamp: 2015-09-24 14:31:06 +0500 > message: > > MDEV-4829 BEFORE INSERT triggers dont issue 1406 error. > Fixed as it's done in MySQL 5.7. > The Strict_error_handler introduced to intercept such states and is activated > in the sp_head::execute_trigger() Hmm... This is, basically, a new implementation of the strict mode. But enabled only for triggers, which is kind of weird. And potentially very fragile and inconsistent, as it's a strict mode we're talking about, it's confusing and not very consistent on itself. Having two different implementations of it doesn't help to make it clear or consistent. If you're introducing a new implementation - use it consistently everywhere. If that won't change existing intentional behavior. And please talk to Bar about his recent experiences in this area. There were also few comments about the code, see below. > diff --git a/sql/sp_head.cc b/sql/sp_head.cc > index a117963..ff6d2b4 100644 > --- a/sql/sp_head.cc > +++ b/sql/sp_head.cc > @@ -54,6 +54,35 @@ > > #include <my_user.h> > > +/** > + A simple RAII wrapper around Strict_error_handler. > +*/ > +class Strict_error_handler_wrapper > +{ > + THD *m_thd; > + Strict_error_handler m_strict_handler; > + bool m_active; > + > +public: > + Strict_error_handler_wrapper(THD *thd, sp_head *sp_head) > + : m_thd(thd), m_active(false) > + { > + if (sp_head->m_sql_mode & (MODE_STRICT_ALL_TABLES | > + MODE_STRICT_TRANS_TABLES)) > + { > + m_thd->push_internal_handler(&m_strict_handler); > + m_active= true; > + } > + } > + > + ~Strict_error_handler_wrapper() > + { > + if (m_active) > + m_thd->pop_internal_handler(); > + } > +}; Oh, really? A wrapper per handler? Make it a template, please, so that it could be used for any error handler. Like this: Error_handler_wraper<Strict_error_handler> strict_error_handler(thd, this); Also, it would be good if you do a second commit using this wrapper in other places, where appropriate. > @@ -1576,6 +1605,9 @@ sp_head::execute_trigger(THD *thd, > DBUG_ENTER("sp_head::execute_trigger"); > DBUG_PRINT("info", ("trigger %s", m_name.str)); > > + // Push Strict_error_handler if the SP was created in STRICT mode. > + Strict_error_handler_wrapper strict_wrapper(thd, this); > + > #ifndef NO_EMBEDDED_ACCESS_CHECKS > Security_context *save_ctx= NULL; > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 9162969..ca0699c 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -96,6 +96,74 @@ bool No_such_table_error_handler::safely_trapped_errors() > > > /** > + Implementation of STRICT mode. > + Upgrades a set of given conditions from warning to error. > +*/ > +bool Strict_error_handler::handle_condition_ext(THD *thd, > + uint sql_errno, > + const char *sqlstate, > + Sql_condition::enum_warning_level *level, > + const char *msg, > + Sql_condition **cond_hdl) > +{ > + /* STRICT MODE should affect only the below statements */ Really? > + switch (thd->lex->sql_command) > + { > + case SQLCOM_CREATE_TABLE: > + case SQLCOM_DROP_INDEX: > + case SQLCOM_INSERT: > + case SQLCOM_REPLACE: > + case SQLCOM_REPLACE_SELECT: > + case SQLCOM_INSERT_SELECT: > + case SQLCOM_UPDATE: > + case SQLCOM_UPDATE_MULTI: > + case SQLCOM_DELETE: > + case SQLCOM_DELETE_MULTI: > + case SQLCOM_ALTER_TABLE: > + case SQLCOM_LOAD: > + case SQLCOM_CALL: > + case SQLCOM_END: > + case SQLCOM_SET_OPTION: > + case SQLCOM_SELECT: > + break; > + default: > + return false; > + } > + > + switch (sql_errno) > + { > + case ER_TRUNCATED_WRONG_VALUE: > + case ER_WRONG_VALUE_FOR_TYPE: > + case ER_WARN_DATA_OUT_OF_RANGE: > + case ER_DIVISION_BY_ZERO: > + case ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: > + case WARN_DATA_TRUNCATED: > + case ER_DATA_TOO_LONG: > + case ER_BAD_NULL_ERROR: > + case ER_NO_DEFAULT_FOR_FIELD: > + case ER_TOO_LONG_KEY: > + case ER_WRONG_ARGUMENTS: > + case ER_NO_DEFAULT_FOR_VIEW_FIELD: > + case ER_WARN_NULL_TO_NOTNULL: > + case ER_CUT_VALUE_GROUP_CONCAT: > + case ER_DATETIME_FUNCTION_OVERFLOW: > + case ER_WARN_TOO_FEW_RECORDS: > + if ((*level == Sql_condition::WARN_LEVEL_WARN) && > + thd->transaction.stmt.m_unsafe_rollback_flags == 0 || > + (thd->variables.sql_mode & MODE_STRICT_ALL_TABLES)) > + { > + (*level)= Sql_condition::WARN_LEVEL_ERROR; > + thd->killed= KILL_BAD_DATA; > + } > + break; > + default: > + break; > + } > + return false; > +} > + > + > +/** > This internal handler is used to trap ER_NO_SUCH_TABLE and > ER_WRONG_MRG_TABLE errors during CHECK/REPAIR TABLE for MERGE > tables. > diff --git a/sql/sql_base.h b/sql/sql_base.h > index 1c0029d..c5a76c5 100644 > --- a/sql/sql_base.h > +++ b/sql/sql_base.h > @@ -676,5 +676,30 @@ class No_such_table_error_handler : public Internal_error_handler > int m_unhandled_errors; > }; > > +/** > + This internal handler implements upgrade from SL_WARNING to SL_ERROR "SL_ERROR"? "SL_WARNING"? > + for the error codes affected by STRICT mode. Currently STRICT mode does > + not affect SELECT statements. > +*/ > + > +class Strict_error_handler : public Internal_error_handler > +{ > +protected: > + bool handle_condition(THD *thd, > + uint sql_errno, > + const char* sqlstate, > + Sql_condition::enum_warning_level level, > + const char* msg, > + Sql_condition ** cond_hdl) > + { DBUG_ASSERT(FALSE); return 0; } > +public: > + bool handle_condition_ext(THD *thd, > + uint sql_errno, > + const char* sqlstate, > + Sql_condition::enum_warning_level *level, > + const char* msg, > + Sql_condition ** cond_hdl); 1. why not to change handle_condition() ? 2. strict mode may change the sql_errno, handle_condition_ext doesn't support it. > +}; > + > > #endif /* SQL_BASE_INCLUDED */ Regards, Sergei

1 2

Re: [Maria-developers] [Commits] d3f10e3: MDEV-8431 Feedback plugin needs an option for http proxy.
by Sergei Golubchik 26 Sep '15

26 Sep '15

Hi, Holyfoot! On Sep 21, holyfoot(a)askmonty.org wrote: > revision-id: d3f10e30ec39bccf1172c7b2eb159f40e412bdfd (mariadb-10.0.21-14-gd3f10e3) > parent(s): abd31ca2b6cde024772fcac0f4953219d76c6344 > committer: Alexey Botchkov > timestamp: 2015-09-21 14:06:15 +0500 > message: > > MDEV-8431 Feedback plugin needs an option for http proxy. > 'feedback_proxy_server' system variable added to specify the > proxy server. Not a dynamic one. How did you test this feature? > diff --git a/plugin/feedback/feedback.cc b/plugin/feedback/feedback.cc > index f644bd5..f76c52a 100644 > --- a/plugin/feedback/feedback.cc > +++ b/plugin/feedback/feedback.cc > @@ -250,6 +251,9 @@ static int init(void *p) > > prepare_linux_info(); > > + proxy_url= proxy_server ? > + Url::create(proxy_server, strlen(proxy_server)) : 0; Some diagnostics would be nice here. At least "feedback: invalid proxy url: '%.*s'". See how ivalid feedback urls are reported. > + > url_count= 0; > if (*url) > { > @@ -347,6 +354,9 @@ static MYSQL_SYSVAR_ULONG(send_timeout, send_timeout, PLUGIN_VAR_RQCMDARG, > static MYSQL_SYSVAR_ULONG(send_retry_wait, send_retry_wait, PLUGIN_VAR_RQCMDARG, > "Wait this many seconds before retrying a failed send.", > NULL, NULL, 60, 1, 60*60*24, 1); > +static MYSQL_SYSVAR_STR(proxy_server, proxy_server, > + PLUGIN_VAR_READONLY | PLUGIN_VAR_RQCMDARG, > + "Proxy server address.", NULL, NULL,0); Mention that it must be an url. "Proxy server URL" or "Proxy server address, in the correct URI format", or something. > static struct st_mysql_sys_var* settings[] = { > MYSQL_SYSVAR(server_uid), > diff --git a/plugin/feedback/url_http.cc b/plugin/feedback/url_http.cc > index f214f7a..eade67c 100644 > --- a/plugin/feedback/url_http.cc > +++ b/plugin/feedback/url_http.cc > @@ -148,9 +148,12 @@ int Url_http::send(const char* data, size_t data_length) > my_socket fd= INVALID_SOCKET; > char buf[1024]; > uint len= 0; > + Url_http *proxy_loc= (Url_http *) proxy_url; > > addrinfo *addrs, *addr, filter= {0, AF_UNSPEC, SOCK_STREAM, 6, 0, 0, 0, 0}; > - int res= getaddrinfo(host.str, port.str, &filter, &addrs); > + int res= proxy_url ? > + getaddrinfo(proxy_loc->host.str, proxy_loc->port.str, &filter, &addrs) : > + getaddrinfo(host.str, port.str, &filter, &addrs); > First, you don't know whether proxy_url is Url_http. It might be easier to have an option --feedback-http-proxy=host:port (not the url). It also solves the potential future confusion if we'll have more than one transport. Second, I don't think proxies work this way at all, for a http proxy, you need to specify the host in th GET request, like POST http://foo.bar/bla/bla So, I think, a better implementation would be to take proxy into account not in Url_http::send(), but already in http_create(). And create Url_http objects for feedback url with proxy's host and port, and with the appropriately adjusted path. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] efcbbed: MDEV-8682 - CSV engine does not properly process "", in quotes
by Sergei Golubchik 26 Sep '15

26 Sep '15

Hi, Sergey! On Sep 22, Sergey Vojtovich wrote: > revision-id: efcbbed977dfcd9ee0ed035cbb0df4404483b0c8 (mariadb-10.1.7-53-gefcbbed) > parent(s): 7cbecadf9b59d06f4d9a9e776f99418b273c2ebc > committer: Sergey Vojtovich > timestamp: 2015-09-22 16:52:30 +0400 > message: > > MDEV-8682 - CSV engine does not properly process "", in quotes > > Added per-table boolean IETF_QUOTES variable to CSV storage engine. It allows to > enable IETF-compatible parsing of embedded quote and comma characters. Disabled > by default. > > This patch is based on Percona revision: > https://github.com/percona/percona-server/commit/b32fbf02766f8db00f9ff4e6dc… > > Note that original patch adds server variable, while this patch adds per-table > variable. I agree that a per-table option is safer. Ok to push. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 5f461a9: MDEV-7640: CHANGE MASTER TO doesn't work with prepared statements
by Kristian Nielsen 21 Sep '15

21 Sep '15

nirbhay(a)mariadb.com (Nirbhay Choubey) writes: > revision-id: 5f461a94a5aa2e1a20fa559ca612d96b53201e4e > parent(s): 29ac245dd04c5416d57a90b0ab3c4d08cbc4d723 > committer: Nirbhay Choubey > branch nick: 5.5 > timestamp: 2015-09-16 18:10:06 -0400 > message: > > MDEV-7640: CHANGE MASTER TO doesn't work with prepared statements > > When CHANGE MASTER was executed as a PS, its attributes were wrongly > getting reset toward the end of PREPARE. As a result, the subsequent > executions had no effect. Fixed by making sure that the CHANGE MASTER > attributes are preserved during the lifetime of the PS. Hm, this MDEV seems now assigned to me, but I did not see any mail from you about it? Did you expect a review? But in any case, this code around prepare/execute is not something I am familiar with. I seem to recall that Sanja worked on several issues related to incorrect prepare/execute, maybe he can have some ideas or suggest a reviewer. Apparently this is something that used to work but started to break sometimes in 5.5. Do you know how the problem was introduced? - Kristian (sadder by the hour over devs not using the mailing list).

2 1

[Maria-developers] Super Read Only in MariaDB (10.1) ?
by Jean-François Gagné 19 Sep '15

19 Sep '15

Hi, a super_read_only option is making its way in MySQL 5.7: * http://www.tocker.ca/2015/07/09/mysql-5-7-8-now-featuring-super_read_only-a… * http://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_… Are there plans to have a similar option in MariaDB (it would be great to have it in MariaDB 10.1) ? Many thanks, -- Jean-François Gagné System Engineer Booking.com BV Rembrandt Square Office Herengracht 597, 1017 CE Amsterdam

2 1

[Maria-developers] New programmer
by Acha Bill 18 Sep '15

18 Sep '15

Hello guys, I'm Acha Bill. Final year undergraduate in Computer Engineering. I would like to be an active contributor to your ongoing projects. I have good knowledge of C/C++ and MySQL. As this is my first approach to open source, any guidance will be much appreciated. Bill.

3 3

Re: [Maria-developers] Please review MDEV-8468 MDEV-8466
by Sergei Golubchik 17 Sep '15

17 Sep '15

Hi, Alexander! On Jul 21, Alexander Barkov wrote: > > I have one question about "HANDLER FOR SQLWARNING". > It does not catch messages of level "Note". > Looks like a bug. Please confirm, I'll file a report if so. Hmm, I don't know. According to the standard, SQLWARNING should match sqlstates in the category W, while SQLEXCEPTION should be for sqlstates in the category X. But we don't do that, 22007 is an exception (category X), but we treat it as a warning. So, the standard doesn't help to answer your question. We seem to use the logic where a level "error" means an SQL exception and a level "warning" means an SQL warning. Using this logic, "notes" are neither exceptions nor warnings, and neither SQLEXCEPTION nor SQLWARNING should match them. I'd say it's not a bug. The patch itself looks great, these changes in behavior are very welcomed. Just one small comment regarding THD::check_edom_and_truncation(), see below. > diff --git a/sql/sql_class.h b/sql/sql_class.h > index a8d8444..c19174b 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -3270,6 +3270,53 @@ class THD :public Statement, > (!transaction.stmt.modified_non_trans_table || > (variables.sql_mode & MODE_STRICT_ALL_TABLES))); > } > + /** > + Check string-to-number conversion and produce a warning if > + - could not convert any digits (EDOM-alike error) > + - found garbage at the end of the string > + See also Field_num::check_edom_and_truncation() for a similar function. > + > + @param thd - the thread > + @param type - name of the data type (e.g. "INTEGER", "DECIMAL", "DOUBLE") > + @param edom - of a EDOM-alike error happened > + @param cs - character set of the original string > + @param str - the original string > + @param end_of_num - the position where the string-to-number routine stopped. > + @param end - the end of the string > + > + Unlike Field_num::check_edom_and_truncation(), this function does not > + distinguish between EDOM and truncation and reports the same warning for > + both cases. Perhaps we should eventually print different warnings, to make > + the explicit CAST work closer to the implicit cast in Field_xxx::store(). Why haven't you done it now? It causes too many changes in tests? > + */ > + void check_edom_and_truncation(const char *type, bool edom, > + CHARSET_INFO *cs, > + const char *str, > + const char *end_of_num, > + const char *end) Move this out of THD please. You don't need to call current_thd every time for it, but only when a warning should be issued. > + { > + DBUG_ASSERT(str <= end_of_num); > + DBUG_ASSERT(end_of_num <= end); > + if (edom || > + (end_of_num < end && !check_if_only_end_space(cs, end_of_num, end))) > + { > + /* > + We can use err.ptr() here as ErrConvString is guranteed to put an > + end \0 here. > + */ > + push_warning_printf(this, Sql_condition::WARN_LEVEL_WARN, > + ER_TRUNCATED_WRONG_VALUE, > + ER(ER_TRUNCATED_WRONG_VALUE), type, > + ErrConvString(str, end - str, cs).ptr()); > + } > + else if (end_of_num < end) > + { > + push_warning_printf(this, Sql_condition::WARN_LEVEL_NOTE, > + ER_TRUNCATED_WRONG_VALUE, > + ER(ER_TRUNCATED_WRONG_VALUE), type, > + ErrConvString(str, end - str, cs).ptr()); > + } > + } Regards, Sergei

2 1

[Maria-developers] unsubscribe
by Abdul Hakeem 16 Sep '15

16 Sep '15

1 0

Re: [Maria-developers] [Commits] a1f31cf: MDEV-6756: map a linux pid (child pid) to a connection id shown in the output of SHOW PROCESSLIST
by Sergei Golubchik 16 Sep '15

16 Sep '15

Hi, Sanja! Looks good, thanks. Ok to push! On Sep 15, sanja(a)mariadb.com wrote: > revision-id: a1f31cfa8d0f04cb166813786f7854db3dbcf715 (mariadb-10.1.6-12-ga1f31cf) > parent(s): 86a3613d4e981c341e38291c9eeec5dc9f836fae > committer: Oleksandr Byelkin > timestamp: 2015-09-15 21:50:28 +0200 > message: > > MDEV-6756: map a linux pid (child pid) to a connection id shown in the > output of SHOW PROCESSLIST > > Added tid (thread ID) for system where it is present. > > ps -eL -o tid,pid,command > > shows the thread on Linux Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 8bf23ef: MDEV-8208: Sporadic SEGFAULT on startup
by Sergei Golubchik 16 Sep '15

16 Sep '15

Hi, Nirbhay! On Aug 19, Nirbhay Choubey wrote: > revision-id: 8bf23efc914f18c687067e9314cadf4f36a85eec > parent(s): 71d1f35847a575239deff856590bf6f13afd74ed > committer: Nirbhay Choubey > branch nick: 5.5-galera.b8208 > timestamp: 2015-08-19 17:17:56 -0400 > message: > > MDEV-8208: Sporadic SEGFAULT on startup A couple of questions: > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 2008fc5..4c05999 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -4584,6 +4579,29 @@ static int init_server_components() > } > plugins_are_initialized= TRUE; /* Don't separate from init function */ > > +#ifdef WITH_WSREP > + /* Wait for wsrep threads to get created. */ > + if (wsrep_creating_startup_threads == 1) { > + mysql_mutex_lock(&LOCK_thread_count); > + while (wsrep_running_threads < 2) > + { > + mysql_cond_wait(&COND_thread_count, &LOCK_thread_count); > + } > + > + /* Now is the time to initialize threads for queries. */ > + THD *tmp; > + I_List_iterator<THD> it(threads); > + while ((tmp= it++)) > + if (tmp->wsrep_applier == true) > + tmp->init_for_queries(); Do you really need to iterate over all THD's? There is only one applier thread, right? And it's THD is not stored anywhere, only in the global list? > + mysql_mutex_unlock(&LOCK_thread_count); > + } > +#endif > + > have_csv= plugin_status(STRING_WITH_LEN("csv"), > MYSQL_STORAGE_ENGINE_PLUGIN); > have_ndbcluster= plugin_status(STRING_WITH_LEN("ndbcluster"), > @@ -4890,12 +4902,21 @@ pthread_handler_t start_wsrep_THD(void *arg) > //thd->version= refresh_version; > thd->proc_info= 0; > thd->command= COM_SLEEP; > - thd->set_time(); > - thd->init_for_queries(); > + > + if (plugins_are_initialized) > + { > + thd->init_for_queries(); > + } What happens if plugins are not initialized yet? thd->init_for_queries() will be done in that THD loop above? In that case why do you need to do it here? Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 6ed7307: MDEV-8576: Bootstrap should ignore --enforce-storage-engine option
by Sergei Golubchik 16 Sep '15

16 Sep '15

Hi, Jan! On Sep 16, Jan Lindström wrote: > revision-id: 6ed73070e120e1be5456288556019db5bd3f9f60 > parent(s): 6cc1befcf26b2dc43e5cd7ad85d29c576073b1c5 > committer: Jan Lindström > branch nick: 10.1-tests > timestamp: 2015-09-16 11:47:34 +0300 > message: > > MDEV-8576: Bootstrap should ignore --enforce-storage-engine option > > Allow enforce-storage-engine="" option and use that on mysql_install_db > when doing bootstrap. Agree. After I commented on the previous patch it occurred to me that --disable-enforce-storage-engine is a bad idea, because --enable-enforce-storage-engine makes no sense, and one normally expects --enable-xxx for every --disable-xxx. So --enforce-storage-engine="" is a better way of switching this feature off. > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index b45d4e1..ee3b870 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -4843,7 +4843,10 @@ static void add_file_to_crash_report(char *file) > static int init_default_storage_engine_impl(const char *opt_name, > char *engine_name, plugin_ref *res) > { > - if (!engine_name) > + /* Storage engine can be NULL and enforced storage engine > + can be empty. */ > + if (!engine_name || > + (strcmp(opt_name, "enforced_storage_engine") == 0 && strlen(engine_name) == 0)) > { > *res= 0; > return 0; In fact, default_tmp_storage_engine can be NULL too. See in mysqld.cc: if (default_tmp_storage_engine && !*default_tmp_storage_engine) default_tmp_storage_engine= NULL; if (init_default_storage_engine(default_tmp_storage_engine, tmp_table_plugin)) unireg_abort(1); And it looks like it's important to have default_tmp_storage_engine=NULL, not leave it as an empty string. Because Sys_var_plugin::do_check() doesn't allow empty strings as plugins names, but translates NULL to "no plugin". So, you either need to set enforced_storage_engine=NULL if it was an empty string, or you need to fix Sys_var_plugin::do_check() to treat empty strings as "no plugin" too. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 7f7cbb1: MDEV-6756: map a linux pid (child pid) to a connection id shown in the output of SHOW PROCESSLIST
by Sergei Golubchik 15 Sep '15

15 Sep '15

Hi, Sanja! There's an issue with cmake detection and with tests. But the code looks fine, thanks! On Sep 14, sanja(a)mariadb.com wrote: > revision-id: 7f7cbb132fa0a40687e4b25a9c9e163c93ec6d15 (mariadb-10.1.6-12-g7f7cbb1) > parent(s): 86a3613d4e981c341e38291c9eeec5dc9f836fae > committer: Oleksandr Byelkin > timestamp: 2015-09-14 12:47:57 +0200 > message: > > MDEV-6756: map a linux pid (child pid) to a connection id shown in the > output of SHOW PROCESSLIST > > Added tid (thread ID) for system where it is present. > > ps -eL -o tid,pid,command > > shows the thread on Linux > > diff --git a/mysql-test/r/information_schema.result b/mysql-test/r/information_schema.result > index 2a918ad..ef889c7 100644 > --- a/mysql-test/r/information_schema.result > +++ b/mysql-test/r/information_schema.result > @@ -2066,3 +2066,7 @@ Variable_name Value > Opened_tables 3 > drop database mysqltest; > drop database db1; > +set @TID= (SELECT tid FROM information_schema.processlist); > +select (SELECT tid FROM information_schema.processlist) = @TID; > +(SELECT tid FROM information_schema.processlist) = @TID > +1 That seems pretty useless as a test :) > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index bf161b2..0478b33 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -69,6 +69,10 @@ > #include "sql_connect.h" > #include "my_atomic.h" > > +#ifdef HAVE_SYS_SYSCALL_H Pardon me? Did you see whether your code actually work? You check here whether HAVE_SYS_SYSCALL_H is defined, but you don't have CHECK_INCLUDE_FILES(sys/syscall.h HAVE_SYS_SYSCALL_H) in configure.cmake and HAVE_SYS_SYSCALL_H in config.h.cmake. So, I don't see how your code can possibly work as HAVE_SYS_SYSCALL_H should be never defined in your patch. Please, when fixing it, make sure that your test *fails* without TID support (I suspect you'll have to enable it only on linux). And then you could remove that useless test above. > +#include <sys/syscall.h> > +#endif Regards, Sergei

2 2

[Maria-developers] MDEV-3929 Add system variable explicit_defaults_for_timestamp for compatibility with MySQL
by Alexander Barkov 15 Sep '15

15 Sep '15

Hi Sergei, Please review a patch that merges a new read-only variables --explicit-defaults-for-timestamp from MySQL. Note, the original MySQL patch updates many tests, to make the tests independent from the compiled-in default --explicit-defaults-for-timestamp value. I guess Oracle is planning to change the default value to ON eventually. I merged these changes for tests as well. As a result, all tests pass in both "mtr" and "mtr --mysqld=--explicit-defaults-for-timestamp", which I think is good. In addition to the original MySQL changes in mtr, I also added two tests that describe specific behavior: sys_vars.explicit_defaults_for_timestamp_off sys_vars.explicit_defaults_for_timestamp_on Note, the patch assumes that the server and the slave have the same --explicit-defaults-for-timeatamp. Monty suggested to make this variable dynamic and replication-safe. We added a separate task for this: MDEV-8455 Make --explicit-defaults-for-timestamp dynamic I plan to do it in the next 10.1 sprint round. Thanks.

2 8

Re: [Maria-developers] [Commits] 8983790: MDEV-7970: EXPLAIN FORMAT=JSON does not print HAVING
by Oleksandr Byelkin 14 Sep '15

14 Sep '15

On 14.09.15 21:59, Sergey Petrunia wrote: > Hi Sanja, > > The big comment: I see HAVING being printed for EXPLAIN FORMAT=JSON but not for > ANALYZE FORMAT=JSON. > > For example: > > MariaDB [test]> explain format=json select count(*) from t1 where t1.a between 0 and 1000 group by b having count(*) > 0\G > *************************** 1. row *************************** > EXPLAIN: { > "query_block": { > "select_id": 1, > "having_condition": "(count(0) > 0)", > "filesort": { > "temporary_table": { > "function": "buffer", > "table": { > "table_name": "t1", > "access_type": "ALL", > "rows": 1000, > "filtered": 100, > "attached_condition": "(t1.a between 0 and 1000)" > } > } > } > } > } > > MariaDB [test]> analyze format=json select count(*) from t1 where t1.a between 0 and 1000 group by b having count(*) > 0\G > *************************** 1. row *************************** > ANALYZE: { > "query_block": { > "select_id": 1, > "r_loops": 1, > "r_total_time_ms": 49.417, > "filesort": { > "r_loops": 1, > "r_total_time_ms": 0.4251, > "r_used_priority_queue": false, > "r_output_rows": 101, > "r_buffer_size": "2Kb", > "temporary_table": { > "table": { > "table_name": "t1", > "access_type": "ALL", > "r_loops": 1, > "rows": 1000, > "r_rows": 1000, > "r_total_time_ms": 18.597, > "filtered": 100, > "r_filtered": 100, > "attached_condition": "(t1.a between 0 and 1000)" > } > } > } > } > } oops, I'll fix. > > Smaller comments below. > > On Wed, Sep 09, 2015 at 04:29:54PM +0200, sanja(a)mariadb.com wrote: >> revision-id: 8983790df9a16520e69905fc4c35d20f7c935c3e (mariadb-10.1.6-119-g8983790) >> parent(s): 3fcd84c2891f4d2c881f13b53bc5a2401870b80a >> committer: Oleksandr Byelkin >> timestamp: 2015-09-09 16:29:50 +0200 >> message: >> >> MDEV-7970: EXPLAIN FORMAT=JSON does not print HAVING >> >> Printing non-trivial HAVING added. >> >> --- >> mysql-test/r/explain_json.result | 84 ++++++++++++++++++++++++++++++++++++++++ >> mysql-test/t/explain_json.test | 23 ++++++++++- >> sql/sql_explain.cc | 15 ++++++- >> sql/sql_explain.h | 7 +++- >> sql/sql_select.cc | 2 + >> 5 files changed, 128 insertions(+), 3 deletions(-) >> >> diff --git a/mysql-test/r/explain_json.result b/mysql-test/r/explain_json.result >> index 0507cee..db2726d 100644 >> --- a/mysql-test/r/explain_json.result >> +++ b/mysql-test/r/explain_json.result >> @@ -799,6 +799,7 @@ EXPLAIN >> { >> "query_block": { >> "select_id": 2, >> + "having_condition": "trigcond(<is_not_null_test>(t1.a))", >> "full-scan-on-null_key": { >> "table": { >> "table_name": "t1", >> @@ -1070,3 +1071,86 @@ EXPLAIN >> } >> } >> drop table t1; >> +# >> +# MDEV-7970: EXPLAIN FORMAT=JSON does not print HAVING >> +# >> +create table t0(a int); >> +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); >> +create table t1(a int); >> +insert into t1 select A.a + B.a* 10 + C.a * 100 from t0 A, t0 B, t0 C; >> +create table t2 ( >> +a int, >> +b int, >> +key (a) >> +); >> +insert into t2 select A.a*1000 + B.a, A.a*1000 + B.a from t0 A, t1 B; >> +# normal HAVING >> +explain format=json select a, max(b) as TOP from t2 group by a having TOP > a; >> +EXPLAIN >> +{ >> + "query_block": { >> + "select_id": 1, >> + "having_condition": "(TOP > t2.a)", >> + "filesort": { >> + "temporary_table": { >> + "function": "buffer", >> + "table": { >> + "table_name": "t2", >> + "access_type": "ALL", >> + "rows": 10000, >> + "filtered": 100 >> + } >> + } >> + } >> + } >> +} >> +# HAVING is always TRUE (not printed) >> +explain format=json select a, max(b) as TOP from t2 group by a having 1<>2; >> +EXPLAIN >> +{ >> + "query_block": { >> + "select_id": 1, >> + "filesort": { >> + "temporary_table": { >> + "function": "buffer", >> + "table": { >> + "table_name": "t2", >> + "access_type": "ALL", >> + "rows": 10000, >> + "filtered": 100 >> + } >> + } >> + } >> + } >> +} >> +# HAVING is always FALSE (intercepted by message) >> +explain format=json select a, max(b) as TOP from t2 group by a having 1=2; >> +EXPLAIN >> +{ >> + "query_block": { >> + "select_id": 1, >> + "table": { >> + "message": "Impossible HAVING" >> + } >> + } >> +} >> +# HAVING is absent >> +explain format=json select a, max(b) as TOP from t2 group by a; >> +EXPLAIN >> +{ >> + "query_block": { >> + "select_id": 1, >> + "filesort": { >> + "temporary_table": { >> + "function": "buffer", >> + "table": { >> + "table_name": "t2", >> + "access_type": "ALL", >> + "rows": 10000, >> + "filtered": 100 >> + } >> + } >> + } >> + } >> +} >> +drop table t0, t1, t2; >> diff --git a/mysql-test/t/explain_json.test b/mysql-test/t/explain_json.test >> index 3e6f34b..0870817 100644 >> --- a/mysql-test/t/explain_json.test >> +++ b/mysql-test/t/explain_json.test >> @@ -278,4 +278,25 @@ explain format=json select count(distinct a1,a2,b,c) from t1 where (a2 >= 'b') a >> >> drop table t1; >> >> - >> +--echo # >> +--echo # MDEV-7970: EXPLAIN FORMAT=JSON does not print HAVING >> +--echo # >> +create table t0(a int); >> +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); >> +create table t1(a int); >> +insert into t1 select A.a + B.a* 10 + C.a * 100 from t0 A, t0 B, t0 C; >> +create table t2 ( >> + a int, >> + b int, >> + key (a) >> +); >> +insert into t2 select A.a*1000 + B.a, A.a*1000 + B.a from t0 A, t1 B; > This seems to be excessive. I don't see any examples using key(a), I don't see a need to > populate a table with 1M rows. I was not the man who made the test suite, usually sotuation is reverse somebody require to keep the original. But I agree, I'll fix. >> +--echo # normal HAVING >> +explain format=json select a, max(b) as TOP from t2 group by a having TOP > a; >> +--echo # HAVING is always TRUE (not printed) >> +explain format=json select a, max(b) as TOP from t2 group by a having 1<>2; >> +--echo # HAVING is always FALSE (intercepted by message) >> +explain format=json select a, max(b) as TOP from t2 group by a having 1=2; >> +--echo # HAVING is absent >> +explain format=json select a, max(b) as TOP from t2 group by a; >> +drop table t0, t1, t2; >> diff --git a/sql/sql_explain.cc b/sql/sql_explain.cc >> index 15ee67a..5f60aee 100644 >> --- a/sql/sql_explain.cc >> +++ b/sql/sql_explain.cc >> @@ -862,7 +862,20 @@ void Explain_select::print_explain_json(Explain_query *query, >> writer->add_member("const_condition"); >> write_item(writer, exec_const_cond); >> } >> - >> + /* we do not print always TRUE HAVING */ > This doesn't parse. Did you mean: > " Do not print HAVING clause if it always evaluates to TRUE" ? yes > >> + if (having || (having_value == Item::COND_FALSE)) >> + { >> + writer->add_member("having_condition"); >> + if (likely(having)) >> + write_item(writer, having); >> + else >> + { >> + /* Normally we should not go this branch, left just for safety */ >> + DBUG_ASSERT(having_value == Item::COND_FALSE); >> + writer->add_str("0"); >> + } >> + } >> + >> Filesort_tracker *first_table_sort= NULL; >> bool first_table_sort_used= false; >> int started_objects= 0; >> diff --git a/sql/sql_explain.h b/sql/sql_explain.h >> index f92a576..434be47 100644 >> --- a/sql/sql_explain.h >> +++ b/sql/sql_explain.h >> @@ -209,6 +209,7 @@ class Explain_select : public Explain_basic_join >> Explain_select(MEM_ROOT *root, bool is_analyze) : >> Explain_basic_join(root), >> message(NULL), >> + having(NULL), having_value(Item::COND_UNDEF), >> using_temporary(false), using_filesort(false), >> time_tracker(is_analyze), >> ops_tracker(is_analyze) >> @@ -231,7 +232,11 @@ class Explain_select : public Explain_basic_join >> >> /* Expensive constant condition */ >> Item *exec_const_cond; >> - >> + >> + /* HAVING condition */ >> + COND *having; >> + Item::cond_result having_value; >> + >> /* Global join attributes. In tabular form, they are printed on the first row */ >> bool using_temporary; >> bool using_filesort; >> diff --git a/sql/sql_select.cc b/sql/sql_select.cc >> index 3f2c2ea..be4b076 100644 >> --- a/sql/sql_select.cc >> +++ b/sql/sql_select.cc >> @@ -24159,6 +24159,8 @@ int JOIN::save_explain_data_intern(Explain_query *output, bool need_tmp_table, >> xpl_sel->using_filesort= true; >> >> xpl_sel->exec_const_cond= exec_const_cond; >> + xpl_sel->having= having; >> + xpl_sel->having_value= having_value; >> >> JOIN_TAB* const first_top_tab= join->first_breadth_first_optimization_tab(); >> JOIN_TAB* prev_bush_root_tab= NULL; > BR > Sergei

1 0

Re: [Maria-developers] [Commits] 93a7c11: MDEV-8380: Subquery parse error
by Sergei Golubchik 14 Sep '15

14 Sep '15

Hi, Sanja! On Aug 31, sanja(a)mariadb.com wrote: > revision-id: 93a7c11c51e2f5c0ca98600924172e452b6188b6 (mariadb-10.1.6-24-g93a7c11) > parent(s): d07b780db239a909e667edf5657e3633e80d1451 > committer: Oleksandr Byelkin > timestamp: 2015-08-31 13:10:01 +0200 > message: > > MDEV-8380: Subquery parse error > > backport mysql parser fixes > 0034963fbf199696792491bcb79d5f0731c98804 > 5948561812bc691bd0c13cf518a3fe77d9daf920 > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 6c835c4..1d9aca9 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -546,6 +546,7 @@ void lex_start(THD *thd) > lex->limit_rows_examined_cnt= ULONGLONG_MAX; > lex->var_list.empty(); > lex->stmt_var_list.empty(); > + lex->proc_list.elements=0; why? this wasn't in the patches you've backported. > > lex->is_lex_started= TRUE; > DBUG_VOID_RETURN; > @@ -852,6 +853,29 @@ bool Lex_input_stream::get_text(LEX_STRING *dst, int pre_skip, int post_skip) > } > > > +uint Lex_input_stream::get_lineno(const char *raw_ptr) > +{ > + DBUG_ASSERT(m_buf <= raw_ptr && raw_ptr < m_end_of_query); > + if (!(m_buf <= raw_ptr && raw_ptr < m_end_of_query)) > + return 1; > + > + uint ret= 1; > + const CHARSET_INFO *cs= m_thd->charset(); > + for (const char *c= m_buf; c < raw_ptr; c++) > + { > + uint mb_char_len; > + if (use_mb(cs) && (mb_char_len= my_ismbchar(cs, c, m_end_of_query))) > + { > + c+= mb_char_len - 1; // skip the rest of the multibyte character > + continue; // we don't expect '\n' there > + } > + if (*c == '\n') > + ret++; > + } > + return ret; > +} why? this doesn't seem to be used anywhere. > + > + > /* > ** Calc type of integer; long integer, longlong integer or real. > ** Returns smallest type that match the string. > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 09a3d5e..529c4d5 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2087,6 +2087,8 @@ class Lex_input_stream > CHARSET_INFO *txt_cs, > const char *end_ptr); > > + uint get_lineno(const char *raw_ptr); > + > /** Current thread. */ > THD *m_thd; > > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 5e58ceb..f7eed09 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -6932,11 +6932,28 @@ mysql_new_select(LEX *lex, bool move_down) > } > else > { > + bool const outer_most= (lex->current_select->master_unit() == &lex->unit); > + if (outer_most && lex->result) > + { > + my_error(ER_WRONG_USAGE, MYF(0), "UNION", "INTO"); > + DBUG_RETURN(TRUE); > + } > + if (lex->proc_list.elements!=0) > + { > + my_error(ER_WRONG_USAGE, MYF(0), "UNION", > + "SELECT ... PROCEDURE ANALYSE()"); > + DBUG_RETURN(TRUE); > + } why couldn't parser do it? > if (lex->current_select->order_list.first && !lex->current_select->braces) > { > my_error(ER_WRONG_USAGE, MYF(0), "UNION", "ORDER BY"); > DBUG_RETURN(1); > } > + if (lex->current_select->explicit_limit && !lex->current_select->braces) > + { > + my_error(ER_WRONG_USAGE, MYF(0), "UNION", "LIMIT"); > + DBUG_RETURN(1); > + } why couldn't parser do it? > select_lex->include_neighbour(lex->current_select); > SELECT_LEX_UNIT *unit= select_lex->master_unit(); > if (!unit->fake_select_lex && unit->add_fake_select_lex(lex->thd)) > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 0e2db26..b79c2ce 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -1,5 +1,5 @@ > /* > - Copyright (c) 2000, 2015, Oracle and/or its affiliates. > + Copyrig ht(c) 2000, 2015, Oracle and/or its affiliates. typo? > Copyright (c) 2010, 2015, MariaDB > > This program is free software; you can redistribute it and/or modify > @@ -1001,10 +1001,10 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > %parse-param { THD *thd } > %lex-param { THD *thd } > /* > - Currently there are 164 shift/reduce conflicts. > + Currently there are 160 shift/reduce conflicts. > We should not introduce new conflicts any more. > */ > -%expect 164 > +%expect 160 > > /* > Comments for TOKENS. > @@ -1731,6 +1731,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > opt_default_time_precision > case_stmt_body opt_bin_mod > opt_if_exists_table_element opt_if_not_exists_table_element > + opt_into opt_procedure_clause > > %type <object_ddl_options> > create_or_replace > @@ -5548,7 +5549,7 @@ create_select: > { > Select->parsing_place= NO_MATTER; > } > - opt_select_from > + table_expression > { > /* > The following work only with the local list, the global list > @@ -5943,11 +5944,6 @@ merge_insert_types: > | LAST_SYM { $$= MERGE_INSERT_TO_LAST; } > ; > > -opt_select_from: > - opt_limit_clause {} > - | select_from opt_select_lock_type > - ; > - > udf_type: > STRING_SYM {$$ = (int) STRING_RESULT; } > | REAL {$$ = (int) REAL_RESULT; } > @@ -8397,6 +8393,7 @@ select_paren_derived: > Lex->current_select->set_braces(true); > } > SELECT_SYM select_part2_derived > + table_expression > { > if (setup_select_in_parentheses(Lex)) > MYSQL_YYABORT; > @@ -8414,7 +8411,45 @@ select_init2: > union_clause > ; > > +/* > + Theoretically we can merge all 3 right hand sides of the select_part2 > + rule into one, however such a transformation adds one shift/reduce > + conflict more. > +*/ > select_part2: > + select_options_and_item_list > + opt_order_clause > + opt_limit_clause > + opt_select_lock_type > + | select_options_and_item_list into opt_select_lock_type > + | select_options_and_item_list > + opt_into > + from_clause > + opt_where_clause > + opt_group_clause > + opt_having_clause > + opt_order_clause > + opt_limit_clause > + opt_procedure_clause > + opt_into > + opt_select_lock_type why not table_expression here? > + { > + if ($2 && $10) > + { > + /* double "INTO" clause */ > + my_error(ER_WRONG_USAGE, MYF(0), "INTO", "INTO"); > + MYSQL_YYABORT; > + } > + if ($9 && ($2 || $10)) > + { > + /* "INTO" with "PROCEDURE ANALYSE" */ > + my_error(ER_WRONG_USAGE, MYF(0), "PROCEDURE", "INTO"); > + MYSQL_YYABORT; > + } > + } Why couldn't parser do that? > + ; > + > +select_options_and_item_list: > { > LEX *lex= Lex; > SELECT_LEX *sel= lex->current_select; > @@ -8426,27 +8461,36 @@ select_part2: > { > Select->parsing_place= NO_MATTER; > } > - select_into opt_select_lock_type > ; > > -select_into: > - opt_order_clause opt_limit_clause {} > - | into > - | select_from > - | into select_from > - | select_from into > +table_expression: > + opt_from_clause > + opt_where_clause > + opt_group_clause > + opt_having_clause > + opt_order_clause > + opt_limit_clause > + opt_procedure_clause > + opt_select_lock_type > + ; > + > +from_clause: > + FROM table_reference_list > + ; > + > +opt_from_clause: > + /* empty */ > + | from_clause > ; > > -select_from: > - FROM join_table_list > +table_reference_list: > + join_table_list > { > Select->context.table_list= > Select->context.first_name_resolution_table= > Select->table_list.first; > } > - opt_where_clause opt_group_clause opt_having_clause > - opt_order_clause opt_limit_clause opt_procedure_clause > - | FROM DUAL_SYM opt_where_clause opt_limit_clause > + | DUAL_SYM > /* oracle compatibility: oracle always requires FROM clause, > and DUAL is system table without fields. > Is "SELECT 1 FROM DUAL" any better than "SELECT 1" ? > @@ -10888,9 +10932,7 @@ select_derived_union: > last select in the union. > */ > Lex->pop_context(); > - } > - opt_union_order_or_limit > - { > + > if ($1 != NULL) > { > my_parse_error(ER_THD(thd, ER_SYNTAX_ERROR)); > @@ -10910,12 +10952,14 @@ select_init2_derived: > my_parse_error(ER_THD(thd, ER_SYNTAX_ERROR)); > MYSQL_YYABORT; > } > + /* > if (sel->linkage == UNION_TYPE && > sel->master_unit()->first_select()->braces) > { > my_parse_error(ER_THD(thd, ER_SYNTAX_ERROR)); > MYSQL_YYABORT; > } > + */ what was wrong with that? if it's not needed anymore, delete it, don't just comment it out. > } > ; > > @@ -10932,7 +10976,6 @@ select_part2_derived: > { > Select->parsing_place= NO_MATTER; > } > - opt_select_from opt_select_lock_type > ; > > /* handle contents of parentheses in join expression */ > @@ -10980,7 +11023,7 @@ select_derived2: > { > Select->parsing_place= NO_MATTER; > } > - opt_select_from > + table_expression > ; > > get_select_lex: > @@ -11556,7 +11599,7 @@ choice: > ; > > opt_procedure_clause: > - /* empty */ > + /* empty */ { $$= false; } > | PROCEDURE_SYM ident /* Procedure name */ > { > LEX *lex=Lex; > @@ -11585,6 +11628,9 @@ opt_procedure_clause: > Lex->uncacheable(UNCACHEABLE_SIDEEFFECT); > } > '(' procedure_list ')' > + { > + $$= true; > + } > ; > > procedure_list: > @@ -11661,6 +11707,11 @@ select_outvar: > } > ; > > +opt_into: > + /* empty */ { $$= false; } > + | into { $$= true; } > + ; > + > into: > INTO > { > @@ -12496,7 +12547,8 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_BINLOG_EVENTS; > - } opt_limit_clause > + } > + opt_limit_clause > | RELAYLOG_SYM optional_connection_name EVENTS_SYM binlog_in binlog_from > { > LEX *lex= Lex; > @@ -15836,17 +15888,19 @@ union_option: > > query_specification: > SELECT_SYM select_init2_derived > - { > + table_expression > + { > $$= Lex->current_select->master_unit()->first_select(); > } > | '(' select_paren_derived ')' > + opt_union_order_or_limit > { > $$= Lex->current_select->master_unit()->first_select(); > } > ; > > query_expression_body: > - query_specification opt_union_order_or_limit > + query_specification > | query_expression_body > UNION_SYM union_option > { > @@ -15854,7 +15908,6 @@ query_expression_body: > MYSQL_YYABORT; > } > query_specification > - opt_union_order_or_limit > { > Lex->pop_context(); > $$= $1; > diff --git a/mysql-test/r/parser.result b/mysql-test/r/parser.result > index 1d29f2c..01cc9d7 100644 > --- a/mysql-test/r/parser.result > +++ b/mysql-test/r/parser.result > @@ -650,3 +650,230 @@ CREATE TABLE t1(a INT); > SELECT * FROM t1 JOIN ((SELECT 1 AS b) UNION ALL (SELECT 2 AS b) ORDER BY b DESC) s1 WHERE a=1; > a b > DROP TABLE t1; > +# > +# Test of collective fix for three parser bugs: > +# > +# Bug #17727401, Bug #17426017, Bug #17473479: > +# The server accepts wrong syntax and then fails in different ways > +# > +CREATE TABLE t1 (i INT); > +# bug #17426017 > +SELECT (SELECT EXISTS(SELECT * LIMIT 1 ORDER BY VALUES (c00))); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY VALUES (c00)))' at line 1 > +# bug#17473479 > +CREATE TABLE a(a int); > +CREATE TABLE b(a int); > +DELETE FROM b ORDER BY(SELECT 1 FROM a ORDER BY a ORDER BY a); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY a)' at line 1 > +DROP TABLE a, b; > +# bug #17727401 > +SELECT '' IN (SELECT '1' c FROM t1 ORDER BY '' ORDER BY '') FROM t1; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY '') FROM t1' at line 1 > +# regression & coverage tests > +# uniform syntax for FROM DUAL clause: > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +FOR UPDATE; > +1 > +1 > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +PROCEDURE ANALYSE() FOR UPDATE; > +ERROR HY000: Can't use ORDER clause with this procedure > +SELECT 1 FROM > +(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +FOR UPDATE) a; > +1 > +1 > +SELECT 1 FROM > +(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +PROCEDURE ANALYSE() FOR UPDATE) a; > +ERROR HY000: Incorrect usage of PROCEDURE and subquery > +SELECT 1 FROM t1 > +WHERE EXISTS(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +FOR UPDATE); > +1 > +SELECT 1 FROM t1 > +WHERE EXISTS(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +PROCEDURE ANALYSE() FOR UPDATE); > +ERROR HY000: Incorrect usage of PROCEDURE and subquery > +SELECT 1 FROM t1 > +UNION > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +FOR UPDATE; > +1 > +1 > +SELECT 1 FROM t1 > +UNION > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +PROCEDURE ANALYSE() FOR UPDATE; > +ERROR HY000: Incorrect usage of PROCEDURE and subquery > +SELECT 1 FROM DUAL PROCEDURE ANALYSE() > +UNION > +SELECT 1 FROM t1; > +ERROR HY000: Incorrect usage of UNION and SELECT ... PROCEDURE ANALYSE() > +(SELECT 1 FROM t1) > +UNION > +(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +FOR UPDATE); > +1 > +1 > +(SELECT 1 FROM t1) > +UNION > +(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > +PROCEDURE ANALYSE() FOR UPDATE); > +ERROR HY000: Incorrect usage of PROCEDURE and subquery > +# "FOR UPDATE" tests > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1; > +1 > +SELECT 1 FROM t1 FOR UPDATE UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1; > +1 > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1 FOR UPDATE; > +1 > +# "INTO" clause tests > +SELECT 1 FROM t1 INTO @var17727401; > +Warnings: > +Warning 1329 No data - zero rows fetched, selected, or processed > +SELECT 1 FROM DUAL INTO @var17727401; > +SELECT 1 INTO @var17727401; > +SELECT 1 INTO @var17727401 FROM t1; > +Warnings: > +Warning 1329 No data - zero rows fetched, selected, or processed > +SELECT 1 INTO @var17727401 FROM DUAL; > +SELECT 1 INTO @var17727401_1 FROM t1 INTO @var17727401_2; > +ERROR HY000: Incorrect usage of INTO and INTO > +SELECT 1 INTO @var17727401_1 FROM DUAL > +INTO @var17727401_2; > +ERROR HY000: Incorrect usage of INTO and INTO > +SELECT 1 INTO @var17727401 FROM t1 WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1; > +Warnings: > +Warning 1329 No data - zero rows fetched, selected, or processed > +SELECT 1 FROM t1 WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1 INTO @var17727401; > +Warnings: > +Warning 1329 No data - zero rows fetched, selected, or processed > +SELECT 1 FROM t1 WHERE 1 INTO @var17727401 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1' at line 1 > +SELECT 1 INTO @var17727401_1 > +FROM t1 WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1 > +INTO @var17727401_2; > +ERROR HY000: Incorrect usage of INTO and INTO > +SELECT (SELECT 1 FROM t1 INTO @var17727401); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INTO @var17727401)' at line 1 > +SELECT 1 FROM (SELECT 1 FROM t1 INTO @var17727401) a; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INTO @var17727401) a' at line 1 > +SELECT EXISTS(SELECT 1 FROM t1 INTO @var17727401); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INTO @var17727401)' at line 1 > +SELECT 1 FROM t1 INTO @var17727401 UNION SELECT 1 FROM t1 INTO t1; > +ERROR HY000: Incorrect usage of UNION and INTO > +(SELECT 1 FROM t1 INTO @var17727401) UNION (SELECT 1 FROM t1 INTO t1); > +ERROR HY000: Incorrect usage of UNION and INTO > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 INTO @var17727401; > +Warnings: > +Warning 1329 No data - zero rows fetched, selected, or processed > +SELECT 1 INTO @var17727401 FROM t1 PROCEDURE ANALYSE(); > +ERROR HY000: Incorrect usage of PROCEDURE and INTO > +SELECT 1 FROM t1 PROCEDURE ANALYSE() INTO @var17727401; > +ERROR HY000: Incorrect usage of PROCEDURE and INTO > +# ORDER and LIMIT clause combinations > +(SELECT 1 FROM t1 ORDER BY 1) ORDER BY 1; > +1 > +(SELECT 1 FROM t1 LIMIT 1) LIMIT 1; > +1 > +((SELECT 1 FROM t1 ORDER BY 1) ORDER BY 1) ORDER BY 1; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY 1) ORDER BY 1' at line 1 > +((SELECT 1 FROM t1 LIMIT 1) LIMIT 1) LIMIT 1; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'LIMIT 1) LIMIT 1' at line 1 > +(SELECT 1 FROM t1 ORDER BY 1) LIMIT 1; > +1 > +(SELECT 1 FROM t1 LIMIT 1) ORDER BY 1; > +1 > +((SELECT 1 FROM t1 ORDER BY 1) LIMIT 1) ORDER BY 1); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'LIMIT 1) ORDER BY 1)' at line 1 > +((SELECT 1 FROM t1 LIMIT 1) ORDER BY 1) LIMIT 1); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY 1) LIMIT 1)' at line 1 > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1; > +1 > +SELECT (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1); > +(SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1) > +NULL > +SELECT 1 FROM (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1) a; > +1 > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1; > +1 > +SELECT (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1); > +(SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1) > +NULL > +SELECT 1 FROM (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1) a; > +1 > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1; > +1 > +SELECT (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1); > +(SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1) > +NULL > +SELECT 1 FROM (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1) a; > +1 > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1 ORDER BY 1; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY 1' at line 1 > +SELECT (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1 ORDER BY 1); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY 1)' at line 1 > +SELECT 1 FROM (SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1 ORDER BY 1) a; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY 1) a' at line 1 > +SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT (SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1); > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM (SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1) a; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1; > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT (SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1); > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT 1 FROM (SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1) a; > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT 1 FROM t1 ORDER BY 1 LIMIT 1 UNION SELECT 1 FROM t1; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT (SELECT 1 FROM t1 ORDER BY 1 LIMIT 1 UNION SELECT 1 FROM t1); > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM (SELECT 1 FROM t1 ORDER BY 1 LIMIT 1 UNION SELECT 1 FROM t1) a; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM t1 LIMIT 1 ORDER BY 1 UNION SELECT 1 FROM t1; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY 1 UNION SELECT 1 FROM t1' at line 1 > +SELECT (SELECT 1 FROM t1 LIMIT 1 ORDER BY 1 UNION SELECT 1 FROM t1); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY 1 UNION SELECT 1 FROM t1)' at line 1 > +SELECT 1 FROM (SELECT 1 FROM t1 LIMIT 1 ORDER BY 1 UNION SELECT 1 FROM t1) a; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 ORDER BY 1; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT (SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 ORDER BY 1); > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM (SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 ORDER BY 1) a; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 LIMIT 1; > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT (SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 LIMIT 1); > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT 1 FROM (SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 LIMIT 1) a; > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 ORDER BY 1; > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT (SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 ORDER BY 1); > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT 1 FROM (SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 ORDER BY 1) a; > +ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 LIMIT 1; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT (SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 LIMIT 1); > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +SELECT 1 FROM (SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 LIMIT 1) a; > +ERROR HY000: Incorrect usage of UNION and ORDER BY > +DROP TABLE t1; > +# > +# MDEV-8380: Subquery parse error > +# > +CREATE TABLE t1 ( a INT); > +INSERT INTO t1 VALUES ( 2 ); > +SELECT * > +FROM ( (SELECT a FROM t1 ORDER BY a) UNION (SELECT 1 as b ORDER BY b ) ) AS a1 > +WHERE a1.a = 1 OR a1.a = 2; > +a > +2 > +1 > +DROP TABLE t1; > diff --git a/mysql-test/r/subselect.result b/mysql-test/r/subselect.result > index 3732f62..b9f9fb0 100644 > --- a/mysql-test/r/subselect.result > +++ b/mysql-test/r/subselect.result > @@ -5132,8 +5132,9 @@ SELECT a FROM t1 WHERE a = 2 > a > 1 > 2 > -SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')' at line 1 > +SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a) q; > +a > +1 > SELECT * FROM (SELECT 1 a UNION (SELECT 1 a)) alias; > a > 1 > @@ -5175,7 +5176,7 @@ a 1 > 1 1 > 2 1 > SELECT * FROM t1 JOIN ((SELECT 1 UNION SELECT 1)) ON 1; > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')) ON 1' at line 1 > +ERROR 42000: Every derived table must have its own alias > SELECT * FROM t1 JOIN (t1 t1a UNION SELECT 1) ON 1; > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') ON 1' at line 1 > SELECT * FROM t1 JOIN ((t1 t1a UNION SELECT 1)) ON 1; > @@ -5294,7 +5295,10 @@ SELECT ((SELECT 1 UNION SELECT 1 UNION SELECT 1)); > SELECT * FROM ( SELECT 1 UNION ( SELECT 1 UNION SELECT 1 ) ); > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'UNION SELECT 1 ) )' at line 1 > SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') UNION SELECT 1 )' at line 1 > +ERROR 42000: Every derived table must have its own alias > +SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ) a; > +1 > +1 > SELECT * FROM ( SELECT 1 UNION SELECT 1 UNION SELECT 1 ) a; > 1 > 1 > diff --git a/mysql-test/r/subselect_no_exists_to_in.result b/mysql-test/r/subselect_no_exists_to_in.result > index 140b790..902c45f 100644 > --- a/mysql-test/r/subselect_no_exists_to_in.result > +++ b/mysql-test/r/subselect_no_exists_to_in.result > @@ -5134,8 +5134,9 @@ SELECT a FROM t1 WHERE a = 2 > a > 1 > 2 > -SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')' at line 1 > +SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a) q; > +a > +1 > SELECT * FROM (SELECT 1 a UNION (SELECT 1 a)) alias; > a > 1 > @@ -5177,7 +5178,7 @@ a 1 > 1 1 > 2 1 > SELECT * FROM t1 JOIN ((SELECT 1 UNION SELECT 1)) ON 1; > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')) ON 1' at line 1 > +ERROR 42000: Every derived table must have its own alias > SELECT * FROM t1 JOIN (t1 t1a UNION SELECT 1) ON 1; > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') ON 1' at line 1 > SELECT * FROM t1 JOIN ((t1 t1a UNION SELECT 1)) ON 1; > @@ -5296,7 +5297,10 @@ SELECT ((SELECT 1 UNION SELECT 1 UNION SELECT 1)); > SELECT * FROM ( SELECT 1 UNION ( SELECT 1 UNION SELECT 1 ) ); > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'UNION SELECT 1 ) )' at line 1 > SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') UNION SELECT 1 )' at line 1 > +ERROR 42000: Every derived table must have its own alias > +SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ) a; > +1 > +1 > SELECT * FROM ( SELECT 1 UNION SELECT 1 UNION SELECT 1 ) a; > 1 > 1 > diff --git a/mysql-test/r/subselect_no_mat.result b/mysql-test/r/subselect_no_mat.result > index 99ac9f4..89034b7 100644 > --- a/mysql-test/r/subselect_no_mat.result > +++ b/mysql-test/r/subselect_no_mat.result > @@ -5132,8 +5132,9 @@ SELECT a FROM t1 WHERE a = 2 > a > 1 > 2 > -SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')' at line 1 > +SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a) q; > +a > +1 > SELECT * FROM (SELECT 1 a UNION (SELECT 1 a)) alias; > a > 1 > @@ -5175,7 +5176,7 @@ a 1 > 1 1 > 2 1 > SELECT * FROM t1 JOIN ((SELECT 1 UNION SELECT 1)) ON 1; > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')) ON 1' at line 1 > +ERROR 42000: Every derived table must have its own alias > SELECT * FROM t1 JOIN (t1 t1a UNION SELECT 1) ON 1; > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') ON 1' at line 1 > SELECT * FROM t1 JOIN ((t1 t1a UNION SELECT 1)) ON 1; > @@ -5294,7 +5295,10 @@ SELECT ((SELECT 1 UNION SELECT 1 UNION SELECT 1)); > SELECT * FROM ( SELECT 1 UNION ( SELECT 1 UNION SELECT 1 ) ); > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'UNION SELECT 1 ) )' at line 1 > SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') UNION SELECT 1 )' at line 1 > +ERROR 42000: Every derived table must have its own alias > +SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ) a; > +1 > +1 > SELECT * FROM ( SELECT 1 UNION SELECT 1 UNION SELECT 1 ) a; > 1 > 1 > diff --git a/mysql-test/r/subselect_no_opts.result b/mysql-test/r/subselect_no_opts.result > index 7ecd40c..6e90aac 100644 > --- a/mysql-test/r/subselect_no_opts.result > +++ b/mysql-test/r/subselect_no_opts.result > @@ -5128,8 +5128,9 @@ SELECT a FROM t1 WHERE a = 2 > a > 1 > 2 > -SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')' at line 1 > +SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a) q; > +a > +1 > SELECT * FROM (SELECT 1 a UNION (SELECT 1 a)) alias; > a > 1 > @@ -5171,7 +5172,7 @@ a 1 > 1 1 > 2 1 > SELECT * FROM t1 JOIN ((SELECT 1 UNION SELECT 1)) ON 1; > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')) ON 1' at line 1 > +ERROR 42000: Every derived table must have its own alias > SELECT * FROM t1 JOIN (t1 t1a UNION SELECT 1) ON 1; > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') ON 1' at line 1 > SELECT * FROM t1 JOIN ((t1 t1a UNION SELECT 1)) ON 1; > @@ -5290,7 +5291,10 @@ SELECT ((SELECT 1 UNION SELECT 1 UNION SELECT 1)); > SELECT * FROM ( SELECT 1 UNION ( SELECT 1 UNION SELECT 1 ) ); > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'UNION SELECT 1 ) )' at line 1 > SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') UNION SELECT 1 )' at line 1 > +ERROR 42000: Every derived table must have its own alias > +SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ) a; > +1 > +1 > SELECT * FROM ( SELECT 1 UNION SELECT 1 UNION SELECT 1 ) a; > 1 > 1 > diff --git a/mysql-test/r/subselect_no_scache.result b/mysql-test/r/subselect_no_scache.result > index c70db6f..1c18817 100644 > --- a/mysql-test/r/subselect_no_scache.result > +++ b/mysql-test/r/subselect_no_scache.result > @@ -5138,8 +5138,9 @@ SELECT a FROM t1 WHERE a = 2 > a > 1 > 2 > -SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')' at line 1 > +SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a) q; > +a > +1 > SELECT * FROM (SELECT 1 a UNION (SELECT 1 a)) alias; > a > 1 > @@ -5181,7 +5182,7 @@ a 1 > 1 1 > 2 1 > SELECT * FROM t1 JOIN ((SELECT 1 UNION SELECT 1)) ON 1; > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')) ON 1' at line 1 > +ERROR 42000: Every derived table must have its own alias > SELECT * FROM t1 JOIN (t1 t1a UNION SELECT 1) ON 1; > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') ON 1' at line 1 > SELECT * FROM t1 JOIN ((t1 t1a UNION SELECT 1)) ON 1; > @@ -5300,7 +5301,10 @@ SELECT ((SELECT 1 UNION SELECT 1 UNION SELECT 1)); > SELECT * FROM ( SELECT 1 UNION ( SELECT 1 UNION SELECT 1 ) ); > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'UNION SELECT 1 ) )' at line 1 > SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') UNION SELECT 1 )' at line 1 > +ERROR 42000: Every derived table must have its own alias > +SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ) a; > +1 > +1 > SELECT * FROM ( SELECT 1 UNION SELECT 1 UNION SELECT 1 ) a; > 1 > 1 > diff --git a/mysql-test/r/subselect_no_semijoin.result b/mysql-test/r/subselect_no_semijoin.result > index 6a7d6b3..de9c4e8 100644 > --- a/mysql-test/r/subselect_no_semijoin.result > +++ b/mysql-test/r/subselect_no_semijoin.result > @@ -5128,8 +5128,9 @@ SELECT a FROM t1 WHERE a = 2 > a > 1 > 2 > -SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')' at line 1 > +SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a) q; > +a > +1 > SELECT * FROM (SELECT 1 a UNION (SELECT 1 a)) alias; > a > 1 > @@ -5171,7 +5172,7 @@ a 1 > 1 1 > 2 1 > SELECT * FROM t1 JOIN ((SELECT 1 UNION SELECT 1)) ON 1; > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')) ON 1' at line 1 > +ERROR 42000: Every derived table must have its own alias > SELECT * FROM t1 JOIN (t1 t1a UNION SELECT 1) ON 1; > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') ON 1' at line 1 > SELECT * FROM t1 JOIN ((t1 t1a UNION SELECT 1)) ON 1; > @@ -5290,7 +5291,10 @@ SELECT ((SELECT 1 UNION SELECT 1 UNION SELECT 1)); > SELECT * FROM ( SELECT 1 UNION ( SELECT 1 UNION SELECT 1 ) ); > ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'UNION SELECT 1 ) )' at line 1 > SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') UNION SELECT 1 )' at line 1 > +ERROR 42000: Every derived table must have its own alias > +SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ) a; > +1 > +1 > SELECT * FROM ( SELECT 1 UNION SELECT 1 UNION SELECT 1 ) a; > 1 > 1 > diff --git a/mysql-test/r/union.result b/mysql-test/r/union.result > index ef1749e..9b7a1c5 100644 > --- a/mysql-test/r/union.result > +++ b/mysql-test/r/union.result > @@ -89,6 +89,10 @@ id select_type table type possible_keys key key_len ref rows filtered Extra > NULL UNION RESULT <union1,2> ALL NULL NULL NULL NULL NULL NULL Using filesort > Warnings: > Note 1003 (select `test`.`t1`.`a` AS `a`,`test`.`t1`.`b` AS `b` from `test`.`t1` limit 2) union all (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` order by `test`.`t2`.`a` limit 1) order by `b` desc > +select count(*) from ( > +(select a,b from t1 limit 2) union all (select a,b from t2 order by a)) q; > +count(*) > +6 > (select sql_calc_found_rows a,b from t1 limit 2) union all (select a,b from t2 order by a) limit 2; > a b > 1 a > @@ -96,6 +100,10 @@ a b > select found_rows(); > found_rows() > 6 > +select count(*) from ( > +select a,b from t1 union all select a,b from t2) q; > +count(*) > +8 > select sql_calc_found_rows a,b from t1 union all select a,b from t2 limit 2; > a b > 1 a > @@ -308,12 +316,20 @@ create table t1 (a int); > insert into t1 values (1),(2),(3); > create table t2 (a int); > insert into t2 values (3),(4),(5); > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1) UNION all (SELECT * FROM t2)) q; > +COUNT(*) > +6 > (SELECT SQL_CALC_FOUND_ROWS * FROM t1) UNION all (SELECT * FROM t2) LIMIT 1; > a > 1 > select found_rows(); > found_rows() > 6 > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2)) q; > +COUNT(*) > +4 > (SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2) LIMIT 2; > a > 1 > @@ -321,6 +337,10 @@ a > select found_rows(); > found_rows() > 4 > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2)) q; > +COUNT(*) > +4 > (SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2); > a > 1 > @@ -330,6 +350,10 @@ a > select found_rows(); > found_rows() > 4 > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1) UNION all (SELECT * FROM t2 LIMIT 1)) q; > +COUNT(*) > +4 > (SELECT SQL_CALC_FOUND_ROWS * FROM t1) UNION all (SELECT * FROM t2 LIMIT 1); > a > 1 > @@ -345,6 +369,16 @@ a > select found_rows(); > found_rows() > 4 > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION SELECT * FROM t2) q; > +COUNT(*) > +4 > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION SELECT * FROM t2 LIMIT 1; > +a > +1 > +select found_rows(); > +found_rows() > +4 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1 UNION all SELECT * FROM t2 LIMIT 2; > ERROR HY000: Incorrect usage of UNION and LIMIT > SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION all SELECT * FROM t2 LIMIT 2; > @@ -354,13 +388,29 @@ a > select found_rows(); > found_rows() > 6 > +SELECT COUNT(*) FROM ( > +SELECT * FROM t1 UNION SELECT * FROM t2) q; > +COUNT(*) > +5 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION SELECT * FROM t2 LIMIT 2; > a > 1 > 2 > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION all SELECT * FROM t2) q; > +COUNT(*) > +4 > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION all SELECT * FROM t2 LIMIT 2; > +a > +1 > +3 > select found_rows(); > found_rows() > -5 > +4 > +SELECT COUNT(*) FROM ( > +SELECT * FROM t1 UNION all SELECT * FROM t2) q; > +COUNT(*) > +6 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION SELECT * FROM t2 LIMIT 100; > a > 1 > @@ -373,10 +423,39 @@ found_rows() > 5 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 100 UNION SELECT * FROM t2; > ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 100) UNION SELECT * FROM t2) q; > +COUNT(*) > +5 > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 100) UNION SELECT * FROM t2; > +a > +1 > +2 > +3 > +4 > +5 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1 UNION SELECT * FROM t2; > ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION SELECT * FROM t2) q; > +COUNT(*) > +4 > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION SELECT * FROM t2; > +a > +1 > +3 > +4 > +5 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1 UNION SELECT * FROM t2 LIMIT 2; > ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION SELECT * FROM t2) q; > +COUNT(*) > +4 > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION SELECT * FROM t2 LIMIT 2; > +a > +1 > +3 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION SELECT * FROM t2 LIMIT 2,2; > a > 3 > @@ -384,8 +463,21 @@ a > select found_rows(); > found_rows() > 5 > +SELECT COUNT(*) FROM ( > +SELECT * FROM t1 UNION SELECT * FROM t2) q; > +COUNT(*) > +5 > SELECT SQL_CALC_FOUND_ROWS * FROM t1 limit 2,2 UNION SELECT * FROM t2; > ERROR HY000: Incorrect usage of UNION and LIMIT > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 limit 2,2) UNION SELECT * FROM t2) q; > +COUNT(*) > +3 > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 limit 2,2) UNION SELECT * FROM t2; > +a > +3 > +4 > +5 > SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a desc LIMIT 1; > a > 5 > diff --git a/mysql-test/t/parser.test b/mysql-test/t/parser.test > index 31c6ab9..0a19b03 100644 > --- a/mysql-test/t/parser.test > +++ b/mysql-test/t/parser.test > @@ -765,3 +765,259 @@ DROP TABLE t1; > CREATE TABLE t1(a INT); > SELECT * FROM t1 JOIN ((SELECT 1 AS b) UNION ALL (SELECT 2 AS b) ORDER BY b DESC) s1 WHERE a=1; > DROP TABLE t1; > + > + > +--echo # > +--echo # Test of collective fix for three parser bugs: > +--echo # > +--echo # Bug #17727401, Bug #17426017, Bug #17473479: > +--echo # The server accepts wrong syntax and then fails in different ways > +--echo # > + > +CREATE TABLE t1 (i INT); > + > +--echo # bug #17426017 > +--error ER_PARSE_ERROR > +SELECT (SELECT EXISTS(SELECT * LIMIT 1 ORDER BY VALUES (c00))); > + > +--echo # bug#17473479 > +CREATE TABLE a(a int); > +CREATE TABLE b(a int); > +--error ER_PARSE_ERROR > +DELETE FROM b ORDER BY(SELECT 1 FROM a ORDER BY a ORDER BY a); > +DROP TABLE a, b; > + > +--echo # bug #17727401 > +--error ER_PARSE_ERROR > +SELECT '' IN (SELECT '1' c FROM t1 ORDER BY '' ORDER BY '') FROM t1; > + > +--echo # regression & coverage tests > + > +--echo # uniform syntax for FROM DUAL clause: > + > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + FOR UPDATE; > + > +--error ER_ORDER_WITH_PROC > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + PROCEDURE ANALYSE() FOR UPDATE; > + > +SELECT 1 FROM > + (SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + FOR UPDATE) a; > + > +--error ER_WRONG_USAGE > +SELECT 1 FROM > + (SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + PROCEDURE ANALYSE() FOR UPDATE) a; > + > +SELECT 1 FROM t1 > + WHERE EXISTS(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + FOR UPDATE); > + > +--error ER_WRONG_USAGE > +SELECT 1 FROM t1 > + WHERE EXISTS(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + PROCEDURE ANALYSE() FOR UPDATE); > + > +SELECT 1 FROM t1 > +UNION > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + FOR UPDATE; > + > +--error ER_WRONG_USAGE > +SELECT 1 FROM t1 > +UNION > +SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + PROCEDURE ANALYSE() FOR UPDATE; > + > +--error ER_WRONG_USAGE > +SELECT 1 FROM DUAL PROCEDURE ANALYSE() > +UNION > +SELECT 1 FROM t1; > + > +(SELECT 1 FROM t1) > +UNION > +(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + FOR UPDATE); > + > +--error ER_WRONG_USAGE > +(SELECT 1 FROM t1) > +UNION > +(SELECT 1 FROM DUAL WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 > + PROCEDURE ANALYSE() FOR UPDATE); > + > +--echo # "FOR UPDATE" tests > + > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1; > +SELECT 1 FROM t1 FOR UPDATE UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1; > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1 FOR UPDATE; > + > + > +--echo # "INTO" clause tests > + > +SELECT 1 FROM t1 INTO @var17727401; > +SELECT 1 FROM DUAL INTO @var17727401; > +SELECT 1 INTO @var17727401; > + > +SELECT 1 INTO @var17727401 FROM t1; > +SELECT 1 INTO @var17727401 FROM DUAL; > + > +--error ER_WRONG_USAGE > +SELECT 1 INTO @var17727401_1 FROM t1 INTO @var17727401_2; > + > +--error ER_WRONG_USAGE > +SELECT 1 INTO @var17727401_1 FROM DUAL > + INTO @var17727401_2; > + > +SELECT 1 INTO @var17727401 FROM t1 WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1; > +SELECT 1 FROM t1 WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1 INTO @var17727401; > + > +--error ER_PARSE_ERROR > +SELECT 1 FROM t1 WHERE 1 INTO @var17727401 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1; > + > +--error ER_WRONG_USAGE > +SELECT 1 INTO @var17727401_1 > + FROM t1 WHERE 1 GROUP BY 1 HAVING 1 ORDER BY 1 LIMIT 1 > + INTO @var17727401_2; > + > +--error ER_PARSE_ERROR > +SELECT (SELECT 1 FROM t1 INTO @var17727401); > +--error ER_PARSE_ERROR > +SELECT 1 FROM (SELECT 1 FROM t1 INTO @var17727401) a; > +--error ER_PARSE_ERROR > +SELECT EXISTS(SELECT 1 FROM t1 INTO @var17727401); > + > +--error ER_WRONG_USAGE > +SELECT 1 FROM t1 INTO @var17727401 UNION SELECT 1 FROM t1 INTO t1; > +--error ER_WRONG_USAGE > +(SELECT 1 FROM t1 INTO @var17727401) UNION (SELECT 1 FROM t1 INTO t1); > + > +SELECT 1 FROM t1 UNION SELECT 1 FROM t1 INTO @var17727401; > + > +--error ER_WRONG_USAGE > +SELECT 1 INTO @var17727401 FROM t1 PROCEDURE ANALYSE(); > + > +--error ER_WRONG_USAGE > +SELECT 1 FROM t1 PROCEDURE ANALYSE() INTO @var17727401; > + > +--echo # ORDER and LIMIT clause combinations > + > +# Limited support for (SELECT ...) ORDER/LIMIT: > + > +(SELECT 1 FROM t1 ORDER BY 1) ORDER BY 1; > +(SELECT 1 FROM t1 LIMIT 1) LIMIT 1; > + > +--error ER_PARSE_ERROR > +((SELECT 1 FROM t1 ORDER BY 1) ORDER BY 1) ORDER BY 1; > +--error ER_PARSE_ERROR > +((SELECT 1 FROM t1 LIMIT 1) LIMIT 1) LIMIT 1; > + > +(SELECT 1 FROM t1 ORDER BY 1) LIMIT 1; > +(SELECT 1 FROM t1 LIMIT 1) ORDER BY 1; > + > +--error ER_PARSE_ERROR > +((SELECT 1 FROM t1 ORDER BY 1) LIMIT 1) ORDER BY 1); > +--error ER_PARSE_ERROR > +((SELECT 1 FROM t1 LIMIT 1) ORDER BY 1) LIMIT 1); > + > +# ORDER/LIMIT and UNION: > + > +let $q=SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1; > +eval $q; > +eval SELECT ($q); > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1; > +eval $q; > +eval SELECT ($q); > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 UNION SELECT 1 FROM t1 ORDER BY 1 LIMIT 1; > +eval $q; > +eval SELECT ($q); > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 UNION SELECT 1 FROM t1 LIMIT 1 ORDER BY 1; > +--error ER_PARSE_ERROR > +eval $q; > +--error ER_PARSE_ERROR > +eval SELECT ($q); > +--error ER_PARSE_ERROR > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1; > +--error ER_WRONG_USAGE > +eval $q; > +--error ER_WRONG_USAGE > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1; > +--error ER_WRONG_USAGE > +eval $q; > +--error ER_WRONG_USAGE > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 ORDER BY 1 LIMIT 1 UNION SELECT 1 FROM t1; > +--error ER_WRONG_USAGE > +eval $q; > +--error ER_WRONG_USAGE > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 LIMIT 1 ORDER BY 1 UNION SELECT 1 FROM t1; > +--error ER_PARSE_ERROR > +eval $q; > +--error ER_PARSE_ERROR > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 ORDER BY 1; > +--error ER_WRONG_USAGE > +eval $q; > +--error ER_WRONG_USAGE > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 LIMIT 1; > +--error ER_WRONG_USAGE > +eval $q; > +--error ER_WRONG_USAGE > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 LIMIT 1 UNION SELECT 1 FROM t1 ORDER BY 1; > +--error ER_WRONG_USAGE > +eval $q; > +--error ER_WRONG_USAGE > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +let $q=SELECT 1 FROM t1 ORDER BY 1 UNION SELECT 1 FROM t1 LIMIT 1; > +--error ER_WRONG_USAGE > +eval $q; > +--error ER_WRONG_USAGE > +eval SELECT ($q); > +--error ER_WRONG_USAGE > +eval SELECT 1 FROM ($q) a; > + > +DROP TABLE t1; > + > +--echo # > +--echo # MDEV-8380: Subquery parse error > +--echo # > +CREATE TABLE t1 ( a INT); > +INSERT INTO t1 VALUES ( 2 ); > +SELECT * > +FROM ( (SELECT a FROM t1 ORDER BY a) UNION (SELECT 1 as b ORDER BY b ) ) AS a1 > +WHERE a1.a = 1 OR a1.a = 2; > +DROP TABLE t1; > diff --git a/mysql-test/t/subselect.test b/mysql-test/t/subselect.test > index 80efe7a..8fb9658 100644 > --- a/mysql-test/t/subselect.test > +++ b/mysql-test/t/subselect.test > @@ -4274,8 +4274,7 @@ SELECT * FROM ( > > # This was not allowed previously. Possibly, it should be allowed on the future. > # For now, the intent is to keep the fix as non-intrusive as possible. > ---error ER_PARSE_ERROR > -SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a); > +SELECT * FROM ((SELECT 1 a) UNION SELECT 1 a) q; > SELECT * FROM (SELECT 1 a UNION (SELECT 1 a)) alias; > SELECT * FROM (SELECT 1 UNION SELECT 1) t1a; > --error ER_PARSE_ERROR > @@ -4310,7 +4309,7 @@ SELECT * FROM (SELECT 1 a UNION SELECT 1 a ORDER BY a LIMIT 1) t1a; > # aliases after. > # > SELECT * FROM t1 JOIN (SELECT 1 UNION SELECT 1) alias ON 1; > ---error ER_PARSE_ERROR > +--error ER_DERIVED_MUST_HAVE_ALIAS > SELECT * FROM t1 JOIN ((SELECT 1 UNION SELECT 1)) ON 1; > --error ER_PARSE_ERROR > SELECT * FROM t1 JOIN (t1 t1a UNION SELECT 1) ON 1; > @@ -4402,8 +4401,9 @@ SELECT ((SELECT 1 UNION SELECT 1 UNION SELECT 1)); > > --error ER_PARSE_ERROR > SELECT * FROM ( SELECT 1 UNION ( SELECT 1 UNION SELECT 1 ) ); > ---error ER_PARSE_ERROR > +--error ER_DERIVED_MUST_HAVE_ALIAS > SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ); > +SELECT * FROM ( ( SELECT 1 UNION SELECT 1 ) UNION SELECT 1 ) a; > SELECT * FROM ( SELECT 1 UNION SELECT 1 UNION SELECT 1 ) a; > > --error ER_PARSE_ERROR > diff --git a/mysql-test/t/union.test b/mysql-test/t/union.test > index b8b040b..56fb2e3 100644 > --- a/mysql-test/t/union.test > +++ b/mysql-test/t/union.test > @@ -26,8 +26,12 @@ select 't1',b,count(*) from t1 group by b UNION select 't2',b,count(*) from t2 g > --error 1250 > (select a,b from t1 limit 2) union all (select a,b from t2 order by a limit 1) order by t1.b; > explain extended (select a,b from t1 limit 2) union all (select a,b from t2 order by a limit 1) order by b desc; > +select count(*) from ( > +(select a,b from t1 limit 2) union all (select a,b from t2 order by a)) q; > (select sql_calc_found_rows a,b from t1 limit 2) union all (select a,b from t2 order by a) limit 2; > select found_rows(); > +select count(*) from ( > +select a,b from t1 union all select a,b from t2) q; > select sql_calc_found_rows a,b from t1 union all select a,b from t2 limit 2; > select found_rows(); > > @@ -206,18 +210,30 @@ create table t2 (a int); > insert into t2 values (3),(4),(5); > > # Test global limits > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1) UNION all (SELECT * FROM t2)) q; > (SELECT SQL_CALC_FOUND_ROWS * FROM t1) UNION all (SELECT * FROM t2) LIMIT 1; > select found_rows(); > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2)) q; > (SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2) LIMIT 2; > select found_rows(); > > # Test cases where found_rows() should return number of returned rows > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2)) q; > (SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION all (SELECT * FROM t2); > select found_rows(); > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1) UNION all (SELECT * FROM t2 LIMIT 1)) q; > (SELECT SQL_CALC_FOUND_ROWS * FROM t1) UNION all (SELECT * FROM t2 LIMIT 1); > select found_rows(); > (SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION SELECT * FROM t2 LIMIT 1; > select found_rows(); > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION SELECT * FROM t2) q; > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION SELECT * FROM t2 LIMIT 1; > +select found_rows(); > > # In these case found_rows() should work > --error ER_WRONG_USAGE > @@ -226,20 +242,41 @@ SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION all SELECT * FROM t2 LIMIT 2; > select found_rows(); > > # The following examples will not be exact > +SELECT COUNT(*) FROM ( > +SELECT * FROM t1 UNION SELECT * FROM t2) q; > SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION SELECT * FROM t2 LIMIT 2; > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION all SELECT * FROM t2) q; > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION all SELECT * FROM t2 LIMIT 2; > select found_rows(); > +SELECT COUNT(*) FROM ( > +SELECT * FROM t1 UNION all SELECT * FROM t2) q; > SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION SELECT * FROM t2 LIMIT 100; > select found_rows(); > --error ER_WRONG_USAGE > SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 100 UNION SELECT * FROM t2; > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 100) UNION SELECT * FROM t2) q; > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 100) UNION SELECT * FROM t2; > --error ER_WRONG_USAGE > SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1 UNION SELECT * FROM t2; > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION SELECT * FROM t2) q; > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION SELECT * FROM t2; > --error ER_WRONG_USAGE > SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1 UNION SELECT * FROM t2 LIMIT 2; > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 LIMIT 1) UNION SELECT * FROM t2) q; > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 LIMIT 1) UNION SELECT * FROM t2 LIMIT 2; > SELECT SQL_CALC_FOUND_ROWS * FROM t1 UNION SELECT * FROM t2 LIMIT 2,2; > select found_rows(); > +SELECT COUNT(*) FROM ( > +SELECT * FROM t1 UNION SELECT * FROM t2) q; > --error ER_WRONG_USAGE > SELECT SQL_CALC_FOUND_ROWS * FROM t1 limit 2,2 UNION SELECT * FROM t2; > +SELECT COUNT(*) FROM ( > +(SELECT * FROM t1 limit 2,2) UNION SELECT * FROM t2) q; > +(SELECT SQL_CALC_FOUND_ROWS * FROM t1 limit 2,2) UNION SELECT * FROM t2; > > # Test some limits with ORDER BY > SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a desc LIMIT 1; Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 6811945e: MDEV-6756: map a linux pid (child pid) to a connection id shown in the output of SHOW PROCESSLIST
by Sergei Golubchik 14 Sep '15

14 Sep '15

Hi, Sanja! On Aug 12, sanja(a)mariadb.com wrote: > revision-id: 6811945eaf6a6e979d92739399b084bf4e00ae2d (mariadb-10.1.6-12-g6811945e) > parent(s): 86a3613d4e981c341e38291c9eeec5dc9f836fae > committer: Oleksandr Byelkin > timestamp: 2015-08-12 23:09:48 +0200 > message: > > MDEV-6756: map a linux pid (child pid) to a connection id shown in the output of SHOW PROCESSLIST > > Added tid & pid for Linux. > > diff --git a/mysql-test/r/information_schema.result b/mysql-test/r/information_schema.result > index 2a918ad..5b3370b 100644 > --- a/mysql-test/r/information_schema.result > +++ b/mysql-test/r/information_schema.result > @@ -2066,3 +2066,18 @@ Variable_name Value > Opened_tables 3 > drop database mysqltest; > drop database db1; > +SHOW PROCESSLIST; > +Id User Host db Command Time State Info Progress > +# root # NULL Query # # SHOW PROCESSLIST 0.000 > +set @@old_mode=""; > +SHOW PROCESSLIST; Eh, no, please, don't change SHOW PROCESSLIST. Only add new columns to I_S.PROCESSLIST. And without OLD_MODE. > diff --git a/sql/CMakeLists.txt b/sql/CMakeLists.txt > index 033a0e0..d1fbaa3 100644 > --- a/sql/CMakeLists.txt > +++ b/sql/CMakeLists.txt > @@ -143,6 +143,10 @@ SET (SQL_SOURCE > ${MYSYS_LIBWRAP_SOURCE} > ) > > +IF (CMAKE_SYSTEM_NAME MATCHES "Linux") > + ADD_DEFINITIONS(-DHAVE_TID_SYSCALL) > +ENDIF() Don't match OS name please, better use #ifdef HAVE_SYS_SYSCALL_H #include <sys/syscall.h> #endif ... #ifdef __NR_gettid ... #endif > + > IF (CMAKE_SYSTEM_NAME MATCHES "Linux" OR > CMAKE_SYSTEM_NAME MATCHES "Windows" OR > CMAKE_SYSTEM_NAME MATCHES "SunOS" OR > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index bf161b2..20c03fe 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -2062,6 +2068,13 @@ bool THD::store_globals() > This allows us to move THD to different threads if needed. > */ > mysys_var->id= thread_id; > +#ifdef HAVE_TID_SYSCALL > + os_pid= getpid(); why would you want to have pid there? isn't it the same for all threads? > + os_thread_id= syscall(__NR_gettid); > +#else > + os_pid= 0; > + os_thread_id= 0; > +#endif > real_id= pthread_self(); // For debugging > mysys_var->stack_ends_here= thread_stack + // for consistency, see libevent_thread_proc > STACK_DIRECTION * (long)my_thread_stack_size; I didn't review the rest of the patch as without old_mode and SHOW PROCESSLIST changes most of it will disappear. Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 5fab5b6: MDEV-8764: Wrong error when encrypted table can't be decrypted.
by Sergei Golubchik 13 Sep '15

13 Sep '15

Hi, Jan! On Sep 08, Jan Lindström wrote: > MDEV-8764: Wrong error when encrypted table can't be decrypted. > > Add a new error message when table is encrypted but decryption > fails. Use this new error message on InnoDB/XtraDB. > > diff --git a/include/my_handler_errors.h b/include/my_handler_errors.h > index a7afcfe..9e452b0 100644 > --- a/include/my_handler_errors.h > +++ b/include/my_handler_errors.h > @@ -94,7 +94,8 @@ static const char *handler_error_messages[]= > "Operation was interrupted by end user (probably kill command?)", > "Disk full", > "Incompatible key or row definition between the MariaDB .frm file and the information in the storage engine. You have to dump and restore the table to fix this", > - "Too many words in a FTS phrase or proximity search" > + "Too many words in a FTS phrase or proximity search", > + "Table encrypted but decryption failed" I'm not sure I like the wording ("table is encrypted" adds no useful information, it's not like there can be another error "table is not encrypted but decryption failed"). But that's up to you. And yes, I agree that this situation deserves a dedicated error message. > }; > > #endif /* MYSYS_MY_HANDLER_ERRORS_INCLUDED */ > diff --git a/sql/handler.cc b/sql/handler.cc > index 1d1dae7..566db8a 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -3518,6 +3519,10 @@ void handler::print_error(int error, myf errflag) > my_error(ER_NO_SUCH_TABLE_IN_ENGINE, errflag, table_share->db.str, > table_share->table_name.str); > DBUG_VOID_RETURN; > + case HA_ERR_DECRYPTION_FAILED: > + my_error(ER_DECRYPTION_FAILED, errflag, table_share->db.str, > + table_share->table_name.str); > + DBUG_VOID_RETURN; Is that needed? I'd rather use the default ER_GET_ERRMSG, which would be Got error 192 'Table encrypted but decryption failed" from InnoDB I think this is perfectly adequate error for internal engine errors. ER_xxx is for upper-level errors. > case HA_ERR_RBR_LOGGING_FAILED: > textno= ER_BINLOG_ROW_LOGGING_FAILED; > break; > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 59908dc..ff72a26 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7136,3 +7136,5 @@ ER_KILL_QUERY_DENIED_ERROR > eng "You are not owner of query %lu" > ger "Sie sind nicht Eigentümer von Abfrage %lu" > rus "Вы не являетесь владельцем запроса %lu" > +ER_DECRYPTION_FAILED 42S02 > + eng "Table '%-.192s.%-.192s' encrypted but decryption failed." In general, please prefer %`s.%`s over '%s.%s' the former uses correct identifier quoting, the second does not Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 7fd5554: MDEV-5171: Add support for --innodb-optimize-keys to mysqldump.
by Sergei Golubchik 12 Sep '15

12 Sep '15

Hi, Jan! First, we've discussed adding support for ALTER TABLE ... DISABLE KEYS to InnoDB. Did you try that? That would've been a better approach, generally I'd like to avoid adding plugin-specific options to a mysqldump utility (although, of course, InnoDB is not just any plugin, so we can always make an exception for it). On Jul 27, Jan Lindström wrote: > revision-id: 7fd55542893b6d9e75406e39b7be672d8a8f473c > parent(s): a6ab8ef9d7d465a3f0bc1cff6034ca97575629dd > committer: Jan Lindström > branch nick: 10.0-git > timestamp: 2015-07-27 13:02:57 +0300 > message: > > MDEV-5171: Add support for --innodb-optimize-keys to mysqldump. > > Merged revision 93 from bzr+ssh://bazaar.launchpad.net/+branch/percona-server/5.6/ > authored by Alexey Kopytov. > > However, internal parser is rewriten to support unquoted identifiers and > multi-column keys. > > This patch expands the applicability of InnoDB fast index creation to > mysqldump, ALTER TABLE and OPTIMIZE TABLE as follows: > > 1. mysqldump has now a new option, --innodb-optimize-keys, which changes > the way InnoDB tables are dumped so that secondary and foreign keys are > created after loading the data thus taking advantage of fast index > creation. > > This part of the patch is an implementation of the feature request > reported as MySQL bug #49120. > > More specifically: > > - KEY, UNIQUE KEY and CONSTRAINT specifications are omitted from CREATE > TABLE corresponding to InnoDB tables. > > - an additional ALTER TABLE is issued after dumping the data to create > the previously omitted keys. > > Delaying foreign key creation does not introduce any additional risks as > mysqldump always prepends its output with SET FOREIGN_KEY_CHECKS=0 anyway. > > 2. When ALTER TABLE requires a table copy, secondary keys are now dropped > and recreated later after copying the data. The following restrictions > apply: > > - only non-unique keys can be involved in this optimization > > - if the table contains foreign keys, or a foreign key is being added as > a part of the current ALTER TABLE statement, the optimization is > disabled for all keys. > > This part of the patch is an implementation of the feature request > reported as MySQL bug #57583. > > 3. As OPTIMIZE TABLE is mapped to ALTER TABLE ... ENGINE=InnoDB for > InnoDB tables, it now also benefits from fast index creation with the > same restrictions as for ALTER TABLE. > This is a good comment, but a bit confusing. It should explicitly say somehow that item 1 is what this patch implements and items 2 and 3 are existing InnoDB features. Currently the comment creates an impression that all 1-3 are implemented in the patch. > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index e4683ab..b8874e7 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -84,12 +86,30 @@ > #define IGNORE_DATA 0x01 /* don't dump data for this table */ > #define IGNORE_INSERT_DELAYED 0x02 /* table doesn't support INSERT DELAYED */ > > +typedef enum { > + KEY_TYPE_NONE, > + KEY_TYPE_PRIMARY, > + KEY_TYPE_UNIQUE, > + KEY_TYPE_NON_UNIQUE, > + KEY_TYPE_FOREIGN > +} key_type_t; > + > /* Chars needed to store LONGLONG, excluding trailing '\0'. */ > #define LONGLONG_LEN 20 > > /* Max length GTID position that we will output. */ > #define MAX_GTID_LENGTH 1024 > > +/* Max length of identifier (columnname, keyname, tablename, etc). */ > +#define MAX_ID_LENGTH 65 Use SAFE_NAME_LEN instead, it's defined in mysql_com.h that mysqldump.c already includes > + > +/* Data structure for holding position information > +for simple keyword parser. */ > +typedef struct { > + char* m_start; > + char* m_pos; > +} simple_parser_t; > + > static void add_load_option(DYNAMIC_STRING *str, const char *option, > const char *option_value); > static ulong find_set(TYPELIB *lib, const char *x, uint length, > @@ -2843,6 +3296,9 @@ static uint get_table_structure(char *table, char *db, char *table_type, > > row= mysql_fetch_row(result); > > + if (opt_innodb_optimize_keys && !opt_comments && !strcmp(table_type, "InnoDB")) > + skip_secondary_keys(row[1], table); Okay, so opt_innodb_optimize_keys needs !opt_comments. Please, add a check, like if (opt_innodb_optimize_keys && opt_comments) { fprintf(stderr, "%s: You can't use --innodb-optimize-keys and " "--comments at the same time.\n", my_progname_short); return(EX_USAGE); } a good place for a check like this is get_options() function. > + > is_log_table= general_log_or_slow_log_tables(db, table); > if (is_log_table) > row[1]+= 13; /* strlen("CREATE TABLE ")= 13 */ The parser looked fine on the first glance, but I didn't review every single line of it. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 7206e6f: MDEV-8577: With enforce-storage-engine mysql_upgrade corrupts the schema:
by Sergei Golubchik 11 Sep '15

11 Sep '15

Hi, Jan! On Aug 08, Jan Lindström wrote: > MDEV-8577: With enforce-storage-engine mysql_upgrade corrupts the schema: > ALTER TABLE should either bypass enforce-storage-engine, or mysql_upgrade > should refuse to run > > Allow user to alter contents of existing table without enforcing > storage engine. However, enforce storage engine on ALTER TABLE x ENGINE=y; Good idea, I like this behavior. > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index 69c87dd..60c4407 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -9801,8 +9801,12 @@ static bool check_engine(THD *thd, const char *db_name, > *new_engine= ha_checktype(thd, req_engine, no_substitution); > DBUG_ASSERT(*new_engine); > > - /* Enforced storage engine should not be used on bootstrap */ > - if (!in_bootstrap) > + /* Enforced storage engine should not be used on bootstrap > + and in ALTER TABLE that does not use explicit ENGINE = x to > + avoid unwanted unrelated changes.*/ > + if (!in_bootstrap && Note that !in_bootstrap belongs to another bug fix, please don't accidentally push it together with this one. > + !(thd->lex->sql_command == SQLCOM_ALTER_TABLE && > + !(create_info->used_fields & HA_CREATE_USED_ENGINE))) > enf_engine= thd->variables.enforced_table_plugin ? > plugin_hton(thd->variables.enforced_table_plugin) : NULL; Looks good, ok to push! Regards, Sergei

1 0

[Maria-developers] MDEV-8372 Use helper methods introduced in MDEV-7824 all around the code
by Alexander Barkov 11 Sep '15

11 Sep '15

Hi Sergei, Please review my patch for mdev-8372. I found only a few places to reuse the new methods. Also, had to add some more helper methods to make the code clearer. Thanks.

2 3

Re: [Maria-developers] [Commits] 32b1497: MDEV-7990: ERROR 1526 when procedure executed for second time ALTER TABLE partition ... pMAX values less than MAXVALUE
by Sergei Golubchik 11 Sep '15

11 Sep '15

Hi, Sanja! On Aug 06, sanja(a)mariadb.com wrote: > revision-id: 32b14970a1b88571aa96f604e5c59ab6bf1b298c > parent(s): aa5095627e2619bdad7916d33d1016802a84a9e1 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-08-06 14:00:48 +0200 > message: > > MDEV-7990: ERROR 1526 when procedure executed for second time ALTER > TABLE partition ... pMAX values less than MAXVALUE > > Added cleaning partition data which are saved in LEX structure to make > possible correct re-execution of a SP or PS using the code. This seems to be wrong now. It's an old comment, but now your fix works differently, doesn't it? The patch is ok to push, thanks! Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 0a1b60a: MDEV-8450: PATCH] Wrong macro expansion in Query_cache::send_result_to_client()
by Sergey Vojtovich 06 Sep '15

06 Sep '15

Hi Sanja, ok to push, a few minor notes JFYI inline. I found only one real bug, but generally I agree that we should fix these fragile macros. On Tue, Sep 01, 2015 at 11:47:09AM +0200, sanja(a)mariadb.com wrote: > revision-id: 0a1b60a8aee4bce3235fd5a1d8cb2231b51ff5cc (mariadb-10.0.21-7-g0a1b60a) > parent(s): 4b41e3c7f33714186c97a6cc2e6d3bb93b050c61 > committer: Oleksandr Byelkin > timestamp: 2015-09-01 11:47:06 +0200 > message: > > MDEV-8450: PATCH] Wrong macro expansion in Query_cache::send_result_to_client() > > Expression in macro protected by () > > diff --git a/include/maria.h b/include/maria.h > index 16a9bea..cdaeade 100644 > --- a/include/maria.h > +++ b/include/maria.h > @@ -43,7 +43,7 @@ extern "C" { > #define MARIA_NAME_IEXT ".MAI" > #define MARIA_NAME_DEXT ".MAD" > /* Max extra space to use when sorting keys */ > -#define MARIA_MAX_TEMP_LENGTH 2*1024L*1024L*1024L > +#define MARIA_MAX_TEMP_LENGTH (2*1024L*1024L*1024L) > /* Possible values for maria_block_size (must be power of 2) */ > #define MARIA_KEY_BLOCK_LENGTH 8192 /* default key block length */ > #define MARIA_MIN_KEY_BLOCK_LENGTH 1024 /* Min key block length */ Was this used at all? May be remove it? > diff --git a/sql/log_slow.h b/sql/log_slow.h > index 2ae07da..3ae2060 100644 > --- a/sql/log_slow.h > +++ b/sql/log_slow.h > @@ -16,23 +16,23 @@ > /* Defining what to log to slow log */ > > #define LOG_SLOW_VERBOSITY_INIT 0 > -#define LOG_SLOW_VERBOSITY_INNODB 1 << 0 > -#define LOG_SLOW_VERBOSITY_QUERY_PLAN 1 << 1 > -#define LOG_SLOW_VERBOSITY_EXPLAIN 1 << 2 > +#define LOG_SLOW_VERBOSITY_INNODB (1 << 0) Was LOG_SLOW_VERBOSITY_INNODB used at all? May be remove it? > +#define LOG_SLOW_VERBOSITY_QUERY_PLAN (1 << 1) > +#define LOG_SLOW_VERBOSITY_EXPLAIN (1 << 2) > > #define QPLAN_INIT QPLAN_QC_NO > > -#define QPLAN_ADMIN 1 << 0 > -#define QPLAN_FILESORT 1 << 1 > -#define QPLAN_FILESORT_DISK 1 << 2 > -#define QPLAN_FULL_JOIN 1 << 3 > -#define QPLAN_FULL_SCAN 1 << 4 > -#define QPLAN_QC 1 << 5 > -#define QPLAN_QC_NO 1 << 6 > -#define QPLAN_TMP_DISK 1 << 7 > -#define QPLAN_TMP_TABLE 1 << 8 > -#define QPLAN_FILESORT_PRIORITY_QUEUE 1 << 9 > +#define QPLAN_ADMIN (1 << 0) > +#define QPLAN_FILESORT (1 << 1) > +#define QPLAN_FILESORT_DISK (1 << 2) > +#define QPLAN_FULL_JOIN (1 << 3) > +#define QPLAN_FULL_SCAN (1 << 4) > +#define QPLAN_QC (1 << 5) > +#define QPLAN_QC_NO (1 << 6) > +#define QPLAN_TMP_DISK (1 << 7) > +#define QPLAN_TMP_TABLE (1 << 8) > +#define QPLAN_FILESORT_PRIORITY_QUEUE (1 << 9) > > /* ... */ > -#define QPLAN_MAX ((ulong) 1) << 31 /* reserved as placeholder */ > +#define QPLAN_MAX (((ulong) 1) << 31) /* reserved as placeholder */ > log_slow.h was fixed in rev.203f4d419 in my 10.0. Did you use outdated tree? I may be wrong, but there seem to be no real bug with QPLAN_QC_NO currently. Just because nobody sets it. So in fact it clears QPLAN_ADMIN, which FWICS is never set at this point. But it is a good to have it fixed nevertheless. Btw, QPLAN_QC_NO doesn't seem to be used in fact. May be remove it? > diff --git a/storage/archive/archive_test.c b/storage/archive/archive_test.c > index d01c1e0..bb052b8 100644 > --- a/storage/archive/archive_test.c > +++ b/storage/archive/archive_test.c > @@ -33,7 +33,7 @@ > > #define ARCHIVE_ROW_HEADER_SIZE 4 > > -#define BUFFER_LEN 1024 + ARCHIVE_ROW_HEADER_SIZE > +#define BUFFER_LEN (1024 + ARCHIVE_ROW_HEADER_SIZE) > > char test_string[BUFFER_LEN]; Alright, there was a bug in this line: assert(write_length != count * BUFFER_LEN); > diff --git a/strings/ctype.c b/strings/ctype.c > index 048fbe3..d8a1dd7 100644 > --- a/strings/ctype.c > +++ b/strings/ctype.c > @@ -264,7 +264,7 @@ static const struct my_cs_file_section_st > } > > #define MY_CS_CSDESCR_SIZE 64 > -#define MY_CS_TAILORING_SIZE 32*1024 > +#define MY_CS_TAILORING_SIZE (32*1024) > #define MY_CS_UCA_VERSION_SIZE 64 > #define MY_CS_CONTEXT_SIZE 64 Was this used at all? May be remove it? Regards, Sergey

2 1

Re: [Maria-developers] [Commits] 62fe87c: MDEV-8663: IF Statement returns multiple values erroneously (or Assertion `!null_value' failed in Item::send(Protocol*, String*))
by Alexander Barkov 04 Sep '15

04 Sep '15

Hi Sanja, From my understanding, the patch fixes consequences of the problem, but not the problem itself. It very likely will still have crashes with slightly modified SQL scripts. The source of the problem is that Item_func_hybrid_result_type::val_decimal() does not return NULL when int_op() or real_op() set null_value to "true". By the way, compare Item_func_hybrid_result_type::val_decimal() with a very similar method Item_func_hybrid_result_type::val_str(). The latter already handle the INT_RESULT and REAL_RESULT cases well and return NULL properly in case of null_value. get_date(), val_int(), val_real() also look fine for me. So the are problems only in val_decimal(). I propose a small patch that fixes the source of the problem. This is the small patch: > @@ -1038,17 +1026,21 @@ my_decimal *Item_func_hybrid_result_type::val_decimal(my_decimal *decimal_value) > case INT_RESULT: > { > longlong result= int_op(); > + if (null_value) > + return NULL; > int2my_decimal(E_DEC_FATAL_ERROR, result, unsigned_flag, decimal_value); > break; > } > case REAL_RESULT: > { > double result= (double)real_op(); > + if (null_value) > + return NULL; > double2my_decimal(E_DEC_FATAL_ERROR, result, decimal_value); > break; > } I also propose an extended patch version, to cover the code with more DBUG_ASSERTs, so in case of the null_value related bugs it crashes much earlier than in Item::send(). It will be easier to debug problems this way. To avoid much code duplication, I added three private helper methods. The extended version of the patch is attached. The scripts from the bug report start to return expected results, with no crashes in DEBUG build, and "mtr --suite=main" works fine. I have not run the entire mtr though, to save time. Greetings. On 09/03/2015 08:00 PM, sanja(a)mariadb.com wrote: > revision-id: 62fe87c5a239c7304b245ce5954c87aecb6382ad (mariadb-5.5.45-1-g62fe87c) > parent(s): fa51f70dc68fe2f3afe943e2c81fcbdb34f16cad > committer: Oleksandr Byelkin > timestamp: 2015-09-03 18:00:43 +0200 > message: > > MDEV-8663: IF Statement returns multiple values erroneously (or Assertion `!null_value' failed in Item::send(Protocol*, String*)) > > keeping contract: NULL value mean NULL pointer in val_str and val_deciman. > > --- > mysql-test/r/func_if.result | 17 +++++++++++++++++ > mysql-test/t/func_if.test | 14 ++++++++++++++ > sql/item_cmpfunc.cc | 6 ++++-- > sql/item_func.cc | 5 +++++ > 4 files changed, 40 insertions(+), 2 deletions(-) > > diff --git a/mysql-test/r/func_if.result b/mysql-test/r/func_if.result > index c7f548a..61f63cc 100644 > --- a/mysql-test/r/func_if.result > +++ b/mysql-test/r/func_if.result > @@ -234,3 +234,20 @@ SELECT if(1, NULL, (SELECT min('hello'))); > if(1, NULL, (SELECT min('hello'))) > NULL > End of 5.2 tests > +# > +# MDEV-8663: IF Statement returns multiple values erroneously > +# (or Assertion `!null_value' failed in Item::send(Protocol*, String*) > +# > +CREATE TABLE `t1` ( > +`datas` VARCHAR(25) NOT NULL > +) ENGINE=InnoDB DEFAULT CHARSET=utf8; > +Warnings: > +Warning 1286 Unknown storage engine 'InnoDB' > +Warning 1266 Using storage engine MyISAM for table 't1' > +INSERT INTO `t1` VALUES ('1,2'), ('2,3'), ('3,4'); > +SELECT IF(FIND_IN_SET('1', `datas`), 1.5, IF(FIND_IN_SET('2', `datas`), 2, NULL)) AS `First`, '1' AS `Second`, '2' AS `Third` FROM `t1`; > +First Second Third > +1.5 1 2 > +2.0 1 2 > +NULL 1 2 > +drop table t1; > diff --git a/mysql-test/t/func_if.test b/mysql-test/t/func_if.test > index 2b89a61..8fdba77 100644 > --- a/mysql-test/t/func_if.test > +++ b/mysql-test/t/func_if.test > @@ -206,6 +206,20 @@ SELECT if(1, NULL, (SELECT min('hello'))); > > --echo End of 5.2 tests > > +--echo # > +--echo # MDEV-8663: IF Statement returns multiple values erroneously > +--echo # (or Assertion `!null_value' failed in Item::send(Protocol*, String*) > +--echo # > +CREATE TABLE `t1` ( > +`datas` VARCHAR(25) NOT NULL > +) ENGINE=InnoDB DEFAULT CHARSET=utf8; > + > +INSERT INTO `t1` VALUES ('1,2'), ('2,3'), ('3,4'); > + > +SELECT IF(FIND_IN_SET('1', `datas`), 1.5, IF(FIND_IN_SET('2', `datas`), 2, NULL)) AS `First`, '1' AS `Second`, '2' AS `Third` FROM `t1`; > + > +drop table t1; > + > --disable_query_log > # Restore timezone to default > set time_zone= @@global.time_zone; > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 998cb1c..4f07318 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -2692,7 +2692,8 @@ Item_func_if::str_op(String *str) > String *res=arg->val_str(str); > if (res) > res->set_charset(collation.collation); > - null_value=arg->null_value; > + if (null_value=arg->null_value) > + res= NULL; > return res; > } > > @@ -2703,7 +2704,8 @@ Item_func_if::decimal_op(my_decimal *decimal_value) > DBUG_ASSERT(fixed == 1); > Item *arg= args[0]->val_bool() ? args[1] : args[2]; > my_decimal *value= arg->val_decimal(decimal_value); > - null_value= arg->null_value; > + if (null_value= arg->null_value) > + value= NULL; > return value; > } > > diff --git a/sql/item_func.cc b/sql/item_func.cc > index 2f51308..cbb8aa0 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -924,6 +924,7 @@ String *Item_func_hybrid_result_type::val_str(String *str) > ltime.time_type= mysql_type_to_time_type(field_type()); > str->length(my_TIME_to_str(&ltime, const_cast<char*>(str->ptr()), decimals)); > str->set_charset(&my_charset_bin); > + DBUG_ASSERT(!null_value); > return str; > } > return str_op(&str_value); > @@ -932,6 +933,7 @@ String *Item_func_hybrid_result_type::val_str(String *str) > case IMPOSSIBLE_RESULT: > DBUG_ASSERT(0); > } > + DBUG_ASSERT(!null_value || (str == NULL)); > return str; > } > > @@ -1071,7 +1073,10 @@ my_decimal *Item_func_hybrid_result_type::val_decimal(my_decimal *decimal_value) > } > String *res; > if (!(res= str_op(&str_value))) > + { > + null_value= 1; > return NULL; > + } > > str2my_decimal(E_DEC_FATAL_ERROR, (char*) res->ptr(), > res->length(), res->charset(), decimal_value); > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

1 0

[Maria-developers] [GSoC] MDEV-4674 Port InnoDB memcached interface to MariaDB - Merging the code
by Piotr Jurkiewicz 02 Sep '15

02 Sep '15

Hi again, there are some issues which I think needs to be resolved before merging the code. 1. Bundle libevent or not? Memcached daemon is built with the help of libevent library. MySQL decided to bundle this library inside its sources, in 'libevent' top level directory. This allows to compile and run the plugin on machines without installed system-wide libevent library. The bundled libevent is used by default during the compilation of plugin. Optionally, it can be specified in CMake variable to use system-wide libevent instead. It needs to be decided whether MariaDB should bundle libevent as well inside its sources. Possible outcomes: a) Leave bundled libevent as is, in the 'libevent' top level directory. b) Move bundled libevent into the 'daemon_memcached' plugin directory, in order to not clutter the top level directory. c) Remove bundled libevent. After that, system libevent headers will become compile-time dependency and system libevent shared library runtime dependency of the plugin. 2. How to transfer InnoDB API callback array pointer to the plugin? InnoDB Memcached interface engine needs to obtain the pointer to InnoDB API callback array in some way. MySQL originally implemented that in 'sql/sql_plugin.cc:plugin_initialize()' in the following way: https://github.com/piotrjurkiewicz/mariadb-server/commit/4eb8fd9c211f396694… As you can see, this is not a clean solution. Possible outcomes: a) Leave the MySQL's solution as is. b) Figure out a better way to transfer the pointer. 3. Where should I post the documentation? 4. How should I prepare the pull request? Should I rebase my branch on the top of current branch before that? I looking forward for your suggestions and decisions. Piotr

2 1

Re: [Maria-developers] MDEV-8389 patch review (part#1)
by Sergey Petrunia 28 Aug '15

28 Aug '15

Hi Diwas, On Thu, Aug 27, 2015 at 09:23:55PM +0530, Diwas Joshi wrote: > hello sergei, I am attaching two patches with this mail. > MDEV-8389_first is the original patch with mysql_create_table() method used > to create tables. I have fixed the crash that was occurring with queries > like select name from f21('aaa') where name like '%foo%'; So, this patch is > giving the results. > MDEV-8389_second contains use of create_tmp_table() so far the table is > being created, but the queries can't see it so it is still giving errors > like table not present. I guess we'll have to fix the Name Resolution for > queries to be able to use the created table. > > Also both patches have fix for MDEV-8650. I've looked at the patch. Unfortunately, it gives little confidence that it will work in all cases. There are a lot of weird things. To name a few: > + tmp_table_param->schema_table = 1; why? ... > + switch (field->sql_type) > + { > + case MYSQL_TYPE_VARCHAR: > + case MYSQL_TYPE_VAR_STRING: > + case MYSQL_TYPE_STRING: > + case MYSQL_TYPE_SET: > + field->pack_flag= 0; > + break; ... This big switch statement is copied from Create_field::init_for_tmp_table. Why is it copied (and not factored out?) If there is a need to copy the code from that function into here, why do you not also copy this part: pack_flag|= (maybe_null ? FIELDFLAG_MAYBE_NULL : 0) | (is_unsigned ? 0 : FIELDFLAG_DECIMAL); > + TABLE *vtable= create_virtual_tmp_table(thd, sp->m_cols_list); ... > + if (! (table = create_tmp_table(thd, tmp_table_param, *types, > + (ORDER*) 0, 1, 1, > + TMP_TABLE_ALL_COLUMNS, HA_POS_ERROR, > + sp->m_table_alias.str))) why create_virtual_tmp_table and then create_tmp_table? 'vtable' is created and then not freed. I look at the comment near create_virtual_tmp_table() definition and see: The table is created in THD mem_root, so are the table's fields. Consequently, if you don't BLOB fields, you don't need to free it. But what if the table does have blob fields? > + table->file->extra(HA_EXTRA_IGNORE_DUP_KEY); What's this? I grep for HA_EXTRA_IGNORE_DUP_KEY and find this explanation: HA_EXTRA_IGNORE_DUP_KEY: HA_EXTRA_NO_IGNORE_DUP_KEY: Informs the handler to we will not stop the transaction if we get an duplicate key errors during insert/upate. Do we expect to get duplicate key errors when inserting into the table? > +bool execute_table_function(THD *thd, sp_head *sp, List<Item> *value_list) > +{ > + /* > + We never write CALL statements into binlog: > + - If the mode is non-prelocked, each statement will be logged > + separately. > + - If the mode is prelocked, the invoking statement will care > + about writing into binlog. > + So just execute the statement. How is CALL statement relevant to executing a table function? I think at this point it is clear that this patch needs more work before it can be pushed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] MDEV-8389 patch review (part#1)
by Sergey Petrunia 27 Aug '15

27 Aug '15

On Sat, Aug 22, 2015 at 02:10:58AM +0530, Diwas Joshi wrote: > Hello Sergei, sorry for late reply, i was busy finishing my office > work yesterday. Please find some of my doubts below (I would come on > irc to discuss but I am travelling today and will be working overnight > in the train, I hope to be on irc tomorrow). > > Regarding use of storage engine-- If one uses SET > default_tmp_storage_engine, to set the default storage engine for > temporary tables in the current session, it is using that default > storage engine. In my function I have used > create_info.use_default_db_type(thd) which does the work. > (Referencing https://dev.mysql.com/doc/refman/5.6/en/storage-engine-setting.html) > > Regarding the name resolution error. I didn't know it but if there is > already a table present in the database with same name, temporary > table is still being created and used int the function. so that is one > problem solved. For the problem of a function being used multiple > times in a query, I was just curious as how would we solve it if we > switch to create_tmp_table(). As I mentioned before, the solution should be: >> - Patch the name resolution code so that the statements inside a Table Function >> see the return table of that Table Function. "Name resolution" is a process of finding which table should be used when a query refers to table named TBL. The process currently is: 1. Look at user-created temporary tables with name TBL. 2. If none found, look at the regular database tables in the current database with the name TBL. Our problem is that tables created with mysql_create_table() are visible to step#1, and all parts of the query can see them. This includes those parts of the query that we don't want to see the table: other stored procedures, the query that invokes the table function, etc. Temporary tables that are created with create_tmp_table() are not visible to step #1. We will not have the problem of temporary table being visible where it should not be visible. We will have the problem though, that temporary table must be visible where it needs to be visible. This is achieved by modifying step#1 in Name Resolution code to also look at the name of the returned table when we are in a table function. > Still looking into crash because of select name from f21('aaa') where > name like '%foo%' and errors MDEV-8649 and MDEV-8650; > Other errors in the patch have been fixed. > BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MDEV-8643 Valgrind error in mroonga/storage.index_multiple_column_unique_datetime_index_read
by Alexander Barkov 27 Aug '15

27 Aug '15

Hi Kentoku and Kouhei, FYI, I just filed this bug report: https://mariadb.atlassian.net/browse/MDEV-8643 Can you please have a look into this? Thanks!

1 1

[Maria-developers] New Hacker
by Aniket Shukla 27 Aug '15

27 Aug '15

Hey MariaDB team,this is Aniket.I'm a final year undegraduate student and would like to contribute to MariaDB.Since this is my first open source project, I have no idea where to begin.I have good knowledge of python,c++ and have benn using MySQL for my previous projects. Any help would be welcomed.

2 1

[Maria-developers] [GSoC] MDEV-4674 Port InnoDB memcached interface to MariaDB - Final report
by Piotr Jurkiewicz 25 Aug '15

25 Aug '15

Hi all, as the pencils down date has passed, I am posting a report on what I have during the whole project. Repository of my project is available here: https://github.com/piotrjurkiewicz/mariadb-server/commits/10.1-memcached Some stats: 33 commits before the midterm 60 commits after the midterm 93 commits total What have been done in details: =============================== BEFORE MIDTERM: --------------- 1. Code of InnoDB Memcached plugin has been imported from MySQL into the MariaDB tree. 2. There were some differences between MariaDB and MySQL, which blocked the plugin from out-of-the-box compilation and correct operation. They have been resolved. 3. Test suite for the plugin has been imported from MySQL. Compatibility changes were introduced into these test cases which required them. The test suite was modified to assign memcached port number dynamically from mtr worker's pool, what allows to run the same test in parallel for many combinations. The plugin fully passes the test suite, both for 'innodb_plugin' and 'xtradb' combinations. 4. Some enhancements and bug fixes have been introduced into the plugin: - detection of memcached daemon initialization has been fixed -- this eliminated unnecessary wait (15 seconds) on plugin shutdown and memory leak (in cases when plugin failed to initialize properly before) - memcached arithmetic commands (incr/decr) handling has been improved -- now numeric columns values are handled in a consistent and predictable way - incorrect handling of unsigned not null integers columns has been fixed AFTER MIDTERM: -------------- 5. Memcached daemon code has been updated to the latest version from upstream memcached/engine-pu branch. 6. Plugin architecture has been changed to "one table per socket", as originally proposed in MDEV-4674 bug report. This allows usage of well known security solutions (firewalls, unix file permissions) to restrict access to different tables for different users. Now user can specify which tables should be accessible on which sockets. Memcached daemon will listen on all these sockets and execute commands on appropriate table, depending on which socket client connected. Supported protocols are TCP, UDP and UNIX stream sockets. Sockets are configured in 'daemon_memcached/containers' table: INSERT INTO containers VALUES ("tcp://*:11211", "db0", "table0", "key_col", "val_col", "c3", "c4", "c5", "PRIMARY", "|"); INSERT INTO containers VALUES ("udp://localhost:11222", "db0", "table1", "key_col", "val_col", "c3", "c4", "c5", "PRIMARY", "|"); INSERT INTO containers VALUES ("unix:/tmp/mc.sock", "db1", "table0", "key_col", "val_col", "c3", "c4", "c5", "PRIMARY", "|"); 7. Plugin has been decoupled from InnoDB. Now plugin is split into two parts: - generic 'daemon_memcached.so' - InnoDB specific 'daemon_memcached_engine_ib.so' The first library is a MariaDB plugin, which should be installed with command: install plugin daemon_memcached soname "daemon_memcached.so"; This library is generic. It provides Memcached protocol handling and it loads the second library, which is API-specific engine for accessing the database. Which engine library should be loaded can be specified in 'daemon_memcached_engine_lib_name' MariaDB server option. Now, the only existing engine library is daemon_memcached_engine_ib.so, which utilizes InnoDB API to access the data. It can operate on InnoDB and XtraDB tables. 8. Some enhancements and bug fixes have been introduced into the plugin: - CMake options and #cmakedefines associated with the plugin have been cleaned up - embedded default Memcached engine has been removed - duplicated code in Memcached source has been removed - fixed MySQL-originating bug with flush_all of non default table -- command flush_all always flushed the default table, not the current one - removed support for in-connection table switching -- after introduction of one table/port architecture it was redundant - plugin shutdown time has been improved -- from constant time of 2 seconds to average time of 0.5 second - logging of plugin has been refactored -- now plugin send log messages to MariaDB log instead of stderr - fixed Memcached-originating bug with get_thread_stats() -- bug resulted in out of bound array accesses and writes, what leaded to SEGFAULT - fixed MySQL-originating crash during 'flush_all' memcached command bug -- crash happened when table was heavily loaded with DML operations and because of that ib_cb_cursor_set_lock() failed to obtain a lock on the table, returning DB_LOCK_WAIT_TIMEOUT error - fixed MySQL-originating multiple memcached_sync_count for the same transaction bug -- the memcached_sync_count was incremented multiple times for the same transaction and after the end of transaction, it was not reset to zero, what blocked TRUNCATE command on any InnoDB table, on which Memcached had operated during program lifetime, even after uninstalling the plugin - support for numeric values has been finished -- now BIGINT overflows in the same way as smaller types, test to check over/under-flowing of all integer types has been added For details of all changes please see commit messages associated with them: https://github.com/piotrjurkiewicz/mariadb-server/commits/10.1-memcached ----------------------------------------------------------------------- Moreover, I planned to implement a new interface inside the Memcached plugin, which will use Handler API for database access. This would make possible to use the plugin to access databases based on storage engines other than InnoDB/XtraDB. However, during testing I discovered many serious bugs, either in Memcached or InnoDB API engine, which originated from upstream Memcached or MySQL and resulted in crashes or deadlocks. I decided that first I should fix them before implementing new functionalities. I lost a lot of time on investigating these bugs and I was unable to finish the implementation of Handler API engine. Nevertheless, I think the whole plugin is ready to be merged into MariaDB. All user-facing changes (like one port/table architecture) have been completed. Eventual later introduction of Handler API engine will not be associated with any user-facing compatibility changes. It will just lift the requirement that all databases accessed via the Memcached plugin must be InnoDB/XtraDB. Now I am going to write a separate email to coordinate the merge of my code into MariaDB repository. Piotr

1 0

Re: [Maria-developers] MDEV-7502 Provisioning of slave - review
by Kristian Nielsen 24 Aug '15

24 Aug '15

Martin Kaluznik <martin.kaluznik(a)gmail.com> writes: > good to do now. And I would also like to ask you for review of > existing code. > > Github link: https://github.com/f4rnham/server/tree/MDEV-7502 > Diff: https://github.com/MariaDB/server/compare/10.1...f4rnham:MDEV-7502.diff Please find my review inline in patch, below. General stuff: > I have created new log event as way to signalize slave successful end > of provisioning, it is last event IO thread writes to relay log before > termination, and last event SQL thread reads from relay log. Test > cases are using Slave_SQL_Running value from SHOW SLAVE STATUS command > to wait until provisioning is completed. It works there, but I am not > sure about its convinience in real world usage. Hm, I am not sure either. Another option would be for LOAD DATA FROM MASTER to block until provisioning is done (failed or successful). Other than review, please check Buildbot results carefully for any failures. Pay particular attention to the valgrind build (for memory leaks). Also pay special attention to random failures in your new test cases that happen only occasionally. Such sporadic / random failures often turn up in replication tests. You need in test cases that use debug_dbug: --source include/have_debug.inc What is the largest data set you tested with? Please do a (manual) test with a large data set. For example, you could run sysbench with a 10GB or 50GB database, let it run for say 10 minutes, and 5 minutes in you start a LOAD DATA FROM MASTER on a slave off the server running the sysbench. Please check comments below on how (and if) the whole approach with provisioning in parallel with active transactions is correct and makes the slave always identical with the master at the end. Is there any issue with foreign keys? An insert to one table could fail if a foreign key constraint against another table is not yet satisfied due to rows not yet provisioned. A CREATE TABLE referencing another table might fail. Maybe look at if mysqldump is doing something to avoid these kinds of issues. A minimal solution is to investigate possible problems, and document them as limitations. Eg. it is reasonable to document that users must disable foreign keys during provisioning, if necessary. What happens if the server that does LOAD DATA FROM MASTER provisioning has --log-bin and --log-slave-updates? What happens if a third-level slave is connected to the server that does LOAD DATA FROM MASTER? I think this needs to be tested. Comments on the patch: > +BOOLEAN _db_keyword_locked_(CODE_STATE *cs, const char *keyword, int strict) Can you explain the race that results without this locking? Is it because _db_keyword_() is scanning the list without any locking, so if the list changes during the scan we can crash? > +#define DBUG_EXECUTE_LOCKED(keyword,a1) \ > + do {if (_db_keyword_locked_(0, (keyword), 0)) { a1 }} while(0) You need empty definition for non-debug build (but I guess you already saw in buildbot run). > +--let $slave_param= Slave_SQL_Running > +--let $slave_param_value= No > +--let $slave_error_param= Last_SQL_Errno > +--source include/wait_for_slave_param.inc Is there a reason not just to use include/wait_for_slave_sql_to_stop.inc here? > +# Ensure that master and slave log names and positions are equal > +--connection master > +--let $binlog_file_master= query_get_value(SHOW MASTER STATUS, File, 1) > +--let $binlog_pos_master= query_get_value(SHOW MASTER STATUS, Position, 1) > +--connection slave > +--let $binlog_file_slave= query_get_value(SHOW SLAVE STATUS, Master_Log_File, 1) > +--let $binlog_pos_slave= query_get_value(SHOW SLAVE STATUS, Read_Master_Log_Pos, 1) Is there a reason you do not use GTIDs here instead (or in addition)? > +# echo Binlog file name master - $binlog_file_master; > +# echo Binlog file name slave - $binlog_file_slave; > +# echo Binlog position master - $binlog_pos_master; > +# echo Binlog position slave - $binlog_pos_slave; Please remove this left-over debugging code. > +++ b/mysql-test/suite/rpl/t/rpl_provisioning_base.inc > +eval CREATE TABLE test.test_1 (a int, PRIMARY KEY (a)) ENGINE=$engine_type; > +eval CREATE TABLE test.test_2 (a int, b int, c int, PRIMARY KEY (a)) ENGINE=$engine_type; Try adding also some foreign key contraints to the table for the test. > +# Start provisioning > +--connection slave > +LOAD DATA FROM MASTER; At this point, master has full binlogs. So just START SLAVE would do the same thing, just by re-playing the binlog. I think the test should use FLUSH LOGS on the master a couple of times during the initial setup of data, and then PURGE BINARY LOGS to delete some (but not all) of that binlog data, to test that the slave can still provision without complete binary logs. > +SET GLOBAL debug_dbug="+d,provisioning_wait"; > +SET GLOBAL debug_dbug="-d,provisioning_wait"; Earlier, when I did this, I ended up with tons of debug output in the error log after setting the "-d,..." which is annoying. Setting an empty "+d" option was enabling debug trace. Maybe this is now fixed, if so then all the better. But if you get this problem (with stray debug trace in error log), you can use the following pattern instead: SET @old_dbug= @@GLOBAL.debug_dbug; ... SET GLOBAL debug_dbug= @old_dbug; > +# > +# Attempt to start provisioning while not in GTID mode > +# Is this an old comment? It seems none of the tests really use any GTID. Does provisioning work in GTID mode? It should be tested. > +# > +# Attempt to start provisioning while SQL thread is running > +# What if IO thread is running? Should probably be tested also. > +# Creation of database `test` depends purely on timing and it has nothing to do with this test case > +--disable_warnings > +DROP DATABASE IF EXISTS test; What is the purpose of this? It sounds like there is a potential race in the test case? Maybe there is a better way to handle it. > +START SLAVE; > + > +# Cleanup after test case > +--connection slave > +STOP SLAVE; > +RESET SLAVE; > +# Creation of database `test` depends purely on timing and it has nothing to do with this test case It is better to use the include/start_slave.inc and include/stop_slave.inc. They prevent some races, which might solve some of the problems that your comment about timing seems to hint at. Also, it is usually best to synchronise the slave with the master before stopping, to avoid it being stopped at a random point. > +# > +# Bug: Deadlock occurs when IO thread fails with error before dump request > +# What do you mean? That the current patch has a bug triggered by this test case? Or that there was a bug, fixed now, and this is a test for it? Please explain better in the comment. > diff --git a/mysql-test/suite/rpl/t/rpl_provisioning_stress.test b/mysql-test/suite/rpl/t/rpl_provisioning_stress.test > new file mode 100644 > index 0000000..8141015 > --- /dev/null > +++ b/mysql-test/suite/rpl/t/rpl_provisioning_stress.test > @@ -0,0 +1,48 @@ > +--skip Test requires manual setup > + > +# Stress test for MDEV-7502 Automatic provisioning of slave > +# Using Random Query Generator v2.2.0 release, downloadable from https://launchpad.net/randgen/+download I think it is great that you made this kind of test. But maybe it would be better to put it somewhere else, to not have a test in the normal test suite that is always unconditionally skipped. Eg. it could be put in /tests/, and one would copy it into the test suite, instead of removing the --skip. > +void Provisioning_done_log_event::print(FILE* file, > + print_header(&cache, print_event_info, FALSE); > + my_b_write_string(&cache, "\tProvisioning done\n"); Hm, this can occur in the relay log, but not in the binlog, right? I think it is ok not to have a test of this in mysql-test-run, we generally can only test the "short-form" in there to avoid dependency on time and such. Just be sure to check manually that the output is correct. > @@ -9718,23 +9760,29 @@ int Rows_log_event::do_apply_event(rpl_group_info *rgi) > + // In provisioning mode, silently ignore table not found error > + if (!rgi->thd->slave_thread || !rli->mi->provisioning_mode || > + !provisioning_error_code(actual_error)) Better use thd than rgi->thd here (just for consistency). > + // Ignore certain errors during provisioning What thoughts did you have on whether just ignoring these kinds of errors would always result in the correct result of provisioning? I suppose since we are copying data fresh from the master, it should not be possible that we miss a "real" error in this way. Will things work correctly, if we have multiple rows in a single Rows_log_event, and some of them succeed while others fail? In both MyISAM and InnoDB? Can an argument be made to prove that, if we ignore an insert or update here, we are sure to later write the correct data as part of provisioning? For both InnoDB and MyISAM? Similarly, can it be proven that it is impossible to ignore a delete and still later wrongly provision the deleted row? I suppose it would be something with binlog ordering and isolation modes for the InnoDB case - and some table locking semantics wrt. MyISAM? > + > + /* > + Semaphore used by LOAD DATA FROM MASTER command for test cases, its role is > + to ensure, that slave's IO thread is connected to master before test case > + can continue. > + volatile bool dump_requested_semaphore; Shouldn't this be under #ifndef DBUG_OFF ? > diff --git a/sql/rpl_provisioning.cc b/sql/rpl_provisioning.cc > +void quote_name(char const *name, char *buff) Try to use append_identifier() instead. Or maybe my_snprintf("%`s"). I notice that append_identifier() handles a configuration where "" is used to quote identifiers instead of ``, you should probably test if this causes a bug in your code, btw. > +provisioning_send_info::provisioning_send_info(THD *thd_arg) > + : thd(thd_arg) > + , connection(new Ed_connection(thd)) > + , row_batch_end(NULL) > + , row_buffer(my_malloc(ROW_BUFFER_DEFAULT_SIZE, MYF(0))) > + , row_buffer_size(ROW_BUFFER_DEFAULT_SIZE) > + , phase(PROV_PHASE_INIT) > + , error(0) > + , error_text(NULL) > + , event_conversion_cache(NULL) > +{ > +} So this constructor will probably cause the server to crash in case of out-of-memory, right? Usually, code tries to fail gracefully with ER_OUT_OF_RESOURCES / ER_OUTOFMEMORY if malloc() fails (though its not well tested, probably). > +void provisioning_send_info::close_tables() > +{ > + trans_commit_stmt(thd); I see no call to commit the entire transaction (as opposed to just the statement here). This may be intentional (autocommit?), in any case please add a comment here explaining how the transaction is committed wrt. statement-transaction and full transaction. Maybe add a test where the server is running with auto-commit globally disabled? Though I am not sure if the short mysql-test-run.pl tests would notice it if everything was done as one long transaction (though large production sites undobtedly would ;-). > +bool provisioning_send_info::send_query_log_event(DYNAMIC_STRING const *query, Is there a particular reason that you chose DYNAMIC_STRING over the String class? I think String is more widely used for building queries and such? A common use pattern with String is to allocate a reasonably-sized buffer on the stack and initialise a String object from it. Then in the common case of smallish result, no my_malloc() calls are needed. > + if (cs_info) > + { > + // thd->variables.character_set_client->number > + int2store(evt.charset, cs_info->cs_client); > + // thd->variables.collation_connection->number > + int2store(evt.charset + 2, cs_info->cl_connection); > + // cs_info->cl_server > + int2store(evt.charset + 4, thd->variables.collation_server->number); > + // thd->variables.collation_database->number > + evt.charset_database_number= cs_info->cl_db; Can you explain why you need to consider the charsets/collations of the connection and client? Could you not just always use the system character set for generating queries? > + if (my_strcasecmp(system_charset_info, "VIEW", type->str) == 0) > + { > + char name_buff[NAME_LEN * 2 + 3]; > + DYNAMIC_STRING *str= (DYNAMIC_STRING*)my_malloc(sizeof(DYNAMIC_STRING), > + MYF(0)); > + > + quote_name(database->str, name_buff); > + init_dynamic_string(str, name_buff, > + // Both lengths + dot and quotes, will be reallocated > + // only in special cases > + database->length + name->length + 5, > + 5); > + dynstr_append(str, "."); > + quote_name(name->str, name_buff); > + dynstr_append(str, name_buff); > + > + views.push_back(str); > + } > + else if (my_strcasecmp(system_charset_info, "BASE TABLE", type->str) == 0) > + tables.push_back(my_strdup(name->str, MYF(0))); Why do you need to prepend the database name in VIEW case but not in TABLE case? Please add a comment explaining that. > + else > + DBUG_ASSERT(false && > + "Unexpected type of table returned by 'SHOW FULL TABLES'"); Probably you should add an error return here, that will cause provisioning to fail in the non-debug build, rather than silently press on. > +bool provisioning_send_info::event_to_packet(Log_event &evt, String &packet) > +{ > + // Reset cache for writing > + reinit_io_cache(event_conversion_cache, WRITE_CACHE, 0, false, true); > + > + if (evt.write(event_conversion_cache)) > + { > + error_text= "Failed to write event to conversion cache"; > + return true; > + } > + > + if (reinit_io_cache(event_conversion_cache, READ_CACHE, 0, false, false)) > + { > + error_text= "Failed to switch event conversion cache mode"; > + return true; > + } > + > + packet.set("\0", 1, &my_charset_bin); > + if (packet.append(event_conversion_cache, > + event_conversion_cache->write_pos - event_conversion_cache->buffer)) > + { > + error_text= "Failed to write event data to packet"; > + return true; > + } Hm. So first the event is constructed in-memory. Then it is written to an IO_CACHE. Then it is read back in again into a byte buffer. And then it is written to the slave's client socket. Hm, I see, the client socket we have only as a NET, not an IO_CACHE. But the Log_event subclasses need an IO_CACHE to construct a serialised event. Hm, that's clunky, but I don't see an easy way to avoid it. But please add a comment explaining why this is necessary. Also, doesn't the IO_CACHE spill to disk if the data written becomes large? So wouldn't a very large event (eg. a row containing a large BLOB) fail here? Because the IO_CACHE only contains part of the data in-memory? I am not 100% sure about how this works, please investigate and test with a large row (if you did not already). > + // Set current server as source of event, when slave registers on master, > + // it overwrites thd->variables.server_id with its own server id - and it is > + // then used when writing event > + int4store(&packet[SERVER_ID_OFFSET + 1], global_system_variables.server_id); > + return false; I think this will fail if event checksums are enabled? Or rather, the patch does not seem to handle binlog checksums at all. Try adding a test that enables binlog checksums on master and slave, and check that everything works. And also - why do you need to change this in the first place? Doesn't the server_id get set correctly when the event is initially created? > +bool provisioning_send_info::send_event(Log_event &evt) > +{ > + String packet; > + > + // Slave will accept this event even if it modifies data structures > + evt.flags|= LOG_EVENT_PROVISIONING_F; > + > + if (event_to_packet(evt, packet)) > + return true; > + > + if (my_net_write(&thd->net, (uchar*)packet.ptr(), packet.length())) > + { > + error_text= "Failed to send event packet to slave"; > + return true; > + } Yeah... It would be so much more elegant if we could wrap the thd->net in an IO_CACHE, and write directly to the client. Did you investigate if this would be a possibility, without too much extra work? > +int8 provisioning_send_info::send_table_data() > + if ((error= hdl->prepare_range_scan(row_batch_end, NULL)) != 0) > + if (table->s->primary_key == MAX_KEY) > + { > + error_text= "Table does not contain primary key"; > + close_tables(); > + return 1; > + } It seems more natural to me to check for missing primary key _before_ prepare_range_scan(). (Otherwise one wonders how it is possible to prepare a range scan on a table without _any_ key? I suppose it works to do a table scan instead, but better do the check first to avoid having to wonder). > + while (packed_rows < PROV_ROW_BATCH_SIZE && > + (error= hdl->read_range_next()) == 0); I think there should be two stop criterias. One is max number of rows reached, another is size of rows. At least, you need to protect against exceeding --max-packet-size. Otherwise the slave will be unable to receive the too large packet. Please add a test case that checks large rows and that it correctly splits row events to not exceed max-packet-size (of course at least one row must fit in one Rows_log_event). Check the code for normal binlogging to see how it splits up Row_log_events to avoid creating too large an event. > + table->default_column_bitmaps(); > + close_tables(); So the critical issue here is: how can we be sure that we will not miss rows during provisioning, by racing between provisioning events and normal binlog events? This needs to be clearly thought out, and then explained in a comment somewhere in the code. For example, you write the events to the slave client socket before closing the table and releasing locks. This seems critical (and correct) - because this ensures that if there is a DELETE made just after the provisioning read of the table, that DELETE will be sent only _after_ sending the provisioning rows (so we do not lose the DELETE). This should be explained in a comment. But what about isolation level (for InnoDB)? Don't we need to run with a sufficiently high isolation level to be correct? Intuitively, I thought so, but maybe not, at least you have nothing in the patch about isolation level that I can see. What about this scenario? 1. Provisioning reads rows 20-29, sends them to slave. 2. An UPDATE runs, it updates row 32 to be now row 25, it is written to the binary log and sent to the slave. 3. Provisioning reads rows 30-39, sends them to the slave. On the slave, the UPDATE from 2 should fail - because row 32 does not exist at this point. But then, it seems to me that the row 25 will never appear on the slave, right? So provisioning will be incorrect. Or am I missing something? I wonder if the solution to this particular case is that during provisioning, an UPDATE (at least one that changes primary key value) must be logged as a DELETE (of the old row) + an INSERT (of the new row). Alternatively, maybe the slave can during provisioning execute the UPDATE regardless of the missing before image, still writing the after image (row 25) to the table. More generally, we really need an explanation of how the provisioning ensures that the end result on the slave is identical to the master - otherwise there may be other problems like this hidden. Also, it would be really good if you could make a test case for the above potential issue. You can get some ideas from how I use DBUG_EXECUTE_IF together with DEBUG_SYNC in the rpl_parallel_*.test test cases. But I know it can be a lot of work to get such tricky race-proviking test cases to work reliably. > diff --git a/sql/rpl_provisioning.h b/sql/rpl_provisioning.h > +// FIXME - Farnham > +// Header > + > +#pragma once We do not use this #pragma once in mariadb (maybe that is what you intended to fix with the FIXME, so mentioning just in case). I assume you will take care of other such FIXME's in the patch yourself before pushing. > +/* > + How much rows from table will be sent in one provisioning tick > + FIXME - Farnham > + Dynamic value > +*/ > +#define PROV_ROW_BATCH_SIZE 5 Right. Probably just a matter of adding something to sql/sys_vars.cc > - if (rpl_load_gtid_slave_state(thd)) > + if (!mi->provisioning_mode && rpl_load_gtid_slave_state(thd)) Actually, what should happen with the GTID position for LOAD DATA FROM MASTER? Just clearing it probably is not correct, in case user wants to LOAD DATA FROM MASTER from multiple masters. Maybe the user should clear it herself, if necessary, and then it will be updated by the normal replication stream from master. But why do you skip loading it here for provisioning mode? That does not seem right. First, it will normally be loaded already anyway. And second, if not loaded, then just having the wrong value in-memory (as compared to in the mysql.gtid_slave_pos table) does not seem correct? > + DBUG_EXECUTE_IF("provisioning_test_running", > + if (!slave_errno) > + while (!mi->dump_requested_semaphore) > + /* pass */;); Maybe put a my_sleep(20000) in with the /* pass */. > + case PROVISIONING_DONE_EVENT: > + mi->abort_slave= true; > + /* > + Fall through to default case ... and through debug code, nothing wrong > + should happen > + */ That seems a bit fragile. Better do like in other places: goto default_action; > + // In provisioning mode, master starts sending data from current position > + if (flags & BINLOG_DUMP_PROVISIONING_MODE) So provisioning assumes non-GTID mode? Maybe that is ok... But it should still be tested what happens if provisioning is attempted in GTID mode. Perhaps either GTID or non-GTID mode can work the same, since provisioning does not need to start at any particular position - and at the end of provisioning, both GTID and non-GTID position should be correct. > +bool > +Ed_connection::execute_direct(char *sql_text, size_t length) > +{ > + LEX_STRING tmp_query; > + tmp_query.str= sql_text; > + tmp_query.length= length ? length : strlen(sql_text); > + > + return execute_direct(tmp_query); > +} You are not actually ever calling this without passing length. So please make this an inline method in the .h file, with mandatory length argument, that does not check for zero length or call strlen(). - Kristian.

1 0

[Maria-developers] Fixing Some Trivial Bugs
by Alangi Derick 21 Aug '15

21 Aug '15

Hi, I just found some trivial bugs on jira that i will be fixing for the time being. Once i am done with each one, i will submit a patch or push to my forked repo so it can be merged. My question is, after i fix the bugs, will they accepted, if yes i should continue working on them thanks. I need a respond from the development team. Regards Alangi Derick Ndimnain

3 4

[Maria-developers] [Patch] Fix to mysql.server init script
by Alangi Derick 21 Aug '15

21 Aug '15

This is a fix to the bug in the mysql.server init script which uses a wrong server name (MySQL) instead of MariaDB. It is not fixed and this is the patch below. Regards Alangi Derick Ndimnain

1 0

[Maria-developers] Please review: MDEV-8433 Make field<'broken-string' use indexes
by Alexander Barkov 20 Aug '15

20 Aug '15

Hi Sergei, Please review my patch for MDEV-8433. I had doubts about one detail: how to make Field::store() that we're currently creating a value for range search rather than regular doing a INSERT/UPDATE. I could think of two options: 1. Using sql_mode, similar to how temporal types pass MODE_INVALID_DATES. 2. Using a new parameter to this method: int Field::store(const char *str, uint length, CHARSET_INFO *cs); The former needed less changes, so I chose to use sql_mode. But probably N2 would be better. Please suggest. Thanks.

1 0

Re: [Maria-developers] [JIRA] (MDEV-7063) Split String into logical components
by Alexander Barkov 20 Aug '15

20 Aug '15

Hi Rasmus, I noticed you put this task on Sprint 10.1.7-2. Did I miss some discussion about this? From my understanding, Sergei and Monty were not enthusiastic about this task (but I still think it should be helpful). Thanks. On 08/19/2015 06:26 PM, Rasmus Johansson (JIRA) wrote: > > [ https://mariadb.atlassian.net/browse/MDEV-7063?page=com.atlassian.jira.plug… ] > > Rasmus Johansson updated MDEV-7063: > ----------------------------------- > Sprint: 10.1.7-2 > >> Split String into logical components >> ------------------------------------ >> >> Key: MDEV-7063 >> URL: https://mariadb.atlassian.net/browse/MDEV-7063 >> Project: MariaDB Server >> Issue Type: Task >> Reporter: Alexander Barkov >> Assignee: Michael Widenius >> Fix For: 10.1 >> >> >> The string class currently consists of the following components: >> - memory allocation routines (using mysys) >> - binary string routines >> - character set aware routines. >> Storage engines (e.g. ConnectSE) often need String-alike classes, but cannot use the current String implementation because of their own memory management system. So they have to reimplement the same set of functionality, but with their own memory managers. This is bug prone and makes duplicate code. >> Also, in some cases it is useful to have character-set unaware strings (just array of bytes), and static strings without allocation. >> This task is to split String into logical pieces: >> 1. class Static_binary_string >> 2. class Charset >> 3. class Static_character_string (#1 + #2) >> 4. class Memory_mysys >> 5. template<typename MEM> class String_with_alloc >> 6. class String (#3 + #5) >> ConnectSE will use #5 (templace class String_with_alloc), in combination with their own memory allocator. > > > > -- > This message was sent by Atlassian JIRA > (v7.0.0-OD-01-193#70101) >

1 0

Re: [Maria-developers] MDEV-8389 patch review
by Sergey Petrunia 19 Aug '15

19 Aug '15

Also, I'm getting a crash when trying to debug a slightly complex example: delimiter | CREATE FUNCTION f10(param1 VARCHAR(11)) RETURNS TABLE return_table(name VARCHAR(11)) deterministic BEGIN insert into return_table values('foo'); END| DELIMITER ; select name from f21('aaa') where name like '%foo%'; Causes a crash: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff7efa700 (LWP 5142)] 0x0000555555cb6000 in Item_field::fix_fields (this=0x7fff80006db8, thd=0x555558161ca0, reference=0x7fff80007070) at /home/psergey/dev-git/10.1-gsoc-repo/sql/item.cc:4928 (gdb) wher #0 0x0000555555cb6000 in Item_field::fix_fields (this=0x7fff80006db8, thd=0x555558161ca0, reference=0x7fff80007070) at /home/psergey/dev-git/10.1-gsoc-repo/sql/item.cc:4928 #1 0x0000555555cf86e8 in Item_func::fix_fields (this=0x7fff80006fe0, thd=0x555558161ca0, ref=0x7fff8000b060) at /home/psergey/dev-git/10.1-gsoc-repo/sql/item_func.cc:208 #2 0x0000555555cd9747 in Item_func_like::fix_fields (this=0x7fff80006fe0, thd=0x555558161ca0, ref=0x7fff8000b060) at /home/psergey/dev-git/10.1-gsoc-repo/sql/item_cmpfunc.cc:4856 #3 0x00005555559ff1b0 in setup_conds (thd=0x555558161ca0, tables=0x7fff80006780, leaves=..., conds=0x7fff8000b060) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_base.cc:8657 #4 0x0000555555aeb4f7 in setup_without_group (thd=0x555558161ca0, ref_pointer_array=0x7fff8000b198, tables=0x7fff80006780, leaves=..., fields=..., all_fields=..., conds=0x7fff8000b060, order=0x0, group=0x0, hidden_group_fields=0x7fff8000af40, reserved=0x555558165ffc) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_select.cc:631 #5 0x0000555555aa6d99 in JOIN::prepare (this=0x7fff8000ac48, rref_pointer_array=0x555558165fd8, tables_init=0x7fff80006780, wild_num=0, conds_init=0x7fff80006fe0, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x555558165d60, unit_arg=0x555558165660) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_select.cc:786 #6 0x0000555555aafcab in mysql_select (thd=0x555558161ca0, rref_pointer_array=0x555558165fd8, tables=0x7fff80006780, wild_num=0, fields=..., conds=0x7fff80006fe0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7fff8000ac28, unit=0x555558165660, select_lex=0x555558165d60) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_select.cc:3301 #7 0x0000555555aa5fd7 in handle_select (thd=0x555558161ca0, lex=0x555558165598, result=0x7fff8000ac28, setup_tables_done_option=0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_select.cc:371 #8 0x0000555555a65fbf in execute_sqlcom_select (thd=0x555558161ca0, all_tables=0x7fff80006780) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:5799 #9 0x0000555555a5c25f in mysql_execute_command (thd=0x555558161ca0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:2939 #10 0x0000555555a69554 in mysql_parse (thd=0x555558161ca0, rawbuf=0x7fff800064a8 "select name from f21('aaa') where name like '%foo%'", length=51, parser_state=0x7ffff7ef90f0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:7175 #11 0x0000555555a584a7 in dispatch_command (command=COM_QUERY, thd=0x555558161ca0, packet=0x555558168ef1 "select name from f21('aaa') where name like '%foo%'", packet_length=51) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:1470 #12 0x0000555555a571c1 in do_command (thd=0x555558161ca0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:1093 #13 0x0000555555b95b3d in do_handle_one_connection (thd_arg=0x555558161ca0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_connect.cc:1347 #14 0x0000555555b95882 in handle_one_connection (arg=0x555558161ca0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_connect.cc:1258 #15 0x00007ffff691de9a in start_thread (arg=0x7ffff7efa700) at pthread_create.c:308 #16 0x00007ffff604e3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #17 0x0000000000000000 in ?? () (gdb) I am nut sure about the cause of this. It needs to be investigated. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] MDEV-8389 patch review
by Sergey Petrunia 19 Aug '15

19 Aug '15

On Wed, Aug 12, 2015 at 01:58:40AM +0530, Diwas Joshi wrote: > hello sergei, > After I mailed you the last patch, I kept working further. I guess the code > has come to a point where it is giving some results. So, I did the > following things: > Earlier we were storing all the parameters in a select query inside > lex->value_list, but since there can be multiple functions in a query, I > declared a list for parameters inside table_list so we can store params > corresponding to each table function in it. Good. > Then instead of function name i created a table with table alias defined in > the function and change the table name to table alias inside select query > as well. This makes it possible for the queries inside the function > definition to access the temporary table we created. There is a problem > with this approach that there might already be a table present in the > database with that name, I am still thinking of a solution for it but we > can discuss a solution for it later. Yes, you're right. There is a problem here. It is not limited to the already existing tables with the same name. Let's work with this example: delimiter | CREATE FUNCTION f10(param1 VARCHAR(11)) RETURNS TABLE return_table(name VARCHAR(11)) deterministic BEGIN insert into return_table values('foo'); END| DELIMITER ; MariaDB [test]> select name from f10('aaa'); +------+ | name | +------+ | foo | +------+ 1 row in set (0.29 sec) Good. However, attempts to invoke the same function multiple times fail: MariaDB [test]> select name from f10('aaa') as TBL1 , f10('bbb') as TBL2; ERROR 1050 (42S01): Table 'return_table' already exists Your code uses this approach: 1. Create a temporary table in a manner similar to how user-specified CREATE TEMPORARY TABLE works. 2. Rely on regular MySQL's name resolution process to "see" the temporary table. The properties of this solution are: - If a temporary table has the same name as a regular db table, temporary table will "shadow" the regular table (this is ok). - user-created temporary tables are per-thread. Other threads do not see the temp. table (good). However, everything that executes within the thread will see the temp. table. For example, if a query calls two Table Functions, they will both see each others return tables. This is a problem. I think, a better approach would be: - switch to something like create_tmp_table() for creation of temporary tables. Tables that are created this way are normally not visible to the name resolution code. - Patch the name resolution code so that the statements inside a Table Function see the return table of that Table Function. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] MDEV-8389 patch review
by Sergey Petrunia 19 Aug '15

19 Aug '15

Hi Diwas, While playing around, I've found and filed: https://mariadb.atlassian.net/browse/MDEV-8649 https://mariadb.atlassian.net/browse/MDEV-8650 I've filed these separately because I think they are not related to MDEV-8389. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] MDEV-8389 patch review (part#1)
by Sergey Petrunia 19 Aug '15

19 Aug '15

Hi Diwas, Please find part #1 of the review below. I'll provide more soon. > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 2998972..a9dc162 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2466,6 +2466,8 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > lex->first_lists_tables_same(); > /* should be assigned after making first tables same */ > all_tables= lex->query_tables; > + > + check_and_process_table_functions(thd, all_tables); > /* set context for commands which do not use setup_tables */ > select_lex-> > context.resolve_in_table_list_only(select_lex-> > @@ -5581,6 +5583,7 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > } > THD_STAGE_INFO(thd, stage_query_end); > thd->update_stats(); > + //clear_table_functions(thd, all_tables); Please remove the comment. > > goto finish; > > @@ -5697,7 +5700,7 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > thd->mdl_context.release_transactional_locks(); > } > #endif /* WITH_WSREP */ > - > + clear_table_functions(thd, all_tables); > DBUG_RETURN(res || thd->is_error()); > } > > @@ -5709,7 +5712,7 @@ static bool execute_sqlcom_select(THD *thd, TABLE_LIST *all_tables) > bool res; > /* assign global limit variable if limit is not given */ > { > - SELECT_LEX *param= lex->unit.global_parameters(); > + SELECT_LEX *param= lex->unit.global_parameters(); No need for this dummy change. > if (!param->explicit_limit) > param->select_limit= > new (thd->mem_root) Item_int((ulonglong) thd->variables.select_limit); > @@ -9222,3 +9225,182 @@ CHARSET_INFO *find_bin_collation(CHARSET_INFO *cs) > } > return cs; > } > + > +/* > + @brief > + Check for table functions and create corresponding tables to be used in query. > + > + @param thd Thread handle > + @param table_list A list of tables being processed in the query. > + > + @return > + False on success > + @retval > + TRUE on error > +*/ > +bool check_and_process_table_functions(THD *thd, TABLE_LIST *table_list) > +{ > + TABLE_LIST *tables = table_list; > + int err= 0; > + while(tables) > + { > + if (!tables->is_table_function || tables->table_name == NULL) > + { > + tables = tables->next_local; > + continue; > + } > + sp_head *sp; > + sp_name *name; > + LEX_STRING lex_db; > + LEX_STRING lex_name; > + lex_db.length= strlen(tables->db); > + lex_name.length= strlen(tables->table_name); > + lex_db.str= thd->strmake(tables->db, lex_db.length); > + lex_name.str= thd->strmake(tables->table_name, lex_name.length); > + name= new sp_name(lex_db, lex_name, true); > + name->init_qname(thd); > + int err=0; Please switch err from int to bool. > + > + if (!err && (sp = sp_find_routine(thd,TYPE_ENUM_FUNCTION, name, Why assign 'err' and then immediately check it? The code looks redundant. > + &thd->sp_proc_cache, FALSE))) //check if the table is a table-valued function. Please move the comment a few lines down, where the check is actually done. > + { > + if (sp->m_type == TYPE_ENUM_FUNCTION) > + my_error(ER_SP_TABLE_VALUE_EXPECTED, MYF(0)); > + LEX *old_lex= thd->lex, newlex; > + thd->lex= &newlex; > + lex_start(thd); > + ulonglong saved_mode= thd->variables.sql_mode; > + Table_specification_st create_info; > + Alter_info alter_info; > + create_info.init(); > + alter_info.reset(); > + create_info.default_table_charset= NULL; > + TABLE_LIST *create_table; > + create_info.options|= HA_LEX_CREATE_TMP_TABLE; > + LEX_STRING tablename,dbname; > + if (strlen(sp->m_table_alias.str) > 0) { > + tablename.str= sp->m_table_alias.str; > + tablename.length= strlen(sp->m_table_alias.str); > + } sp->m_table_alias is the alias of the return table of the table function. When can it have zero length? Please add a comment about this here, and also add a testcase in the .test file > + else { > + tablename.str= tables->table_name; > + tablename.length= strlen(tables->table_name); > + } > + dbname.str= tables->db; > + dbname.length= strlen(tables->db); > + Table_ident *new_ident= new Table_ident(thd, dbname, tablename, 0); > + if (!thd->lex->select_lex.add_table_to_list(thd, new_ident, NULL, > + TL_OPTION_UPDATING, > + TL_WRITE, MDL_EXCLUSIVE)) > + err= 1; > + create_info.use_default_db_type(thd); > + thd->lex->query_tables->open_strategy= TABLE_LIST::OPEN_STUB; > + thd->lex->create_info.default_table_charset= NULL; > + thd->lex->name= null_lex_str; > + create_table= thd->lex->create_last_non_select_table= thd->lex->last_table(); > + thd->lex->current_select= &thd->lex->select_lex; > + create_table_set_open_action_and_adjust_tables(thd->lex); > + alter_info.create_list= sp->m_cols_list; > + err= mysql_create_table(thd, create_table, &create_info, &alter_info); Ok, so now it is using mysql_create_table instead of parsing the table definition. This is a step forward. However, I still think that mysql_create_table() does too much. It is a function that used for regular CREATE TABLE statements. When I trace a query, I still see that the created temporary table uses InnoDB storage engine. I think, more effort is needed to make temporary table creation work in a more similar to how other parts of code create temporary tables. I think I have also found a problem with how name resolution process works. I'll elaborate in a separate email. > + thd->variables.option_bits|= OPTION_KEEP_LOG; What is this for? please add a comment. > + if (!err) > + close_thread_tables(thd); > + execute_table_function(thd,sp,tables->table_func_params); > + thd->variables.sql_mode= saved_mode; > + lex_end(thd->lex); > + thd->lex= old_lex; > + if (strlen(sp->m_table_alias.str) > 0) { > + tables->table_name= sp->m_table_alias.str; > + tables->mdl_request.init(tables->mdl_request.key.mdl_namespace(), > + tables->db,sp->m_table_alias.str, > + tables->mdl_request.type,tables->mdl_request.duration); > + } What does the above statement do? Please add a comment. > + } > + tables = tables->next_local; > + } > + close_thread_tables(thd); > + return err; > +} ... > +bool execute_table_function(THD *thd, sp_head *sp, List<Item> *value_list) > +{ > + /* bits that should be cleared in thd->server_status */ > + uint bits_to_be_cleared= 0; > + if (sp->m_flags & sp_head::MULTI_RESULTS) > + { > + if (!(thd->client_capabilities & CLIENT_MULTI_RESULTS)) > + { > + /* The client does not support multiple result sets being sent back */ > + my_error(ER_SP_BADSELECT, MYF(0), sp->m_qname.str); > + return 1; > + } > + /* > + If SERVER_MORE_RESULTS_EXISTS is not set, > + then remember that it should be cleared > + */ > + bits_to_be_cleared= (~thd->server_status & > + SERVER_MORE_RESULTS_EXISTS); > + thd->server_status|= SERVER_MORE_RESULTS_EXISTS; > + } Does the above make any sense here? This is a piece of logic from code that executes stored *procedures*. A stored procedure may have multiple SELECT statements. Executing this stored procedure with "CALL sp(...)" may produce multiple result sets. So this logic checks that the client is able to receive multiple resultsets. A stored *function* may not emit resultsets. It must return a scalar value, in our case it must return a table. > + ha_rows select_limit= thd->variables.select_limit; > + thd->variables.select_limit= HA_POS_ERROR; > + > + /* > + We never write CALL statements into binlog: > + - If the mode is non-prelocked, each statement will be logged > + separately. > + - If the mode is prelocked, the invoking statement will care > + about writing into binlog. > + So just execute the statement. > + */ > + int res= sp->execute_procedure(thd, value_list); > + > + thd->variables.select_limit= select_limit; > + thd->server_status&= ~bits_to_be_cleared; > + > + if (res) > + { > + DBUG_ASSERT(thd->is_error() || thd->killed); > + return 1; // Substatement should already have sent error > + } > + > + my_ok(thd, (thd->get_row_count_func() < 0) ? 0 : thd->get_row_count_func()); What is the above for? As far as I understand, the code in this function was based on do_execute_sp(). Which is not a problem, however, do_execute_sp() is used to handle "CALL stored_proc" or compound statements, which should produce "Ok" if they have not produced a resultset. There is no such need when using a table function. Upper query will produce a resultset which it will return to the client. > + thd->reset_query_timer(); What is this for? > + thd->lex->unit.cleanup(); > + thd->mdl_context.release_transactional_locks(); > + thd->reset_for_next_command(); > + return 0; > +} > \ No newline at end of file ... > diff --git a/sql/table.h b/sql/table.h > index 622a3b2..5eddd3d 100644 > --- a/sql/table.h > +++ b/sql/table.h > @@ -2010,6 +2010,7 @@ struct TABLE_LIST > List<Item> used_items; > /* Sublist (tail) of persistent used_items */ > List<Item> persistent_used_items; > + List<Item> *table_func_params; //parameters used for table function. > Item **materialized_items; > > /* View creation context. */ > @@ -2059,7 +2060,7 @@ struct TABLE_LIST > from I_S should be done by create_sort_index() or by JOIN::exec.) > */ > enum enum_schema_table_state schema_table_state; > - > + bool is_table_function; //Check whether the table is actually a table function. Do I get it right that a TABLE_LIST object that denotes a Table Function invocation always has table_func_params != NULL? If yes, please remove 'bool is_table_function' and add a 'bool is_table_function()' function instead which checks for table_func_params!=NULL. > /* Something like a "query plan" for reading INFORMATION_SCHEMA table */ > IS_table_read_plan *is_table_read_plan; > > @@ -2277,6 +2278,18 @@ struct TABLE_LIST > return false; > } > void set_lock_type(THD* thd, enum thr_lock_type lock); > + void fill_table_func_params(List<Item> param_list) > + { > + table_func_params= new List<Item>; > + List_iterator_fast<Item> it(param_list); > + Item *param; > + param= it++; > + while(param) > + { > + table_func_params->push_back(param); > + param= it++; > + } > + } > > private: > bool prep_check_option(THD *thd, uint8 check_opt_type); BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] MDEV-7502 Provisioning of slave - review
by Kristian Nielsen 17 Aug '15

17 Aug '15

Martin Kaluznik <martin.kaluznik(a)gmail.com> writes: > Earlier you mentioned, that you are able to get my code on build > server to try compile and test runs on various platforms, it would be Ok. I have added you as a developer on https://github.com/MariaDB/server. The way it works is that you push your changes to that repository in a branch that starts with "bb-", eg. "bb-10.1-f4rnham". Then Buildbot will pick up your changes and start building (can take some hours or even a day or two in busy periods). Your builds will appear then on this page: https://buildbot.askmonty.org/buildbot/grid?branch=bb-10.1-f4rnham (Please be careful about not accidentally pushing to main branches (or any other branches) in the main repository until we've completed the review process.) Then when the builds are done, you can use the web interface to check for any failures on special platforms or test configuration, and fix them as appropriate. Meanwhile, I will read through all of your patch, and get back to you with comments. > dbug.c,my_dbug.h > When line like SET GLOBAL debug_dbug="+d,flag"; is executed in > test case, it acquires THR_LOCK_dbug, but when it is tested for > (DBUG_EXECUTE_IF) no lock is used - and it causes data races in > replication threads Hm. The way it usually works is that the test cases SET GLOBAL debug_dbug while the slave threads are stopped. That way there should be no races? You might be right that there is a need for a lock, but I would prefer if that was made as a separate change from MDEV-7502, if possible. But it can be tricky to properly control the slave threads for testing, I will keep this point in mind when I check your test cases, and see if I can come up with concrete suggestions where needed. > Edited *.test,*.result > Updated old test cases, most of them because I have added new log > event type, what increased length of format description event and all > positions in test cases have shifted Right, annoying but expected. The Buildbot might catch a few more that are missed by a standard test run. - Kristian.

3 2

[Maria-developers] Unexpected contents of mysql.gtid_slave_pos
by Giuseppe Maxia 13 Aug '15

13 Aug '15

Using MariaDB 10.0.20, when I set a multi-master environment in which one of the masters has log-slave-updated enabled, I get this strange behavior: -- node#1 select @@server_id; +-------------+ | @@server_id | +-------------+ | 101 | +-------------+ use test; create table t1 (i int not null primary key, msg varchar(50), d date, t time, dt datetime, ts timestamp); use mysql; select * from gtid_slave_pos; +-----------+--------+-----------+--------+ | domain_id | sub_id | server_id | seq_no | +-----------+--------+-----------+--------+ | 0 | 1 | 101 | 1 | +-----------+--------+-----------+--------+ Why is this happening? Is there a reason for the server to log its own transactions? The transaction originated from this same server, but it is effectively filtered off when it comes back from the other node (otherwise we would have a conflict). Note: I get the same behavior if I use multi-source setup where one of the nodes has log-slave-updates enabled. Using different domains for each master does not affect the outcome. Thanks Giuseppe

2 4

[Maria-developers] Interested in Contributing Code to Maria DB.
by Alangi Derick 08 Aug '15

08 Aug '15

Hi, I am Alangi Derick and a computer engineering student(specialty: software engineering) from the University of Buea. I am a junior and i code in C. I have found interest in Maria DB. I have used many DBMS but i will like to develop for MariaDB since it is Open source and it is the one that has caught my interest. I have some contributions to make to this organisation in terms of the Maria DB project. I am very proficient in my skills of programming and will first of all start by fixing some bugs in the Maria DB code base before writing a whole module(implementation of my idea) to add to maria db. I will like if the entire community welcomes and support me so as to make Maria DB a better DBMS for everyone. Thanks and hope to participate in the development. Regards Alangi Derick Ndimnain

2 1

Re: [Maria-developers] [Commits] 686761f: MDEV-6066: Merge new defaults from 5.6 and 5.7 (part 1 (no hash serch) v2)
by Sergei Golubchik 07 Aug '15

07 Aug '15

Hi, Sanja! Summary: The implementation looks very much ok, all comments below are about details, nothing big. Well done! On Jul 14, sanja(a)mariadb.com wrote: > revision-id: 686761fbb0fc36e4fa781b68aba24af21599822d > parent(s): 302bf7c4664b904482ecc133476e822d497b114d > committer: Oleksandr Byelkin > branch nick: server could you update your post-commit trigger? thanks. > timestamp: 2015-07-14 00:11:25 +0200 > message: > > MDEV-6066: Merge new defaults from 5.6 and 5.7 (part 1 (no hash serch) v2) > diff --git a/mysql-test/extra/binlog_tests/mysqlbinlog_row_engine.inc b/mysql-test/extra/binlog_tests/mysqlbinlog_row_engine.inc > index 95440ab..c072d50 100644 > --- a/mysql-test/extra/binlog_tests/mysqlbinlog_row_engine.inc > +++ b/mysql-test/extra/binlog_tests/mysqlbinlog_row_engine.inc > @@ -16,6 +16,7 @@ > # > > --source include/have_log_bin.inc > +set sql_mode=""; please split this big commit in 'new defaults' and 'autoset' commits. > > SET NAMES 'utf8'; > #SHOW VARIABLES LIKE 'character_set%'; > diff --git a/mysql-test/r/analyze_stmt_privileges.result b/mysql-test/r/analyze_stmt_privileges.result > index d382b0b..05a35e7 100644 > --- a/mysql-test/r/analyze_stmt_privileges.result > +++ b/mysql-test/r/analyze_stmt_privileges.result > @@ -7,7 +7,7 @@ use db; > create table t1 (i int, c varchar(8)); > insert into t1 values (1,'foo'),(2,'bar'),(3,'baz'),(4,'qux'); > create view v1 as select * from t1 where i > 1; > -grant ALL on db.v1 to u1@localhost; > +set statement sql_mode="" for grant ALL on db.v1 to u1@localhost; heh. I think it would've been less confusing to put here an explicit CREATE USER statement > connect con1,localhost,u1,,; > select * from db.t1; > ERROR 42000: SELECT command denied to user 'u1'@'localhost' for table 't1' > diff --git a/mysql-test/r/blackhole_plugin.result b/mysql-test/r/blackhole_plugin.result > index 4ef9fa0..ab106f8 100644 > --- a/mysql-test/r/blackhole_plugin.result > +++ b/mysql-test/r/blackhole_plugin.result > @@ -1,7 +1,9 @@ > +set sql_mode=""; you could've put --error 1286 instead. > CREATE TABLE t1(a int) ENGINE=BLACKHOLE; > Warnings: > Warning 1286 Unknown storage engine 'BLACKHOLE' > Warning 1266 Using storage engine MyISAM for table 't1' > +set sql_mode=default; > DROP TABLE t1; > INSTALL PLUGIN blackhole SONAME 'ha_blackhole.so'; > INSTALL PLUGIN BLACKHOLE SONAME 'ha_blackhole.so'; > diff --git a/mysql-test/r/change_user.result b/mysql-test/r/change_user.result > index 18c53a5..bd87398 100644 > --- a/mysql-test/r/change_user.result > +++ b/mysql-test/r/change_user.result > @@ -1,3 +1,5 @@ > +set sql_mode=""; > +set global secure_auth=0; > grant select on test.* to test_nopw; > grant select on test.* to test_oldpw identified by password "09301740536db389"; > grant select on test.* to test_newpw identified by "newpw"; you could've used "create user" instead of "grant select on test.* to" and that would work without resetting sql_mode > diff --git a/mysql-test/r/concurrent_innodb_safelog.result b/mysql-test/r/concurrent_innodb_safelog.result > index 24a84af..03d20f4 100644 > --- a/mysql-test/r/concurrent_innodb_safelog.result > +++ b/mysql-test/r/concurrent_innodb_safelog.result > @@ -1,4 +1,5 @@ > SET GLOBAL TRANSACTION ISOLATION LEVEL REPEATABLE READ; > +SET SQL_MODE=""; same. basically I'd suggest to fix tests instead of setting sql_mode="". i suspect it's mostly adding create user or replacing grant with create user. sometimes it's about removing engine=xxx and you're doing that already. > SELECT @@global.tx_isolation; > @@global.tx_isolation > REPEATABLE-READ > diff --git a/mysql-test/r/func_compress.result b/mysql-test/r/func_compress.result > index 9fde006..4763d87 100644 > --- a/mysql-test/r/func_compress.result > +++ b/mysql-test/r/func_compress.result > @@ -1,3 +1,4 @@ > +set global max_allowed_packet=1048576; again, I'd rather fix the test > select @test_compress_string:='string for test compress function aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa '; > @test_compress_string:='string for test compress function aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ' > string for test compress function aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa > diff --git a/mysql-test/r/host_cache_size_functionality.result b/mysql-test/r/host_cache_size_functionality.result > index e7f9e09..ffa8d71 100644 > --- a/mysql-test/r/host_cache_size_functionality.result > +++ b/mysql-test/r/host_cache_size_functionality.result > @@ -9,7 +9,7 @@ SELECT COUNT(@@GLOBAL.Host_Cache_Size) > set @Default_host_cache_size=128; > select @@global.Host_Cache_Size=@Default_host_cache_size; > @@global.Host_Cache_Size=@Default_host_cache_size > -1 > +0 > 1 Expected Now that looks weird. > '#---------------------WL6372_VAR_6_02----------------------#' > # Restart server with Host_Cache_Size 1 > diff --git a/mysql-test/r/index_intersect.result b/mysql-test/r/index_intersect.result > index 1337c3f..edc52db 100644 > --- a/mysql-test/r/index_intersect.result > +++ b/mysql-test/r/index_intersect.result > @@ -1035,7 +1032,7 @@ EXPLAIN > SELECT * FROM t1 > WHERE (f1 < 535 OR f1 > 985) AND ( f4='r' OR f4 LIKE 'a%' ) ; > id select_type table type possible_keys key key_len ref rows Extra > -1 SIMPLE t1 range PRIMARY,f4 f4 35 NULL # Using index condition; Using where > +1 SIMPLE t1 index_merge PRIMARY,f4 PRIMARY,f4 4,39 NULL # Using sort_intersect(PRIMARY,f4); Using where why? > SELECT * FROM t1 > WHERE (f1 < 535 OR f1 > 985) AND ( f4='r' OR f4 LIKE 'a%' ) ; > f1 f4 f5 > diff --git a/mysql-test/r/mysqld--help.result b/mysql-test/r/mysqld--help.result > index fc22af6..1f2a8cf 100644 > --- a/mysql-test/r/mysqld--help.result > +++ b/mysql-test/r/mysqld--help.result > @@ -890,6 +890,7 @@ The following options may be given as the first argument: > write privileges to the mysql.user table. > --secure-auth Disallow authentication for accounts that have old > (pre-4.1) passwords > + (Defaults to on; use --skip-secure-auth to disable.) You know what, it would be nice if the user would know what options support the --autoset prefix. I think here's a good place for it, the help for back_log can end with gets very many connection requests in a very short time (Automatically configured unless set explicitly) Of course, this line should be automatically generated by my_print_help(), not manually added to the help text of every autoset variable. > --secure-file-priv=name > Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to > files within specified directory > diff --git a/mysql-test/r/selectivity.result b/mysql-test/r/selectivity.result > index 620bdc6..4238359 100644 > --- a/mysql-test/r/selectivity.result > +++ b/mysql-test/r/selectivity.result > @@ -1298,10 +1298,10 @@ Note 1003 select `test`.`t1`.`a` AS `a`,`test`.`t1`.`b` AS `b`,`test`.`t2`.`c` A > select * from t1, t2, t1 as t3 > where t1.b=t2.c and t2.d=t3.a and t3.b<5 and t1.a < 2000; > a b c d a b > -1495 89366 89366 28296 28296 3 > -961 24512 24512 85239 85239 4 > 1063 89366 89366 28296 28296 3 > +1495 89366 89366 28296 28296 3 > 221 56120 56120 28296 28296 3 > +961 24512 24512 85239 85239 4 why? > set optimizer_use_condition_selectivity=3; > explain extended > select * from t1, t2, t1 as t3 > @@ -1315,10 +1315,10 @@ Note 1003 select `test`.`t1`.`a` AS `a`,`test`.`t1`.`b` AS `b`,`test`.`t2`.`c` A > select * from t1, t2, t1 as t3 > where t1.b=t2.c and t2.d=t3.a and t3.b<5 and t1.a < 2000; > a b c d a b > -961 24512 24512 85239 85239 4 > -1495 89366 89366 28296 28296 3 > 1063 89366 89366 28296 28296 3 > +1495 89366 89366 28296 28296 3 > 221 56120 56120 28296 28296 3 > +961 24512 24512 85239 85239 4 why? > set optimizer_use_condition_selectivity=@save_optimizer_use_condition_selectivity; > drop table t1,t2; > set histogram_type=@save_histogram_type; > diff --git a/mysql-test/r/selectivity_innodb.result b/mysql-test/r/selectivity_innodb.result > index 0acbb46..daf2807 100644 > --- a/mysql-test/r/selectivity_innodb.result > +++ b/mysql-test/r/selectivity_innodb.result > @@ -1308,10 +1308,10 @@ Note 1003 select `test`.`t1`.`a` AS `a`,`test`.`t1`.`b` AS `b`,`test`.`t2`.`c` A > select * from t1, t2, t1 as t3 > where t1.b=t2.c and t2.d=t3.a and t3.b<5 and t1.a < 2000; > a b c d a b > -1495 89366 89366 28296 28296 3 > -961 24512 24512 85239 85239 4 > 1063 89366 89366 28296 28296 3 > +1495 89366 89366 28296 28296 3 > 221 56120 56120 28296 28296 3 > +961 24512 24512 85239 85239 4 why? > set optimizer_use_condition_selectivity=3; > explain extended > select * from t1, t2, t1 as t3 > @@ -1325,10 +1325,10 @@ Note 1003 select `test`.`t1`.`a` AS `a`,`test`.`t1`.`b` AS `b`,`test`.`t2`.`c` A > select * from t1, t2, t1 as t3 > where t1.b=t2.c and t2.d=t3.a and t3.b<5 and t1.a < 2000; > a b c d a b > -961 24512 24512 85239 85239 4 > -1495 89366 89366 28296 28296 3 > 1063 89366 89366 28296 28296 3 > +1495 89366 89366 28296 28296 3 > 221 56120 56120 28296 28296 3 > +961 24512 24512 85239 85239 4 why? > set optimizer_use_condition_selectivity=@save_optimizer_use_condition_selectivity; > drop table t1,t2; > set histogram_type=@save_histogram_type; > diff --git a/mysql-test/suite/funcs_1/r/is_columns_is_embedded.result b/mysql-test/suite/funcs_1/r/is_columns_is_embedded.result > index c314165..bd49492 100644 > --- a/mysql-test/suite/funcs_1/r/is_columns_is_embedded.result > +++ b/mysql-test/suite/funcs_1/r/is_columns_is_embedded.result > @@ -167,9 +167,9 @@ def information_schema GEOMETRY_COLUMNS MAX_PPR 12 0 NO tinyint NULL NULL 3 0 NU > def information_schema GEOMETRY_COLUMNS SRID 13 0 NO smallint NULL NULL 5 0 NULL NULL NULL smallint(5) > def information_schema GEOMETRY_COLUMNS STORAGE_TYPE 9 0 NO tinyint NULL NULL 3 0 NULL NULL NULL tinyint(2) > def information_schema GLOBAL_STATUS VARIABLE_NAME 1 NO varchar 64 192 NULL NULL NULL utf8 utf8_general_ci varchar(64) > -def information_schema GLOBAL_STATUS VARIABLE_VALUE 2 NO varchar 1024 3072 NULL NULL NULL utf8 utf8_general_ci varchar(1024) > +def information_schema GLOBAL_STATUS VARIABLE_VALUE 2 NO varchar 2048 6144 NULL NULL NULL utf8 utf8_general_ci varchar(2048) > def information_schema GLOBAL_VARIABLES VARIABLE_NAME 1 NO varchar 64 192 NULL NULL NULL utf8 utf8_general_ci varchar(64) > -def information_schema GLOBAL_VARIABLES VARIABLE_VALUE 2 NO varchar 1024 3072 NULL NULL NULL utf8 utf8_general_ci varchar(1024) > +def information_schema GLOBAL_VARIABLES VARIABLE_VALUE 2 NO varchar 2048 6144 NULL NULL NULL utf8 utf8_general_ci varchar(2048) why? > def information_schema INDEX_STATISTICS INDEX_NAME 3 NO varchar 192 576 NULL NULL NULL utf8 utf8_general_ci varchar(192) > def information_schema INDEX_STATISTICS ROWS_READ 4 0 NO bigint NULL NULL 19 0 NULL NULL NULL bigint(21) > def information_schema INDEX_STATISTICS TABLE_NAME 2 NO varchar 192 576 NULL NULL NULL utf8 utf8_general_ci varchar(192) > diff --git a/mysql-test/suite/perfschema/r/stage_mdl_global.result b/mysql-test/suite/perfschema/r/stage_mdl_global.result > index 1a6f51a..de5df8f 100644 > --- a/mysql-test/suite/perfschema/r/stage_mdl_global.result > +++ b/mysql-test/suite/perfschema/r/stage_mdl_global.result > @@ -6,6 +6,8 @@ user1 statement/sql/flush flush tables with read lock > username event_name nesting_event_type > username event_name nesting_event_type > user1 stage/sql/init STATEMENT > +user1 stage/sql/Waiting for query cache lock STATEMENT > +user1 stage/sql/init STATEMENT why? > user1 stage/sql/query end STATEMENT > user1 stage/sql/closing tables STATEMENT > user1 stage/sql/freeing items STATEMENT > diff --git a/mysys/my_getopt.c b/mysys/my_getopt.c > index 88fa3d4..1d32833 100644 > --- a/mysys/my_getopt.c > +++ b/mysys/my_getopt.c > @@ -47,14 +47,15 @@ static char *check_struct_option(char *cur_arg, char *key_name); > order of their arguments must correspond to each other. > */ > static const char *special_opt_prefix[]= > -{"skip", "disable", "enable", "maximum", "loose", 0}; > +{"skip", "disable", "enable", "maximum", "loose", "autoset", 0}; So, I'm not very happy with the --autoset- prefix. Because it cannot be used from the SQL. In SQL we have to use SET variable=AUTO; or something like that. And it would be good to use the same approach on the command line and from SQL. On the other hand, =AUTO has its problems too - it conflicts with the AUTO value of the enum/set variables. I don't have a good answer to this :( > static const uint special_opt_prefix_lengths[]= > -{ 4, 7, 6, 7, 5, 0}; > +{ 4, 7, 6, 7, 5, 7, 0}; > enum enum_special_opt > -{ OPT_SKIP, OPT_DISABLE, OPT_ENABLE, OPT_MAXIMUM, OPT_LOOSE}; > +{ OPT_SKIP, OPT_DISABLE, OPT_ENABLE, OPT_MAXIMUM, OPT_LOOSE, OPT_AUTOSET}; > > char *disabled_my_option= (char*) "0"; > char *enabled_my_option= (char*) "1"; > +char *autoset_my_option= (char*) ""; > > /* > This is a flag that can be set in client programs. 0 means that > @@ -458,6 +462,36 @@ int handle_options(int *argc, char ***argv, > } > argument= optend; > } > + else if (option_is_autoset) > + { > + if (optend) > + { > + my_getopt_error_reporter(ERROR_LEVEL, > + "%s: automatic setup request of " > + "option '--%s' cannot take an argument", > + my_progname, optp->name); heh, another argument agains --autoset- prefix. With the =AUTO value one cannot possibly have this error. So, perhaps, I'm *slighty* leaning towards the =AUTO vs --autoset. > + > + DBUG_RETURN(EXIT_NO_ARGUMENT_ALLOWED); > + } > + /* > + We support automatic setup only via get_one_option and only for > + marked options. > + */ > + if (!get_one_option || > + !(optp->var_type & GET_AUTO)) > + { > + my_getopt_error_reporter(option_is_loose ? > + WARNING_LEVEL : ERROR_LEVEL, > + "%s: automatic setup request is " > + "unsupported by option '--%s'", > + my_progname, optp->name); > + if (!option_is_loose) > + return EXIT_ARGUMENT_INVALID; > + continue; > + } > + else > + argument= autoset_my_option; > + } > else if (optp->arg_type == REQUIRED_ARG && !optend) > { > /* Check if there are more arguments after this one, > diff --git a/sql/set_var.cc b/sql/set_var.cc > index c65ca3d..2251c36 100644 > --- a/sql/set_var.cc > +++ b/sql/set_var.cc > @@ -1075,7 +1076,7 @@ int fill_sysvars(THD *thd, TABLE_LIST *tables, COND *cond) > // NUMERIC_MAX_VALUE > // NUMERIC_BLOCK_SIZE > bool is_unsigned= true; > - switch (var->option.var_type) > + switch (vartype) ah, yes. thanks. > { > case GET_INT: > case GET_LONG: > @@ -1176,3 +1177,20 @@ void mark_sys_var_value_origin(void *ptr, enum sys_var::where here) > DBUG_ASSERT(found); // variable must have been found > } > > +enum sys_var::where get_sys_var_value_origin(void *ptr) > +{ > + DBUG_ASSERT(!mysqld_server_started); // only to be used during startup > + > + for (uint i= 0; i < system_variable_hash.records; i++) > + { > + sys_var *var= (sys_var*) my_hash_element(&system_variable_hash, i); > + if (var->option.value == ptr) > + { > + return var->value_origin; //first match > + } > + } > + > + DBUG_ASSERT(1); // variable must have been found you mean DBUG_ASSERT(0); here > + return sys_var::CONFIG; > +} > + > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 43a3f0e..21a1198 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -4316,6 +4316,27 @@ static int init_common_variables() > } > #endif /* HAVE_SOLARIS_LARGE_PAGES */ > > + > + /* Fix host_cache_size. */ > + if (IS_SYSVAR_AUTOSIZE(&host_cache_size)) > + { > + if (max_connections <= 628 - 128) > + SYSVAR_AUTOSIZE(host_cache_size, 128 + max_connections); > + else if (max_connections <= ((ulong)(2000 - 628)) * 20 + 500) > + SYSVAR_AUTOSIZE(host_cache_size, 628 + ((max_connections - 500) / 20)); > + else > + SYSVAR_AUTOSIZE(host_cache_size, 2000); > + } > + > + /* Fix back_log */ > + if (back_log == 0 || IS_SYSVAR_AUTOSIZE(&back_log)) why back_log==0 ? > + { > + if ((900 - 50) * 5 >= max_connections) > + SYSVAR_AUTOSIZE(back_log, (50 + max_connections / 5)); > + else > + SYSVAR_AUTOSIZE(back_log, 900); > + } > + > /* connections and databases needs lots of files */ > { > uint files, wanted_files, max_open_files; > diff --git a/sql/sys_vars.ic b/sql/sys_vars.ic > index 7c0359a..b04bdb5 100644 > --- a/sql/sys_vars.ic > +++ b/sql/sys_vars.ic > @@ -226,6 +227,7 @@ class Sys_var_integer: public sys_var > typedef Sys_var_integer<int, GET_INT, SHOW_SINT> Sys_var_int; > typedef Sys_var_integer<uint, GET_UINT, SHOW_UINT> Sys_var_uint; > typedef Sys_var_integer<ulong, GET_ULONG, SHOW_ULONG> Sys_var_ulong; > +typedef Sys_var_integer<ulong, (GET_ULONG|GET_AUTO), SHOW_ULONG> Sys_var_aulong; Hmm, why did you make it a new type? I think the *type* of the variable doesn't depend on the auto-set behavior. The type defines storage size, how to print the value, etc. While auto-set is just a behavior modifier. I'd rather put it in flags: - READ_ONLY GLOBAL_VAR(back_log), CMD_LINE(REQUIRED_ARG), + AUTO_SET READ_ONLY GLOBAL_VAR(back_log), CMD_LINE(REQUIRED_ARG), > typedef Sys_var_integer<ha_rows, GET_HA_ROWS, SHOW_HA_ROWS> Sys_var_harows; > typedef Sys_var_integer<ulonglong, GET_ULL, SHOW_ULONGLONG> Sys_var_ulonglong; > typedef Sys_var_integer<long, GET_LONG, SHOW_SLONG> Sys_var_long; > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 7a7b9d1..6d90305 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -360,12 +360,12 @@ static Sys_var_mybool Sys_automatic_sp_privileges( > GLOBAL_VAR(sp_automatic_privileges), > CMD_LINE(OPT_ARG), DEFAULT(TRUE)); > > -static Sys_var_ulong Sys_back_log( > +static Sys_var_aulong Sys_back_log( > "back_log", "The number of outstanding connection requests " > "MySQL can have. This comes into play when the main MySQL thread " s/MySQL/MariaDB/ I suppose > "gets very many connection requests in a very short time", > READ_ONLY GLOBAL_VAR(back_log), CMD_LINE(REQUIRED_ARG), > - VALID_RANGE(1, 65535), DEFAULT(150), BLOCK_SIZE(1)); > + VALID_RANGE(0, 65535), DEFAULT(150), BLOCK_SIZE(1)); > > static Sys_var_charptr Sys_basedir( > "basedir", "Path to installation directory. All paths are " > @@ -2751,16 +2743,38 @@ static bool check_query_cache_type(sys_var *self, THD *thd, set_var *var) > my_error(ER_QUERY_CACHE_IS_DISABLED, MYF(0)); > return true; > } > - if (var->type != OPT_GLOBAL && > - global_system_variables.query_cache_type == 0 && > - var->value->val_int() != 0) > + > + if (var->type != OPT_GLOBAL && global_system_variables.query_cache_type == 0) > { > - my_error(ER_QUERY_CACHE_IS_GLOBALY_DISABLED, MYF(0)); > - return true; > - } > + uint value= 0 /*default is off*/; > + if (var->value) > + { > + if (var->value->result_type() == INT_RESULT) > + value= var->value->val_int(); > + else > + { > + char buff[STRING_BUFFER_USUAL_SIZE]; > + String str(buff, sizeof(buff), system_charset_info), *res; > + if (!(res=var->value->val_str(&str))) > + return true; > + if (res->length() != 3 || > + my_toupper(res->charset(), res->ptr()[0]) != 'O' || > + my_toupper(res->charset(), res->ptr()[1]) != 'F' || > + my_toupper(res->charset(), res->ptr()[2]) != 'F') eh? query_cache_type is ENUM variable, there is no need to compare string values manually. > + value= 1; // set to something not 0 > + } > + } > > + if (value != 0) > + { > + my_error(ER_QUERY_CACHE_IS_GLOBALY_DISABLED, MYF(0)); > + return true; > + } > + } > return false; > } > + > + > static bool fix_query_cache_type(sys_var *self, THD *thd, enum_var_type type) > { > if (type != OPT_GLOBAL) > diff --git a/unittest/mysys/CMakeLists.txt b/unittest/mysys/CMakeLists.txt > index 8c09a46..6564e09 100644 > --- a/unittest/mysys/CMakeLists.txt > +++ b/unittest/mysys/CMakeLists.txt > @@ -13,7 +13,7 @@ > # along with this program; if not, write to the Free Software > # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > > -MY_ADD_TESTS(bitmap base64 my_vsnprintf my_atomic my_rdtsc lf my_malloc > +MY_ADD_TESTS(bitmap base64 my_vsnprintf my_atomic my_rdtsc lf my_malloc my_getopt Ah, that's great, thanks! Long overdue. > LINK_LIBRARIES mysys) > > MY_ADD_TESTS(ma_dyncol Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 69704b6: MDEV-6887 Optimize sys_vars.sysvar_* tests
by Sergei Golubchik 06 Aug '15

06 Aug '15

Hi, Elena! On Aug 06, elenst(a)montyprogram.com wrote: > revision-id: 69704b6c6f895fa706f83bcd3bcefe267aaf8e57 > parent(s): afd59b575a75ebbc57f71ce2865fdff85e3e233b > committer: Elena Stepanova > branch nick: bb-10.1-elenst > timestamp: 2015-08-06 02:55:43 +0300 > message: > > MDEV-6887 Optimize sys_vars.sysvar_* tests > > diff --git a/mysql-test/suite/sys_vars/inc/sysvars_server.inc b/mysql-test/suite/sys_vars/inc/sysvars_server.inc > index cb06b40..98bffd4 100644 > --- a/mysql-test/suite/sys_vars/inc/sysvars_server.inc > +++ b/mysql-test/suite/sys_vars/inc/sysvars_server.inc > @@ -11,8 +12,9 @@ set sql_mode=ansi_quotes; > # global_value_origin=SQL > set global div_precision_increment=5; > > ---replace_regex /^\/\S+/PATH/ > +--replace_regex /^\/\S+/PATH/ /^[a-zA-Z]:[\\\/]\S+/PATH/ /^BIGINT\b/<INT or BIGINT>/ /^INT\b/<INT or BIGINT>/ /^(?:18446744073709551615\b|4294967295\b)/<32-bit or 64-bit MAX UNSIGNED>/ /^(?:9223372036854775807\b|2147483647\b)/<32-bit or 64-bit MAX SIGNED>/ /^(?:18446744073709547520\b|4294963200\b)/<32-bit or 64-bit MAX UNSIGNED ADJUSTED>/ /^(?:9223372036853727232\b|2146435072\b)/<32-bit or 64-bit MAX SIGNED ADJUSTED>/ Aren't you removing too much with these replacements? It's kinda good to get rid of 32bit diff files (as they aren't easy to update), but I wouldn't replace every INT or BIGINT with <INT or BIGINT>. May be, move them to a separate select, like let type=BIGINT; if (32-bit) { let type=INT; } --replace $type <INT-or-BIGINT> select * from information_schema.system_variables where variable_name in (list of ulong variables); and something similar for other replacements. In fact, there's obscure feature of let $repl=/foo/bar/ /abc/def; replace_regex $repl; so you can have sysvar32bitrepl.inc that will set up $replace_regexes variable: let type=BIGINT; let max=18446744073709551615; if (32-bit) { let type=INT; let max=4294967295; } let replace_regexes=/$type/INT or BIGINT/ /$max/<32-bit or 64-bit MAX UNSIGNED>/; Then in every sysvar*.test file you can do, like --source sysvar32bitrepl --replace_regex $replace_regexes select * from information_schema.system_variables where variable_name in (list of ulong variables); Regards, Sergei

2 2

Re: [Maria-developers] MDEV-8389 patch review
by Sergey Petrunia 06 Aug '15

06 Aug '15

Hi Diwas, On Wed, Aug 05, 2015 at 08:47:28PM +0530, Diwas Joshi wrote: > I have removed the errors from previous patch, I only got those errors when > i compiled with valgrind. Everything seems to be working fine now. Also I > have fixed the test case for select * from mysql.proc. Also added a > function that removes the tables after they have been used. I have added > the errors that we discussed about and also test cases corresponding to > these errors. Please review. > The big problems in the patch are: 1. check_and_process_table_functions() is at the wrong place. I debug a query, "SELECT * FROM tbl" , where 'tbl' is a regular table. I see check_and_process_table_functions() invoke sp_find_routine() for "test.tbl". Why does this happen? The query makes it clear that the table is not a table function. This is not acceptable. 2. Temporary table 2.1 The temporary table that is created uses InnoDB storage engine. This doesn't follow any other kind of temporary table. When execution engine creates temporary tables (e.g. for VIEWs, or GROUP BY), these temporary tables are either Aria or MyISAM tables, depending on the server settings. We should follow that. 2.2 Temporary table is created by building the "CREATE TEMPORARY TABLE (...)" string and passing it to the parser. Did you try looking into creating the temp. table directly from the data strucutres you have? No other code has to build "CREATE TABLE" statements and then parse them back. I suspect Table Functions don't need to do this either. The rest of the feedback is below: > diff --git a/mysql-test/t/sp-error.test b/mysql-test/t/sp-error.test > index ccf5b98..2ad4144 100644 > --- a/mysql-test/t/sp-error.test > +++ b/mysql-test/t/sp-error.test > @@ -3846,7 +3846,23 @@ DROP FUNCTION f1; > DROP FUNCTION f2; > DROP TABLE t1; > > ---error 1235 > -SELECT * FROM f1('xyz'); > ---error 1064 > -SELECT * FROM t1, f1('xyz',t1.a); > +delimiter |; > +CREATE FUNCTION f1() RETURNS INTEGER > +BEGIN > + RETURN 1; > +END| > +delimiter ;| > +--error 1979 Please use error names, not error numbers. you can write --error ER_SP_SCALAR_VALUE_EXPECTED. > +SELECT * FROM f1(); > +DROP FUNCTION f1; > + > +delimiter |; > +CREATE FUNCTION f1(a INT, b VARCHAR(11)) > +RETURNS TABLE t1(id INT, name VARCHAR(11)) > +BEGIN > +INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > +END| > +delimiter ;| > +--error 1978 Same as above. > +SELECT f1(1,'AA'); > +DROP FUNCTION f1; > diff --git a/mysql-test/t/sp.test b/mysql-test/t/sp.test > index 5c42df0..45db6a3 100644 > --- a/mysql-test/t/sp.test > +++ b/mysql-test/t/sp.test > @@ -9398,6 +9398,6 @@ END| > DELIMITER ;| > > --vertical_results > -SELECT * FROM mysql.proc WHERE db = database() and name = 'f1'; > - > +SELECT db, name, type, specific_name, language, sql_data_access, is_deterministic, security_type, param_list, returns, body, definer, sql_mode, comment, character_set_client, collation_connection, db_collation, body_utf8 FROM mysql.proc WHERE db = database() and name = 'f1'; The change is ok. Please also format the query so that its width is limited 80 chars. > +SELECT * FROM f1(10,'XYZ'); > DROP FUNCTION f1; > diff --git a/sql/item_func.cc b/sql/item_func.cc > index 3e78be8..ad18e2e 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -6574,7 +6574,12 @@ void my_missing_function_error(const LEX_STRING &token, const char *func_name) > context->process_error(thd); > DBUG_RETURN(TRUE); > } > - > + > + if (m_sp->m_type == TYPE_ENUM_TABLE){ > + my_error(ER_SP_SCALAR_VALUE_EXPECTED, MYF(0)); > + context->process_error(thd); > + DBUG_RETURN(TRUE); > + } > /* > A Field need to be attached to a Table. > Below we "create" a dummy table by initializing > @@ -6588,7 +6593,7 @@ void my_missing_function_error(const LEX_STRING &token, const char *func_name) > dummy_table->copy_blobs= TRUE; > share->table_cache_key = empty_name; > share->table_name = empty_name; > - > + Please don't make dummy changes like this. > if (!(sp_result_field= m_sp->create_result_field(max_length, name, > dummy_table))) > { > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 2998972..6f94720 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2467,6 +2467,8 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > /* should be assigned after making first tables same */ > all_tables= lex->query_tables; > /* set context for commands which do not use setup_tables */ > + if (lex->sql_command == SQLCOM_SELECT) > + check_and_process_table_functions(thd, all_tables); Why limit table function uses to SELECTs ? A user can write an INSERT.. SELECT statement INSERT INTO table SELECT * FROM table_func('argument'); or a multi-table UPDATE statement: UPDATE regular_table, table_func('arg') SET regular_table.col= ...; In general, there should not be a limit. I think, it's ok to introduce the limit within the scope of this MDEV entry. In that case, you need to add a comment with 'TODO-DiwasJoshi:...' saying that the limitation is temporary. > select_lex-> > context.resolve_in_table_list_only(select_lex-> > table_list.first); > @@ -5697,7 +5699,8 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > thd->mdl_context.release_transactional_locks(); > } > #endif /* WITH_WSREP */ > - > + if (lex->sql_command == SQLCOM_SELECT) > + clear_table_functions(thd, all_tables); This looks like an odd place to do cleanups. Let's get back to this once other issues are resolved. > DBUG_RETURN(res || thd->is_error()); > } > > @@ -9222,3 +9225,185 @@ CHARSET_INFO *find_bin_collation(CHARSET_INFO *cs) > } > return cs; > } > + > +/* > + @brief > + Check for table functions and create corresponding tables to be used in query. > + > + @param thd Thread handle > + @param table_list A list of tables being processed in the query. > + > + @return > + False on success > + @retval > + TRUE on error > +*/ > +bool check_and_process_table_functions(THD *thd, TABLE_LIST *table_list) > +{ > + TABLE_LIST *tables = table_list; > + int err= 0; > + while(tables) > + { > + if (tables->table_name == NULL) > + { > + tables = tables->next_local; > + continue; > + } > + sp_head *sp; > + sp_name *name; > + LEX_STRING lex_db; > + LEX_STRING lex_name; > + lex_db.length= strlen(tables->db); > + lex_name.length= strlen(tables->table_name); > + lex_db.str= thd->strmake(tables->db, lex_db.length); > + lex_name.str= thd->strmake(tables->table_name, lex_name.length); > + name= new sp_name(lex_db, lex_name, true); > + name->init_qname(thd); > + int err=0; > + > + if (!err && (sp = sp_find_routine(thd,TYPE_ENUM_FUNCTION, name, > + &thd->sp_proc_cache, FALSE))) //check if the table is a table-valued function. > + { > + if (sp->m_type == TYPE_ENUM_FUNCTION) > + my_error(ER_SP_TABLE_VALUE_EXPECTED, MYF(0)); > + LEX *old_lex= thd->lex, newlex; > + thd->lex= &newlex; > + ulonglong sql_mode= 0 , saved_mode= thd->variables.sql_mode; > + String defstr; //query in defstr will be parsed to create temporary table. > + defstr.append(STRING_WITH_LEN("CREATE TEMPORARY TABLE ")); > + defstr.append(tables->table_name); > + defstr.append(STRING_WITH_LEN(" (")); > + List_iterator_fast<Create_field> it(sp->m_cols_list); > + > + TABLE table; > + TABLE_SHARE share; > + bzero((char*) &table, sizeof(table)); > + bzero((char*) &share, sizeof(share)); > + table.in_use= thd; > + table.s = &share; > + Create_field *cols_field; > + cols_field= it++; > + while (cols_field) > + { > + uint unused1= 0; > + sp_prepare_create_field(thd, cols_field); > + prepare_create_field(cols_field, &unused1, HA_CAN_GEOMETRY); > + uint field_length; > + Field *field; > + String tmp_result(512); > + field_length= cols_field->length; > + cols_field->flags= 0; > + field= make_field(table.s, /* TABLE_SHARE ptr */ > + (uchar*) 0, /* field ptr */ > + field_length, /* field [max] length */ > + (uchar*) "", /* null ptr */ > + 0, /* null bit */ > + cols_field->pack_flag, > + cols_field->sql_type, > + cols_field->charset, > + cols_field->geom_type, cols_field->srid, > + Field::NONE, /* unreg check */ > + cols_field->interval, > + cols_field->field_name); > + field->vcol_info= cols_field->vcol_info; > + field->stored_in_db= cols_field->stored_in_db; > + if (field) > + field->init(&table); > + defstr.append(cols_field->field_name); > + defstr.append(STRING_WITH_LEN(" ")); > + field->sql_type(tmp_result); > + defstr.append(tmp_result); > + cols_field= it++; > + if(cols_field) > + defstr.append(STRING_WITH_LEN(", ")); > + delete field; > + } > + defstr.append(STRING_WITH_LEN(")")); > + newlex.create_info.DDL_options_st::init(); > + newlex.current_select= NULL; > + Parser_state parser_state; > + thd->variables.sql_mode= sql_mode; > + err= parser_state.init(thd, defstr.c_ptr_safe(), defstr.length()); > + if (!err) { > + lex_start(thd); > + err= parse_sql(thd, &parser_state, NULL, TRUE) || thd->lex == NULL; > + } > + if (!err) > + { > + Table_specification_st create_info(thd->lex->create_info); > + Alter_info alter_info(thd->lex->alter_info, thd->mem_root); > + thd->lex->create_info.default_table_charset= NULL; > + SELECT_LEX *select_lex= &thd->lex->select_lex; > + TABLE_LIST *create_table= select_lex->table_list.first; > + if (append_file_to_dir(thd, &create_info.data_file_name, > + create_table->table_name) || > + append_file_to_dir(thd, &create_info.index_file_name, > + create_table->table_name)) > + { > + err= 1; > + } > + if (!(create_info.used_fields & HA_CREATE_USED_ENGINE)) > + create_info.use_default_db_type(thd); > + > + err= mysql_create_table(thd, create_table, &create_info, &alter_info); > + thd->variables.option_bits|= OPTION_KEEP_LOG; > + } > + thd->variables.sql_mode= saved_mode; > + lex_end(thd->lex); > + thd->lex= old_lex; > + } > + tables = tables->next_local; > + } > + if (!err) > + close_thread_tables(thd); > + return err; > +} > + > +/* > + @brief > + Drop tables created for table functions. > + > + @param thd Thread handle > + @param table_list A list of tables being processed in the query. > + > + @return > + False on success > + @retval > + TRUE on error > +*/ > +bool clear_table_functions(THD *thd, TABLE_LIST *table_list) > +{ > + TABLE_LIST *tables = table_list; > + int err= 0; > + while(tables) > + { > + if (tables->table_name == NULL) > + { > + tables = tables->next_local; > + continue; > + } > + sp_head *sp; > + sp_name *name; > + LEX_STRING lex_db; > + LEX_STRING lex_name; > + lex_db.length= strlen(tables->db); > + lex_name.length= strlen(tables->table_name); > + lex_db.str= thd->strmake(tables->db, lex_db.length); > + lex_name.str= thd->strmake(tables->table_name, lex_name.length); > + name= new sp_name(lex_db, lex_name, true); > + name->init_qname(thd); > + int err=0; > + if (!err && (sp = sp_find_routine(thd,TYPE_ENUM_FUNCTION, name, > + &thd->sp_proc_cache, FALSE)) && This function is supposed to just close the temporary tables. Why does it need to search for SPs? > + sp->m_type== TYPE_ENUM_TABLE) //check if the table is a table-valued function. > + { > + if(tables->table) > + close_temporary_table(thd,tables->table,TRUE,TRUE); > + if (thd->is_error()) > + err= 1; > + tables->table= NULL; > + } > + tables= tables->next_local; > + } > + return err; > +} BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Best way of starting multi-source connections with active slave.
by Giuseppe Maxia 06 Aug '15

06 Aug '15

Hi, I am setting up multi-source replication with four nodes, where nodes 1, 2, and 3 are masters and node 4 is the slave of the first 3. The slave is *active*, meaning that it has a binary log and runs transactions of its own. All is well if I run 'reset master' in all nodes before setting up replication. CHANGE MASTER 'node1' TO master_host='host1', master_port=3306, master_user='some_slave', master_password='some_slave', master_use_gtid=current_pos; CHANGE MASTER 'node2' TO master_host='host2', master_port=3306, master_user='some_slave', master_password='some_slave', master_use_gtid=current_pos; CHANGE MASTER 'node3' TO master_host='host3', master_port=3306, master_user='some_slave', master_password='some_slave', master_use_gtid=current_pos; start all slaves; However, if host 4 has a binary log and has logged one or more transactions before starting replication, we get a conflict. Got fatal error 1236 from master when reading data from binary log: 'Error: connecting slave requested to start from GTID 0-104-1, which is not in the master's binlog. Since the master's binlog contains GTIDs with higher sequence numbers, it probably means that the slave has diverged due to executing extra erroneous transactions' I get the same result if I use 'slave_pos' instead of 'current_pos'. Notice that the changes in the slave do not conflict with anything that happens in the masters. For example: #1 node4 (fan-in slave): create user distribution_user identified by 'something'; #2 node4 CHANGE MASTER ... #3 node4 start all slaves #4 node1 (master): create table t1 #5 node2 (master): create table t2 #6 node3 (master): create table t3 This sequence of events fails. If I execute event #1 after #2 and #3, then all works as expected. Or it all works if I say master_use_gtid=no, but that would defeat the purpose of having GTIDs, or am I missing something? Is there a way of enabling multi-source replication with the slave having non-conflicting transactions already logged? Thanks Giuseppe

3 3

Re: [Maria-developers] 3eceb51: MDEV-7990: ERROR 1526 when procedure executed for second time ALTER TABLE partition ... pMAX values less than MAXVALUE
by Sergei Golubchik 05 Aug '15

05 Aug '15

Hi, Sanja! On May 06, sanja(a)mariadb.com wrote: > revision-id: 3eceb51e697d5a514e70c374a06e7eef6b5a37a0 > parent(s): aa5095627e2619bdad7916d33d1016802a84a9e1 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-05-06 13:19:22 +0200 > message: > > MDEV-7990: ERROR 1526 when procedure executed for second time ALTER > TABLE partition ... pMAX values less than MAXVALUE > > Added cleaning partition data which are saved in LEX structure to make > possible correct re-execution of a SP or PS using the code. No, this is wrong. See prep_alter_part_table(). The first thing it does it to copy thd->lex->part_info to thd->work_part_info. It tries to do a deep copy to be able to modify it freely, but apparently the copy isn't deep enough. I see two options 1. Do a deeper copy. Duplicate all value lists. 2. Make part_column_list_val elements immutable. That would be preferred, of course, but I'm not sure it's possible. Regards, Sergei

2 2

Re: [Maria-developers] MDEV-8389 patch review
by Sergey Petrunia 03 Aug '15

03 Aug '15

On Sun, Aug 02, 2015 at 08:51:44PM +0530, Diwas Joshi wrote: > So what would be the expected result of query select f1(1,'aa'). since it > will be a table shouldn't we use select * from f1(1,'aa'). I know it > shouldn't crash but what should be the output of that query? > My first guess is that since we are storing this is as functions too(as we > decided earlier) it is crashing so we'll have to check for table functions > in the execution cycle of functions Agree. When a table function is invoked in the context where a scalar value is expected, the statement should fail with an error (Please look in sql/share/errmsg.txt and pick an error code. If none works, please add a new error code/message). However, this is not what I wanted to complain about (I pasted the wrong example). Let's consider a testcase where a table function is used in the scalar context (and should return an empty table): == example starts == delimiter | CREATE FUNCTION f1(param1 VARCHAR(11)) RETURNS TABLE t1(name VARCHAR(11)) deterministic BEGIN INSERT INTO t1 SELECT a FROM t2 WHERE id = param1; END| DELIMITER ; select 1 from f1('aaa'); == example ends == This crashes for me, too. The crash is in your function: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff7eb8700 (LWP 12653)] 0x0000555555b4c56b in Object_creation_ctx::set_n_backup (this=0xb, thd=0x55555ac4aeb0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/table.cc:83 (gdb) wher #0 0x0000555555b4c56b in Object_creation_ctx::set_n_backup (this=0xb, thd=0x55555ac4aeb0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/table.cc:83 #1 0x0000555555a6d280 in parse_sql (thd=0x55555ac4aeb0, parser_state=0x7ffff7eb4580, creation_ctx=0xb, do_pfs_digest=false) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:9100 #2 0x0000555555a6dcbb in check_and_process_table_functions (thd=0x55555ac4aeb0, table_list=0x7fff5c007630) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:9336 #3 0x0000555555a5ab77 in mysql_execute_command (thd=0x55555ac4aeb0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:2471 #4 0x0000555555a6942e in mysql_parse (thd=0x55555ac4aeb0, rawbuf=0x7fff5c007408 "select 1 from f1('aaa')", length=23, parser_state=0x7ffff7eb7100) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:7174 #5 0x0000555555a5838b in dispatch_command (command=COM_QUERY, thd=0x55555ac4aeb0, packet=0x55555ac51f71 "select 1 from f1('aaa')", packet_length=23) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:1470 #6 0x0000555555a570a5 in do_command (thd=0x55555ac4aeb0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:1093 #7 0x0000555555b95929 in do_handle_one_connection (thd_arg=0x55555ac4aeb0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_connect.cc:1347 #8 0x0000555555b9566e in handle_one_connection (arg=0x55555ac4aeb0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_connect.cc:1258 #9 0x00007ffff691de9a in start_thread (arg=0x7ffff7eb8700) at pthread_create.c:308 #10 0x00007ffff604e3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #11 0x0000000000000000 in ?? () Please check what's going on. (If you don't get the crash, compile and run under valgrind). > Also i removed the test case SELECT * FROM mysql.proc WHERE db = database() > and name = 'f1' because it was failing as it is writing time to result file > so the test passes if we use --record since it writes new file but fails > for ./mysql-test-run sp.test with differences in reject and result file Good point. However, the solution should be not to remove the testcase, but rather make the testcase output not depend on the date/time when the testcase was run. Why did .result file have time/date. Let's look: -SELECT * FROM mysql.proc WHERE db = database() and name = 'f1'; ... -definer root@localhost -created 2015-07-03 16:36:36 -modified 2015-07-03 16:36:36 -sql_mode -comment -character_set_client latin1 ... It did so, because the SELECT selects 'created' and 'modified' columns from mysql.proc. The testcase doesn't really need these columns, please make the SELECT not to select them. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 1

Re: [Maria-developers] MDEV-8389 patch review
by Sergey Petrunia 02 Aug '15

02 Aug '15

Hi Diwas, I've got a crash when trying the first example. The example was: CREATE FUNCTION f1(a INT, b VARCHAR(11)) RETURNS TABLE t1(id INT, name VARCHAR(11)) DETERMINISTIC BEGIN INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; END|; delimiter ; select f1(1,'aa'); The crash was: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff7eb8700 (LWP 24655)] 0x0000555555c6a82a in Field_str::Field_str (this=0x7fff5c007d30, ptr_arg=0x0, len_arg=4294967295, null_ptr_arg=0x55555644299c "", null_bit_arg=1 '\001', unireg_check_arg=Field::NONE, field_name_arg=0x7fff5c006780 "f1(1,'aa')", charset_arg=0x0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/field.cc:1845 (gdb) wher #0 0x0000555555c6a82a in Field_str::Field_str (this=0x7fff5c007d30, ptr_arg=0x0, len_arg=4294967295, null_ptr_arg=0x55555644299c "", null_bit_arg=1 '\001', unireg_check_arg=Field::NONE, field_name_arg=0x7fff5c006780 "f1(1,'aa')", charset_arg=0x0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/field.cc:1845 #1 0x0000555555ae6820 in Field_longstr::Field_longstr (this=0x7fff5c007d30, ptr_arg=0x0, len_arg=4294967295, null_ptr_arg=0x55555644299c "", null_bit_arg=1 '\001', unireg_check_arg=Field::NONE, field_name_arg=0x7fff5c006780 "f1(1,'aa')", charset_arg=0x0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/field.h:1200 #2 0x0000555555c7d85c in Field_blob::Field_blob (this=0x7fff5c007d30, ptr_arg=0x0, null_ptr_arg=0x55555644299c "", null_bit_arg=1 '\001', unireg_check_arg=Field::NONE, field_name_arg=0x7fff5c006780 "f1(1,'aa')", share=0x7fff5c0062a0, blob_pack_length=4, cs=0x0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/field.cc:7415 #3 0x0000555555c85725 in make_field (share=0x7fff5c0062a0, ptr=0x0, field_length=2779096485, null_pos=0x55555644299c "", null_bit=1 '\001', pack_flag=2779096485, field_type=MYSQL_TYPE_NULL, field_charset=0x0, geom_type=Field::GEOM_GEOMETRY, srid=0, unireg_check=Field::NONE, interval=0x0, field_name=0x7fff5c006780 "f1(1,'aa')") at /home/psergey/dev-git/10.1-gsoc-repo/sql/field.cc:9861 #4 0x0000555555db4d35 in sp_head::create_result_field (this=0x7fff5c01c398, field_max_length=0, field_name=0x7fff5c006780 "f1(1,'aa')", table=0x7fff5c005710) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sp_head.cc:885 #5 0x0000555555d0d2b3 in Item_func_sp::init_result_field (this=0x7fff5c0055e0, thd=0x55555acf2800) at /home/psergey/dev-git/10.1-gsoc-repo/sql/item_func.cc:6593 #6 0x0000555555d0dc99 in Item_func_sp::fix_fields (this=0x7fff5c0055e0, thd=0x55555acf2800, ref=0x7fff5c006778) at /home/psergey/dev-git/10.1-gsoc-repo/sql/item_func.cc:6844 #7 0x00005555559fd1ed in setup_fields (thd=0x55555acf2800, ref_pointer_array=0x7fff5c007c90, fields=..., mark_used_columns=MARK_COLUMNS_READ, sum_func_list=0x7fff5c007a80, allow_sum_func=true) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_base.cc:7925 #8 0x0000555555aa6a83 in JOIN::prepare (this=0x7fff5c007750, rref_pointer_array=0x55555acf6b28, tables_init=0x0, wild_num=0, conds_init=0x0, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x55555acf68b0, unit_arg=0x55555acf61c0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_select.cc:782 #9 0x0000555555aafa67 in mysql_select (thd=0x55555acf2800, rref_pointer_array=0x55555acf6b28, tables=0x0, wild_num=0, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7fff5c007730, unit=0x55555acf61c0, select_lex=0x55555acf68b0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_select.cc:3301 #10 0x0000555555aa5d93 in handle_select (thd=0x55555acf2800, lex=0x55555acf60f8, result=0x7fff5c007730, setup_tables_done_option=0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_select.cc:371 #11 0x0000555555a65e98 in execute_sqlcom_select (thd=0x55555acf2800, all_tables=0x0) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:5798 #12 0x0000555555a5c151 in mysql_execute_command (thd=0x55555acf2800) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:2939 #13 0x0000555555a6942e in mysql_parse (thd=0x55555acf2800, rawbuf=0x7fff5c0051b8 "select f1(1,'aa')", length=17, parser_state=0x7ffff7eb7100) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:7174 #14 0x0000555555a5838b in dispatch_command (command=COM_QUERY, thd=0x55555acf2800, packet=0x55555acf9b41 "select f1(1,'aa')", packet_length=17) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:1470 #15 0x0000555555a570a5 in do_command (thd=0x55555acf2800) at /home/psergey/dev-git/10.1-gsoc-repo/sql/sql_parse.cc:1093 (gdb) list 1840 const char *field_name_arg, CHARSET_INFO *charset_arg) 1841 :Field(ptr_arg, len_arg, null_ptr_arg, null_bit_arg, 1842 unireg_check_arg, field_name_arg) 1843 { 1844 field_charset= charset_arg; 1845 if (charset_arg->state & MY_CS_BINSORT) 1846 flags|=BINARY_FLAG; 1847 field_derivation= DERIVATION_IMPLICIT; 1848 } 1849 (gdb) p charset_arg $10 = (CHARSET_INFO *) 0x0 Another piece of feedback: Please don't change old tests into new tests. Tests that were made for previous MDEVs should stay. New tasks should add new tests. I'll continue to look at the patch. On Sun, Aug 02, 2015 at 06:51:26PM +0530, Diwas Joshi wrote: > I am attaching the patch for MDEV-8389. As discussed earlier, I am creating > tables in the query corresponding to table functions before they are being > accessed. I am checking for each table if its a table-function using > sp_find_routine. If it is then we make a table for it. Going by the same > mechanism used in sp_find_routine to parse the function, i parse a create > table query which initializes create_info and alter_info which i later use > in mysql_create_table() to create temporary tables. I also added a test > case for it. Seems to be working fine. > > I guess we'll also need a function that will delete the tables create after > use. I am working on it right now, but i guess that won't be needed in this > patch. > Please review the patch. > > regards > Diwas Joshi > diff --git a/mysql-test/r/sp-error.result b/mysql-test/r/sp-error.result > index cacfeea..4373925 100644 > --- a/mysql-test/r/sp-error.result > +++ b/mysql-test/r/sp-error.result > @@ -2866,7 +2866,3 @@ SELECT @msg; > DROP FUNCTION f1; > DROP FUNCTION f2; > DROP TABLE t1; > -SELECT * FROM f1('xyz'); > -ERROR 42000: This version of MariaDB doesn't yet support 'Queries using Table Functions' > -SELECT * FROM t1, f1('xyz',t1.a); > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1 > diff --git a/mysql-test/r/sp.result b/mysql-test/r/sp.result > index f06939b..d0f6578 100644 > --- a/mysql-test/r/sp.result > +++ b/mysql-test/r/sp.result > @@ -7943,29 +7943,7 @@ RETURNS TABLE t1(id INT, name VARCHAR(11)) > BEGIN > INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > END| > -SELECT * FROM mysql.proc WHERE db = database() and name = 'f1'; > -db test > -name f1 > -type FUNCTION > -specific_name f1 > -language SQL > -sql_data_access CONTAINS_SQL > -is_deterministic NO > -security_type DEFINER > -param_list a INT, b VARCHAR(11) > -returns TABLE t1(id int(11), name varchar(11)) > -body BEGIN > -INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > -END > -definer root@localhost > -created 2015-07-03 16:36:36 > -modified 2015-07-03 16:36:36 > -sql_mode > -comment > -character_set_client latin1 > -collation_connection latin1_swedish_ci > -db_collation latin1_swedish_ci > -body_utf8 BEGIN > -INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > -END > +SELECT * FROM f1(10,'XYZ'); > +id name > +DROP TABLE f1; > DROP FUNCTION f1; > diff --git a/mysql-test/t/sp-error.test b/mysql-test/t/sp-error.test > index ccf5b98..c1a21c8 100644 > --- a/mysql-test/t/sp-error.test > +++ b/mysql-test/t/sp-error.test > @@ -3845,8 +3845,3 @@ SELECT @msg; > DROP FUNCTION f1; > DROP FUNCTION f2; > DROP TABLE t1; > - > ---error 1235 > -SELECT * FROM f1('xyz'); > ---error 1064 > -SELECT * FROM t1, f1('xyz',t1.a); > diff --git a/mysql-test/t/sp.test b/mysql-test/t/sp.test > index 5c42df0..906c91c 100644 > --- a/mysql-test/t/sp.test > +++ b/mysql-test/t/sp.test > @@ -9397,7 +9397,6 @@ END| > > DELIMITER ;| > > ---vertical_results > -SELECT * FROM mysql.proc WHERE db = database() and name = 'f1'; > - > +SELECT * FROM f1(10,'XYZ'); > +DROP TABLE f1; > DROP FUNCTION f1; > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 2998972..dd16c36 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2467,6 +2467,8 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > /* should be assigned after making first tables same */ > all_tables= lex->query_tables; > /* set context for commands which do not use setup_tables */ > + if (lex->sql_command == SQLCOM_SELECT) > + check_and_process_table_functions(thd, all_tables); > select_lex-> > context.resolve_in_table_list_only(select_lex-> > table_list.first); > @@ -9222,3 +9224,142 @@ CHARSET_INFO *find_bin_collation(CHARSET_INFO *cs) > } > return cs; > } > + > +/* > + @brief > + Check for table functions and create corresponding tables to be used in query. > + > + @param thd Thread handle > + @param table_list A list of tables being processed in the query. > + > + @return > + False on success > + @retval > + TRUE on error > +*/ > +bool check_and_process_table_functions(THD *thd, TABLE_LIST *table_list) > +{ > + TABLE_LIST *tables = table_list; > + int err= 0; > + while(tables) > + { > + if (tables->table_name == NULL) > + { > + tables = tables->next_local; > + continue; > + } > + sp_head *sp; > + sp_name *name; > + LEX_STRING lex_db; > + LEX_STRING lex_name; > + lex_db.length= strlen(tables->db); > + lex_name.length= strlen(tables->table_name); > + lex_db.str= thd->strmake(tables->db, lex_db.length); > + lex_name.str= thd->strmake(tables->table_name, lex_name.length); > + name= new sp_name(lex_db, lex_name, true); > + name->init_qname(thd); > + Object_creation_ctx *creation_ctx; > + int err=0; > + > + if (!err && (sp = sp_find_routine(thd,TYPE_ENUM_FUNCTION, name, > + &thd->sp_proc_cache, FALSE)) && > + sp->m_type== TYPE_ENUM_TABLE) //check if the table is a table-valued function. > + { > + LEX *old_lex= thd->lex, newlex; > + thd->lex= &newlex; > + ulonglong sql_mode= 0 , saved_mode= thd->variables.sql_mode; > + String defstr; //query in defstr will be parsed to create temporary table. > + defstr.append(STRING_WITH_LEN("CREATE TEMPORARY TABLE ")); > + defstr.append(tables->table_name); > + defstr.append(STRING_WITH_LEN(" (")); > + List_iterator_fast<Create_field> it(sp->m_cols_list); > + > + TABLE table; > + TABLE_SHARE share; > + bzero((char*) &table, sizeof(table)); > + bzero((char*) &share, sizeof(share)); > + table.in_use= thd; > + table.s = &share; > + Create_field *cols_field; > + cols_field= it++; > + while (cols_field) > + { > + uint unused1= 0; > + sp_prepare_create_field(thd, cols_field); > + prepare_create_field(cols_field, &unused1, HA_CAN_GEOMETRY); > + uint field_length; > + Field *field; > + String tmp_result(512); > + field_length= cols_field->length; > + cols_field->flags= 0; > + field= make_field(table.s, /* TABLE_SHARE ptr */ > + (uchar*) 0, /* field ptr */ > + field_length, /* field [max] length */ > + (uchar*) "", /* null ptr */ > + 0, /* null bit */ > + cols_field->pack_flag, > + cols_field->sql_type, > + cols_field->charset, > + cols_field->geom_type, cols_field->srid, > + Field::NONE, /* unreg check */ > + cols_field->interval, > + cols_field->field_name); > + field->vcol_info= cols_field->vcol_info; > + field->stored_in_db= cols_field->stored_in_db; > + if (field) > + field->init(&table); > + defstr.append(cols_field->field_name); > + defstr.append(STRING_WITH_LEN(" ")); > + field->sql_type(tmp_result); > + defstr.append(tmp_result); > + cols_field= it++; > + if(cols_field) > + defstr.append(STRING_WITH_LEN(", ")); > + delete field; > + } > + defstr.append(STRING_WITH_LEN(")")); > + char saved_cur_db_name_buf[SAFE_NAME_LEN+1]; > + LEX_STRING saved_cur_db_name= { saved_cur_db_name_buf, > + sizeof(saved_cur_db_name_buf) }; > + newlex.create_info.DDL_options_st::init(); > + Stored_program_creation_ctx *creation_ctx; > + char definer_user_name_holder[USERNAME_LENGTH + 1]; > + LEX_STRING definer_user_name= { definer_user_name_holder, USERNAME_LENGTH }; > + char definer_host_name_holder[HOSTNAME_LENGTH + 1]; > + LEX_STRING definer_host_name= { definer_host_name_holder, HOSTNAME_LENGTH }; > + newlex.current_select= NULL; > + Parser_state parser_state; > + thd->variables.sql_mode= sql_mode; > + err= parser_state.init(thd, defstr.c_ptr_safe(), defstr.length()); > + if (!err) { > + lex_start(thd); > + err= parse_sql(thd, &parser_state, creation_ctx) || thd->lex == NULL; > + } > + if (!err) > + { > + Table_specification_st create_info(thd->lex->create_info); > + Alter_info alter_info(thd->lex->alter_info, thd->mem_root); > + thd->lex->create_info.default_table_charset= NULL; > + SELECT_LEX *select_lex= &thd->lex->select_lex; > + TABLE_LIST *create_table= select_lex->table_list.first; > + if (append_file_to_dir(thd, &create_info.data_file_name, > + create_table->table_name) || > + append_file_to_dir(thd, &create_info.index_file_name, > + create_table->table_name)) > + { > + err= 1; > + } > + if (!(create_info.used_fields & HA_CREATE_USED_ENGINE)) > + create_info.use_default_db_type(thd); > + > + err= mysql_create_table(thd, create_table, &create_info, &alter_info); > + thd->variables.option_bits|= OPTION_KEEP_LOG; > + } > + thd->variables.sql_mode= saved_mode; > + lex_end(thd->lex); > + thd->lex= old_lex; > + } > + tables = tables->next_local; > + } > + return err; > +} > \ No newline at end of file > diff --git a/sql/sql_parse.h b/sql/sql_parse.h > index 76e1b7a..6b89b35 100644 > --- a/sql/sql_parse.h > +++ b/sql/sql_parse.h > @@ -121,7 +121,7 @@ bool push_new_name_resolution_context(THD *thd, > Item *normalize_cond(Item *cond); > Item *negate_expression(THD *thd, Item *expr); > bool check_stack_overrun(THD *thd, long margin, uchar *dummy); > - > +bool check_and_process_table_functions(THD *thd, TABLE_LIST *table_list); > /* Variables */ > > extern const char* any_db; > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index d7d7d7b..dd1f9f6 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -10802,7 +10802,6 @@ table_factor: > Yacc_state *state= & thd->m_parser_state->m_yacc; > if (state->table_factor_part_type== 2) > { > - mysql_init_select(lex); > List_iterator_fast<Item> it(lex->value_list); > Item *item; > while ((item = it++)) > @@ -10817,9 +10816,6 @@ table_factor: > MYSQL_YYABORT; > sel->add_joined_table($$); > lex->pop_context(); > - > - my_error(ER_NOT_SUPPORTED_YET, MYF(0), > - "Queries using Table Functions"); //to be removed later > } > else > { -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] 2430d1d: MDEV-7821 - Server crashes in Item_func_group_concat::fix_fields on 2nd
by Oleksandr Byelkin 31 Jul '15

31 Jul '15

Hi! On 30.07.15 16:59, Sergey Vojtovich wrote: > revision-id: 2430d1d7ccfcf429e45dbfe655a24d57a79ebf6c (mariadb-5.5.44-14-g2430d1d) > parent(s): 392df76bc3a40a5dd1956b12628dd6489a37be36 > committer: Sergey Vojtovich > timestamp: 2015-07-30 18:51:44 +0400 > message: > > MDEV-7821 - Server crashes in Item_func_group_concat::fix_fields on 2nd > execution of PS > > GROUP_CONCAT() with ORDER BY column position may crash server on PS reexecution. > > The problem was that arguments array of GROUP_CONCAT() was adjusted to point to > temporary elements (resolved ORDER BY fields) during first execution. > > This patch expands rev. 08763096cb to restore original arguments array as well. OK to push! > > --- > mysql-test/r/func_gconcat.result | 16 ++++++++++++++++ > mysql-test/t/func_gconcat.test | 11 +++++++++++ > sql/item_sum.cc | 3 +++ > 3 files changed, 30 insertions(+) > > diff --git a/mysql-test/r/func_gconcat.result b/mysql-test/r/func_gconcat.result > index f12a0c1..0bc31a5 100644 > --- a/mysql-test/r/func_gconcat.result > +++ b/mysql-test/r/func_gconcat.result > @@ -1103,3 +1103,19 @@ ORDER BY field; > field > c,c > drop table t3, t2, t1; > +# > +# MDEV-7821 - Server crashes in Item_func_group_concat::fix_fields on 2nd > +# execution of PS > +# > +CREATE TABLE t1(a INT); > +INSERT INTO t1 VALUES(1),(2); > +PREPARE stmt FROM "SELECT GROUP_CONCAT(t1a.a ORDER BY 1, t1a.a=0) FROM t1 AS t1a, t1 AS t1b GROUP BY t1a.a"; > +EXECUTE stmt; > +GROUP_CONCAT(t1a.a ORDER BY 1, t1a.a=0) > +1,1 > +2,2 > +EXECUTE stmt; > +GROUP_CONCAT(t1a.a ORDER BY 1, t1a.a=0) > +1,1 > +2,2 > +DROP TABLE t1; > diff --git a/mysql-test/t/func_gconcat.test b/mysql-test/t/func_gconcat.test > index 42a3076..5550eeb 100644 > --- a/mysql-test/t/func_gconcat.test > +++ b/mysql-test/t/func_gconcat.test > @@ -821,3 +821,14 @@ FROM ( SELECT * FROM t2 ) AS sq2, t3 > ORDER BY field; > > drop table t3, t2, t1; > + > +--echo # > +--echo # MDEV-7821 - Server crashes in Item_func_group_concat::fix_fields on 2nd > +--echo # execution of PS > +--echo # > +CREATE TABLE t1(a INT); > +INSERT INTO t1 VALUES(1),(2); > +PREPARE stmt FROM "SELECT GROUP_CONCAT(t1a.a ORDER BY 1, t1a.a=0) FROM t1 AS t1a, t1 AS t1b GROUP BY t1a.a"; > +EXECUTE stmt; > +EXECUTE stmt; > +DROP TABLE t1; > diff --git a/sql/item_sum.cc b/sql/item_sum.cc > index d8970ca..a24307b 100644 > --- a/sql/item_sum.cc > +++ b/sql/item_sum.cc > @@ -3300,6 +3300,8 @@ void Item_func_group_concat::cleanup() > from Item_func_group_concat::setup() to point to runtime > created objects, we need to reset them back to the original > arguments of the function. > + > + The very same applies to args array. > */ > ORDER **order_ptr= order; > for (uint i= 0; i < arg_count_order; i++) > @@ -3307,6 +3309,7 @@ void Item_func_group_concat::cleanup() > (*order_ptr)->item= &args[arg_count_field + i]; > order_ptr++; > } > + memcpy(args, orig_args, sizeof(Item *) * arg_count); > DBUG_VOID_RETURN; > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] [MDEV-8554] Weird behavior of multi-table UPDATE and THD::save_prep_leaf_list
by Sergey Petrunia 30 Jul '15

30 Jul '15

Hello Igor, I was debugging https://mariadb.atlassian.net/browse/MDEV-8554 The patch is about save_prep_leaf_list is not being cleaned up by a statement. I committed a patch that makes that bug go away. However, the way that statements work with THD::save_prep_leaf_list look weird to me. Since it was you who added save_prep_leaf_list, I wanted to ask you: Consider an example: CREATE TABLE t1 (a INT) ENGINE=MyISAM; INSERT INTO t1 VALUES (1),(2); # Not necessary, the table can be empty CREATE TABLE t2 (b INT) ENGINE=MyISAM; INSERT INTO t2 VALUES (3),(4); # Not necessary, the table can be empty CREATE TABLE t3 (c INT) ENGINE=MyISAM; INSERT INTO t3 VALUES (5),(6); # Not necessary, the table can be empty CREATE OR REPLACE ALGORITHM=MERGE VIEW v3 AS SELECT * FROM t3; Now, let's debug how THD::save_prep_leaf_list changes and is used: UPDATE t1, t2 SET a = 1 WHERE a IN ( SELECT 0 FROM v3 ); We see: 1. Breakpoint 4, THD::THD (this=0x59ff6f0) at /home/psergey/dev-git/10.0/sql/sql_class.cc:1062 save_prep_leaf_list= FALSE; 2. Breakpoint 1, THD::reset_for_next_command (this=0x59ff6f0) at /home/psergey/dev-git/10.0/sql/sql_parse.cc:6272 thd->save_prep_leaf_list= false; ^^^ I have added this line in my fix 3. Breakpoint 5, LEX::save_prep_leaf_tables (this=0x5a02f80) at /home/psergey/dev-git/10.0/sql/sql_lex.cc:4132 bool LEX::save_prep_leaf_tables() { if (!thd->save_prep_leaf_list) return FALSE; ^^^ This is where save_prep_leaf_list is used. 4. Breakpoint 2, mysql_derived_merge (thd=0x59ff6f0, lex=0x5a02f80, derived=0x7fff68008618) at /home/psergey/dev-git/10.0/sql/sql_derived.cc:379 if (thd->lex->sql_command == SQLCOM_UPDATE_MULTI || thd->lex->sql_command == SQLCOM_DELETE_MULTI) thd->save_prep_leaf_list= TRUE; ^^^ this is where save_prep_leaf_list is changed by this statement. 5. (no break). The statement finished. That is, the statement first checks save_prep_leaf_list, and then uses it. This looks weird. Any thoughts? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] 9f8a458: MDEV-7846: Server crashes in Item_subselect::fix_fields or fails with Thread stack overrun
by Sergei Golubchik 30 Jul '15

30 Jul '15

Hi, Sanja! On Apr 23, sanja(a)mariadb.com wrote: > revision-id: 9f8a458fb2d07298810bb5d9824ce728017bfb32 > parent(s): e540d023e2ec6f37efc9ab695ccdfd4a6744ad64 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-04-23 20:08:57 +0200 > message: > > MDEV-7846: Server crashes in Item_subselect::fix_fields or fails with > Thread stack overrun > > Substitute into transformed subselects original left expression and > than register its change in case it was substituted. Looks mostly ok to me. But please: 1. Add comments (the patch is attached) 2. Add a test case with row items 3. In your test case - if you remove GROUP BY you get a crash again. This might be a different bug, though. Regards, Sergei

1 0

[Maria-developers] mysql_stmt_prepare_start returns with error code 2006
by András Szabó 29 Jul '15

29 Jul '15

Hi Folks, I am currently working on a c++11 asio based async connector for mariadb. The library takes advantage of mariadb’s async interface (mysql_*_start, mysql_*_cont functions). The whole point would be to prevent blocking in mysql operations and make operations cancellable via a reasonable interface. Some example code: TEST_CASE("tmp3") { init_db(); asio::io_service io; amdb::connector conn(io); bool saved = false; entity e(42, 3.14, "our entity"); amdb::auth_info auth("root"); conn.async_connect(amdb::null_endpoint, "edbtest", auth, amdb::default_flags, [&] (const std::error_code& ec) { REQUIRE(!ec); conn.async_write_entity(e, [&] (const std::error_code &ec) { INFO(ec); REQUIRE(!ec); if (!ec) { saved = true; } io.stop(); }); }); io.run(); REQUIRE(saved); } async_write_entity() is an extra feature of the lib, which is generating an sql “batch” of prepared statements to persist hierarchical objects and executes that batch. Unfortunately I am receiving an error 2006 from my very first mysql_stmt_prepare call. This error roughly translates to "connection to mysql server is gone”. I cannot really spot the issue with my operation, could someone give me a hint about this error (typical causes, what to check, etc etc.)? I am a bit clueless here how to tackle this problem. Since the batch generation/execution code is a bit convoluted and complex, i would not paste it here now, but if someone can point me in the right direction i can share relevant parts. Thanks in advance Andras -- András Szabó Sent with Airmail

2 2

Re: [Maria-developers] 4728581: MDEV-7652 - More explanatory ERROR and WARNING messages when loading plugins
by Sergei Golubchik 29 Jul '15

29 Jul '15

Hi, Sergey! Looks ok, just a couple of comments about the messages. If you're agree - please, fix and push. On Jul 28, Sergey Vojtovich wrote: > revision-id: 4728581813000522d05ab589df8e5f8cb6ad1ee6 > parent(s): cf30074c3ff6815d85bbd864b28adfe7949d340c > committer: Sergey Vojtovich > branch nick: mariadb Please, update the git post-commit hook (bzr-pull from mariadb-tools). I've just changed it to report a branch better than just "mariadb". > timestamp: 2015-07-28 10:18:55 +0400 > message: > > MDEV-7652 - More explanatory ERROR and WARNING messages when loading plugins > with plugin-load-add that are already registered at mysql.plugin > > - issue just one error message, without this extra warning > - don't abuse ER_UDF_EXISTS, instead add a proper error message for plugins > - report started initialization for each plugin source > > --- > sql/share/errmsg-utf8.txt | 3 +++ > sql/sql_plugin.cc | 15 +++++++++++---- > 2 files changed, 14 insertions(+), 4 deletions(-) > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 6954170..374e31f 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7111,3 +7111,6 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > ER_TABLE_DEFINITION_TOO_BIG > eng "The definition for table %`s is too big" > +ER_PLUGIN_EXISTS > + eng "Plugin '%-.192s' already exists" > + rus "Плагин '%-.192s' уже существует" May be "already installed" ? The command is INSTALL PLUGIN. For UDF it was CREATE FUNCTION, that's why "already exists", I suppose. > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index dc40870..21efeb0 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -1546,6 +1546,9 @@ int plugin_init(int *argc, char **argv, int flags) > /* > First we register builtin plugins > */ > + if (global_system_variables.log_warnings >= 9) > + sql_print_information("Initializing mandatory plugins"); > + > for (builtins= mysql_mandatory_plugins; *builtins || mandatory; builtins++) > { > if (!*builtins) Here you should've put "Initializing optional plugins". Actually, a better message would be "Initializing other built-in plugins" or " Initializing remaining built-in plugins". > @@ -1627,11 +1630,17 @@ int plugin_init(int *argc, char **argv, int flags) > { > I_List_iterator<i_string> iter(opt_plugin_load_list); > i_string *item; > + if (global_system_variables.log_warnings >= 9) > + sql_print_information("Initializinig plugins specified by command line"); "Initializinig plugins specified on the command line" > while (NULL != (item= iter++)) > plugin_load_list(&tmp_root, item->ptr); > > if (!(flags & PLUGIN_INIT_SKIP_PLUGIN_TABLE)) > + { > + if (global_system_variables.log_warnings >= 9) > + sql_print_information("Initializing installed plugins"); > plugin_load(&tmp_root); > + } > } Regards, Sergei

2 2

[Maria-developers] Using oqgraph/connect engines in mariadb-galera-server
by Nirbhay Choubey 28 Jul '15

28 Jul '15

Hi Serg, Svoj! I am looking into an issue where accessing oqgraph/connect engines from galera server leads to errors like "Unknown option '<table option>'" and segfaults. The issue being the differences in handlerton structure in galera and non-galera server codes. Since the connect/oqgraph packages are built from non-galera server, when they are loaded into galera server, the engine's handlerton gets improperly initialized. https://mariadb.atlassian.net/browse/MDEV-8240 I can think of the following potential solutions : a) Club connect/oqgraph and mariadb-galera-server into a single package. (this would mean adding new dependencies to mariadb-galera-server) b) New packages for connect/oqgraph out of mariadb-galera build (mariadb-connect-engine-galera-10.0, mariadb-oqgraph-engine-galera-10.0) Do you see any other option? Also, we should add some checks for plugin load to fail in first place in such scenarios. Best, Nirbhay

2 1

[Maria-developers] GTID: On the difference between master_use_gtid = slave_pos or current_pos
by Kristian Nielsen 27 Jul '15

27 Jul '15

I was asked about the difference between master_use_gtid = slave_pos or current_pos. This comes up often, and I thought the below text, which I used to answer the question earlier, might be helpful. Ian, do you think you could incorporate this in some form into the GTID documentation in the knowledgebase? Hope this helps, - Kristian. ----------------------------------------------------------------------- First, consider a slave S2 replicating from a master S1. We stop S2, and then later restart it. Where should it resume replication? Clearly at the point of the last transaction replicated before stopping, which is @@gtid_slave_pos. Now suppose we take down S1 for maintenance and promote S2 as the new master. So we are doing direct transactions on S2. Later we take S1 back up, and now we connect S1 as a slave to S2. Where should S1 start replicating? Clearly at the point of the last transaction in S1's binlog, which is the last transaction made while S1 was the master. This is @@gtid_binlog_pos. Now suppose the user does STOP SLAVE on S1, does a direct transaction on the slave S1, then START SLAVE. That direct transaction was only intended locally, and it is not on the master S2. So when S1 reconnects, it needs to connect at its @@gtid_slave_pos this time, not at the @@gtid_binlog_pos. So we see, only the user knows if a direct transaction made on a server is meant as a cluster-wide transaction, or a local transaction only. And the "correct" behaviour is _different_ depending on this. Normally, direct local transactions in the slave binlog are considered sloppy, because of all the possibilities for errors they introduce. So it is recommended to run slaves in read-only mode, and set @@sql_log_bin=0 if doing local transactions. And gtid_strict_mode=1 is provided to enforce this recommended mode of operation. In this case, user can use current_pos, which will automatically use either @@gtid_slave_pos or @@gtid_binlog_pos, whatever is appropriate. However, some (or many) users like to "be sloppy" and do direct transactions on a slave not intended as master transactions. And for those we provide gtid_strict_mode=0 and master_use_gtid=slave_pos. This makes slave connect always use the position of the last replicated transaction, ignoring any local transactions in the binlog. In this case, when a master is changed to be a slave, it is necessary to manually set @@gtid_slave_pos=@@gtid_binlog_pos. Explicitly marking the updates on the old master to be intended as master transactions, not local transactions. -----------------------------------------------------------------------

2 1

[Maria-developers] PLEASE REVIEW: (MDEV-7574) Security definer views don't work with CONNECT ODBC tables
by Alexander Barkov 26 Jul '15

26 Jul '15

Hello Sergei, Please review a patch that fixes the problem described in MDEV-7574. The idea is that a SELECT from a view over a CONNECT table now checks FILE privileges of the view definer (unless CREATE VIEW states SQL SECURITY INVOKER). It looks like a very good idea and gives more possible security options. The administrator can create a VIEW and give access to it to some user, without giving FILE privilege to this user. I'm not really sure about the patch, but it seems to do the trick :) From what I understood, FILE_ACL is written (among the other privileges) into thd->security_ctx.privilege in TABLE_LIST::prepare_security(). In case of a DEFINER view, thd->security_ctx.privilege is filled exactly with the definer privileges, and to the invoker privileges otherwise. So inside ha_connect::check_privileges() the fact that there is FILE_ACL in thd->security_ctx.privilege means that TABLE_LIST::prepare_security() was previously called and FILE_ACL is set to DEFINER or INVOKER, according to the view definition. This is exactly what we need. I'm not sure about the opposite: if there is no FILE_ACL in thd->security_ctx.privilege, what does it mean? Does it mean that there is no FILE_ACL for the effective user? Or can it also mean that TABLE_LIST::prepare_security() was not called? Thanks. On 02/12/2015 04:04 AM, Olivier Bertrand wrote: > Hi Alexander, > > Can you take care of this issue? I know you wrote the > ha_connect::check_privileges function and I personnally have not the > faintest idea about how this must be checked. > > Thanks, > Olivier > > > -------- Message transféré -------- > Sujet : [JIRA] (MDEV-7574) Security definer views don't work with > CONNECT ODBC tables > Date : Wed, 11 Feb 2015 23:21:00 +0200 (EET) > De : Elena Stepanova (JIRA) <jira(a)mariadb.atlassian.net> > Pour : bertrandop(a)gmail.com > > > > [https://mariadb.atlassian.net/browse/MDEV-7574?page=com.atlassian.jira.plug… ] > > Elena Stepanova reassigned MDEV-7574: > ------------------------------------- > > Assignee: Olivier Bertrand > Fix Version/s: 10.0 > >> Security definer views don't work with CONNECT ODBC tables >> ---------------------------------------------------------- >> >> Key: MDEV-7574 >> URL:https://mariadb.atlassian.net/browse/MDEV-7574 >> Project: MariaDB Server >> Issue Type: Bug >> Components: Storage Engine - Connect >> Affects Versions: 10.0.16 >> Reporter: Geoff Montee >> Assignee: Olivier Bertrand >> Labels: connect-engine >> Fix For: 10.0 >> >> >> One possible way to get around the requirement for having the FILE privilege to access ODBC tables with CONNECT would be to have them called indirectly via a security definer view. However, it does not currently work. >> Create a security definer view to access the ODBC table, then create a new user: >> {code} >> [gmontee@localhost ~]$ mysql -u root tmp >> Reading table information for completion of table and column names >> You can turn off this feature to get a quicker startup with -A >> Welcome to the MariaDB monitor. Commands end with ; or \g. >> Your MariaDB connection id is 16 >> Server version: 10.0.15-MariaDB-log MariaDB Server >> Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others. >> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. >> MariaDB [tmp]> SHOW CREATE TABLE datetime_table; >> +----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >> | Table | Create Table | >> +----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >> | datetime_table | CREATE TABLE `datetime_table` ( >> `id` int(10) NOT NULL, >> `modifiedon` datetime DEFAULT NULL >> ) ENGINE=CONNECT DEFAULT CHARSET=latin1 CONNECTION='DSN=connect_test_azure;UID=connect_test;PWD=Password1' `TABLE_TYPE`='ODBC' `TABNAME`='dbo.datetime_table' | >> +----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >> 1 row in set (0.00 sec) >> MariaDB [tmp]> DROP USER 'connecttest'@'localhost'; >> Query OK, 0 rows affected (0.00 sec) >> MariaDB [tmp]> CREATE OR REPLACE >> -> DEFINER = CURRENT_USER >> -> SQL SECURITY DEFINER >> -> VIEW datetime_view >> -> AS SELECT * FROM datetime_table; >> Query OK, 0 rows affected (0.00 sec) >> MariaDB [tmp]> CREATE USER 'connecttest'@'localhost'; >> Query OK, 0 rows affected (0.00 sec) >> MariaDB [tmp]> GRANT SELECT ON datetime_view TO 'connecttest'@'localhost'; >> Query OK, 0 rows affected (0.00 sec) >> MariaDB [tmp]> \q >> Bye >> {code} >> Now connect with the new user, and try to use the view: >> {code} >> [gmontee@localhost ~]$ mysql -u connecttest tmp >> Reading table information for completion of table and column names >> You can turn off this feature to get a quicker startup with -A >> Welcome to the MariaDB monitor. Commands end with ; or \g. >> Your MariaDB connection id is 17 >> Server version: 10.0.15-MariaDB-log MariaDB Server >> Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others. >> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. >> MariaDB [tmp]> SELECT * FROM datetime_view; >> ERROR 1045 (28000): Access denied for user 'connecttest'@'localhost' (using password: NO) >> MariaDB [tmp]> \q >> Bye >> {code} >> It didn't work, so give the user privileges on the underlying ODBC table: >> {code} >> [gmontee@localhost ~]$ mysql -u root tmp >> Reading table information for completion of table and column names >> You can turn off this feature to get a quicker startup with -A >> Welcome to the MariaDB monitor. Commands end with ; or \g. >> Your MariaDB connection id is 18 >> Server version: 10.0.15-MariaDB-log MariaDB Server >> Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others. >> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. >> MariaDB [tmp]> GRANT FILE ON *.* TO 'connecttest'@'localhost'; >> Query OK, 0 rows affected (0.00 sec) >> MariaDB [tmp]> GRANT SELECT ON datetime_table TO 'connecttest'@'localhost'; >> Query OK, 0 rows affected (0.00 sec) >> MariaDB [tmp]> \q >> Bye >> {code} >> Now try using the view again: >> {code} >> [gmontee@localhost ~]$ mysql -u connecttest tmp >> Reading table information for completion of table and column names >> You can turn off this feature to get a quicker startup with -A >> Welcome to the MariaDB monitor. Commands end with ; or \g. >> Your MariaDB connection id is 19 >> Server version: 10.0.15-MariaDB-log MariaDB Server >> Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others. >> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. >> MariaDB [tmp]> SELECT * FROM datetime_view; >> +----+---------------------+ >> | id | modifiedon | >> +----+---------------------+ >> | 1 | 2014-01-01 00:00:00 | >> | 2 | 2016-01-01 00:00:00 | >> +----+---------------------+ >> 2 rows in set (0.24 sec) >> {code} > > > > -- > This message was sent by Atlassian JIRA > (v6.4-OD-14-082#64012) > > >

2 9

Re: [Maria-developers] [Commits] 0b82e6d: MDEV-8399 - [PATCH] Missing Sanity Checks for memory allocation in MariaDB
by Sergei Golubchik 23 Jul '15

23 Jul '15

Hi, Sergey! On Jul 16, Sergey Vojtovich wrote: > revision-id: 0b82e6d85a18fb5fa9cc54ad7a854cb2eecfce03 > parent(s): 94bc50650eb490e8d844acfc700f75992857509a > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-07-16 16:20:17 +0400 > message: > > MDEV-8399 - [PATCH] Missing Sanity Checks for memory allocation in MariaDB > > - since param is quite small store it on stack > - fixed a few memory leaks I'm not going to review that, I don't care about missing sanity checks or memory leaks in mysys test programs :) Do whatever you want with it (it = the code inside #ifdef MAIN ... #endif) Regards, Sergei

1 0

Re: [Maria-developers] [Commits] a6ab8ef: MDEV-5997 - MySQL bug#11759114 - '51401: GRANT TREATS NONEXISTENT
by Sergei Golubchik 23 Jul '15

23 Jul '15

Hi, Sergey! On Jul 23, Sergey Vojtovich wrote: > revision-id: a6ab8ef9d7d465a3f0bc1cff6034ca97575629dd > parent(s): d897015d74d82464a21ed154f606adc69503c372 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-07-23 16:45:20 +0400 > message: > > MDEV-5997 - MySQL bug#11759114 - '51401: GRANT TREATS NONEXISTENT > FUNCTIONS/PRIVILEGES DIFFERENTLY' ok to push Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 74b4ead: MDEV-5096 - Wrong error message on attempt to kill somebody else's query ID
by Sergei Golubchik 23 Jul '15

23 Jul '15

Hi, Sergey! On Feb 09, svoj(a)mariadb.org wrote: > revision-id: 74b4ead8ef32eb2bb96b17dafac514ffba28963a > parent(s): f13939061d468a47985dee0268652a6fe2db7862 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-02-09 13:41:24 +0400 > message: > > MDEV-5096 - Wrong error message on attempt to kill somebody else's query ID > > Attempting to kill query owned by different user now returns better error: > "You are not owner of query N" instead of "You are not owner of thread N". Ok to push! Nice test. Regards, Sergei

1 0

[Maria-developers] MariaDB parallel replication
by Kristian Nielsen 18 Jul '15

18 Jul '15

Some interesting work is starting to be done by users to test parallel replication on real workloads. One issue that comes up is the performance on workloads that have a relatively high number of lock conflicts between transactions. I made some patches (against latest 10.1) to help investigate and improve the issues, and I wanted to mention them here in case someone wants to comment on or experiment with them: 1. Adding status variables for parallel replication. http://lists.askmonty.org/pipermail/commits/2015-July/008131.html This patch adds new status variables that measure the time spent by parallel replication worker threads being - Idle (waiting for work from the SQL thread). - Processing relay log events. - Waiting for a prior group commit to finish (measuring the overhead of insufficient parallelism recorded on the master, in conservative mode, or the overhead of serialisation around DDL, in optimistic/aggressive modes). - Waiting for the immediately prior transaction to commit (measuring the overhead of in-order parallel replication). - Rolling back and re-executing events due to deadlocks. It would be interesting to see how the different numbers compare on various workloads and parallel replication modes. 2. More aggressive retry of conflicting transactions. http://lists.askmonty.org/pipermail/commits/2015-July/008133.html When we get a deadlock and a transaction retry, the current MariaDB waits for all prior transactions to commit before retrying. The logic is that since we already got a conflict, there is a high risk that an immediate retry will just give another conflict. So if we have T1 T2 T3 T4, and T4 conflicts with T1, we roll back T4, wait for T3 to commit, and only then retry T4. If we have many such conflicts, we could end up wasting a lot of times on such waits. This patch changes aggressive mode so that T4 will only wait for T1 to commit, then it will retry. This allows T4 to run in parallel with T2 and T3. It will be interesting to see if this improves throughput in aggressive mode in some workloads, and also to see if/how much it increases the number of transaction retries and associated overhead. 3. Debug patch to log all row lock waits. http://lists.askmonty.org/pipermail/commits/2015-July/008132.html This is not a patch suitable for production use. It adds an option --gtid-log-all-lock-conflicts. When enabled, whenever one parallel replication transaction needs to wait on the InnoDB row lock of another, it will log a line to the error log, including the GTIDs of the transactions. This could be used to correlate back with the binlog and study exactly which transactions it is that are conflicting with each other. Such results could again be highly interesting. - Kristian.

1 1

[Maria-developers] MDEV-8441 Bad SHOW CREATE TABLE output for a table with a virtual column
by Alexander Barkov 15 Jul '15

15 Jul '15

Hi Sanja, Please review a patch for MDEV-8441. Thanks.

2 1

[Maria-developers] MDEV-4682 plugin - qc_info - add Query Cache flags - not-GSoc2015 patch for review
by Daniel Black 14 Jul '15

14 Jul '15

This expands out all the query cache info from the flags that wasn't previously available in the information_schema.query_cache_info table by the query_cache_info plugin. https://github.com/MariaDB/server/pull/67 -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

1 0

Re: [Maria-developers] Likely a dead optimizer related code
by Michael Widenius 13 Jul '15

13 Jul '15

Hi! > Hi Igor, > > I noticed that during Item_cond::fix_fields() and > Item_func_between::fix_fields() update these optimizer > related SELECT_LEX members: > > thd->lex->current_select->cond_count and > thd->lex->current_select->between_count > > The purpose of these members is to allocate optimizer related buffers > in: > - update_ref_and_keys() in sql_select.cc and > - check_func_dependency() in opt_table_elimination,cc > and it seems they have no any other purposes.. Correct. > From my understanding, this is a dead code and the collected values are > never used: > > - cond_count and_between_count are later always initialized to 0 > before the walk(count_sargable_conds) call in update_ref_and_keys(). And the call to check_func_dependency() is done after this call. > - moreover, using the values collected during fix_fields() would not be > correct to allocate optimized related buffers, because the code > in fix_fields() does not distinguish between different query parts, > so the items can be just a part of the SELECT list rather than a > part of WHERE/HAVING/ON. This would note be a problem as the count was supposed to be an upper bound. As it's not normal that you have many IF or BETWEEN in the SELECT part, this was not a problem. I looked at the patch and it's ok to commit. Regards, Monty

2 1

Re: [Maria-developers] Probably in a bug in simple_pred()
by Michael Widenius 09 Jul '15

09 Jul '15

Hi! > From: Alexander Barkov <bar(a)mariadb.org> > Hi Sergey, > > I probably found a bug in simple_pred() in opt_sum.cc. > > In this piece of code: > >> case 3: >> /* field BETWEEN const AND const */ >> item= func_item->arguments()[0]->real_item(); >> if (item->type() == Item::FIELD_ITEM) >> { >> args[0]= item; >> for (int i= 1 ; i <= 2; i++) >> { >> item= func_item->arguments()[i]->real_item(); >> if (!item->const_item()) >> return 0; >> args[i]= item; >> if (check_item1_shorter_item2(args[0], args[1])) > > > Should't it be > > if (check_item1_shorter_item2(args[0], args[i])) > > ? > >> return 0; >> } >> } > > > If this is a bug, can you please help to make an SQL scrip which would > be affected by this bug? yes, it's a bug. Here is a script that shows that: drop table if exists t1; create table t1 (a varchar(10), key (a)); insert into t1 values("bar"),("Cafe"); explain select min(a) from t1 where a between "a" and "Cafe2"; explain select min(a) from t1 where a between "a" and "Cafeeeeeeeeeeeeeeeeeeeeeeeeee"; explain select min(a) from t1 where a between "abbbbbbbbbbbbbbbbbbbb" and "Cafe2"; The output is: 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | Select tables optimized away | For the two first cases and: | 1 | SIMPLE | t1 | index | a | a | 13 | NULL | 2 | Using where; Using index | for the last The second case should have the same output as the third. Will fix and push into 10.1 Regards, Monty

1 0

[Maria-developers] MDEV-4119 patch comments
by Sergey Petrunia 07 Jul '15

07 Jul '15

Hi Monty, Please find my comments on the patch for MDEV-4119 below. Sorry for the delay. > diff --git a/sql/item.cc b/sql/item.cc > index b4de973..3b1f8b7 100644 > --- a/sql/item.cc > +++ b/sql/item.cc > @@ -1705,69 +1705,98 @@ class Item_aggregate_ref : public Item_ref > > @param thd Thread handler > @param ref_pointer_array Pointer to array of reference fields > - @param fields All fields in select > + @param fields All fields in select > @param ref Pointer to item > - @param skip_registered <=> function be must skipped for registered > - SUM items > + @param split_flags Zero or more of the following flags > + SPLIT_FUNC_SKIP_REGISTERED: > + Function be must skipped for registered SUM > + SUM items > + SPLIT_FUNC_SELECT > + We are called on the select level and have to > + register items operated on sum function Typo, it's not SPLIT_FUNC_SELECT, it is SPLIT_SUM_SELECT > > @note > - This is from split_sum_func2() for items that should be split > - > All found SUM items are added FIRST in the fields list and > we replace the item with a reference. > > + If this is an item in the SELECT list then we also have to split out > + all arguments to functions used together with the sum function. > + For example in case of SELECT A*sum(B) we have to split out both > + A and sum(B). > + This is not needed for ORDER BY, GROUP BY or HAVING as all references > + to items in the select list are already of type REF > + > thd->fatal_error() may be called if we are out of memory > */ > > void Item::split_sum_func2(THD *thd, Item **ref_pointer_array, > List<Item> &fields, Item **ref, > - bool skip_registered) > + uint split_flags) > { > - /* An item of type Item_sum is registered <=> ref_by != 0 */ > - if (type() == SUM_FUNC_ITEM && skip_registered && > - ((Item_sum *) this)->ref_by) > - return; > - if ((type() != SUM_FUNC_ITEM && with_sum_func) || > - (type() == FUNC_ITEM && > - (((Item_func *) this)->functype() == Item_func::ISNOTNULLTEST_FUNC || > - ((Item_func *) this)->functype() == Item_func::TRIG_COND_FUNC))) > + if (unlikely(type() == SUM_FUNC_ITEM)) > { > - /* Will split complicated items and ignore simple ones */ > - split_sum_func(thd, ref_pointer_array, fields); > + /* An item of type Item_sum is registered if ref_by != 0 */ > + if ((split_flags & SPLIT_SUM_SKIP_REGISTERED) && > + ((Item_sum *) this)->ref_by) > + return; > } > - else if ((type() == SUM_FUNC_ITEM || (used_tables() & ~PARAM_TABLE_BIT)) && > - type() != SUBSELECT_ITEM && > - (type() != REF_ITEM || > - ((Item_ref*)this)->ref_type() == Item_ref::VIEW_REF)) > + else > { > - /* > - Replace item with a reference so that we can easily calculate > - it (in case of sum functions) or copy it (in case of fields) > - > - The test above is to ensure we don't do a reference for things > - that are constants (PARAM_TABLE_BIT is in effect a constant) > - or already referenced (for example an item in HAVING) > - Exception is Item_direct_view_ref which we need to convert to > - Item_ref to allow fields from view being stored in tmp table. > - */ > - Item_aggregate_ref *item_ref; > - uint el= fields.elements; > - /* > - If this is an item_ref, get the original item > - This is a safety measure if this is called for things that is > - already a reference. > - */ > - Item *real_itm= real_item(); > + /* Not a SUM() function */ > + if (unlikely((!with_sum_func && !(split_flags & SPLIT_SUM_SELECT)))) > + { > + /* > + This is not a SUM function and there are no SUM functions inside. > + Nothing more to do. > + */ > + return; > + } > + if (likely(with_sum_func || > + (type() == FUNC_ITEM && > + (((Item_func *) this)->functype() == > + Item_func::ISNOTNULLTEST_FUNC || > + ((Item_func *) this)->functype() == > + Item_func::TRIG_COND_FUNC)))) > + { > + /* Will call split_sum_func2() for all items */ > + split_sum_func(thd, ref_pointer_array, fields, split_flags); > + return; > + } > > - ref_pointer_array[el]= real_itm; > - if (!(item_ref= new Item_aggregate_ref(&thd->lex->current_select->context, > - ref_pointer_array + el, 0, name))) > - return; // fatal_error is set > - if (type() == SUM_FUNC_ITEM) > - item_ref->depended_from= ((Item_sum *) this)->depended_from(); > - fields.push_front(real_itm); > - thd->change_item_tree(ref, item_ref); > + if (unlikely((!(used_tables() & ~PARAM_TABLE_BIT) || > + type() == SUBSELECT_ITEM || > + (type() == REF_ITEM && > + ((Item_ref*)this)->ref_type() != Item_ref::VIEW_REF)))) > + return; > } > + > + /* > + Replace item with a reference so that we can easily calculate > + it (in case of sum functions) or copy it (in case of fields) > + > + The test above is to ensure we don't do a reference for things > + that are constants (PARAM_TABLE_BIT is in effect a constant) > + or already referenced (for example an item in HAVING) > + Exception is Item_direct_view_ref which we need to convert to > + Item_ref to allow fields from view being stored in tmp table. > + */ > + Item_aggregate_ref *item_ref; > + uint el= fields.elements; > + /* > + If this is an item_ref, get the original item > + This is a safety measure if this is called for things that is > + already a reference. > + */ > + Item *real_itm= real_item(); > + > + ref_pointer_array[el]= real_itm; > + if (!(item_ref= new Item_aggregate_ref(&thd->lex->current_select->context, > + ref_pointer_array + el, 0, name))) > + return; // fatal_error is set > + if (type() == SUM_FUNC_ITEM) > + item_ref->depended_from= ((Item_sum *) this)->depended_from(); > + fields.push_front(real_itm); > + thd->change_item_tree(ref, item_ref); > } > > > diff --git a/sql/item.h b/sql/item.h > index 8254359..44649bf 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -74,6 +74,10 @@ char_to_byte_length_safe(uint32 char_length_arg, uint32 mbmaxlen_arg) > } > > > +/* Bits for the split_sum_func() function */ > +#define SPLIT_SUM_SKIP_REGISTERED 1 /* Skip registered funcs */ > +#define SPLIT_SUM_SELECT 2 /* SELECT item; Split all parts */ > + > /* > "Declared Type Collation" > A combination of collation and its derivation. > @@ -1169,10 +1173,10 @@ class Item: public Type_std_attributes > return false; > } > virtual void split_sum_func(THD *thd, Item **ref_pointer_array, > - List<Item> &fields) {} > + List<Item> &fields, uint flags) {} > /* Called for items that really have to be split */ > void split_sum_func2(THD *thd, Item **ref_pointer_array, List<Item> &fields, > - Item **ref, bool skip_registered); > + Item **ref, uint flags); > virtual bool get_date(MYSQL_TIME *ltime, ulonglong fuzzydate); > bool get_time(MYSQL_TIME *ltime) > { return get_date(ltime, TIME_TIME_ONLY | TIME_INVALID_DATES); } > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 7b34e64..caa645d 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -4575,12 +4575,13 @@ void Item_cond::traverse_cond(Cond_traverser traverser, > */ > > void Item_cond::split_sum_func(THD *thd, Item **ref_pointer_array, > - List<Item> &fields) > + List<Item> &fields, uint flags) > { > List_iterator<Item> li(list); > Item *item; > while ((item= li++)) > - item->split_sum_func2(thd, ref_pointer_array, fields, li.ref(), TRUE); > + item->split_sum_func2(thd, ref_pointer_array, fields, li.ref(), > + flags | SPLIT_SUM_SKIP_REGISTERED); Why not clear SPLIT_SUM_SELECT flag here? The Item_cond may be at the select list, but its parts are definitely not in the select list? The same applies to Item_row::split_sum_func and Item_func::split_sum_func. > } > > > diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h > index 7d89cf5..ceab597 100644 > --- a/sql/item_cmpfunc.h > +++ b/sql/item_cmpfunc.h > @@ -1759,7 +1759,8 @@ class Item_cond :public Item_bool_func > SARGABLE_PARAM **sargables); > SEL_TREE *get_mm_tree(RANGE_OPT_PARAM *param, Item **cond_ptr); > virtual void print(String *str, enum_query_type query_type); > - void split_sum_func(THD *thd, Item **ref_pointer_array, List<Item> &fields); > + void split_sum_func(THD *thd, Item **ref_pointer_array, List<Item> &fields, > + uint flags); > friend int setup_conds(THD *thd, TABLE_LIST *tables, TABLE_LIST *leaves, > COND **conds); > void top_level_item() { abort_on_null=1; } > diff --git a/sql/item_func.cc b/sql/item_func.cc > index 3e78be8..684724b 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -419,11 +419,12 @@ Item *Item_func::compile(Item_analyzer analyzer, uchar **arg_p, > */ > > void Item_func::split_sum_func(THD *thd, Item **ref_pointer_array, > - List<Item> &fields) > + List<Item> &fields, uint flags) > { > Item **arg, **arg_end; > for (arg= args, arg_end= args+arg_count; arg != arg_end ; arg++) > - (*arg)->split_sum_func2(thd, ref_pointer_array, fields, arg, TRUE); > + (*arg)->split_sum_func2(thd, ref_pointer_array, fields, arg, > + flags | SPLIT_SUM_SKIP_REGISTERED); Why not clear SPLIT_SUM_SELECT flag here? (same question as in Item_cond::split_sum_func) > } > > ........ > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 4a50003..ce02484 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -1944,8 +1954,9 @@ int JOIN::init_execution() > /* > Enable LIMIT ROWS EXAMINED during query execution if: > (1) This JOIN is the outermost query (not a subquery or derived table) > - This ensures that the limit is enabled when actual execution begins, and > - not if a subquery is evaluated during optimization of the outer query. > + This ensures that the limit is enabled when actual execution begins, > + and not if a subquery is evaluated during optimization of the outer > + query. > (2) This JOIN is not the result of a UNION. In this case do not apply the > limit in order to produce the partial query result stored in the > UNION temp table. > @@ -2562,8 +2573,12 @@ void JOIN::exec_inner() > skip_sort_order= 0; > } > bool made_call= false; > + SQL_SELECT *select= join_tab[const_tables].select; This line causes a crash in a few tests: main.select_pkeycache main.select main.select_jcl6 main.subselect3 main.subselect3_jcl6 The reason is that execution gets here when the join is degenerate and join_tab==NULL. > if (order && > - (order != group_list || !(select_options & SELECT_BIG_RESULT)) && > + (order != group_list || !(select_options & SELECT_BIG_RESULT) || > + (select && select->quick && > + select->quick->get_type() == QUICK_SELECT_I::QS_TYPE_GROUP_MIN_MAX)) > + && > (const_tables == table_count || > ((simple_order || skip_sort_order) && > (made_call=true) && BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] MDEV-8433 Make field<'broken-string' use indexes
by Alexander Barkov 07 Jul '15

07 Jul '15

Hi Sergei, I created a task according to our discussion a few weeks ago: https://mariadb.atlassian.net/browse/MDEV-8433 Which version should we do this task in? Thanks.

1 0

Re: [Maria-developers] GSoc2015: MDEV-8356 first patch
by Sergey Petrunia 03 Jul '15

03 Jul '15

Hi Diwas, On Fri, Jul 03, 2015 at 02:54:28PM +0530, Diwas Joshi wrote: > hello sergey, I have fixed the issues from the previous patch in this one. > Please find some feedback below. After you have addressed it, please commit the change and push it to your tree. The next task to focus on after this one is MDEV-8389. > diff --git a/mysql-test/r/sp-error.result b/mysql-test/r/sp-error.result > index 897895e..cacfeea 100644 > --- a/mysql-test/r/sp-error.result > +++ b/mysql-test/r/sp-error.result > @@ -2866,12 +2866,6 @@ SELECT @msg; > DROP FUNCTION f1; > DROP FUNCTION f2; > DROP TABLE t1; > -CREATE FUNCTION f1(a INT, b VARCHAR(11)) > -RETURNS TABLE t1(id INT, name VARCHAR(11)) > -BEGIN > -INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > -END| > -ERROR 42000: This version of MariaDB doesn't yet support 'Table functions' > SELECT * FROM f1('xyz'); > ERROR 42000: This version of MariaDB doesn't yet support 'Queries using Table Functions' > SELECT * FROM t1, f1('xyz',t1.a); > diff --git a/mysql-test/r/sp.result b/mysql-test/r/sp.result > index 3f28df9..2a931b1 100644 > --- a/mysql-test/r/sp.result > +++ b/mysql-test/r/sp.result > @@ -7937,3 +7937,17 @@ return 1; > end | > ERROR 0A000: Not allowed to return a result set from a function > drop table t1,t2; > +# Tests for Table functions > +CREATE FUNCTION f1(a INT, b VARCHAR(11)) > +RETURNS TABLE t1(id INT, name VARCHAR(11)) > +BEGIN > +INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > +END| Please add "--vertical_results" here so that the SELECT results are printed vertically. It will be easier to see what the return value is. > +SELECT * FROM mysql.proc WHERE db = database() and name = 'f1'; > +db name type specific_name language sql_data_access is_deterministic security_type param_list returns body definer created modified sql_mode comment character_set_client collation_connection db_collation body_utf8 > +test f1 FUNCTION f1 SQL CONTAINS_SQL NO DEFINER a INT, b VARCHAR(11) TABLE t1(id int(11), name varchar(11)) BEGIN > +INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > +END root@localhost 2015-07-03 12:18:04 2015-07-03 12:18:04 latin1 latin1_swedish_ci latin1_swedish_ci BEGIN > +INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > +END > +DROP FUNCTION f1; -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Buildbot branch 10.0-custombld
by Kristian Nielsen 02 Jul '15

02 Jul '15

Hi Rasmus, The branch 10.0-custombld seems to no longer build binaries (after the transition to git?): https://github.com/MariaDB/server/tree/10.0-custombld https://buildbot.askmonty.org/buildbot/grid?branch=10.0-custombld&category=… Can you please fix Buildbot so that this branch works again for producing custom builds of all packages? This is urgent, as it is blocking further progress of MDEV-8302. Thanks, - Kristian.

1 1

Re: [Maria-developers] GSoc2015: MDEV-8356 first patch
by Sergey Petrunia 02 Jul '15

02 Jul '15

Hi Diwas, On Thu, Jul 02, 2015 at 04:26:58PM +0530, Diwas Joshi wrote: > for the return type i am storing a string containing TABLE(<col > definitions>). eg. forCREATE FUNCTION f1(a INT, b VARCHAR(11)) > RETURNS TABLE t1(id INT, name VARCHAR(11)) BEGIN set @a=3; END| > we store > TABLE t1(id INT, name VARCHAR(11)) Ok. > for the type of function we can store it as TYPE_ENUM_FUNCTION only and > later see if it creates a problem, although we can always differentiate > between the functions based on the return type stored above. Yes, let's keep it this way for now. > > I included the table alias with the return type string as you suggested. > There is a warning that i am getting, which i am failing to figure the > reason for but I am still trying. > Please review. > The first thing: please add a testcase that checks what the code code does. I can think of doing this: CREATE table_function .... < whatever here> select * from mysql.proc where name='table_function'; This will show what was stored in the system table. This will also make it clear if/what warning gets produced. Since you've changed CREATE FUNCTION table_function ... not to return error, I guess some of testcases you previously added will need to be modified, too. Overall the patch seems to be ok but I would like to look at the final version of it before giving the approval. > diff --git a/sql/sp.cc b/sql/sp.cc > index 9fd17b0..3413ba1 100644 > --- a/sql/sp.cc > +++ b/sql/sp.cc > @@ -925,21 +925,66 @@ class Bad_db_error_handler : public Internal_error_handler > bzero((char*) &share, sizeof(share)); > table.in_use= thd; > table.s = &share; > - field= sp->create_result_field(0, 0, &table); > - field->sql_type(result); > - > - if (field->has_charset()) > + if (sp->m_type == TYPE_ENUM_FUNCTION) > { > - result.append(STRING_WITH_LEN(" CHARSET ")); > - result.append(field->charset()->csname); > - if (!(field->charset()->state & MY_CS_PRIMARY)) > + field= sp->create_result_field(0, 0, &table); > + field->sql_type(result); > + > + if (field->has_charset()) > { > - result.append(STRING_WITH_LEN(" COLLATE ")); > - result.append(field->charset()->name); > + result.append(STRING_WITH_LEN(" CHARSET ")); > + result.append(field->charset()->csname); > + if (!(field->charset()->state & MY_CS_PRIMARY)) > + { > + result.append(STRING_WITH_LEN(" COLLATE ")); > + result.append(field->charset()->name); > + } > } > + delete field; > + } > + else if (sp->m_type == TYPE_ENUM_TABLE) > + { > + List_iterator_fast<Create_field> it(sp->m_cols_list); > + Create_field *cols_field; > + cols_field= it++; > + result.append(STRING_WITH_LEN("TABLE ")); > + result.append(sp->m_table_alias.str); > + result.append(STRING_WITH_LEN("(")); > + while (cols_field) > + { > + uint unused1= 0; > + sp_prepare_create_field(thd, cols_field); > + prepare_create_field(cols_field, &unused1, HA_CAN_GEOMETRY); > + uint field_length; > + Field *field; > + String tmp_result(64); > + field_length= cols_field->length; > + cols_field->flags= 0; > + field= make_field(table.s, /* TABLE_SHARE ptr */ > + (uchar*) 0, /* field ptr */ > + field_length, /* field [max] length */ > + (uchar*) "", /* null ptr */ > + 0, /* null bit */ > + cols_field->pack_flag, > + cols_field->sql_type, > + cols_field->charset, > + cols_field->geom_type, cols_field->srid, > + Field::NONE, /* unreg check */ > + cols_field->interval, > + cols_field->field_name); > + field->vcol_info= cols_field->vcol_info; > + field->stored_in_db= cols_field->stored_in_db; > + if (field) > + field->init(&table); > + field->sql_type(tmp_result); > + result.append(tmp_result); > + cols_field= it++; > + if(cols_field) > + result.append(STRING_WITH_LEN(", ")); > + delete field; > + } > + result.append(STRING_WITH_LEN(")")); > } > - > - delete field; > } > > > @@ -1020,7 +1065,8 @@ class Bad_db_error_handler : public Internal_error_handler > char definer_buf[USER_HOST_BUFF_SIZE]; > LEX_STRING definer; > ulonglong saved_mode= thd->variables.sql_mode; > - MDL_key::enum_mdl_namespace mdl_type= type == TYPE_ENUM_FUNCTION ? > + MDL_key::enum_mdl_namespace mdl_type= type == (type == TYPE_ENUM_FUNCTION || > + type == TYPE_ENUM_TABLE) ? > MDL_key::FUNCTION : MDL_key::PROCEDURE; Looks reasonable. > > CHARSET_INFO *db_cs= get_default_db_collation(thd, sp->m_db.str); > @@ -1036,7 +1082,8 @@ class Bad_db_error_handler : public Internal_error_handler > retstr.set_charset(system_charset_info); > > DBUG_ASSERT(type == TYPE_ENUM_PROCEDURE || > - type == TYPE_ENUM_FUNCTION); > + type == TYPE_ENUM_FUNCTION || > + type == TYPE_ENUM_TABLE); > > /* Grab an exclusive MDL lock. */ > if (lock_object_name(thd, mdl_type, sp->m_db.str, sp->m_name.str)) > @@ -1117,7 +1164,8 @@ class Bad_db_error_handler : public Internal_error_handler > > store_failed= store_failed || > table->field[MYSQL_PROC_MYSQL_TYPE]-> > - store((longlong)type, TRUE); > + store((longlong)(type == TYPE_ENUM_TABLE ? > + TYPE_ENUM_FUNCTION : type), TRUE); > > store_failed= store_failed || > table->field[MYSQL_PROC_FIELD_SPECIFIC_NAME]-> > @@ -1145,7 +1193,7 @@ class Bad_db_error_handler : public Internal_error_handler > table->field[MYSQL_PROC_FIELD_PARAM_LIST]-> > store(sp->m_params.str, sp->m_params.length, system_charset_info); > > - if (sp->m_type == TYPE_ENUM_FUNCTION) > + if (sp->m_type == TYPE_ENUM_FUNCTION || sp->m_type == TYPE_ENUM_TABLE) > { > sp_returns_type(thd, retstr, sp); > > @@ -1153,7 +1201,7 @@ class Bad_db_error_handler : public Internal_error_handler > table->field[MYSQL_PROC_FIELD_RETURNS]-> > store(retstr.ptr(), retstr.length(), system_charset_info); > } > - > + String body_with_alias(64); What is this for? It's not used anywhere. > store_failed= store_failed || > table->field[MYSQL_PROC_FIELD_BODY]-> > store(sp->m_body.str, sp->m_body.length, system_charset_info); > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index b53d939..d7d7d7b 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -16392,12 +16392,6 @@ sf_tail: > Lex_input_stream *lip= YYLIP; > > lex->sphead->set_body_start(thd, lip->get_cpp_tok_start()); > - > - if (Lex->sphead->m_type== TYPE_ENUM_TABLE) //to be removed later. > - { > - my_error(ER_NOT_SUPPORTED_YET, MYF(0), "Table functions"); > - MYSQL_YYABORT; > - } > } > sp_proc_stmt /* $15 */ > { BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] GSoC2015: MDEV-8356 patch for review
by Diwas Joshi 02 Jul '15

02 Jul '15

Hello all, I am attaching a patch for MDEV-8356 <https://mariadb.atlassian.net/browse/MDEV-8356> as part of my work under GSoC2015 implementing table valued functions. Please feel free to point out mistakes, if any. regards Diwas Joshi

1 0

[Maria-developers] [GSoC] MDEV-4674 Port InnoDB memcached interface to MariaDB - Midterm report
by Piotr Jurkiewicz 02 Jul '15

02 Jul '15

Hi all, as the midterm is approaching, I am posting a report on what I have done so far. The project is going as scheduled. You can see the timeline in my proposal: http://www.google-melange.com/gsoc/proposal/public/google/gsoc2015/piotrjur… All goals planned before the midterm have been reached. InnoDB Memcached plugin ported from MySQL works now with MariaDB (both with InnoDB and XtraDB engines). That means that MariaDB is now on par with MySQL on this issue. My repository with the code is available here: https://github.com/piotrjurkiewicz/mariadb-server What have been done in details: 1. Code of InnoDB Memcached plugin has been imported from MySQL into the MariaDB tree. 2. There were some differences between MariaDB and MySQL, which blocked the plugin from out-of-the-box compilation and correct operation. They have been resolved. 3. Test suite for the plugin has been imported from MySQL. Compatibility changes were introduced into these test cases which required them. The test suite was modified to assign memcached port number dynamically from mtr worker's pool, what allows to run the same test in parallel for many combinations. The plugin fully passes the test suite, both for 'innodb_plugin' and 'xtradb' combinations. 4. Some enhancements and bug fixes have been introduced into the plugin: - detection of memcached daemon initialization has been fixed -- this eliminated unnecessary wait (15 seconds) on plugin shutdown and memory leak (in cases when plugin failed to initialize properly before) - memcached arithmetic commands (incr/decr) handling has been improved -- now numeric columns values are handled in a consistent and predictable way - incorrect handling of unsigned not null integers columns has been fixed For details of all changes please see commit messages associated with them: https://github.com/piotrjurkiewicz/mariadb-server/commits/10.1-memcached After the midterm, I plan to focus on further enhancements of the plugin, as stated in my proposal: 1. Change plugin architecture to "one table per socket", as originally proposed in MDEV-4674 bug report. This will allow usage of well known security solutions (firewalls, unix file permissions) to restrict access to different tables for different users. 2. Implement a new interface inside the Memcached plugin, which will use Handler API for database access. This will make possible to use the plugin with database engines other than InnoDB/XtraDB. Ultimately, this interface will replace existing InnoDB direct API interface (if there will be no significant performance differences). Therefore, I would like to as you to point me to Handler API documentation (if documentation such exists). I know that HandlerSocket plugin uses the Handler API to access databases. Is HandlerSocket plugin a good example of Handler API usage to follow? Piotr

1 0

[Maria-developers] How to read of innodb_locks table by using low level interface?
by 김경남 01 Jul '15

01 Jul '15

Hi, Sergei I'm developing a mariadb monitoring plugin type agent. We're planning to contribute our monitoring solution for mariadb in one or two months. Our solution is consist of plugin type agent, APM Server and Dashboard. Our plugin type agent collect data by using low level interface instead of using query. I can collect a information_schema.global_status and performance_schema.event_xxxx_table for your help in the past. By referencing the source of feedback.cc, I can collect data from global_status using fill_stauts function. I hava a question. I need to collect data from information_schema.innodb_locks for lock monitoring. But I found that I must use trx_i_s_common_fill_table in i_s.cc for reading of innodb_locks table and that fuction is static. so I only access that function by function pointer. Debugging source for many days, I guess that I must use ST_SCHEMA TABLE's fill_table member variable for pointing to trx_i_s_common_fille_table function. Right? But I don't know how to that. I attemped in various ways for many days. but I failed. If possible. please let me know how to read data of information_schema.innodb_locks using native api. Thanks in advance. Regards, Sergei. _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

1 0

[Maria-developers] sprint tasks for the next week
by Alexander Barkov 01 Jul '15

01 Jul '15

Hi Sergei, I'd like to do these tasks: - MDEV-8215 Asian MB3 charsets: compare broken bytes as "greater than any non-broken character" - MDEV-8036 Fix all collations to compare broken bytes as "greater than any non-broken character" to close the group of tasks related to handling bad bytes in comparison. Any objections? How do I mark tasks as "sprint"? Thanks.

1 0

Re: [Maria-developers] MDEV-8356 analysis and ideas
by Sergey Petrunia 29 Jun '15

29 Jun '15

Hi Diwas, Nice analysis. Please find my comments below. On Mon, Jun 29, 2015 at 10:01:56AM +0530, Diwas Joshi wrote: > hello sergey, so i was reading further on MDEV-8356 last night. There are > two main things here: type of sf and return type of sf. Right now the > create function... is giving errors at > > store_failed= store_failed || > table->field[MYSQL_PROC_MYSQL_TYPE]-> > store((longlong)type, TRUE); > > in sf_create_routine(), which is right since we have been been using > TYPE_ENUM_TABLE for m_type so far and mysql.proc takes > enum('FUNCTION','PROCEDURE') for type. This can be solved by either > changing this enum in mysql.proc definition and including TABLE, or storing > table_valued functions as FUNCTION as well. I think first option is better, > including TABLE in the enum, since this will help us in distinguishing > these functions while executing later if needed. It is not clear for me which option is better. On one hand, there is your argument that adding 'TABLE' into the enum will make it easier to tell table functions from regular function. On the other hand, try grepping for 'TYPE_ENUM_FUNCTION' in sql/*.{h,cc}. You find a lot of code that checks if m_type == TYPE_ENUM_FUNCTION. You will need to review each check to see what should be done for TYPE_ENUM_FUNCTION. If we can't decide towards one option or another, let's use whatever you prefer right now. If we discover that that works poorly, we will be able to reverse the decision later on. > The second thing is about return type of function. Right now > sp_returns_type() method is being used to store the return type of > functions. But in our case we have a list of create_fields: > List<Create_field> m_cols_list keeping list of attribute names and their > types for the table. We need to store all these, for this we can either > alter mysql.proc definition to make a field appropriate for storing this > list. The type of the field storing return type right now is longblob which > as i read further on it is a binary large object that can hold a variable > amount of data and are treated as binary strings (byte strings). Maybe we > can try to find a way to store all the attribute information in this field > only, maybe putting in some delimiters between different field types or > some other way since after all its a string. I am still not sure about this > problem. I tried creating this function: CREATE FUNCTION f2(a INT, b VARCHAR(11)) RETURNS VARCHAR(32) DETERMINISTIC BEGIN set @a=3; RETURN @a; END| and mysql.proc.returns has: returns: varchar(32) CHARSET latin1 So, it is a text defition of the return type. It seems natural if for table functions we Is it a problem to do similar thing for Table functions? e.g for a function CREATE FUNCTION f1(a INT, b VARCHAR(11)) RETURNS TABLE t1(id INT, name VARCHAR(11)) BEGIN set @a=3; END| store 'TABLE t1(id INT, name VARCHAR(11)' ? I guess it depends on where the value of mysql.proc.returns is used. Please research this. > > Also, another thing is to store the table alias m_table_alias, because > queries inside the function will be using it. There is no field present > specifically for this in mysql.proc. So either we can create one. There is > a field specific_name in it, as i saw in the code is right now storing the > same thing as name field. So maybe we can use this field to store the table > alias. I would prefer that m_table_alias was stored as part of the expression in mysql.proc.returns field. Can we try that option first? If we figure that can't work, we can think of other options. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MariaDB multi-source replication testing at Booking.com
by Karoly Nagy 29 Jun '15

29 Jun '15

Dear MariaDB developers, I'm Károly Nagy working for Booking.com currently testing multi-source replication functionality of MariaDB. Kristian suggested I should reach out to you on this mailing list regarding my questions. We're seeing very high and fluctuating mutex contentions while replicating from two sources (Oracle MySQL 5.6) to a single MariaDB slave. You can see that on the graphs below. The spin waits are relatively [1] aligned but the mutex rounds [2] are 5-10 times higher than it is on the two sources combined together and not consistent. The sources have a relatively constant pattern while the target has dips around 2.5k and spikes up to 8k. The os waits are in completely different order of magnitude [3]. The scenario where values were captured: * Multi-source target is replicating the full dataset of `source 2` and a subset of `source 1` (the hot data) - MariaDB 10.0.16 * Both sources are MySQL 5.6 being part of their replication chain as slaves with log_slave_updates * Source 2 is in normal mode - Oracle MySQL 5.6.17 * Source 1 is catching up from a 1 day replication delay - Oracle MySQL 5.6.24 * All the slaves are warm having the buffer pool fully populated Is this behavior expected? Could you give us some insights on why we could see these results? If there is any more information you need please let me know. Thank you for your help in advance! Every metric is on /10 seconds basis. [1] Mutex spin waits [2] Mutex rounds [3] Mutex OS waits Best regards, -- Károly Nagy System engineer Booking.com <http://booking.com/> BV Rembrandt Square Office, Herengracht 597, 1017 CE Amsterdam Direct +31 (0)20 715 8403

3 4

Re: [Maria-developers] [Commits] 132935a: MDEV-8339 - Server crash during table cache eviction
by Sergei Golubchik 27 Jun '15

27 Jun '15

Hi, Sergey! On Jun 25, svoj(a)mariadb.org wrote: > revision-id: 132935a3ef6578f5fabc765ff6ac966cd3798819 > parent(s): 1f4a89b10ca8df61ec51fcc87e1203dc72579759 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-25 14:21:16 +0400 > message: > > MDEV-8339 - Server crash during table cache eviction > > TDC_element::free_tables_back() had pre-lfhash leftover code, which referenced > TDC_element::free_tables via TDC_element::share. This is not correct as share > may be NULL (newly inserted or to be removed), instead access free_tables > directly. ok to push Regards, Sergei

1 0

[Maria-developers] MDEV-8214 Asian MB2 charsets: compare broken bytes as "greater than any non-broken character"
by Alexander Barkov 26 Jun '15

26 Jun '15

Hi Sergei, For simplicity I decided to split: MDEV-8036 Fix all collations to compare broken bytes as "greater than any non-broken character" into sub-tasks. Please review a patch for the first sub-task MDEV-8214. The overall plan is: - MDEV-8214, for the Asian charsets with mbmaxlen=2 - MDEV-8215, for the Asian charsets with mbmaxlen=3, based on MDEV-8214 - MDEV-8036, the rest (Unicode character sets) Thanks.

2 4

[Maria-developers] MDEV-7502 Automatic provisioning of slave - error reporting
by Martin Kaluznik 26 Jun '15

26 Jun '15

Hello Kristian, with general idea working (sending of database / table structures + table data), my next plan is to improve / introduce error handling in existing code along with error handling on slave side (ddl events not from provisioning, ignore errors when updating not yet existing rows, ...). What are general guidelines for introduction of new error codes? Is there anything else I need to do except adding it to /sql/share/errmsg-utf8.txt ? I would create only one new error code - ER_PROVISIONING_FAILED - and place specific messages inside formatted text - same way as ER_MASTER_FATAL_ERROR_READING_BINLOG works. It doesn't go well with translations, but... you can decide. Maybe create different error codes at least for errors, where we are expecting user to fix them - table without index, statement mode. Also, how to tell slave from master that provisioning finished? Is introduction of new error code ER_PROVISIONING_DONE acceptable solution? The error itself would never produce error message in logs. Alternatively I can reuse / create similar event to Stop_log_event, note - provisioning is completed after SQL thread processes whole relay log, not when IO thread receives this event, so even reuse would require additional code. And side question, is there any log event other than Query_log_event which can produce ddl statement? Martin

2 1

Re: [Maria-developers] [Commits] 2cf747e: MDEV-8219: enforce_storage_engine cannot be set globally
by Sergei Golubchik 26 Jun '15

26 Jun '15

Hi, Jan! On Jun 26, Jan Lindström wrote: > revision-id: 2cf747e885603122b2adbefc131516cd12dc953c > parent(s): d3b7eb7b99ef7dae79c20b5beb63acdfdf6d8046 > committer: Jan Lindström > branch nick: 10.1-bugs > timestamp: 2015-06-26 08:56:50 +0300 > message: > > MDEV-8219: enforce_storage_engine cannot be set globally > > Change session only variable enforce_storage_engine to be > session variable and make sure that also global value > is used if session variable is not set. Ok to push, thanks! Regards, Sergei

1 0

[Maria-developers] Please review: MDEV-7824 [Bug #68041] Zero date can be inserted in strict no-zero mode through a default value
by Alexander Barkov 25 Jun '15

25 Jun '15

Hi Sergei, Please review a patch for mdev-7824. It's based on a MySQL patch for http://bugs.mysql.com/bug.php?id=68041 and is a blocker for: MDEV-3929 Add full support for auto-initialized/updated timestamp and datetime Thanks.

2 9

[Maria-developers] MDEV-6877 - binlog_row_image implementation advice
by Vicențiu Ciorbaru 25 Jun '15

25 Jun '15

Hi Sergei! This past week I've done quite a bit of reading regarding the replication code and how the binlog is used by both mysqlbinlog and the slaves in order to do replication. I think I now have a good enough understanding of the flow of the code for both MySQL and MariaDB to actually start coding the feature into MariaDB. I do have a couple of unknowns still. I don't quite understand what's the use for the TABLE_MAP event. My intuition tells me its some sort of id we assign to a table name (test.t1 for example) that gets used afterwards in the binlog row events. We seem to be using a bitmap of columns that we want to log into the binlog. MySQL does extend the TABLE class to flip the bits according to the binlog_row_image variable. That seems like the first place to start with the addition. I could not find any other "hidden" logic that we use to choose how we log row operations in the binlog. What I would like to know is if there are any other pitfalls that I am supposed to watch out for, or if there is anything else I may be missing regarding the logging part. When it comes to actually doing the replication, I concluded that Log_event::do_apply_event is the place where we begin translating the event from the binlog to an SQL statement. Another concern that I have is how to test the implementation. Right now, I run the test, get the binlog file and print it with mysqlbinlog and see if the output is what I expect. Do we have something within mysql-test-run that automates this? Regards, Vicentiu

2 7

Re: [Maria-developers] [Commits] 9f00950: MDEV-7631 : Invalid WSREP_SST rows appear in mysqld-bin.index file
by Sergei Golubchik 25 Jun '15

25 Jun '15

Hi, Nirbhay! On Jun 24, Nirbhay Choubey wrote: > revision-id: 9f00950d2ad9be5533e66a5fa87a795a9574585f > parent(s): 0f4478105f5027ce589a0f08c6019d906d632029 > committer: Nirbhay Choubey > branch nick: 10.0-galera.b7631 > timestamp: 2015-06-24 23:25:22 -0400 > message: > > MDEV-7631 : Invalid WSREP_SST rows appear in mysqld-bin.index file > > Try not to release 0 (STDIN_FILENO) file descriptor as it can > incorrectly get reused by streams opened later. This really deserved a more descriptive comment. There's nothing wrong when fd 0 is reused by other streams. The problem was that because of a bug (arguably) in old glibc in CentOS6/RHEL6 freopen remapped stdout/stderr to fd 0 and 1. Which is, on itself, is not a problem either, but when mysqld forked other processes, these children had no access to stdout and stderr. it's too late to recommit now and, luckily, MDEV-7631 has all the information about this issue. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] aff5606: Add close-on-exec flag to open(), socket(), accept() & fopen().
by Sergei Golubchik 24 Jun '15

24 Jun '15

Hi, Nirbhay! On Jun 24, Nirbhay Choubey wrote: > revision-id: aff560696cac5e081d19f4460ccd9c906ddc9d84 > parent(s): 70714d3597ec9ffa742cc9e173d4ad32f294cc51 > committer: Nirbhay Choubey > branch nick: 10.0-galera.b7631 > timestamp: 2015-06-24 11:58:11 -0400 > message: > > Add close-on-exec flag to open(), socket(), accept() & fopen(). > > @@ -1026,7 +1026,8 @@ static inline void inline_mysql_socket_register( > (&state, socket_listen.m_psi, PSI_SOCKET_CONNECT, (size_t)0, src_file, src_line); > > /* Instrumented code */ > - socket_accept.fd= accept(socket_listen.fd, addr, &addr_length); > + socket_accept.fd= accept4(socket_listen.fd, addr, &addr_length, > + SOCK_CLOEXEC); Note that, according to the manpage, "accept4() is a nonstandard Linux extension". It needs a configure-time check and ifdef. Alternatively, don't use it at all, do fcntl(FD_CLOEXEC) instead. > /* Instrumentation end */ > if (locker != NULL) > @@ -1036,7 +1037,8 @@ static inline void inline_mysql_socket_register( > #endif > { > /* Non instrumented code */ > - socket_accept.fd= accept(socket_listen.fd, addr, &addr_length); > + socket_accept.fd= accept4(socket_listen.fd, addr, &addr_length, > + SOCK_CLOEXEC); > } Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 99ca8b9: MDEV-8219: enforce_storage_engine cannot be set globally
by Sergei Golubchik 24 Jun '15

24 Jun '15

Hi, Jan! On Jun 23, Jan Lindström wrote: > revision-id: 99ca8b9ea8ee1b756fbf1edc567f6d06c9cf63fd > parent(s): d3b7eb7b99ef7dae79c20b5beb63acdfdf6d8046 > committer: Jan Lindström > branch nick: 10.1-bugs > timestamp: 2015-06-23 13:14:46 +0300 > message: > > MDEV-8219: enforce_storage_engine cannot be set globally > > Change session only variable enforce_storage_engine to be > session variable and make sure that also global value > is used if session variable is not set. > > diff --git a/mysql-test/suite/sys_vars/r/sysvars_innodb.result b/mysql-test/suite/sys_vars/r/sysvars_innodb.result > index 927cd05..1c59391 100644 > --- a/mysql-test/suite/sys_vars/r/sysvars_innodb.result > +++ b/mysql-test/suite/sys_vars/r/sysvars_innodb.result > @@ -355,6 +355,20 @@ NUMERIC_BLOCK_SIZE 1048576 > ENUM_VALUE_LIST NULL > READ_ONLY YES > COMMAND_LINE_ARGUMENT REQUIRED > +VARIABLE_NAME INNODB_BUF_DUMP_STATUS_FREQUENCY > +SESSION_VALUE NULL > +GLOBAL_VALUE 0 > +GLOBAL_VALUE_ORIGIN COMPILE-TIME > +DEFAULT_VALUE 0 > +VARIABLE_SCOPE GLOBAL > +VARIABLE_TYPE BIGINT UNSIGNED > +VARIABLE_COMMENT A number between [0, 100] that tells how oftern buffer pool dump status in percentages should be printed. E.g. 10 means that buffer pool dump status is printed when every 10% of number of buffer pool pages are dumped. Default is 0 (only start and end status is printed). please, no unrelated changes. If you want to fix forgotten result files, do it in a separate commit. > +NUMERIC_MIN_VALUE 0 > +NUMERIC_MAX_VALUE 100 > +NUMERIC_BLOCK_SIZE 0 > +ENUM_VALUE_LIST NULL > +READ_ONLY NO > +COMMAND_LINE_ARGUMENT REQUIRED > VARIABLE_NAME INNODB_BUF_FLUSH_LIST_NOW > SESSION_VALUE NULL > GLOBAL_VALUE OFF > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index 954970d..f18e6a2 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -9772,7 +9772,7 @@ static bool check_engine(THD *thd, const char *db_name, > DBUG_ENTER("check_engine"); > handlerton **new_engine= &create_info->db_type; > handlerton *req_engine= *new_engine; > - handlerton *enf_engine= thd->variables.enforced_table_plugin ? plugin_hton(thd->variables.enforced_table_plugin) : NULL; > + handlerton *enf_engine= thd->variables.enforced_table_plugin ? plugin_hton(thd->variables.enforced_table_plugin) : global_system_variables.enforced_table_plugin ? plugin_hton(global_system_variables.enforced_table_plugin) : NULL; Please, no. Global variable should not be used like that, it simply sets a default value for the session variable for new connections. The way you've done it now, one cannot disable enforced_table_plugin per session at all. > bool no_substitution= thd->variables.sql_mode & MODE_NO_ENGINE_SUBSTITUTION; > *new_engine= ha_checktype(thd, req_engine, no_substitution); Regards, Sergei

1 0

[Maria-developers] GSoc2015
by Diwas Joshi 24 Jun '15

24 Jun '15

hii, please find a patch for MDEV-8343 <https://mariadb.atlassian.net/browse/MDEV-8343> attached to this mail. I have also included a test case in the patch. regards Diwas Joshi

2 2

[Maria-developers] Can't find messagefile '/usr/share/mysql/errmsg.sys'
by Sylvain Raybaud 24 Jun '15

24 Jun '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I built mariadb galera cluster 10.0.17 from source. Starting it with mysql_safe fails with: Can't find messagefile '/usr/share/mysql/errmsg.sys' This file does not exist indeed. It seems to not exist at all in mariadb-galera-10.0.17.tar.gz. Is it supposed to be installed separately? Am I missing something? Regards, - -- Sylvain Raybaud www.green-communications.fr -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVeAYZAAoJEEkkwl4JtJ9yvfoP/jTnvj7V1Ze+E7sFE7Rz4Hsm +4Ga8f9MKCr5VxBZky3wMb8EgWmfcWDHbKqu4httTMH2RS6CCv7aiBeRfrMHMW5I NHl+k9CPUrDs03n4Vg12G2M33IIRfdEBgYf2xZ3sp3kBJ07MUWZPvlVL6ch+Aybn oOVzQ1rbUpxrkX2gj4XZ/wtXmy8vN6s9L3jy+o5HU4JZJcFyp1IpCwCE0OneI81V EiHs8dbpsUstr/32TaQ7gCBN9Hb25i5aVwRCD7UXE6kFt6OD6ifv8mw5KfmdtjHc i/0OotnOOsfj7tgDXKoPOltauGdSADJm2VSp8/nsfd6MR2nxMGZcw/Xbhj5l4FkY jWZf7uRAiKsYlmZEaJlb0WZrCTGSe6pcO82hObhv/vD0O55gKCdOM2rAExRiuxFn IPAuKGAKhzegxXxmmBTQv09tjSY3WTNZfFbDKGDQe4EYmCJ0K14cq37gC+TYI+xb Jr9SI5WBAvVVrCvhRktloEf03jVqXMH3H03DBjxGDB9KRc3Vr6AGhQmg0ixC2ZHC uTbRcx+R5mEn40TVwzLEnsUfiV+0MuJ5YYM5brYYRmXRg4ipwTcts1byKTHpvy69 PrDF1mzStInfex2ntkmw7puX8CQ4uebCLzHWSPODDa1n66D1vaEYCbZV7hEg25dH ox0Iptm4REWGyxgNeQxB =0Ydk -----END PGP SIGNATURE-----

3 4

[Maria-developers] doubt about query cache and write locks
by Roberto Spadim 24 Jun '15

24 Jun '15

hi guys, i'm with a doubt when i execute LOCK TABLE xxx WRITE any query at query cache could be executed without locking (waiting for table lock state) shouldn't the lock table invalidade the table query cache when using a write lock? -- Roberto Spadim

2 2

[Maria-developers] crash - analyze table with histogram
by Roberto Spadim 24 Jun '15

24 Jun '15

Hi guys, i got a crash, could someone help? --- queries: set histogram_size=255,histogram_type=SINGLE_PREC_HB; ANALYZE TABLE 19_org.estoque_itens PERSISTENT FOR ALL; /* Erro SQL (2013): Lost connection to MySQL server during query */ --- log: 150623 12:31:47 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.0.20-MariaDB-log key_buffer_size=2097152000 read_buffer_size=6291456 max_used_connections=22 max_threads=92 thread_count=9 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 3180295 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x7f523b81c008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4b1410a0 thread_stack 0x48000 mysys/stacktrace.c:247(my_print_stacktrace)[0xbcb2ee] sql/signal_handler.cc:153(handle_fatal_signal)[0x71fa8c] /lib/libpthread.so.0[0x7f52ccbe98f0] sql/field.h:703(Field::set_notnull(long long))[0x610c70] sql/sql_statistics.cc:533(Stat_table::update_stat())[0x60dd68] sql/sql_admin.cc:759(mysql_admin_table)[0x67adc1] sql/sql_admin.cc:1183(Sql_cmd_analyze_table::execute(THD*))[0x67c66c] sql/sql_parse.cc:5095(mysql_execute_command(THD*))[0x5986c2] sql/sql_parse.cc:6529(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x59fc86] sql/sql_parse.cc:1310(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x5a1bb7] sql/sql_parse.cc:999(do_command(THD*))[0x5a22f9] sql/sql_connect.cc:1378(do_handle_one_connection(THD*))[0x66c7d4] sql/sql_connect.cc:1295(handle_one_connection)[0x66c912] /lib/libpthread.so.0[0x7f52ccbe1fe7] /lib/libc.so.6(clone+0x6d)[0x7f52cbb822bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f5190884020): is an invalid pointer Connection ID (thread ID): 304 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. -- Roberto Spadim

2 2

Re: [Maria-developers] [Commits] 6050ab6: MDEV-6829 : SELinux/AppArmor policies for Galera server
by Daniel Black 23 Jun '15

23 Jun '15

Nice work. https://mariadb.atlassian.net/browse/MDEV-7637 has some netlink_audit_socket rules that don't appear to be here. Recommend contributing the selinux component to https://github.com/TresysTechnology/refpolicy which distros usually develop their policies from. Does this work for galera multicast? It appears to only allow tcp bind here. note for readme semanage permissive -a mysqld_t - less of a change for enabling just that domain to be permissive. Does any of https://mariadb.com/kb/en/mariadb/what-to-do-if-mariadb-doesnt-start/ need changing? ----- On 18 Jun, 2015, at 11:59 PM, Nirbhay Choubey nirbhay(a)mariadb.com wrote: > revision-id: 6050ab658696925f2a031b901eb398fff65fa92a > parent(s): 9eff9ed5c58e782abf383a52a7e691a55b4798a2 > committer: Nirbhay Choubey > branch nick: 5.5-galera > timestamp: 2015-06-18 09:59:09 -0400 > message: > > MDEV-6829 : SELinux/AppArmor policies for Galera server > > Add SELinux policy and AppArmor profile under policy/. > > --- > policy/apparmor/README | 5 ++ > policy/apparmor/usr.sbin.mysqld | 150 ++++++++++++++++++++++++++++++++++ > policy/apparmor/usr.sbin.mysqld.local | 4 + > policy/selinux/README | 18 ++++ > policy/selinux/mariadb-server.fc | 10 +++ > policy/selinux/mariadb-server.te | 91 +++++++++++++++++++++ > 6 files changed, 278 insertions(+) > > diff --git a/policy/apparmor/README b/policy/apparmor/README > new file mode 100644 > index 0000000..271655f > --- /dev/null > +++ b/policy/apparmor/README > @@ -0,0 +1,5 @@ > +Note: The included AppArmor profiles can be used for MariaDB Galera cluster. > +However, since these profiles had been tested for a limited set of scenarios, > +it is highly recommended to run them in "complain" mode and report any denials > +on mariadb.org/jira. > + > diff --git a/policy/apparmor/usr.sbin.mysqld b/policy/apparmor/usr.sbin.mysqld > new file mode 100644 > index 0000000..307872c > --- /dev/null > +++ b/policy/apparmor/usr.sbin.mysqld > @@ -0,0 +1,150 @@ > +# Last Modified: Fri Mar 1 18:55:47 2013 > +# Based on usr.sbin.mysqld packaged in mysql-server in Ubuntu. > +# This AppArmor profile has been copied under BSD License from > +# Percona XtraDB Cluster, along with some additions. > + > +#include <tunables/global> > + > +/usr/sbin/mysqld flags=(complain) { > + #include <abstractions/base> > + #include <abstractions/mysql> > + #include <abstractions/nameservice> > + #include <abstractions/user-tmp> > + #include <abstractions/winbind> > + > + capability chown, > + capability dac_override, > + capability setgid, > + capability setuid, > + capability sys_rawio, > + capability sys_resource, > + > + network tcp, > + > + /bin/dash rcx, > + /dev/dm-0 r, > + /etc/gai.conf r, > + /etc/group r, > + /etc/hosts.allow r, > + /etc/hosts.deny r, > + /etc/ld.so.cache r, > + /etc/mtab r, > + /etc/my.cnf r, > + /etc/mysql/*.cnf r, > + /etc/mysql/*.pem r, > + /etc/mysql/conf.d/ r, > + /etc/mysql/conf.d/* r, > + /etc/nsswitch.conf r, > + /etc/passwd r, > + /etc/services r, > + /run/mysqld/mysqld.pid w, > + /run/mysqld/mysqld.sock w, > + /sys/devices/system/cpu/ r, > + owner /tmp/** lk, > + /tmp/** rw, > + /usr/lib/mysql/plugin/ r, > + /usr/lib/mysql/plugin/*.so* mr, > + /usr/sbin/mysqld mr, > + /usr/share/mysql/** r, > + /var/lib/mysql/ r, > + /var/lib/mysql/** rwk, > + /var/log/mysql.err rw, > + /var/log/mysql.log rw, > + /var/log/mysql/ r, > + /var/log/mysql/* rw, > + /var/run/mysqld/mysqld.pid w, > + /var/run/mysqld/mysqld.sock w, > + > + > + profile /bin/dash flags=(complain) { > + #include <abstractions/base> > + #include <abstractions/bash> > + #include <abstractions/mysql> > + #include <abstractions/nameservice> > + #include <abstractions/perl> > + > + > + > + /bin/cat rix, > + /bin/dash rix, > + /bin/date rix, > + /bin/grep rix, > + /bin/nc.openbsd rix, > + /bin/netstat rix, > + /bin/ps rix, > + /bin/rm rix, > + /bin/sed rix, > + /bin/sleep rix, > + /bin/tar rix, > + /bin/which rix, > + /dev/tty rw, > + /etc/ld.so.cache r, > + /etc/my.cnf r, > + /proc/ r, > + /proc/*/cmdline r, > + /proc/*/fd/ r, > + /proc/*/net/dev r, > + /proc/*/net/if_inet6 r, > + /proc/*/net/tcp r, > + /proc/*/net/tcp6 r, > + /proc/*/stat r, > + /proc/*/status r, > + /proc/sys/kernel/pid_max r, > + /proc/tty/drivers r, > + /proc/uptime r, > + /proc/version r, > + /sbin/ifconfig rix, > + /sys/devices/system/cpu/ r, > + /tmp/** rw, > + /usr/bin/cut rix, > + /usr/bin/dirname rix, > + /usr/bin/gawk rix, > + /usr/bin/innobackupex rix, > + /usr/bin/mysql rix, > + /usr/bin/perl rix, > + /usr/bin/seq rix, > + /usr/bin/wsrep_sst* rix, > + /usr/bin/wsrep_sst_common r, > + /usr/bin/xtrabackup* rix, > + /var/lib/mysql/ r, > + /var/lib/mysql/** rw, > + /var/lib/mysql/*.log w, > + /var/lib/mysql/*.err w, > + > +# MariaDB additions > + ptrace peer=@{profile_name}, > + > + /bin/hostname rix, > + /bin/ip rix, > + /bin/mktemp rix, > + /bin/ss rix, > + /bin/sync rix, > + /bin/touch rix, > + /bin/uname rix, > + /etc/mysql/*.cnf r, > + /etc/mysql/conf.d/ r, > + /etc/mysql/conf.d/* r, > + /proc/*/attr/current r, > + /proc/*/fdinfo/* r, > + /proc/*/net/* r, > + /proc/locks r, > + /proc/sys/net/ipv4/ip_local_port_range r, > + /run/mysqld/mysqld.sock rw, > + /sbin/ip rix, > + /usr/bin/basename rix, > + /usr/bin/du rix, > + /usr/bin/find rix, > + /usr/bin/lsof rix, > + /usr/bin/my_print_defaults rix, > + /usr/bin/mysqldump rix, > + /usr/bin/pv rix, > + /usr/bin/rsync rix, > + /usr/bin/socat rix, > + /usr/bin/tail rix, > + /usr/bin/timeout rix, > + /usr/bin/xargs rix, > + /usr/bin/xbstream rix, > + } > + # Site-specific additions and overrides. See local/README for details. > + #include <local/usr.sbin.mysqld> > +} > diff --git a/policy/apparmor/usr.sbin.mysqld.local > b/policy/apparmor/usr.sbin.mysqld.local > new file mode 100644 > index 0000000..a0b8a02 > --- /dev/null > +++ b/policy/apparmor/usr.sbin.mysqld.local > @@ -0,0 +1,4 @@ > +# Site-specific additions and overrides for usr.sbin.mysqld.. > +# For more details, please see /etc/apparmor.d/local/README. > +# This AppArmor profile has been copied under BSD License from > +# Percona XtraDB Cluster, along with some additions. > diff --git a/policy/selinux/README b/policy/selinux/README > new file mode 100644 > index 0000000..a8c11c7 > --- /dev/null > +++ b/policy/selinux/README > @@ -0,0 +1,18 @@ > +Note: The included SELinux policy files can be used for MariaDB Galera cluster. > +However, since these policies had been tested for a limited set of scenarios, > +it is highly recommended to run SELinux in "permissive" mode even with these > +policies installed and report any denials on mariadb.org/jira. > + > + > +How to generate and load the policy module of MariaDB Galera cluster ? > + > +* Generate the SELinux policy module. > + # cd <source>/policy/selinux/ > + # make -f /usr/share/selinux/devel/Makefile mariadb-server.pp > + > +* Load the generated policy module. > + # semodule -i /path/to/mariadb-server.pp > + > +* Lastly, run the following command to allow 4568. > + # semanage port -a -t mysqld_port_t -p tcp 4568 > + > diff --git a/policy/selinux/mariadb-server.fc b/policy/selinux/mariadb-server.fc > new file mode 100644 > index 0000000..1a69ecc > --- /dev/null > +++ b/policy/selinux/mariadb-server.fc > @@ -0,0 +1,10 @@ > +# This SELinux file contexts (.fc) file has been copied under BSD License from > +# Percona XtraDB Cluster. > + > +/etc/init\.d/rc\.d/mysql -- > gen_context(system_u:object_r:mysqld_initrc_exec_t,s0) > +/var/lib/mysql/.*\.log -- gen_context(system_u:object_r:mysqld_log_t,s0) > +/var/lib/mysql/.*\.err -- gen_context(system_u:object_r:mysqld_log_t,s0) > +/var/lib/mysql/.*\.pid -- gen_context(system_u:object_r:mysqld_var_run_t,s0) > +/var/lib/mysql/.*\.cnf -- gen_context(system_u:object_r:mysqld_etc_t,s0) > +/usr/bin/xtrabackup.* -- gen_context(system_u:object_r:mysqld_exec_t,s0) > +/usr/bin/wsrep.* -- gen_context(system_u:object_r:mysqld_safe_exec_t,s0) > diff --git a/policy/selinux/mariadb-server.te b/policy/selinux/mariadb-server.te > new file mode 100644 > index 0000000..9c0319c > --- /dev/null > +++ b/policy/selinux/mariadb-server.te > @@ -0,0 +1,91 @@ > +# This SELinux type enforcement (.te) file has been copied under BSD License > +# from Percona XtraDB Cluster, along with some additions. > + > +module mariadb-server 1.0; > + > +require { > + type user_tmp_t; > + type kerberos_port_t; > + type mysqld_safe_t; > + type tmp_t; > + type tmpfs_t; > + type hostname_exec_t; > + type ifconfig_exec_t; > + type sysctl_net_t; > + type proc_net_t; > + type port_t; > + type mysqld_t; > + type var_lib_t; > + type rsync_exec_t; > + type bin_t; > + type shell_exec_t; > + type anon_inodefs_t; > + type fixed_disk_device_t; > + class lnk_file read; > + class process { getattr signull }; > + class unix_stream_socket connectto; > + class capability { sys_resource sys_nice }; > + class tcp_socket { name_bind name_connect }; > + class file { execute setattr read create getattr execute_no_trans write ioctl > open append unlink }; > + class sock_file { create unlink getattr }; > + class blk_file { read write open }; > + class dir { write search getattr add_name read remove_name open }; > + > +# MariaDB additions > + type tram_port_t; > + class process setpgid; > + class netlink_tcpdiag_socket { create nlmsg_read }; > +} > + > + > +#============= mysqld_safe_t ============== > +allow mysqld_safe_t mysqld_t:process signull; > +allow mysqld_safe_t self:capability { sys_resource sys_nice }; > +allow mysqld_safe_t tmp_t:file { create read write open getattr unlink ioctl > setattr }; > +allow mysqld_safe_t tmp_t:dir { write remove_name add_name }; > +allow mysqld_safe_t tmp_t:sock_file { getattr unlink }; > +allow mysqld_safe_t user_tmp_t:sock_file { getattr unlink }; > +allow mysqld_safe_t var_lib_t:dir { write add_name }; > +allow mysqld_safe_t var_lib_t:file { write ioctl setattr create open getattr > append unlink }; > + > +#============= mysqld_t ============== > +allow mysqld_t anon_inodefs_t:file write; > +allow mysqld_t tmp_t:sock_file { create unlink }; > +allow mysqld_t tmpfs_t:dir { write search read remove_name open add_name }; > +allow mysqld_t tmpfs_t:file { write getattr read create unlink open }; > +allow mysqld_t fixed_disk_device_t:blk_file { read write open }; > +allow mysqld_t ifconfig_exec_t:file { read execute open execute_no_trans > getattr }; > + > +#This rule allows connecting on 4444 > +allow mysqld_t kerberos_port_t:tcp_socket { name_bind name_connect }; > + > +allow mysqld_t mysqld_safe_t:dir { getattr search }; > +allow mysqld_t mysqld_safe_t:file { read open }; > +allow mysqld_t self:unix_stream_socket connectto; > +allow mysqld_t port_t:tcp_socket { name_bind name_connect }; > +allow mysqld_t proc_net_t:file { read getattr open }; > +allow mysqld_t sysctl_net_t:dir search; > +allow mysqld_t var_lib_t:file { getattr open append }; > +allow mysqld_t var_lib_t:sock_file { create unlink getattr }; > +allow mysqld_t rsync_exec_t:file { read getattr open execute execute_no_trans > }; > +allow mysqld_t self:process getattr; > +allow mysqld_t hostname_exec_t:file { read getattr execute open > execute_no_trans }; > +allow mysqld_t user_tmp_t:dir { write add_name }; > +allow mysqld_t user_tmp_t:file create; > +allow mysqld_t bin_t:lnk_file read; > +allow mysqld_t tmp_t:file { append create read write open getattr unlink > setattr }; > + > +# Allows too much leeway - the xtrabackup/wsrep rules in fc should fix it, but > +# keep for the moment. > +allow mysqld_t shell_exec_t:file { execute_no_trans getattr read execute open > }; > +allow mysqld_t bin_t:file { getattr read execute open execute_no_trans ioctl }; > + > +# MariaDB additions > +allow mysqld_t self:process setpgid; > +# This rule allows port 4567 > +allow mysqld_t tram_port_t:tcp_socket name_bind; > + > +# Rules related to XtraBackup > +allow mysqld_t self:netlink_tcpdiag_socket { create nlmsg_read }; > +allow mysqld_t sysctl_net_t:file { read getattr open }; > + > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

2 1

[Maria-developers] Likely a dead optimizer related code
by Alexander Barkov 23 Jun '15

23 Jun '15

Hi Igor, I noticed that during Item_cond::fix_fields() and Item_func_between::fix_fields() update these optimizer related SELECT_LEX members: thd->lex->current_select->cond_count and thd->lex->current_select->between_count The purpose of these members is to allocate optimizer related buffers in: - update_ref_and_keys() in sql_select.cc and - check_func_dependency() in opt_table_elimination,cc and it seems they have no any other purposes.. From my understanding, this is a dead code and the collected values are never used: - cond_count and_between_count are later always initialized to 0 before the walk(count_sargable_conds) call in update_ref_and_keys(). - moreover, using the values collected during fix_fields() would not be correct to allocate optimized related buffers, because the code in fix_fields() does not distinguish between different query parts, so the items can be just a part of the SELECT list rather than a part of WHERE/HAVING/ON. Is the attached patch correct? All tests pass. Thanks.

1 1

Re: [Maria-developers] [Commits] d3b7eb7: MDEV-7528 GIS: Functions return NULL instead of specified -1 for NULL arguments.
by Sergei Golubchik 23 Jun '15

23 Jun '15

Hi, Holyfoot! On Jun 23, holyfoot(a)askmonty.org wrote: > revision-id: d3b7eb7b99ef7dae79c20b5beb63acdfdf6d8046 > parent(s): 3e4126e9d6b21d2f330a365b0847999346485863 > committer: Alexey Botchkov > branch nick: server > timestamp: 2015-06-23 11:57:05 +0500 > message: > > MDEV-7528 GIS: Functions return NULL instead of specified -1 for NULL arguments. > The behaviour required by the standard seems too weird to expect. If the function can never return NULL, please set maybe_null accordingly. Regards, Sergei

1 0

[Maria-developers] GSOC [Adding Redis interface to XtraDB]
by Aalekh Nigam 22 Jun '15

22 Jun '15

Hello, I am Aalekh Nigam, working as a GSOC Summer Strudent for the project "Adding Redis interace to XtraDB". Please follow my blog, for weekly update on my project here: http://mimdl.tumblr.com/ Thank you Aalekh

1 0

[Maria-developers] MXS-2: Week 4
by sriram patil 22 Jun '15

22 Jun '15

Hi All, Please find the link to my blog post of fourth week below. <http://srirampatil.tumblr.com/post/122086304290/mxs-2-week-4> MXS-2: Week 4 <http://srirampatil.tumblr.com/post/122086304290/mxs-2-week-4> Feel free to point out mistakes, if any. Also, suggestions ans/or reviews are welcome! Thanks, Sriram

1 0

Re: [Maria-developers] [Commits] 95a99d8: MDEV-8303; Dumping buffer pool noisy in the logs.
by Daniel Black 20 Jun '15

20 Jun '15

Will only get used if documented: https://mariadb.com/kb/en/mariadb/xtradbinnodb-server-system-variables/ + https://mariadb.com/kb/en/mariadb/server-system-variables/ ----- On 18 Jun, 2015, at 6:45 PM, Jan Lindström jan.lindstrom(a)mariadb.com wrote: > revision-id: 95a99d8dd96be43369d5be9cc23735a1ce86674a > parent(s): b94eaff8d90a60b9dea3b5b68fbde3d35563127b > committer: Jan Lindström > branch nick: 10.1-encryption > timestamp: 2015-06-18 08:38:33 +0300 > message: > > MDEV-8303; Dumping buffer pool noisy in the logs. > > Added new dynamic configuration variable innodb_buf_dump_status_frequency > to configure how often buffer pool dump status is printed in the logs. > A number between [0, 100] that tells how oftern buffer pool dump status > in percentages should be printed. E.g. 10 means that buffer pool dump > status is printed when every 10% of number of buffer pool pages are > dumped. Default is 0 (only start and end status is printed). -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

2 1

Re: [Maria-developers] [Commits] d1f5dfe: MDEV-8030 - Apc_target::disable() locks mutex twice
by Sergey Petrunia 19 Jun '15

19 Jun '15

Hi Sergei, Ok to push. I'm sorry for the delay with this. On Wed, May 20, 2015 at 08:09:36AM +0000, svoj(a)mariadb.org wrote: > revision-id: d1f5dfe67703a22f017eb25dd48b1eb5246a9c54 > parent(s): 80333ad847d9f1708fa02fb47a976e96f014bc50 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-05-20 12:04:32 +0400 > message: > > MDEV-8030 - Apc_target::disable() locks mutex twice > > Moved Apc_target::destroy(), Apc_target::enable() and Apc_targe::disable() > definitions to my_apc.h so that they can be inlined. > > Apc_targe::disable() now calls Apc_target::process_apc_requests() only if > there're APC requests. This saves one pthread_mutex_lock() call. > > Overhead change: > Apc_target::disable 0.04% -> out of radar > Apc_target::enable 0.03% -> out of radar > Apc_target::process_apc_requests 0.02% -> out of radar > pthread_mutex_lock 0.43% -> 0.42% > pthread_mutex_unlock 0.26% -> 0.25% > > --- > sql/my_apc.cc | 39 --------------------------------------- > sql/my_apc.h | 27 +++++++++++++++++++++++---- > 2 files changed, 23 insertions(+), 43 deletions(-) > > diff --git a/sql/my_apc.cc b/sql/my_apc.cc > index 1766068..91f5cd3 100644 > --- a/sql/my_apc.cc > +++ b/sql/my_apc.cc > @@ -41,45 +41,6 @@ void Apc_target::init(mysql_mutex_t *target_mutex) > } > > > -/* > - Destroy the target. The target must be disabled when this call is made. > -*/ > -void Apc_target::destroy() > -{ > - DBUG_ASSERT(!enabled); > -} > - > - > -/* > - Enter ther state where the target is available for serving APC requests > -*/ > -void Apc_target::enable() > -{ > - /* Ok to do without getting/releasing the mutex: */ > - enabled++; > -} > - > - > -/* > - Make the target unavailable for serving APC requests. > - > - @note > - This call will serve all requests that were already enqueued > -*/ > - > -void Apc_target::disable() > -{ > - bool process= FALSE; > - DBUG_ASSERT(enabled); > - mysql_mutex_lock(LOCK_thd_data_ptr); > - if (!(--enabled)) > - process= TRUE; > - mysql_mutex_unlock(LOCK_thd_data_ptr); > - if (process) > - process_apc_requests(); > -} > - > - > /* [internal] Put request qe into the request list */ > > void Apc_target::enqueue_request(Call_request *qe) > diff --git a/sql/my_apc.h b/sql/my_apc.h > index dfeef5e..20b1ee4 100644 > --- a/sql/my_apc.h > +++ b/sql/my_apc.h > @@ -50,10 +50,29 @@ class Apc_target > ~Apc_target() { DBUG_ASSERT(!enabled && !apc_calls);} > > void init(mysql_mutex_t *target_mutex); > - void destroy(); > - void enable(); > - void disable(); > - > + > + /* Destroy the target. The target must be disabled when this call is made. */ > + void destroy() { DBUG_ASSERT(!enabled); } > + > + /* Enter ther state where the target is available for serving APC requests */ > + void enable() { enabled++; } > + > + /* > + Make the target unavailable for serving APC requests. > + > + @note > + This call will serve all requests that were already enqueued > + */ > + void disable() > + { > + DBUG_ASSERT(enabled); > + mysql_mutex_lock(LOCK_thd_data_ptr); > + bool process= !--enabled && have_apc_requests(); > + mysql_mutex_unlock(LOCK_thd_data_ptr); > + if (unlikely(process)) > + process_apc_requests(); > + } > + > void process_apc_requests(); > /* > A lightweight function, intended to be used in frequent checks like this: > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] GSoC2015: facing errors working on udf's for table valued functions.
by Diwas Joshi 19 Jun '15

19 Jun '15

hii i have been stuck on some shift/reduce conflict since past few days. It is giving the error for the code i have added for select query using the udf. I think the rule that I defined under table_factor is conflicting with the first rule defined under it. regards

2 1

[Maria-developers] mariadb 10.1.5 crash
by Roberto Spadim 19 Jun '15

19 Jun '15

hi guys, i was testing mariadb with test servers and i hit a crash could anyone check? server has 8GB of ram, i don't know what query is 2015-06-17 14:01:32 140497706510080 [Note] WSREP: Stop replication 2015-06-17 14:01:32 140497706510080 [Note] Event Scheduler: Purging the queue. 0 events 2015-06-17 14:01:32 140497706510080 [Note] /usr/mariadb/mariadb/bin//mysqld: Shutdown complete 150617 14:01:32 mysqld_safe mysqld from pid file /home/mysql/pid/local.pid ended 150617 14:02:38 mysqld_safe Starting mysqld daemon with databases from /home/mysql/data/local/ 2015-06-17 14:02:38 140260849645440 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release. 2015-06-17 14:02:38 140260849645440 [Note] /usr/mariadb/mariadb/bin//mysqld (mysqld 10.1.5-MariaDB-log) starting as process 2321 ... 2015-06-17 14:02:38 140260849645440 [Note] Plugin 'InnoDB' is disabled. 2015-06-17 14:02:38 140260849645440 [Note] Plugin 'FEEDBACK' is disabled. 2015-06-17 14:02:38 140260849645440 [Note] Server socket created on IP: '::'. 2015-06-17 14:02:38 140260849645440 [Note] Event Scheduler: Loaded 0 events 2015-06-17 14:02:38 140260849645440 [Note] WSREP: Read nil XID from storage engines, skipping position init 2015-06-17 14:02:38 140260849645440 [Note] WSREP: wsrep_load(): loading provider library 'none' 2015-06-17 14:02:38 140260849645440 [Note] /usr/mariadb/mariadb/bin//mysqld: ready for connections. Version: '10.1.5-MariaDB-log' socket: '/tmp/mysql.BH.sock' port: 3307 MariaDB Server 150617 17:02:14 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.1.5-MariaDB-log key_buffer_size=629145600 read_buffer_size=4194304 max_used_connections=23 max_threads=102 thread_count=16 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1452041 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x7f90d0353008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f9103539e80 thread_stack 0x48000 mysys/stacktrace.c:247(my_print_stacktrace)[0x7f9106c50b0b] sql/signal_handler.cc:153(handle_fatal_signal)[0x7f91067eadb5] /lib64/libpthread.so.0(+0x10d00)[0x7f910537dd00] sql/sql_plist.h:181(I_P_List_iterator)[0x7f910679cf03] mysys/lf_hash.c:125(lfind)[0x7f9106c5a1a0] sql/table_cache.cc:1084(tdc_iterate(THD*, char (*)(void*, void*), void*, bool))[0x7f910679e4e3] sql/table_cache.cc:267(tc_add_table(THD*, TABLE*))[0x7f910679e808] sql/sql_base.cc:2572(open_table(THD*, TABLE_LIST*, Open_table_context*))[0x7f9106633272] sql/sql_base.cc:4016(open_and_process_table)[0x7f91066370c8] sql/sql_base.cc:5184(open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*))[0x7f9106637394] sql/sql_parse.cc:5718(execute_sqlcom_select)[0x7f910666f083] sql/sql_parse.cc:2938(mysql_execute_command(THD*))[0x7f910667a9ce] sql/sql_parse.cc:7173(mysql_parse)[0x7f910667dc3d] sql/sql_parse.cc:1470(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x7f91066802a8] sql/sql_parse.cc:1095(do_command(THD*))[0x7f910668083b] sql/sql_connect.cc:1347(do_handle_one_connection(THD*))[0x7f91067435ef] sql/sql_connect.cc:1260(handle_one_connection)[0x7f9106743747] /lib64/libpthread.so.0(+0x7654)[0x7f9105374654] /lib64/libc.so.6(clone+0x6d)[0x7f9104a987dd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f90c8424020): is an invalid pointer Connection ID (thread ID): 516 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 150617 17:02:14 mysqld_safe Number of processes running now: 0 150617 17:02:14 mysqld_safe mysqld restarted 2015-06-17 17:02:15 140086665152384 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release. 2015-06-17 17:02:15 140086665152384 [Note] /usr/mariadb/mariadb/bin//mysqld (mysqld 10.1.5-MariaDB-log) starting as process 9785 ... 2015-06-17 17:02:15 140086665152384 [Note] mysqld: Aria engine: starting recovery recovered pages: 0% 10% 21% 31% 41% 51% 78% 90% 100% (0.0 seconds); tables to flush: 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 (0.3 seconds); 2015-06-17 17:02:15 140086665152384 [Note] mysqld: Aria engine: recovery done 2015-06-17 17:02:15 140086665152384 [Note] Plugin 'InnoDB' is disabled. 2015-06-17 17:02:15 140086665152384 [Note] Plugin 'FEEDBACK' is disabled. 2015-06-17 17:02:15 140086665152384 [Note] Recovering after a crash using tc.log 2015-06-17 17:02:15 140086665152384 [Note] Starting crash recovery... 2015-06-17 17:02:15 140086665152384 [Note] Crash recovery finished. 2015-06-17 17:02:15 140086665152384 [Note] Server socket created on IP: '::'. 2015-06-17 17:02:15 140086665152384 [Note] Event Scheduler: Loaded 0 events 2015-06-17 17:02:15 140086665152384 [Note] WSREP: Read nil XID from storage engines, skipping position init 2015-06-17 17:02:15 140086665152384 [Note] WSREP: wsrep_load(): loading provider library 'none' 2015-06-17 17:02:15 140086665152384 [Note] /usr/mariadb/mariadb/bin//mysqld: ready for connections. Version: '10.1.5-MariaDB-log' socket: '/tmp/mysql.BH.sock' port: 3307 MariaDB Server 2015-06-17 17:02:19 140086617798400 [ERROR] mysqld: Table './conquista_comercial/est_atu' is marked as crashed and should be repaired 2015-06-17 17:02:19 140086617798400 [Warning] Checking table: './conquista_comercial/est_atu' 2015-06-17 17:02:19 140086617798400 [ERROR] mysqld: Table './conquista_comercial/est_spa' is marked as crashed and should be repaired 2015-06-17 17:02:19 140086617798400 [Warning] Checking table: './conquista_comercial/est_spa' 2015-06-17 17:02:51 140086617798400 [ERROR] mysqld: Table './conquista_comercial/est_dia' is marked as crashed and should be repaired 2015-06-17 17:02:51 140086617798400 [Warning] Checking table: './conquista_comercial/est_dia' 2015-06-17 17:02:52 140086617798400 [ERROR] mysqld: Table './conquista_comercial/rastreabilidade_impressao' is marked as crashed and should be repaired 2015-06-17 17:02:52 140086617798400 [Warning] Checking table: './conquista_comercial/rastreabilidade_impressao' 2015-06-17 17:29:58 140086663948032 [ERROR] mysqld: Table './conquista_comercial/est_atu_erros' is marked as crashed and should be repaired 2015-06-17 17:29:58 140086663948032 [Warning] Checking table: './conquista_comercial/est_atu_erros' 2015-06-17 19:24:04 140086538364672 [Note] /usr/mariadb/mariadb/bin//mysqld: Normal shutdown 2015-06-17 19:24:04 140086538364672 [Note] WSREP: Stop replication 2015-06-17 19:24:06 140086538364672 [Note] Event Scheduler: Purging the queue. 0 events 2015-06-17 19:24:07 140086538364672 [Note] /usr/mariadb/mariadb/bin//mysqld: Shutdown complete 150617 19:24:07 mysqld_safe mysqld from pid file /home/mysql/pid/local.pid ended -- Roberto Spadim SPAEmpresarial - Software ERP Eng. Automação e Controle

3 4

Re: [Maria-developers] [Commits] 4563e13: MDEV-8156: Assertion failure in file log0crypt.cc line 220 on server restart
by Sergei Golubchik 18 Jun '15

18 Jun '15

Hi, Jan! On Jun 18, Jan Lindström wrote: > revision-id: 4563e1315df6333b280aa441e07d979d8c2ce9f4 > parent(s): eb2c170bb03fb7c98d3bd864bae46ddfb8c3acb5 > committer: Jan Lindström > branch nick: 10.1-encryption > timestamp: 2015-06-18 19:58:57 +0300 > message: > > MDEV-8156: Assertion failure in file log0crypt.cc line 220 on server restart > > Instead of asserting print informative error message to error log > and return failure from innodb_init causing the server to shutdown. A test case would've been nice Regards, Sergei

1 0

Re: [Maria-developers] [Commits] a5799f5: MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
by Sergei Golubchik 18 Jun '15

18 Jun '15

Hi, Sergey! On Jun 18, svoj(a)mariadb.org wrote: > revision-id: a5799f5b087f79a8063cd10242527e9235f87597 > parent(s): 4c336c8ba4fbb4ac1a4fe9680c6899c6a49b6c46 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-18 18:01:56 +0400 > message: > > MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO > > Pass THD to Item_field (and all derivatives) constructors. > This reduces number of pthread_getspecific() calls from 322 to 292. ok to push Regards, Sergei

1 0

Re: [Maria-developers] changing some defaults related to replication
by Kristian Nielsen 18 Jun '15

18 Jun '15

Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Hi, Kristian! > > could you say against some or default changes listed here if they will > be done in 10.1? > > http://dev.mysql.com/doc/refman/5.6/en/server-default-changes.html binlog_checksum NONE CRC32 5.6.6 I am not sure that is a good idea. Binlog checksums are incompatible with earlier slave versions I think (though I guess we supported it all the way back to 5.3?). The implementation is rather bad, and not well tested in MariaDB I think, and could easily have bugs. --binlog-row-event-max-size 1024 8192 5.6.6 Seems like a good idea, 1024 is quite small. Shouldn't it default to something even larger than 8192? Seems safe to change. sync_master_info 0 10000 5.6.6 sync_relay_log 0 10000 5.6.6 sync_relay_log_info 0 10000 5.6.6 Seems safe enough, though I am not sure what the point of this change would be. I think that was all of the replication-related changes. I am not planning myself to change anything about this in 10.1, but I have no strong opinion for or against if someone else wants to do it. - Kristian.

1 0

Re: [Maria-developers] [Commits] 2e08ec2: MDEV-7956 - handler::rebind_psi() takes 0.07% in OLTP RO
by Sergei Golubchik 18 Jun '15

18 Jun '15

Hi, Sergey! On Jun 17, svoj(a)mariadb.org wrote: > revision-id: 2e08ec24623e0b3d3db02333e16658dcfaa0535c > parent(s): b776f8d3a8afff55d5f8208da7bdfce9aa33eb39 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-17 13:09:22 +0400 > message: > > MDEV-7956 - handler::rebind_psi() takes 0.07% in OLTP RO > > Do not call handler::rebind_psi() and handler::unbind_psi() when performance > schema is compiled out. I don't particularly like it, you're just avoding an empty virtual function call, and that in a configuration (no P_S) which no our binaries use. But you can push it if you want. Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 3f0b023: MDEV-7952 - clock_gettime() takes 0.24% in OLTP RO
by Sergei Golubchik 18 Jun '15

18 Jun '15

Hi, Sergey! On Jun 17, svoj(a)mariadb.org wrote: > revision-id: 3f0b02351a53e0fcaee8554c2c852fb3da8fe7da > parent(s): 5d03dd20a4abbfd9777b619dfd51a6927323955a > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-17 18:55:38 +0400 > message: > > MDEV-7952 - clock_gettime() takes 0.24% in OLTP RO > > Initialize abs_timeout when it is about to be used. This saves one my_hrtime() > call on hot path (when we acquire MDL lock without waiting). ok > When filling I_S.PROCESSLIST use THD::start_utime/THD::utime_after_query instead > of THD::start_time. This allows us to save 2 clock_gettime() calls. see below > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 1e4723d..fa3161a 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -2730,7 +2730,7 @@ int fill_schema_processlist(THD* thd, TABLE_LIST* tables, COND* cond) > TABLE *table= tables->table; > CHARSET_INFO *cs= system_charset_info; > char *user; > - my_hrtime_t unow= my_hrtime(); > + ulonglong unow= microsecond_interval_timer(); > DBUG_ENTER("fill_schema_processlist"); > > DEBUG_SYNC(thd,"fill_schema_processlist_after_unow"); > @@ -2793,9 +2793,12 @@ int fill_schema_processlist(THD* thd, TABLE_LIST* tables, COND* cond) > table->field[4]->store(command_name[tmp->get_command()].str, > command_name[tmp->get_command()].length, cs); > /* MYSQL_TIME */ > - ulonglong start_utime= tmp->start_time * HRTIME_RESOLUTION + tmp->start_time_sec_part; > - ulonglong utime= start_utime && start_utime < unow.val > - ? unow.val - start_utime : 0; > + ulonglong utime= tmp->start_utime; > + ulonglong utime_after_query_snapshot= tmp->utime_after_query; > + if (utime < utime_after_query_snapshot) > + utime= utime_after_query_snapshot; // COM_SLEEP > + utime= utime && utime < unow ? unow - utime : 0; 1. Please add an assert here that utime<=unow. It uses microsecond_interval_timer(), which is supposed to be strictly increasing (unlike my_hrtime()). 2. You might need to update mysqld_list_processes() too. > table->field[5]->store(utime / HRTIME_RESOLUTION, TRUE); > /* STATE */ Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 3d1b65b: MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
by Sergei Golubchik 18 Jun '15

18 Jun '15

Hi, Sergey! On Apr 28, svoj(a)mariadb.org wrote: > revision-id: 3d1b65b4923a49f9303501dd546af02982608b2b > parent(s): 8a0f3310275f7b4fa445f907140f677910e18999 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-28 15:20:48 +0400 > message: > > MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO > > Pass THD to find_all_keys() and Item_equal::Item_equal(). > In MRR use table->in_use instead of current_thd. > > This reduces number of pthread_getspecific() calls from 354 to 320. ok to push. although for find_all_keys() it doesn't make any sense, it's a big, complex, and expensive function. One current_thd doesn't change anything. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 2e08ec2: MDEV-7956 - handler::rebind_psi() takes 0.07% in OLTP RO
by Sergei Golubchik 18 Jun '15

18 Jun '15

Hi, Sergey! On Jun 17, svoj(a)mariadb.org wrote: > revision-id: 2e08ec24623e0b3d3db02333e16658dcfaa0535c > parent(s): b776f8d3a8afff55d5f8208da7bdfce9aa33eb39 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-17 13:09:22 +0400 > message: > > MDEV-7956 - handler::rebind_psi() takes 0.07% in OLTP RO > > Do not call handler::rebind_psi() and handler::unbind_psi() when performance > schema is compiled out. > > Overhead change: > handler::rebind_psi 0.04% -> out of radar > handler::unbind_psi 0.03% -> out of radar > open_table 0.21% -> 0.18% > close_thread_table 0.05% -> 0.05% Really? I don't believe it. handler::rebind_psi() and handler::unbind_psi() are empty methods, not doing anything. There's only a cost a virtual function call. And you're saying that it took 15% of the open_table() execution time? Regards, Sergei

2 1

[Maria-developers] [GSoC] MXS-2: Week 2 and 3
by sriram patil 17 Jun '15

17 Jun '15

Hi All, Please find link to my first blog post of second and third week below. <http://srirampatil.tumblr.com/post/121594591925/mxs-2-week-2-and-3> MXS-2: Week 2 and 3 <http://srirampatil.tumblr.com/post/121594591925/mxs-2-week-2-and-3> Feel free to point out mistakes, if any. Also, suggestions ans/or reviews are welcome! Thanks, Sriram

2 2

[Maria-developers] GSoC Progress Report [MDEV-5199]
by Blessing Atie 16 Jun '15

16 Jun '15

Hi Sergei, colin! This past week I have studied the sequence engine to understand how it generates tables on the fly. I also looked at how table functions work in other databases. I am currently working on how to implement the simplest table function from our new plugin API(Still to be completed) and which will do exacly what the sequence engine does but this time in a function call like in <select * from udf_seq(1,10)>. Exams are fast approaching in my University. I will acceletrate work as soon as I finish my Exams which will run from 22nd May to 6th June and after which I will be available to the fullest capacity to put enough time on my Work. In the mean while for this week I plan on completing our plugin declaration structure and also look for other candidate functions that can be implemented. Regards, Blessing Atie.

2 1

[Maria-developers] Sprint 10.0: MDEV-8205 timediff returns null when comparing decimal time to time string value
by Alexander Barkov 16 Jun '15

16 Jun '15

Hi Sergei, I'm not sure what to do with this: > MariaDB [test]> select > -> timediff('2014-01-01 00:00:00' , '2014-01-01 01:00:00' ), > -> timediff(20140101000000.000 , 20140101010000.000 ), > -> timediff(20140101000000.000 , '2014-01-01 01:00:00' ), > -> datediff('2014-01-01 00:00:00' , '2014-01-02 01:00:00' ), > -> datediff(20140101000000.000 , 20140102010000.000 ), > -> datediff(20140101000000.000 , '2014-01-02 01:00:00' )\G > *************************** 1. row *************************** > timediff('2014-01-01 00:00:00' , '2014-01-01 01:00:00' ): -01:00:00 > timediff(20140101000000.000 , 20140101010000.000 ): -01:00:00.000 > timediff(20140101000000.000 , '2014-01-01 01:00:00' ): NULL > datediff('2014-01-01 00:00:00' , '2014-01-02 01:00:00' ): -1 > datediff(20140101000000.000 , 20140102010000.000 ): -1 > datediff(20140101000000.000 , '2014-01-02 01:00:00' ): -1 > 1 row in set (0.03 sec) Notice, it returns NULL in one case. The problem is in this piece of the code: > if (args[0]->get_time(&l_time1) || > args[1]->get_time(&l_time2) || > l_time1.time_type != l_time2.time_type) > return (null_value= 1); get_time() forces decimal-to-time conversion to truncate the date part and return MYSQL_TIMESTAMP_TIME. get_time() does not force string-to-time conversion to truncate the date part. It still returns the full MYSQL_TIMESTAMP_DATETIME. I'm afraid a lot of the code rely in this behavior. If we fix string-to-time conversion, mtr will start to fail in other cases. Any ideas what to do?

2 4

[Maria-developers] MDEV-7502 Automatic provisioning of slave – retrieval of data from master
by Martin Kaluznik 16 Jun '15

16 Jun '15

Hello Kristian, after master-slave connection part, I proceeded to retrieval of data from master. There are two types of data which provisioning master will have to send, meta-data and row data. Meta-data For this I found only one method, pass output of SHOW CREATE * to slave in 'Query_log_event'. When I was looking for alternatives, I have found MySQL online backup work log where they are describing similar/same problem under High Level Architecture – Decision ( http://dev.mysql.com/worklog/task/?id=3574 ). Because of this, I find it unlikely, that there currently is better way. Because meta-data retrieval will take only small fraction of provisioning time, I think, that it could be implemented using 'Ed_connection' class, it executes query in string form and returns result set from within replication thread. Disadvantage of this solution is that query has to go through parser and all steps of generic execution, but on the other hand resulting code will be more independent from exact implementations, their pre-conditions and won’t contain any logical mistakes (locking, etc). Row data For this, same method with 'Ed_connection' could work, but it would be probably too big overhead – parser, result processing. So I created low level loop which goes through records of table and fills them in 'Write_rows_log_event'. Could you please check it, if it looks acceptable and I will then rewrite it to generic form, it is provisioning_send_info::send_table_data() function. https://github.com/f4rnham/server/blob/3d36cbfc20dff73d92ce61168dcc9805d68d… Right now it is working as expected on small test case. It goes through whole table at once and creates multiple row events when one would be enough. The same file also contains test of 'Ed_connection' and test of slave being able to receive and process 'Query_log_event'. Thanks for advice, Martin.

2 1

[Maria-developers] Getting the columns from LEX or THD
by sriram patil 16 Jun '15

16 Jun '15

Hi all, I wanted to retrieve a tables columns i.e its schema definition when a query is executed. I figured out LEX and THD structures store these in different variables for different queries. I wanted to get the same when a simple INSERT query is executed, e.g INSERT INTO <table> VALUES(<values>). Where can I find it? Is there any mechanism to get a tables schema that will work for any kind of query? Thanks, Sriram

2 2

Re: [Maria-developers] 8456b23: MDEV-5309 - RENAME TABLE does not check for existence of the table's engine
by Sergei Golubchik 16 Jun '15

16 Jun '15

Hi, Sergey! On Jun 15, svoj(a)mariadb.org wrote: > revision-id: 8456b231e9079be26c77da2a294ea1dac64d9d76 > parent(s): aad8667f8a4e2a26652494b29ee6c12824865ac8 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-15 15:49:00 +0400 > message: > > MDEV-5309 - RENAME TABLE does not check for existence of the table's engine > > When RENAME TABLE is executed, it apparently does not check whether the engine > is available (unlike ALTER TABLE .. RENAME, which does). It means that if the > engine in question was not loaded on some reason, the table might become > unusable, since the engine won't know about the change. > > With this patch RENAME TABLE fails if storage engine is not available. Ok to push. I'm not completely happy with the "table does not exist" error in this case, but I suspect that doing something reasonable might be not very easy. And it's a corner case anyway. Regards, Sergei

2 1

[Maria-developers] unsubscribe
by Abdul Hakeem 14 Jun '15

14 Jun '15

2 1

Re: [Maria-developers] 186e8e9: MDEV-5171: Add support for --innodb-optimize-keys to mysqldump.
by Sergei Golubchik 14 Jun '15

14 Jun '15

Hi, Jan! On May 04, Jan Lindström wrote: > revision-id: 186e8e91f3ce797258389f6d8729082f7d3ea164 > parent(s): aa5095627e2619bdad7916d33d1016802a84a9e1 > committer: Jan Lindström > branch nick: 10.0-git > timestamp: 2015-05-04 10:23:50 +0300 > message: > > MDEV-5171: Add support for --innodb-optimize-keys to mysqldump. > > Merged revision 93 from bzr+ssh://bazaar.launchpad.net/+branch/percona-server/5.6/ > authored by Alexey Kopytov. > > This patch expands the applicability of InnoDB fast index creation to > mysqldump, ALTER TABLE and OPTIMIZE TABLE as follows: > > 1. mysqldump has now a new option, --innodb-optimize-keys, which changes > the way InnoDB tables are dumped so that secondary and foreign keys are > created after loading the data thus taking advantage of fast index > creation. > > This part of the patch is an implementation of the feature request > reported as MySQL bug #49120. > > More specifically: > > - KEY, UNIQUE KEY and CONSTRAINT specifications are omitted from CREATE > TABLE corresponding to InnoDB tables. > > - an additional ALTER TABLE is issued after dumping the data to create > the previously omitted keys. > > Delaying foreign key creation does not introduce any additional risks as > mysqldump always prepends its output with SET FOREIGN_KEY_CHECKS=0 anyway. > > 2. When ALTER TABLE requires a table copy, secondary keys are now dropped > and recreated later after copying the data. The following restrictions > apply: > > - only non-unique keys can be involved in this optimization > > - if the table contains foreign keys, or a foreign key is being added as > a part of the current ALTER TABLE statement, the optimization is > disabled for all keys. > > This part of the patch is an implementation of the feature request > reported as MySQL bug #57583. > > 3. As OPTIMIZE TABLE is mapped to ALTER TABLE ... ENGINE=InnoDB for > InnoDB tables, it now also benefits from fast index creation with the > same restrictions as for ALTER TABLE. > diff --git a/client/client_priv.h b/client/client_priv.h > index 67a6e82..1846926 100644 > --- a/client/client_priv.h > +++ b/client/client_priv.h > @@ -94,6 +94,7 @@ enum options_client > OPT_SSL_CRL, OPT_SSL_CRLPATH, > OPT_USE_GTID, > OPT_GALERA_SST_MODE, > + OPT_INNODB_OPTIMIZE_KEYS, /* mysqldump */ Not needed, please, remove > OPT_MAX_CLIENT_OPTION /* should be always the last */ > }; > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index e118198..0970082 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -375,6 +386,11 @@ static struct my_option my_long_options[] = > "in dump produced with --dump-slave.", &opt_include_master_host_port, > &opt_include_master_host_port, 0, GET_BOOL, NO_ARG, > 0, 0, 0, 0, 0, 0}, > + {"innodb-optimize-keys", OPT_INNODB_OPTIMIZE_KEYS, Here use 0 instead of OPT_INNODB_OPTIMIZE_KEYS > + "Use InnoDB fast index creation by creating secondary indexes after " > + "dumping the data.", > + &opt_innodb_optimize_keys, &opt_innodb_optimize_keys, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, > {"insert-ignore", OPT_INSERT_IGNORE, "Insert rows with INSERT IGNORE.", > &opt_ignore, &opt_ignore, 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, > 0, 0}, > @@ -2581,6 +2597,359 @@ static inline my_bool general_log_or_slow_log_tables(const char *db, > } > > /* > + Find the first occurrence of a quoted identifier in a given string. Returns > + the pointer to the opening quote, and stores the pointer to the closing quote > + to the memory location pointed to by the 'end' argument, > + > + If no quoted identifiers are found, returns NULL (and the value pointed to by > + 'end' is undefined in this case). > +*/ > + > +static const char *parse_quoted_identifier(const char *str, > + const char **end) > +{ > + const char *from; > + const char *to; > + > + if (!(from= strchr(str, '`'))) > + return NULL; This is wrong, see below. Identifiers ar enot necessarily quoted, so this function should parse both quoted and not quoted identifiers (and renamed to, say, parse_next_identifier) > + > + to= from; > + > + while ((to= strchr(to + 1, '`'))) { > + /* > + Double backticks represent a backtick in identifier, rather than a quote > + character. > + */ > + if (to[1] == '`') > + { > + to++; > + continue; > + } > + > + break; > + } > + > + if (to <= from + 1) > + return NULL; /* Empty identifier */ > + > + *end= to; > + > + return from; > +} > + > +/* > + Parse the specified key definition string and check if the key contains an > + AUTO_INCREMENT column as the first key part. We only check for the first key > + part, because unlike MyISAM, InnoDB does not allow the AUTO_INCREMENT column > + as a secondary key column, i.e. the AUTO_INCREMENT column would not be > + considered indexed for such key specification. > +*/ > +static my_bool contains_autoinc_column(const char *autoinc_column, > + const char *keydef, > + key_type_t type) > +{ > + const char *from, *to; > + uint idnum; > + > + DBUG_ASSERT(type != KEY_TYPE_NONE); > + > + if (autoinc_column == NULL) > + return FALSE; > + > + idnum= 0; > + > + /* > + There is only 1 iteration of the following loop for type == KEY_TYPE_PRIMARY > + and 2 iterations for type == KEY_TYPE_UNIQUE / KEY_TYPE_NON_UNIQUE. > + */ > + while ((from= parse_quoted_identifier(keydef, &to))) > + { > + idnum++; > + > + /* > + Skip the check if it's the first identifier and we are processing a > + secondary key. > + */ > + if ((type == KEY_TYPE_PRIMARY || idnum != 1) && > + !strncmp(autoinc_column, from + 1, to - from - 1)) > + return TRUE; > + > + /* > + Check only the first (for PRIMARY KEY) or the second (for secondary keys) > + quoted identifier. > + */ > + if ((idnum == 1 + MY_TEST(type != KEY_TYPE_PRIMARY))) > + break; > + > + keydef= to + 1; > + } > + > + return FALSE; > +} > + > +/* > + Find a node in the skipped keys list whose name matches a quoted > + identifier specified as 'id_from' and 'id_to' arguments. > +*/ > + > +static LIST *find_matching_skipped_key(const char *id_from, > + const char *id_to) > +{ > + LIST *list; > + size_t id_len; > + > + id_len= id_to - id_from + 1; > + DBUG_ASSERT(id_len > 2); > + > + for (list= skipped_keys_list; list; list= list_rest(list)) > + { > + const char *keydef; > + const char *keyname_from; > + const char *keyname_to; > + size_t keyname_len; > + > + keydef= list->data; > + > + if ((keyname_from= parse_quoted_identifier(keydef, &keyname_to))) > + { > + keyname_len= keyname_to - keyname_from + 1; > + > + if (id_len == keyname_len && > + !strncmp(keyname_from, id_from, id_len)) > + return list; > + } > + } > + > + return NULL; > +} > + > +/* > + Remove secondary/foreign key definitions from a given SHOW CREATE TABLE string > + and store them into a temporary list to be used later. > + > + SYNOPSIS > + skip_secondary_keys() > + create_str SHOW CREATE TABLE output > + has_pk TRUE, if the table has PRIMARY KEY > + (or UNIQUE key on non-nullable columns) > + > + > + DESCRIPTION > + > + Stores all lines starting with "KEY" or "UNIQUE KEY" > + into skipped_keys_list and removes them from the input string. > + Ignoring FOREIGN KEYS constraints when creating the table is ok, because > + mysqldump sets foreign_key_checks to 0 anyway. > +*/ > + > +static void skip_secondary_keys(char *create_str, my_bool has_pk) > +{ > + char *ptr, *strend; > + char *last_comma= NULL; > + my_bool pk_processed= FALSE; > + char *autoinc_column= NULL; > + my_bool has_autoinc= FALSE; > + key_type_t type; > + const char *constr_from; > + const char *constr_to; > + LIST *keydef_node; > + my_bool keys_processed= FALSE; > + > + strend= create_str + strlen(create_str); > + > + ptr= create_str; > + while (*ptr && !keys_processed) > + { please, fix the coding style (here and everywhere in the patch) to follow the rest of the file. That is while { char *tmp, *orig_ptr, c; ... > + char *tmp, *orig_ptr, c; > + > + orig_ptr= ptr; > + /* Skip leading whitespace */ > + while (*ptr && my_isspace(charset_info, *ptr)) > + ptr++; > + > + /* Read the next line */ > + for (tmp= ptr; *tmp != '\n' && *tmp != '\0'; tmp++); > + > + c= *tmp; > + *tmp= '\0'; /* so strstr() only processes the current line */ > + > + if (!strncmp(ptr, "CONSTRAINT ", sizeof("CONSTRAINT ") - 1) && > + (constr_from= parse_quoted_identifier(ptr, &constr_to)) && yuck! identifiers do not have to be quoted. this code needs to be fixed to accept non-quoted identifiers too. > + (keydef_node= find_matching_skipped_key(constr_from, constr_to))) > + { > + char *keydef; > + size_t keydef_len; > + > + /* > + There's a skipped key with the same name as the constraint name. Let's > + put it back before the current constraint definition and remove from the > + skipped keys list. 1. fix the formatting, please 2. why not to create the FK constraint afterwards? > + */ > + keydef= keydef_node->data; > + keydef_len= strlen(keydef) + 5; /* ", \n " */ > + > + memmove(orig_ptr + keydef_len, orig_ptr, strend - orig_ptr + 1); > + memcpy(ptr, keydef, keydef_len - 5); > + memcpy(ptr + keydef_len - 5, ", \n ", 5); > + > + skipped_keys_list= list_delete(skipped_keys_list, keydef_node); > + my_free(keydef); > + my_free(keydef_node); > + > + strend+= keydef_len; > + orig_ptr+= keydef_len; > + ptr+= keydef_len; > + tmp+= keydef_len; > + > + type= KEY_TYPE_NONE; > + } > + else if (!strncmp(ptr, "UNIQUE KEY ", sizeof("UNIQUE KEY ") - 1)) > + type= KEY_TYPE_UNIQUE; > + else if (!strncmp(ptr, "KEY ", sizeof("KEY ") - 1)) > + type= KEY_TYPE_NON_UNIQUE; > + else if (!strncmp(ptr, "PRIMARY KEY ", sizeof("PRIMARY KEY ") - 1)) > + type= KEY_TYPE_PRIMARY; > + else > + type= KEY_TYPE_NONE; > + > + has_autoinc= (type != KEY_TYPE_NONE) ? > + contains_autoinc_column(autoinc_column, ptr, type) : FALSE; > + > + /* Is it a secondary index definition? */ > + if (c == '\n' && > + ((type == KEY_TYPE_UNIQUE && (pk_processed || !has_pk)) || > + type == KEY_TYPE_NON_UNIQUE) && !has_autoinc) > + { > + char *data, *end= tmp - 1; > + > + /* Remove the trailing comma */ > + if (*end == ',') > + end--; > + data= my_strndup(ptr, end - ptr + 1, MYF(MY_FAE)); > + skipped_keys_list= list_cons(data, skipped_keys_list); > + > + memmove(orig_ptr, tmp + 1, strend - tmp); > + ptr= orig_ptr; > + strend-= tmp + 1 - ptr; > + > + /* Remove the comma on the previos line */ > + if (last_comma != NULL) > + { > + *last_comma= ' '; > + } > + } > + else > + { > + char *end; > + > + if (last_comma != NULL && *ptr == ')') > + { > + keys_processed= TRUE; > + } > + else if (last_comma != NULL && !keys_processed) > + { > + /* > + It's not the last line of CREATE TABLE, so we have skipped a key > + definition. We have to restore the last removed comma. > + */ > + *last_comma= ','; > + } > + > + /* > + If we are skipping a key which indexes an AUTO_INCREMENT column, it is > + safe to optimize all subsequent keys, i.e. we should not be checking for > + that column anymore. > + */ > + if (type != KEY_TYPE_NONE && has_autoinc) > + { > + DBUG_ASSERT(autoinc_column != NULL); > + > + my_free(autoinc_column); > + autoinc_column= NULL; > + } > + > + if ((has_pk && type == KEY_TYPE_UNIQUE && !pk_processed) || > + type == KEY_TYPE_PRIMARY) > + pk_processed= TRUE; > + > + if (strstr(ptr, "AUTO_INCREMENT") && *ptr == '`') > + { > + /* > + The first secondary key defined on this column later cannot be > + skipped, as CREATE TABLE would fail on import. Unless there is a > + PRIMARY KEY and it indexes that column. > + */ > + for (end= ptr + 1; > + /* Skip double backticks as they are a part of identifier */ > + *end != '\0' && (*end != '`' || end[1] == '`'); > + end++) > + /* empty */; > + > + if (*end == '`' && end > ptr + 1) > + { > + DBUG_ASSERT(autoinc_column == NULL); > + > + autoinc_column= my_strndup(ptr + 1, end - ptr - 1, MYF(MY_FAE)); Eh? Why does it need to strdup? > + } > + } > + > + *tmp= c; > + > + if (tmp[-1] == ',') > + last_comma= tmp - 1; > + ptr= (*tmp == '\0') ? tmp : tmp + 1; > + } > + } > + > + my_free(autoinc_column); > +} > + > +/* > + Check if the table has a primary key defined either explicitly or > + implicitly (i.e. a unique key on non-nullable columns). > + > + SYNOPSIS > + my_bool has_primary_key(const char *table_name) > + > + table_name quoted table name > + > + RETURNS TRUE if the table has a primary key > + > + DESCRIPTION > +*/ > + > +static my_bool has_primary_key(const char *table_name) > +{ > + MYSQL_RES *res= NULL; > + MYSQL_ROW row; > + char query_buff[QUERY_LENGTH]; > + my_bool has_pk= TRUE; > + > + my_snprintf(query_buff, sizeof(query_buff), > + "SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS WHERE " > + "TABLE_SCHEMA=DATABASE() AND TABLE_NAME='%s' AND " > + "COLUMN_KEY='PRI'", table_name); > + if (mysql_query(mysql, query_buff) || !(res= mysql_store_result(mysql)) || > + !(row= mysql_fetch_row(res))) > + { > + fprintf(stderr, "Warning: Couldn't determine if table %s has a " > + "primary key (%s). " > + "--innodb-optimize-keys may work inefficiently.\n", > + table_name, mysql_error(mysql)); > + goto cleanup; > + } > + > + has_pk= atoi(row[0]) > 0; > + > + cleanup: > + if (res) > + mysql_free_result(res); > + > + return has_pk; > +} > + > +/* > get_table_structure -- retrievs database structure, prints out corresponding > CREATE statement and fills out insert_pat if the table is the type we will > be dumping. > @@ -2659,6 +3029,9 @@ static uint get_table_structure(char *table, char *db, char *table_type, > result_table= quote_name(table, table_buff, 1); > opt_quoted_table= quote_name(table, table_buff2, 0); > > + if (opt_innodb_optimize_keys && !strcmp(table_type, "InnoDB")) > + has_pk= has_primary_key(table); > + see below > if (opt_order_by_primary) > order_by= primary_key_fields(result_table); > > @@ -2838,6 +3211,9 @@ static uint get_table_structure(char *table, char *db, char *table_type, > > row= mysql_fetch_row(result); > > + if (opt_innodb_optimize_keys && !strcmp(table_type, "InnoDB")) > + skip_secondary_keys(row[1], has_pk); > + Not that it makes a lot of difference, but I'd remove the previous hunk, the variable has_pk, and put everything here, as if (opt_innodb_optimize_keys && !strcmp(table_type, "InnoDB")) skip_secondary_keys(row[1], has_primary_key(table)); this removes one if(), one strcmp(), and doesn't invoke has_primary_key() (meaning, one SQL query less) unless all other conditions for skip_secondary_keys() are met. even better - don't do has_primary_key() at all, remove the function altogether, and look for "PRIMARY KEY" in the create string inside skip_secondary_keys(). > is_log_table= general_log_or_slow_log_tables(db, table); > if (is_log_table) > row[1]+= 13; /* strlen("CREATE TABLE ")= 13 */ Regards, Sergei

1 0

Re: [Maria-developers] Features Wish-list in MariaDB 10.1.
by Kristian Nielsen 12 Jun '15

12 Jun '15

So my opinion on these below two points was requested, let's keep it on the public mailing list: > * Crash-Safe replication with filename and position (not depending on > GTIDs) > Something similar as what is described in the following link would > be good: > http://blog.booking.com/better_crash_safe_replication_for_mysql.html > * Have a way to avoid "SELECT binlog_gtid_pos" when a slave connect to > the master in MASTER_USE_GTID = no So my personal opinion on this is that I prefer to focus on improving the MariaDB GTID implementation. Considerable effort was invested into this, and the idea is that future development will focus on GTID mode, keeping non-GTID mode only as a backward compatibility. Significant effort was made to ensure that MariaDB GTID is usable for all existing use-cases of replication. For the first point, GTID replication is already crash-safe. For the second point, for GTID we in any case need a solution for locating GTID position quickly, otherwise GTID replication can have performance problems in case of frequent slave connects (and Jonas Oreland has promised a patch for this any day now ;-). And once this solution is in, the second point should be moot anyway. Of course, others may have a different opinion. For the second case, I suppose the easiest thing is just to remove the code that does SELECT binlog_gtid_pos() from the slave, if no-one cares about that functionality anyway... Hope this helps, - Kristian.

2 2

[Maria-developers] 10.0 sprint: Please advise re MDEV-8109 unexpected CAST result
by Alexander Barkov 11 Jun '15

11 Jun '15

Hi Sergei, I'm thinking about what to do with MDEV-8109 unexpected CAST result. Observations: 1. An empty string in VALUES SET sql_mode='STRICT_ALL_TABLES'; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (a DECIMAL); INSERT INTO t1 VALUES (''); ERROR 1292 (22007): Truncated incorrect DECIMAL value: '' This is OK. In strict mode INSERT with a bad value correctly returns the error. 2. CAST('' AS DECIMAL) SET sql_mode='STRICT_ALL_TABLES'; SELECT CAST('' AS DECIMAL); +---------------------+ | CAST('' AS DECIMAL) | +---------------------+ | 0 | +---------------------+ 1 row in set, 1 warning (0.00 sec) This is also OK. There are no tables involved, and CAST itself does not convert warnings to errors. 3. CAST('' AS DECIMAL) in VALUES: INSERT INTO t1 VALUES(CAST('' AS DECIMAL)); ERROR 1292 (22007): Truncated incorrect DECIMAL value: '' I think this is not OK. I wrote an explicit CAST, so I expect: - CAST to return 0 with a warning, like in #2 - INSERT to write 0 into the table normally, as I'm actually inserting the CAST result, which is 0. 4. COLUMN_GET(AS DECIMAL) in VALUES: SET @aaa = COLUMN_CREATE('price', ''); INSERT INTO t1 VALUES(COLUMN_GET(@aaa, 'price' AS DECIMAL)); ERROR 1918 (22007): Encountered illegal value '' when converting to DECIMAL This is also not OK. I wrote explicit column conversion to DECIMAL. I expect: - COLUMN_GET() to return 0 with a warning, like in #2 - INSERT to write 0 into the table normally, as I'm actually inserting the COLUMN_GET result, which is 0. Do you agree? Thanks.

2 4

Re: [Maria-developers] MariaDB replication event generators?
by Kristian Nielsen 11 Jun '15

11 Jun '15

James Cheng <jcheng(a)tivo.com> writes: > The binlog stream has all the row data, but doesn't have > everything. It doesn't have column names, for example, or primary > keys, or character sets. Using the binlog for replication seems to > assume that you have a copy of the schema beforehand on the receiving > slave. It doesn't seem possible to replicate to a slave using only the > information in the binlogs. It is a common request to be able to do 3rd-party replication from mysqld. However, the replication protocol between master and slave servers was never designed as a public API, it is based on internal data formats and so on. So unfortunately, there does not exist currently a public API for doing external replication, at least not that I am aware of. The binlog stream _does_ have everything of course, eg. in the form of CREATE TABLE statements and so on. However, being internal to the server, they are not very easy to use by an external replication mechanism. - Kristian.

1 0

Re: [Maria-developers] [Commits] 6ff2c0a: MDEV-5309 - RENAME TABLE does not check for existence of the table's engine
by Sergei Golubchik 10 Jun '15

10 Jun '15

Hi, Sergey! On Feb 09, svoj(a)mariadb.org wrote: > revision-id: 6ff2c0a05155f985484e23bd8611eb95f045907f > parent(s): f13939061d468a47985dee0268652a6fe2db7862 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-02-09 16:12:00 +0400 > message: > > MDEV-5309 - RENAME TABLE does not check for existence of the table's engine > > When RENAME TABLE is executed, it apparently does not check whether the engine > is available (unlike ALTER TABLE .. RENAME, which does). It means that if the > engine in question was not loaded on some reason, the table might become > unusable, since the engine won't know about the change. > > With this patch RENAME TABLE fails if storage engine is not available. .... > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index e9a1ae9..f1096da 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -5172,18 +5172,19 @@ bool mysql_create_table(THD *thd, TABLE_LIST *create_table, > error= my_errno; > (void) file->ha_create_partitioning_metadata(to, from, CHF_RENAME_FLAG); > } > - else if (!file || !(error=file->ha_rename_table(from_base, to_base))) > + else if (!file) > { > + error= ER_UNKNOWN_STORAGE_ENGINE; > + } > + else if (!(error= file->ha_rename_table(from_base, to_base))) > + { > if (!(flags & NO_FRM_RENAME) && rename_file_ext(from,to,reg_ext)) > { > error= my_errno; > - if (file) > - { > if (error == ENOENT) > error= 0; // this is ok if file->ha_rename_table() succeeded > else > file->ha_rename_table(to_base, from_base); // Restore old file name > - } > } > } > delete file; May be you'd rather fix it in the caller? Let's state that the first argument of mysql_rename_table() can never be NULL. While mysql_rename_table() is called in many places, the only place where it can actually get a NULL hton is sql_rename.cc, if ha_table_exists() returns hton=NULL. I'd add the check here and simplified mysql_rename_table(). Regards, Sergei

1 0

[Maria-developers] Progress Report on [MDEV5199]: UDF Table Functions.
by Blessing Atie 10 Jun '15

10 Jun '15

Hi Serg, Colin! The following is a rundown on my progress with the above task. I started this project by getting the source of 10.1 from Github, setting my working environment by building and running the tests on ubuntu 14 then proceeded with an analysis by studying plugins and how they interact with the server.(Especially storage engines plugins). With analysis and sources investigations being done, I realized with help from assigned mentor(Serg) that our task is about implementing MYSQL_UDF_PLUGIN with the functionalities of the CONNECT engine which is similar in functionality to table functions in a way. This past week, I worked on the interface for our plugin by declaring a structure with version and some useful init/deinit function pointers to work with our UDFs(table UDF only for now). I am currently studying the Connect engine to see how I can implement its functionalities as a UDF plugin. For this week, I intend to look at other databases for example table functions that will be good candidates that we can implement with our plugin(This was not mentioned in my proposal but I intend to implement a couple of them using our plugin even if we are short of time after the final evaluation). So far, I have had difficulties in communication due to unscheduled powerouts in my city. This has really been an issue to me because my mentor was beginning to loose hope and on the other side, I don't have the chance to ask questions when im faced with a powerout. I really hope my mentors understand with me how difficult it is to code from Cameroon. Please feel free to request for any important details i might have not mentioned which you will want to know. Best Wishes, Blessing Atie

2 1

Re: [Maria-developers] MariaDB replication event generators?
by Kristian Nielsen 10 Jun '15

10 Jun '15

James Cheng <jcheng(a)tivo.com> writes: > On Jun 7, 2015, at 12:05 AM, Kristian Nielsen <knielsen(a)knielsen-hq.org> wrote: > >> James Cheng <jcheng(a)tivo.com> writes: >> >>> I recently read your old blog post from 2010 >>> (http://kristiannielsen.livejournal.com/14305.html) about some ideas >>> for MariaDB replication. In particular, your ideas for replication >>> event generators that would allow plugins to be written to receive >>> replication events. >>> >>> Did this work ever happen in MariaDB? Or anything close/equivalent? If >>> so, could you point me to something so that I can learn more about it? >> >> Unfortunately, nothing like this ever happened. > That's too bad. Are there any interesting details about why it never > happened? Was it simply due to not enough time, not high enough > priority? Was there any technical or organizational resistance to the > actual ideas or functionality? If someone were to build this today and > contribute it back to the community, do you think it would be > accepted? I think there was not enough interest and/or time/priority. For example, Galera cluster would be a prime example for something using this kind of API. But the developers did not prioritise spending time working on such API, rather they hacked in their functionality all over the server. It is the same with other parties in the community, everyone is focused on solving their own narrow problem, not in working together. And MySQL@Oracle, of course, is not interested in anything that involves working with anyone else on the code. So I don't see much chance in the current state of the community of any significant effort being spent neither on developing such an API, nor on using it. - Kristian.

1 0

Re: [Maria-developers] [Commits] 865bbbf: MDEV-3870 - Valgrind warnings on OPTIMIZE MyISAM or Aria TABLE with disabled
by Sergei Golubchik 09 Jun '15

09 Jun '15

Hi, Sergey! On Jun 09, svoj(a)mariadb.org wrote: > revision-id: 865bbbf14d1a8ed500eeeadabe0a7fdfa90420df > parent(s): b8e26b04d4ac22e29a78bca68e101176b6ef507a > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-09 16:08:09 +0400 > message: > > MDEV-3870 - Valgrind warnings on OPTIMIZE MyISAM or Aria TABLE with disabled > keys > > Fixed that OPTIMIZE TABLE against MyISAM/Aria table may write uninitialized key > root position for disabled keys. ok to push, thanks! Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 7948cc3: MDEV-8260 : Issues related to concurrent CTAS
by Sergei Golubchik 09 Jun '15

09 Jun '15

Hi, Nirbhay! On Jun 08, Nirbhay Choubey wrote: > revision-id: 7948cc32b6c12c9696e8b497ae90f123cafcd234 > parent(s): 6d5b723bdc3e04978619b9673fca266e0426916f > committer: Nirbhay Choubey > branch nick: 10.0-galera.ctas > timestamp: 2015-06-08 22:50:26 -0400 > message: > > MDEV-8260 : Issues related to concurrent CTAS > > * Wait for aborted thd (victim) to release MDL locks > * Skip aborting an already aborted thd > * Defer setting OK status in case of CTAS > * Minor cosmetic changes > * Added a test case ... > +bool select_insert::send_eof() > +{ > + DBUG_ENTER("select_insert::send_eof"); > + DBUG_RETURN(prepare_eof() || send_ok_packet()); > } Please, don't do that, don't put function calls in DBUG_RETURN. This will result in dbug traces like > select_insert::send_eof < select_insert::send_eof > select_insert::prepare_eof < select_insert::prepare_eof > select_insert::send_ok_packet < select_insert::send_ok_packet while the correct trace should've be > select_insert::send_eof | > select_insert::prepare_eof | < select_insert::prepare_eof | > select_insert::send_ok_packet | < select_insert::send_ok_packet < select_insert::send_eof So, either write like bool res= prepare_eof() || send_ok_packet(); DBUG_RETURN(res); or fix DBUG_RETURN() macro. Whatever you prefer :) Otherwise the patch is ok. Regards, Sergei

2 1

[Maria-developers] Review requested for MDEV-7818
by Kristian Nielsen 09 Jun '15

09 Jun '15

I need someone to review my below fix for MDEV-7818. This involves the FLUSH TABLES WITH READ LOCK command and associated metadata locking, something that I know little about. Monty suggested the basic semantics - that all currently running parallel replication transactions would be allowed to complete before granting the global read lock. But it really needs a review of the actual implementation, by someone who has an understanding of the FTWRL semantics and implementation. Thanks, - Kristian. Kristian Nielsen <knielsen(a)knielsen-hq.org> writes: > revision-id: 0e609daea4796fafca94f7adf796addfc506168b > parent(s): 22b6c297dc0de6b176238ae820bddb5e70f2c8ab > committer: Kristian Nielsen > branch nick: mariadb > timestamp: 2015-06-09 16:01:53 +0200 > message: > > MDEV-7818: Deadlock occurring with parallel replication and FTWRL > > Problem is that FLUSH TABLES WITH READ LOCK first blocks threads from > starting new commits, then waits for running commits to complete. But > in-order parallel replication needs commits to happen in a particular > order, so this can easily deadlock. > > To fix this problem, this patch introduces a way to temporarily pause > the parallel replication worker threads. Before starting FTWRL, we let > all worker threads complete in-progress transactions, and then > wait. Then we proceed to take the global read lock. Once the lock is > obtained, we unpause the worker threads. Now commits are blocked from > starting by the global read lock, so the deadlock will no longer occur. > > --- > sql/mysqld.cc | 3 + > sql/mysqld.h | 3 + > sql/rpl_parallel.cc | 230 ++++++++++++++++++++++++++++++++++++++++++++++++++-- > sql/rpl_parallel.h | 27 ++++-- > sql/sql_parse.cc | 13 +++ > 5 files changed, 262 insertions(+), 14 deletions(-) > > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index e05c0b6..af8ae2c 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -9514,6 +9514,9 @@ PSI_stage_info stage_waiting_for_prior_transaction_to_commit= { 0, "Waiting for > PSI_stage_info stage_waiting_for_prior_transaction_to_start_commit= { 0, "Waiting for prior transaction to start commit before starting next transaction", 0}; > PSI_stage_info stage_waiting_for_room_in_worker_thread= { 0, "Waiting for room in worker thread event queue", 0}; > PSI_stage_info stage_waiting_for_workers_idle= { 0, "Waiting for worker threads to be idle", 0}; > +PSI_stage_info stage_waiting_for_ftwrl= { 0, "Waiting due to global read lock", 0}; > +PSI_stage_info stage_waiting_for_ftwrl_threads_to_pause= { 0, "Waiting for worker threads to pause for global read lock", 0}; > +PSI_stage_info stage_waiting_for_rpl_thread_pool= { 0, "Waiting while replication worker thread pool is busy", 0}; > PSI_stage_info stage_master_gtid_wait_primary= { 0, "Waiting in MASTER_GTID_WAIT() (primary waiter)", 0}; > PSI_stage_info stage_master_gtid_wait= { 0, "Waiting in MASTER_GTID_WAIT()", 0}; > PSI_stage_info stage_gtid_wait_other_connection= { 0, "Waiting for other master connection to process GTID received on multiple master connections", 0}; > diff --git a/sql/mysqld.h b/sql/mysqld.h > index 156e7f9..8c953c1 100644 > --- a/sql/mysqld.h > +++ b/sql/mysqld.h > @@ -454,6 +454,9 @@ extern PSI_stage_info stage_waiting_for_prior_transaction_to_commit; > extern PSI_stage_info stage_waiting_for_prior_transaction_to_start_commit; > extern PSI_stage_info stage_waiting_for_room_in_worker_thread; > extern PSI_stage_info stage_waiting_for_workers_idle; > +extern PSI_stage_info stage_waiting_for_ftwrl; > +extern PSI_stage_info stage_waiting_for_ftwrl_threads_to_pause; > +extern PSI_stage_info stage_waiting_for_rpl_thread_pool; > extern PSI_stage_info stage_master_gtid_wait_primary; > extern PSI_stage_info stage_master_gtid_wait; > extern PSI_stage_info stage_gtid_wait_other_connection; > diff --git a/sql/rpl_parallel.cc b/sql/rpl_parallel.cc > index 738d0e5..d90507a 100644 > --- a/sql/rpl_parallel.cc > +++ b/sql/rpl_parallel.cc > @@ -280,6 +280,8 @@ do_gco_wait(rpl_group_info *rgi, group_commit_orderer *gco, > rpl_parallel_entry *entry= rgi->parallel_entry; > uint64 wait_count; > > + mysql_mutex_assert_owner(&entry->LOCK_parallel_entry); > + > if (!gco->installed) > { > group_commit_orderer *prev_gco= gco->prev_gco; > @@ -336,6 +338,159 @@ do_gco_wait(rpl_group_info *rgi, group_commit_orderer *gco, > } > > > +static void > +do_ftwrl_wait(rpl_group_info *rgi, > + bool *did_enter_cond, PSI_stage_info *old_stage) > +{ > + THD *thd= rgi->thd; > + rpl_parallel_entry *entry= rgi->parallel_entry; > + uint64 sub_id= rgi->gtid_sub_id; > + > + mysql_mutex_assert_owner(&entry->LOCK_parallel_entry); > + > + if (unlikely(entry->pause_sub_id > 0) && sub_id > entry->pause_sub_id) > + { > + thd->ENTER_COND(&entry->COND_parallel_entry, &entry->LOCK_parallel_entry, > + &stage_waiting_for_ftwrl, old_stage); > + *did_enter_cond= true; > + do > + { > + if (entry->force_abort || rgi->worker_error) > + break; > + if (thd->check_killed()) > + { > + thd->send_kill_message(); > + slave_output_error_info(rgi, thd); > + signal_error_to_sql_driver_thread(thd, rgi, 1); > + break; > + } > + mysql_cond_wait(&entry->COND_parallel_entry, &entry->LOCK_parallel_entry); > + } while (entry->pause_sub_id > 0 && sub_id > entry->pause_sub_id); > + } > + > + if (sub_id > entry->largest_started_sub_id) > + entry->largest_started_sub_id= sub_id; > +} > + > + > +static void > +pool_mark_busy(rpl_parallel_thread_pool *pool) > +{ > + mysql_mutex_assert_owner(&pool->LOCK_rpl_thread_pool); > + DBUG_ASSERT(!pool->busy); > + pool->busy= true; > +} > + > + > +static void > +pool_mark_not_busy(rpl_parallel_thread_pool *pool) > +{ > + mysql_mutex_assert_owner(&pool->LOCK_rpl_thread_pool); > + DBUG_ASSERT(pool->busy); > + pool->busy= false; > + mysql_cond_broadcast(&pool->COND_rpl_thread_pool); > +} > + > + > +void > +rpl_unpause_for_ftwrl(THD *thd) > +{ > + uint32 i; > + rpl_parallel_thread_pool *pool= &global_rpl_thread_pool; > + > + DBUG_ASSERT(pool->busy); > + > + for (i= 0; i < pool->count; ++i) > + { > + rpl_parallel_entry *e; > + rpl_parallel_thread *rpt= pool->threads[i]; > + > + mysql_mutex_lock(&rpt->LOCK_rpl_thread); > + if (!rpt->current_owner) > + { > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + continue; > + } > + e= rpt->current_entry; > + mysql_mutex_lock(&e->LOCK_parallel_entry); > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + e->pause_sub_id= 0; > + mysql_cond_broadcast(&e->COND_parallel_entry); > + } > + > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + pool_mark_not_busy(pool); > + mysql_cond_broadcast(&pool->COND_rpl_thread_pool); > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > +} > + > + > +/* > + . > + > + Note: in case of error return, unpause_for_ftwrl() must _not_ be called. > +*/ > +int > +rpl_pause_for_ftwrl(THD *thd) > +{ > + uint32 i; > + rpl_parallel_thread_pool *pool= &global_rpl_thread_pool; > + int err= 0; > + > + /* > + While the count_pending_pause_for_ftwrl counter is non-zero, the pool > + cannot be shutdown/resized, so threads are guaranteed to not disappear. > + > + This is required to safely be able to access the individual threads below. > + (We cannot lock an individual thread while holding LOCK_rpl_thread_pool, > + as this can deadlock against release_thread()). > + */ > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + pool_mark_busy(pool); > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > + > + for (i= 0; i < pool->count; ++i) > + { > + PSI_stage_info old_stage; > + rpl_parallel_entry *e; > + rpl_parallel_thread *rpt= pool->threads[i]; > + > + mysql_mutex_lock(&rpt->LOCK_rpl_thread); > + if (!rpt->current_owner) > + { > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + continue; > + } > + e= rpt->current_entry; > + mysql_mutex_lock(&e->LOCK_parallel_entry); > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + ++e->need_sub_id_signal; > + if (!e->pause_sub_id) > + e->pause_sub_id= e->largest_started_sub_id; > + thd->ENTER_COND(&e->COND_parallel_entry, &e->LOCK_parallel_entry, > + &stage_waiting_for_ftwrl_threads_to_pause, &old_stage); > + while (e->last_committed_sub_id < e->pause_sub_id && !err) > + { > + if (thd->check_killed()) > + { > + thd->send_kill_message(); > + err= 1; > + break; > + } > + mysql_cond_wait(&e->COND_parallel_entry, &e->LOCK_parallel_entry); > + }; > + --e->need_sub_id_signal; > + thd->EXIT_COND(&old_stage); > + if (err) > + break; > + } > + > + if (err) > + rpl_unpause_for_ftwrl(thd); > + return err; > +} > + > + > #ifndef DBUG_OFF > static int > dbug_simulate_tmp_error(rpl_group_info *rgi, THD *thd) > @@ -856,6 +1011,9 @@ handle_rpl_parallel_thread(void *arg) > > if (unlikely(entry->stop_on_error_sub_id <= rgi->wait_commit_sub_id)) > skip_event_group= true; > + if (likely(!skip_event_group)) > + do_ftwrl_wait(rgi, &did_enter_cond, &old_stage); > + > register_wait_for_prior_event_group_commit(rgi, entry); > > unlock_or_exit_cond(thd, &entry->LOCK_parallel_entry, > @@ -1058,6 +1216,47 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > rpl_parallel_thread **new_list= NULL; > rpl_parallel_thread *new_free_list= NULL; > rpl_parallel_thread *rpt_array= NULL; > + THD *thd; > + PSI_stage_info old_stage; > + int res; > + > + /* > + Wait here while the queue is busy. This is done to make FLUSH TABLES WITH > + READ LOCK work correctly, without incuring extra locking penalties in > + normal operation. FLUSH TABLES WITH READ LOCK needs to lock threads in the > + thread pool, and for this we need to make sure the pool will not go away > + during the operation. The LOCK_rpl_thread_pool is not suitable for > + this. It is taken by release_thread() while holding LOCK_rpl_thread; so it > + must be released before locking any LOCK_rpl_thread lock, or a deadlock > + can occur. > + > + So we protect the infrequent operations of FLUSH TABLES WITH READ LOCK and > + pool size changes with this condition wait. > + */ > + thd= current_thd; > + res= 0; > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + if (thd) > + thd->ENTER_COND(&pool->COND_rpl_thread_pool, &pool->LOCK_rpl_thread_pool, > + &stage_waiting_for_rpl_thread_pool, &old_stage); > + while (pool->busy) > + { > + if (thd->check_killed()) > + { > + thd->send_kill_message(); > + res= 1; > + break; > + } > + mysql_cond_wait(&pool->COND_rpl_thread_pool, &pool->LOCK_rpl_thread_pool); > + } > + if (!res) > + pool_mark_busy(pool); > + if (thd) > + thd->EXIT_COND(&old_stage); > + else > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > + if (res) > + return res; > > /* > Allocate the new list of threads up-front. > @@ -1106,7 +1305,14 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > */ > for (i= 0; i < pool->count; ++i) > { > - rpl_parallel_thread *rpt= pool->get_thread(NULL, NULL); > + rpl_parallel_thread *rpt; > + > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + while ((rpt= pool->free_list) == NULL) > + mysql_cond_wait(&pool->COND_rpl_thread_pool, &pool->LOCK_rpl_thread_pool); > + pool->free_list= rpt->next; > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > + mysql_mutex_lock(&rpt->LOCK_rpl_thread); > rpt->stop= true; > mysql_cond_signal(&rpt->COND_rpl_thread); > mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > @@ -1157,7 +1363,7 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > } > > mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > - mysql_cond_broadcast(&pool->COND_rpl_thread_pool); > + pool_mark_not_busy(pool); > mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > > return 0; > @@ -1182,6 +1388,9 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > } > my_free(new_list); > } > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + pool_mark_not_busy(pool); > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > return 1; > } > > @@ -1445,7 +1654,7 @@ rpl_parallel_thread::loc_free_gco(group_commit_orderer *gco) > > > rpl_parallel_thread_pool::rpl_parallel_thread_pool() > - : count(0), threads(0), free_list(0), inited(false) > + : threads(0), free_list(0), count(0), inited(false), busy(false) > { > } > > @@ -1453,9 +1662,10 @@ rpl_parallel_thread_pool::rpl_parallel_thread_pool() > int > rpl_parallel_thread_pool::init(uint32 size) > { > - count= 0; > threads= NULL; > free_list= NULL; > + count= 0; > + busy= false; > > mysql_mutex_init(key_LOCK_rpl_thread_pool, &LOCK_rpl_thread_pool, > MY_MUTEX_INIT_SLOW); > @@ -1496,8 +1706,14 @@ rpl_parallel_thread_pool::get_thread(rpl_parallel_thread **owner, > rpl_parallel_thread *rpt; > > mysql_mutex_lock(&LOCK_rpl_thread_pool); > - while ((rpt= free_list) == NULL) > + for (;;) > + { > + while (unlikely(busy)) > + mysql_cond_wait(&COND_rpl_thread_pool, &LOCK_rpl_thread_pool); > + if ((rpt= free_list) != NULL) > + break; > mysql_cond_wait(&COND_rpl_thread_pool, &LOCK_rpl_thread_pool); > + } > free_list= rpt->next; > mysql_mutex_unlock(&LOCK_rpl_thread_pool); > mysql_mutex_lock(&rpt->LOCK_rpl_thread); > @@ -1908,7 +2124,7 @@ rpl_parallel::wait_for_workers_idle(THD *thd) > > e= (struct rpl_parallel_entry *)my_hash_element(&domain_hash, i); > mysql_mutex_lock(&e->LOCK_parallel_entry); > - e->need_sub_id_signal= true; > + ++e->need_sub_id_signal; > thd->ENTER_COND(&e->COND_parallel_entry, &e->LOCK_parallel_entry, > &stage_waiting_for_workers_idle, &old_stage); > while (e->current_sub_id > e->last_committed_sub_id) > @@ -1921,7 +2137,7 @@ rpl_parallel::wait_for_workers_idle(THD *thd) > } > mysql_cond_wait(&e->COND_parallel_entry, &e->LOCK_parallel_entry); > } > - e->need_sub_id_signal= false; > + --e->need_sub_id_signal; > thd->EXIT_COND(&old_stage); > if (err) > return err; > diff --git a/sql/rpl_parallel.h b/sql/rpl_parallel.h > index 04f8706..77a4005 100644 > --- a/sql/rpl_parallel.h > +++ b/sql/rpl_parallel.h > @@ -199,12 +199,18 @@ struct rpl_parallel_thread { > > > struct rpl_parallel_thread_pool { > - uint32 count; > struct rpl_parallel_thread **threads; > struct rpl_parallel_thread *free_list; > mysql_mutex_t LOCK_rpl_thread_pool; > mysql_cond_t COND_rpl_thread_pool; > + uint32 count; > bool inited; > + /* > + While FTWRL runs, this counter is incremented to make SQL thread or > + STOP/START slave not try to start new activity while that operation > + is in progress. > + */ > + bool busy; > > rpl_parallel_thread_pool(); > int init(uint32 size); > @@ -219,6 +225,12 @@ struct rpl_parallel_entry { > mysql_mutex_t LOCK_parallel_entry; > mysql_cond_t COND_parallel_entry; > uint32 domain_id; > + /* > + Incremented by wait_for_workers_idle() and rpl_pause_for_ftwrl() to show > + that they are waiting, so that finish_event_group knows to signal them > + when last_committed_sub_id is increased. > + */ > + uint32 need_sub_id_signal; > uint64 last_commit_id; > bool active; > /* > @@ -228,12 +240,6 @@ struct rpl_parallel_entry { > */ > bool force_abort; > /* > - Set in wait_for_workers_idle() to show that it is waiting, so that > - finish_event_group knows to signal it when last_committed_sub_id is > - increased. > - */ > - bool need_sub_id_signal; > - /* > At STOP SLAVE (force_abort=true), we do not want to process all events in > the queue (which could unnecessarily delay stop, if a lot of events happen > to be queued). The stop_count provides a safe point at which to stop, so > @@ -291,6 +297,11 @@ struct rpl_parallel_entry { > The value is ULONGLONG_MAX when no error occured. > */ > uint64 stop_on_error_sub_id; > + /* > + During FLUSH TABLES WITH READ LOCK, transactions with sub_id larger than > + this value must not start, but wait until the global read lock is released. > + */ > + uint64 pause_sub_id; > /* Total count of event groups queued so far. */ > uint64 count_queued_event_groups; > /* > @@ -331,5 +342,7 @@ extern struct rpl_parallel_thread_pool global_rpl_thread_pool; > extern int rpl_parallel_activate_pool(rpl_parallel_thread_pool *pool); > extern int rpl_parallel_inactivate_pool(rpl_parallel_thread_pool *pool); > extern bool process_gtid_for_restart_pos(Relay_log_info *rli, rpl_gtid *gtid); > +extern int rpl_pause_for_ftwrl(THD *thd); > +extern void rpl_unpause_for_ftwrl(THD *thd); > > #endif /* RPL_PARALLEL_H */ > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 5635e9a..ca6b11e 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -4271,6 +4271,17 @@ case SQLCOM_PREPARE: > break; > } > > + if (lex->type & REFRESH_READ_LOCK) > + { > + /* > + We need to pause any parallel replication slave workers during FLUSH > + TABLES WITH READ LOCK. Otherwise we might cause a deadlock, as > + worker threads eun run in arbitrary order but need to commit in a > + specific given order. > + */ > + if (rpl_pause_for_ftwrl(thd)) > + goto error; > + } > /* > reload_acl_and_cache() will tell us if we are allowed to write to the > binlog or not. > @@ -4301,6 +4312,8 @@ case SQLCOM_PREPARE: > if (!res) > my_ok(thd); > } > + if (lex->type & REFRESH_READ_LOCK) > + rpl_unpause_for_ftwrl(thd); > > break; > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] b8e26b0: MDEV-363 - Server crashes in intern_plugin_lock on concurrent installing
by Sergei Golubchik 09 Jun '15

09 Jun '15

Hi, Sergey! On Jun 09, svoj(a)mariadb.org wrote: > revision-id: b8e26b04d4ac22e29a78bca68e101176b6ef507a > parent(s): 08b2e1ae73f6fe055391bf8437b6dff1a889c49c > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-09 13:50:43 +0400 > message: > > MDEV-363 - Server crashes in intern_plugin_lock on concurrent installing > semisync plugin and setting rpl_semi_sync_master_enabled > > There was race condition between INSTALL PLUGIN and SET. It was caused by a > gap in INSTALL PLUGIN when plugin variables were registered but not fully > initialized. Accessing such variables concurrently may reference uninitialized > memory, specifically sys_var_pluginvar::plugin. > > Fixed by initializing sys_var_pluginvar::plugin early, before variable is > registered. ok to push! Regards, Sergei

1 0

[Maria-developers] MDEV-7268 Column of table cannot be converted from type 'decimal(0, ?)' to type ' 'decimal(10, 7)'
by Alexander Barkov 09 Jun '15

09 Jun '15

Hi Sergei, Please review a patch for mdev-7268. Thanks.

2 1

Re: [Maria-developers] 1ed4683: MDEV-8260 : Issues related to concurrent CTAS
by Sergei Golubchik 09 Jun '15

09 Jun '15

Hi, Nirbhay! On Jun 02, Nirbhay Choubey wrote: > revision-id: 1ed46831ed3a1c842b45f7819edd48a233080ce4 > parent(s): 6d5b723bdc3e04978619b9673fca266e0426916f > committer: Nirbhay Choubey > branch nick: 10.0-galera.ctas > timestamp: 2015-06-02 17:59:32 -0400 > message: > > MDEV-8260 : Issues related to concurrent CTAS > > * Wait for aborted thd (victim) to release MDL locks > * Skip aborting an already aborted thd > * Defer setting OK status in case of CTAS > * Minor cosmetic changes > * Added a test case So, I'm only reviewing non-wsrep part, ok? > @@ -3735,24 +3757,16 @@ bool select_insert::send_eof() > table->file->print_error(error,MYF(0)); > DBUG_RETURN(1); > } > - char buff[160]; > - if (info.ignore) > - sprintf(buff, ER(ER_INSERT_INFO), (ulong) info.records, > - (ulong) (info.records - info.copied), > - (long) thd->get_stmt_da()->current_statement_warn_count()); > - else > - sprintf(buff, ER(ER_INSERT_INFO), (ulong) info.records, > - (ulong) (info.deleted+info.updated), > - (long) thd->get_stmt_da()->current_statement_warn_count()); > - row_count= info.copied + info.deleted + > - ((thd->client_capabilities & CLIENT_FOUND_ROWS) ? > - info.touched : info.updated); > - id= (thd->first_successful_insert_id_in_cur_stmt > 0) ? > - thd->first_successful_insert_id_in_cur_stmt : > - (thd->arg_of_last_insert_id_function ? > - thd->first_successful_insert_id_in_prev_stmt : > - (info.copied ? autoinc_value_of_last_inserted_row : 0)); > - ::my_ok(thd, row_count, id, buff); > + > + /* > + Avoid setting OK status for CREATE .. SELECT as it may fail during commit. > + */ > + if (!((thd->lex->sql_command == SQLCOM_CREATE_TABLE) && > + ((&thd->lex->select_lex)->item_list.elements))) { > + prepare_ok_info(thd); > + ::my_ok(thd, ok.row_count, ok.id, ok.message); > + } > + > DBUG_RETURN(0); > } I think we've discussed a slightly different fix, where the parent class (select_insert) wouldn't need to check thd->lex->sql_command. Like, create two methods select_insert::prepare_eof select_insert::send_ok_packet the first method will have everything of the current select_insert::send_eof but without ::my_ok part. Then select_insert::send_eof() will be just bool select_insert::send_eof() { return prepare_eof() || send_ok_packet(); } and select_create::send_eof() will do prepare_eof(), commit and other stuff, send_ok_packet(). In this case you won't need to test for thd->lex->sql_command in the parent class. And you won't need prepare_ok_info(), it'll be inside send_ok_packet(). Regards, Sergei

2 1

Re: [Maria-developers] 87088b9: MDEV-8211 plugins.server_audit fails sporadically in buildbot.
by Sergei Golubchik 08 Jun '15

08 Jun '15

Hi, Holyfoot! On Jun 08, holyfoot(a)askmonty.org wrote: > revision-id: 87088b91f7dfa49a1946454d347d823dab643569 > parent(s): 96b37035e4045a27f64810c2d946bad2d1453a80 > committer: Alexey Botchkov > branch nick: server > timestamp: 2015-06-08 21:44:13 +0500 > message: > > MDEV-8211 plugins.server_audit fails sporadically in buildbot. > This test also should be fixed - delay added so the connection > event doesn't happen before the query. > > --- > mysql-test/suite/plugins/t/thread_pool_server_audit.test | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/mysql-test/suite/plugins/t/thread_pool_server_audit.test b/mysql-test/suite/plugins/t/thread_pool_server_audit.test > index 1a2d729..2c749e2 100644 > --- a/mysql-test/suite/plugins/t/thread_pool_server_audit.test > +++ b/mysql-test/suite/plugins/t/thread_pool_server_audit.test > @@ -13,6 +13,7 @@ set global server_audit_incl_users=null; > set global server_audit_file_path='server_audit.log'; > set global server_audit_output_type=file; > set global server_audit_logging=on; > +--sleep 2 > connect (con1,localhost,root,,mysql); > connection default; > disconnect con1; This is not a good idea. First, delays make mysql-test slower. And most often - unnecessary slower, as the test succeeds without a delay too. But also delays are unreliable, on a slow box two seconds may be not enough. Don't use delays, please, in this case you can wait for the query to complete before opening a new connection. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 5ccbfa0: MDEV-7445: Server crash with Signal 6
by Sergey Petrunia 08 Jun '15

08 Jun '15

On Thu, Apr 23, 2015 at 07:11:09PM +0200, sanja(a)mariadb.com wrote: > revision-id: 5ccbfa02d4da91c3f9080dc1b1627d19cf40eccd > parent(s): c7efc0315b51f214b6423ae4dcb4ddeb1b14e91a > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-04-23 19:11:06 +0200 > message: > > MDEV-7445: Server crash with Signal 6 > > Problem was in rewriting left expression which had 2 references on it. Solved with making subselect reference main. > > Item_in_optimized can have not Item_in_subselect reference in left part so type casting with no check is dangerous. > > Item::cols() should be checked after Item::fix_fields(). > > --- ... > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 998cb1c..51a38f0 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -1442,9 +1442,25 @@ bool Item_in_optimizer::eval_not_null_tables(uchar *opt_arg) > bool Item_in_optimizer::fix_left(THD *thd, Item **ref) > { > DBUG_ENTER("Item_in_optimizer::fix_left"); > - if ((!args[0]->fixed && args[0]->fix_fields(thd, args)) || > - (!cache && !(cache= Item_cache::get_cache(args[0])))) > + Item **ref0= args; > + if (args[1]->type() == Item::SUBSELECT_ITEM && > + ((Item_subselect *)args[1])->is_in_predicate()) > + { > + /* > + left_expr->fix_fields() may cause left_expr to be substituted for > + another item. (e.g. an Item_field may be changed into Item_ref). This > + transformation is undone at the end of statement execution (e.g. the > + Item_ref is deleted). However, Item_in_optimizer::args[0] may keep > + the pointer to the post-transformation item. Because of that, on the > + next execution we need to copy args[1]->left_expr again. > + */ > + ref0= &(((Item_in_subselect *)args[1])->left_expr); > + args[0]= ref0[0]; > + } > + if ((!args[0]->fixed && args[0]->fix_fields(thd, ref0)) || > + (!cache && !(cache= Item_cache::get_cache(ref0[0])))) > DBUG_RETURN(1); > + args[0]= ref0[0]; > DBUG_PRINT("info", ("actual fix fields")); > > cache->setup(args[0]); > @@ -1500,6 +1516,16 @@ bool Item_in_optimizer::fix_left(THD *thd, Item **ref) > bool Item_in_optimizer::fix_fields(THD *thd, Item **ref) > { > DBUG_ASSERT(fixed == 0); > + Item_subselect *sub= 0; > + uint col; > + > + /* > + MAX/MIN optimization can convert the subquery into > + expr + Item_singlerow_subselect > + */ (*) > + if (args[1]->type() == Item::SUBSELECT_ITEM) > + sub= (Item_subselect *)args[1]; > + I was looking for the case where the above if statement is not taken. Eventually I have applied this patch, the patch after it (e540d023e2ec6f37efc9ab695ccdfd4a6744ad64) to get the test cases, then added this at line (*): DBUG_ASSERT(args[1]->type() == Item::SUBSELECT_ITEM); and ran the entire main test suite, with and without --ps-protocol. It worked. Can you add test coverage for this case? > if (fix_left(thd, ref)) > return TRUE; > if (args[0]->maybe_null) > @@ -1507,10 +1533,10 @@ bool Item_in_optimizer::fix_fields(THD *thd, Item **ref) > > if (!args[1]->fixed && args[1]->fix_fields(thd, args+1)) > return TRUE; > - Item_in_subselect * sub= (Item_in_subselect *)args[1]; > - if (args[0]->cols() != sub->engine->cols()) > + if ((sub && ((col= args[0]->cols()) != sub->engine->cols())) || > + (!sub && (args[1]->cols() != (col= 1)))) > { > - my_error(ER_OPERAND_COLUMNS, MYF(0), args[0]->cols()); > + my_error(ER_OPERAND_COLUMNS, MYF(0), col); > return TRUE; > } > if (args[1]->maybe_null) ... BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

Re: [Maria-developers] [Commits] 9f8a458: MDEV-7846: Server crashes in Item_subselect::fix_fields or fails with Thread stack overrun
by Sergey Petrunia 08 Jun '15

08 Jun '15

Hi Sanja, It is very difficult to understand this patch. What is the exact meaning of left_expr_orig? It is declared but has no comments. It is interesting that convert_subq_to_sj has got new code to handle scalar comparison case ( foo IN (SELECT bar ...)), while there is no handling for tuple comparison ( (foo1,foo2) IN (SELECT bar1,bar2 ..)). Is this intentional? On Thu, Apr 23, 2015 at 08:09:00PM +0200, sanja(a)mariadb.com wrote: > revision-id: 9f8a458fb2d07298810bb5d9824ce728017bfb32 > parent(s): e540d023e2ec6f37efc9ab695ccdfd4a6744ad64 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-04-23 20:08:57 +0200 > message: > > MDEV-7846: Server crashes in Item_subselect::fix_fields or fails with Thread stack overrun > > Substitute into transformed subselects original left expression and than register its change in case it was substituted. > > --- > mysql-test/r/subselect.result | 28 +++++++++++++++++++++++++ > mysql-test/r/subselect_no_mat.result | 28 +++++++++++++++++++++++++ > mysql-test/r/subselect_no_opts.result | 28 +++++++++++++++++++++++++ > mysql-test/r/subselect_no_scache.result | 28 +++++++++++++++++++++++++ > mysql-test/r/subselect_no_semijoin.result | 28 +++++++++++++++++++++++++ > mysql-test/t/subselect.test | 34 +++++++++++++++++++++++++++++++ > sql/item_cmpfunc.cc | 8 ++++---- > sql/item_subselect.cc | 8 +++----- > sql/item_subselect.h | 1 + > sql/opt_subselect.cc | 4 +++- > 10 files changed, 185 insertions(+), 10 deletions(-) > > diff --git a/mysql-test/r/subselect.result b/mysql-test/r/subselect.result > index 0ab2d48..cf52ba2 100644 > --- a/mysql-test/r/subselect.result > +++ b/mysql-test/r/subselect.result > @@ -7053,3 +7053,31 @@ WHERE h.host in (SELECT host FROM mysql.user) > ) AS sq > FROM mysql.host h GROUP BY h.host; > sq > +# > +# MDEV-7846:Server crashes in Item_subselect::fix > +#_fields or fails with Thread stack overrun > +# > +CREATE TABLE t1 (column1 INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(9); > +CREATE TABLE t2 (column2 INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (1),(4); > +CREATE TABLE t3 (column3 INT) ENGINE=MyISAM; > +INSERT INTO t3 VALUES (6),(8); > +CREATE TABLE t4 (column4 INT) ENGINE=MyISAM; > +INSERT INTO t4 VALUES (2),(5); > +PREPARE stmt FROM " > +SELECT ( > + SELECT MAX( table1.column1 ) AS field1 > + FROM t1 AS table1 > + WHERE table3.column3 IN ( SELECT table2.column2 AS field2 FROM t2 AS table2 ) > +) AS sq > +FROM t3 AS table3, t4 AS table4 GROUP BY sq > +"; > +EXECUTE stmt; > +sq > +NULL > +EXECUTE stmt; > +sq > +NULL > +deallocate prepare stmt; > +drop table t1,t2,t3,t4; > diff --git a/mysql-test/r/subselect_no_mat.result b/mysql-test/r/subselect_no_mat.result > index 5780351..73a69e9 100644 > --- a/mysql-test/r/subselect_no_mat.result > +++ b/mysql-test/r/subselect_no_mat.result > @@ -7050,6 +7050,34 @@ WHERE h.host in (SELECT host FROM mysql.user) > ) AS sq > FROM mysql.host h GROUP BY h.host; > sq > +# > +# MDEV-7846:Server crashes in Item_subselect::fix > +#_fields or fails with Thread stack overrun > +# > +CREATE TABLE t1 (column1 INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(9); > +CREATE TABLE t2 (column2 INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (1),(4); > +CREATE TABLE t3 (column3 INT) ENGINE=MyISAM; > +INSERT INTO t3 VALUES (6),(8); > +CREATE TABLE t4 (column4 INT) ENGINE=MyISAM; > +INSERT INTO t4 VALUES (2),(5); > +PREPARE stmt FROM " > +SELECT ( > + SELECT MAX( table1.column1 ) AS field1 > + FROM t1 AS table1 > + WHERE table3.column3 IN ( SELECT table2.column2 AS field2 FROM t2 AS table2 ) > +) AS sq > +FROM t3 AS table3, t4 AS table4 GROUP BY sq > +"; > +EXECUTE stmt; > +sq > +NULL > +EXECUTE stmt; > +sq > +NULL > +deallocate prepare stmt; > +drop table t1,t2,t3,t4; > set optimizer_switch=default; > select @@optimizer_switch like '%materialization=on%'; > @@optimizer_switch like '%materialization=on%' > diff --git a/mysql-test/r/subselect_no_opts.result b/mysql-test/r/subselect_no_opts.result > index 4bea029..1512e39 100644 > --- a/mysql-test/r/subselect_no_opts.result > +++ b/mysql-test/r/subselect_no_opts.result > @@ -7048,4 +7048,32 @@ WHERE h.host in (SELECT host FROM mysql.user) > ) AS sq > FROM mysql.host h GROUP BY h.host; > sq > +# > +# MDEV-7846:Server crashes in Item_subselect::fix > +#_fields or fails with Thread stack overrun > +# > +CREATE TABLE t1 (column1 INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(9); > +CREATE TABLE t2 (column2 INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (1),(4); > +CREATE TABLE t3 (column3 INT) ENGINE=MyISAM; > +INSERT INTO t3 VALUES (6),(8); > +CREATE TABLE t4 (column4 INT) ENGINE=MyISAM; > +INSERT INTO t4 VALUES (2),(5); > +PREPARE stmt FROM " > +SELECT ( > + SELECT MAX( table1.column1 ) AS field1 > + FROM t1 AS table1 > + WHERE table3.column3 IN ( SELECT table2.column2 AS field2 FROM t2 AS table2 ) > +) AS sq > +FROM t3 AS table3, t4 AS table4 GROUP BY sq > +"; > +EXECUTE stmt; > +sq > +NULL > +EXECUTE stmt; > +sq > +NULL > +deallocate prepare stmt; > +drop table t1,t2,t3,t4; > set @optimizer_switch_for_subselect_test=null; > diff --git a/mysql-test/r/subselect_no_scache.result b/mysql-test/r/subselect_no_scache.result > index fdb3b12..26cea1f 100644 > --- a/mysql-test/r/subselect_no_scache.result > +++ b/mysql-test/r/subselect_no_scache.result > @@ -7059,6 +7059,34 @@ WHERE h.host in (SELECT host FROM mysql.user) > ) AS sq > FROM mysql.host h GROUP BY h.host; > sq > +# > +# MDEV-7846:Server crashes in Item_subselect::fix > +#_fields or fails with Thread stack overrun > +# > +CREATE TABLE t1 (column1 INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(9); > +CREATE TABLE t2 (column2 INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (1),(4); > +CREATE TABLE t3 (column3 INT) ENGINE=MyISAM; > +INSERT INTO t3 VALUES (6),(8); > +CREATE TABLE t4 (column4 INT) ENGINE=MyISAM; > +INSERT INTO t4 VALUES (2),(5); > +PREPARE stmt FROM " > +SELECT ( > + SELECT MAX( table1.column1 ) AS field1 > + FROM t1 AS table1 > + WHERE table3.column3 IN ( SELECT table2.column2 AS field2 FROM t2 AS table2 ) > +) AS sq > +FROM t3 AS table3, t4 AS table4 GROUP BY sq > +"; > +EXECUTE stmt; > +sq > +NULL > +EXECUTE stmt; > +sq > +NULL > +deallocate prepare stmt; > +drop table t1,t2,t3,t4; > set optimizer_switch=default; > select @@optimizer_switch like '%subquery_cache=on%'; > @@optimizer_switch like '%subquery_cache=on%' > diff --git a/mysql-test/r/subselect_no_semijoin.result b/mysql-test/r/subselect_no_semijoin.result > index cb6d35d..4c6f037 100644 > --- a/mysql-test/r/subselect_no_semijoin.result > +++ b/mysql-test/r/subselect_no_semijoin.result > @@ -7048,5 +7048,33 @@ WHERE h.host in (SELECT host FROM mysql.user) > ) AS sq > FROM mysql.host h GROUP BY h.host; > sq > +# > +# MDEV-7846:Server crashes in Item_subselect::fix > +#_fields or fails with Thread stack overrun > +# > +CREATE TABLE t1 (column1 INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(9); > +CREATE TABLE t2 (column2 INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (1),(4); > +CREATE TABLE t3 (column3 INT) ENGINE=MyISAM; > +INSERT INTO t3 VALUES (6),(8); > +CREATE TABLE t4 (column4 INT) ENGINE=MyISAM; > +INSERT INTO t4 VALUES (2),(5); > +PREPARE stmt FROM " > +SELECT ( > + SELECT MAX( table1.column1 ) AS field1 > + FROM t1 AS table1 > + WHERE table3.column3 IN ( SELECT table2.column2 AS field2 FROM t2 AS table2 ) > +) AS sq > +FROM t3 AS table3, t4 AS table4 GROUP BY sq > +"; > +EXECUTE stmt; > +sq > +NULL > +EXECUTE stmt; > +sq > +NULL > +deallocate prepare stmt; > +drop table t1,t2,t3,t4; > set @optimizer_switch_for_subselect_test=null; > set @join_cache_level_for_subselect_test=NULL; > diff --git a/mysql-test/t/subselect.test b/mysql-test/t/subselect.test > index 3eb056d..00ab48b 100644 > --- a/mysql-test/t/subselect.test > +++ b/mysql-test/t/subselect.test > @@ -5930,3 +5930,37 @@ SELECT > WHERE h.host in (SELECT host FROM mysql.user) > ) AS sq > FROM mysql.host h GROUP BY h.host; > + > + > +--echo # > +--echo # MDEV-7846:Server crashes in Item_subselect::fix > +--echo #_fields or fails with Thread stack overrun > +--echo # > +CREATE TABLE t1 (column1 INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(9); > + > +CREATE TABLE t2 (column2 INT) ENGINE=MyISAM; > + > +INSERT INTO t2 VALUES (1),(4); > + > +CREATE TABLE t3 (column3 INT) ENGINE=MyISAM; > +INSERT INTO t3 VALUES (6),(8); > + > +CREATE TABLE t4 (column4 INT) ENGINE=MyISAM; > +INSERT INTO t4 VALUES (2),(5); > + > + > +PREPARE stmt FROM " > +SELECT ( > + SELECT MAX( table1.column1 ) AS field1 > + FROM t1 AS table1 > + WHERE table3.column3 IN ( SELECT table2.column2 AS field2 FROM t2 AS table2 ) > +) AS sq > +FROM t3 AS table3, t4 AS table4 GROUP BY sq > +"; > + > +EXECUTE stmt; > +EXECUTE stmt; > + > +deallocate prepare stmt; > +drop table t1,t2,t3,t4; > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 51a38f0..29fff8e 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -1455,12 +1455,12 @@ bool Item_in_optimizer::fix_left(THD *thd, Item **ref) > next execution we need to copy args[1]->left_expr again. > */ > ref0= &(((Item_in_subselect *)args[1])->left_expr); > - args[0]= ref0[0]; > } > - if ((!args[0]->fixed && args[0]->fix_fields(thd, ref0)) || > - (!cache && !(cache= Item_cache::get_cache(ref0[0])))) > + if ((!(*ref0)->fixed && (*ref0)->fix_fields(thd, ref0)) || > + (!cache && !(cache= Item_cache::get_cache(*ref0)))) > DBUG_RETURN(1); > - args[0]= ref0[0]; > + if (args[0] != (*ref0)) > + current_thd->change_item_tree(args, (*ref0)); > DBUG_PRINT("info", ("actual fix fields")); > > cache->setup(args[0]); > diff --git a/sql/item_subselect.cc b/sql/item_subselect.cc > index 4c11808..e04f5ff 100644 > --- a/sql/item_subselect.cc > +++ b/sql/item_subselect.cc > @@ -1360,7 +1360,7 @@ Item_in_subselect::Item_in_subselect(Item * left_exp, > upper_item(0) > { > DBUG_ENTER("Item_in_subselect::Item_in_subselect"); > - left_expr= left_exp; > + left_expr_orig= left_expr= left_exp; > func= &eq_creator; > init(select_lex, new select_exists_subselect(this)); > max_columns= UINT_MAX; > @@ -1384,7 +1384,7 @@ Item_allany_subselect::Item_allany_subselect(Item * left_exp, > :Item_in_subselect(), func_creator(fc), all(all_arg) > { > DBUG_ENTER("Item_allany_subselect::Item_allany_subselect"); > - left_expr= left_exp; > + left_expr_orig= left_expr= left_exp; > func= func_creator(all_arg); > init(select_lex, new select_exists_subselect(this)); > max_columns= 1; > @@ -2584,15 +2584,13 @@ Item_in_subselect::select_in_like_transformer(JOIN *join) > arena= thd->activate_stmt_arena_if_needed(&backup); > if (!optimizer) > { > - result= (!(optimizer= new Item_in_optimizer(left_expr, this))); > + result= (!(optimizer= new Item_in_optimizer(left_expr_orig, this))); > if (result) > goto out; > } > > thd->lex->current_select= current->return_after_parsing(); > result= optimizer->fix_left(thd, optimizer->arguments()); > - /* fix_fields can change reference to left_expr, we need reassign it */ > - left_expr= optimizer->arguments()[0]; > thd->lex->current_select= current; > > if (changed) > diff --git a/sql/item_subselect.h b/sql/item_subselect.h > index 592e771..930bd66 100644 > --- a/sql/item_subselect.h > +++ b/sql/item_subselect.h > @@ -449,6 +449,7 @@ class Item_in_subselect :public Item_exists_subselect > Item **having_item); > public: > Item *left_expr; > + Item *left_expr_orig; > /* Priority of this predicate in the convert-to-semi-join-nest process. */ > int sj_convert_priority; > /* > diff --git a/sql/opt_subselect.cc b/sql/opt_subselect.cc > index 5b1a7f2..1363be0 100644 > --- a/sql/opt_subselect.cc > +++ b/sql/opt_subselect.cc > @@ -1593,7 +1593,9 @@ static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > { > nested_join->sj_outer_expr_list.push_back(subq_pred->left_expr); > Item_func_eq *item_eq= > - new Item_func_eq(subq_pred->left_expr, subq_lex->ref_pointer_array[0]); > + new Item_func_eq(subq_pred->left_expr_orig, subq_lex->ref_pointer_array[0]); > + if (subq_pred->left_expr_orig != subq_pred->left_expr) > + thd->change_item_tree(item_eq->arguments(), subq_pred->left_expr); > item_eq->in_equality_no= 0; > sj_nest->sj_on_expr= and_items(sj_nest->sj_on_expr, item_eq); > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] [GSoC] MXS-2
by sriram patil 08 Jun '15

08 Jun '15

Hi All, There is no blog post this week. Could not conclude the task at hand. Will be writing a blog post for 2nd and 3rd week together. Thanks, Sriram

1 0

[Maria-developers] 5.5.44 Sprint: MDEV-8067 correct fix for MySQL Bug # 19699237: UNINITIALIZED VARIABLE IN ITEM_FIELD::STR_RESULT
by Alexander Barkov 08 Jun '15

08 Jun '15

Hi Sergei, Please review my patch for MDEV-8067. Some subselect_(mat|sj) test chanted "EXPLAIN" output. Perhaps the new results are correct. Please advise. Thanks.

1 0

[Maria-developers] Please review MDEV-7269 mysqlbinlog Don't know how to handle column type=0 meta=0 (0000)#
by Alexander Barkov 05 Jun '15

05 Jun '15

Hi Sergei, Please review a joint patch for: MDEV-7269 mysqlbinlog Don't know how to handle column type=0 meta=0 (0000)# MDEV-8267 Add /*old*/ comment into I_S.COLUMN_TYPE for old DECIMAL Thanks.

2 4

[Maria-developers] [maria-developers] MMC Snapin and WMI Provider
by Rolain Djeumen 05 Jun '15

05 Jun '15

Hello to all , I would like to work on the MariaDB MMC Snapin and WMI Provider. Is there anyone in charge of the said feature ?

2 1

Re: [Maria-developers] 91259f9: MDEV-7505 - Too large scale in DECIMAL dynamic column getter crashes mysqld
by Sergei Golubchik 04 Jun '15

04 Jun '15

Hi, svoj! On Jun 04, svoj(a)mariadb.org wrote: > revision-id: 91259f903c2ffac7c7c8fc72385a572eace4fa89 > parent(s): 9bb9bde42dbf6f936faadeaa34cde838bf6b63dd > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-04 16:04:05 +0400 > message: > > MDEV-7505 - Too large scale in DECIMAL dynamic column getter crashes mysqld > > Server may crash if sanity checks of COLUMN_GET() fail. > > COLUMN_GET() description generator expects parent CAST item, which may not have > been created due to failure of sanity checks. Then further attempt to report > an error may crash the server. > > Fixed COLUMN_GET() description generator to handle such case. > > --- > mysql-test/r/dyncol.result | 6 ++++++ > mysql-test/t/AAA.test | 1 + Mistakenly committed file ^^^^ > mysql-test/t/dyncol.test | 6 ++++++ > sql/item_strfunc.cc | 7 +++++++ > 4 files changed, 20 insertions(+) > > diff --git a/mysql-test/t/AAA.test b/mysql-test/t/AAA.test > new file mode 100644 > index 0000000..371db62 > --- /dev/null > +++ b/mysql-test/t/AAA.test > @@ -0,0 +1 @@ > +SELECT COLUMN_GET(`x`, 'y' AS DECIMAL(5,34)); > diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc > index eedf149..7ad4c06 100644 > --- a/sql/item_strfunc.cc > +++ b/sql/item_strfunc.cc > @@ -4467,6 +4467,13 @@ bool Item_dyncol_get::get_date(MYSQL_TIME *ltime, ulonglong fuzzy_date) > > void Item_dyncol_get::print(String *str, enum_query_type query_type) How do you get to Item_dyncol_get::print if cast couldn't be created? The comment below says "see create_func_dyncol_get". This function looks like if (!(res= new (thd->mem_root) Item_dyncol_get(str, num))) return res; // Return NULL return create_func_cast(thd, res, cast_type, c_len, c_dec, cs); and to get Item_dyncol_get without a parent, one needs create_func_cast to return 'res' without wrapping it in a CAST(...). But it never does that. > { > + /* Parent cast doesn't exist yet. Only print dynamic column name. */ > + if (!str->length()) Not safe, if you wrap your COLUMN_GET() in, say, CONCAT(): CONCAT(COLUMN_GET(`x`, 'y' AS DECIMAL(5,34))) then str->length() won't be 0, despite the error in CAST. > + { > + args[1]->print(str, query_type); > + return; > + } > + > /* see create_func_dyncol_get */ > DBUG_ASSERT(str->length() >= 5); > DBUG_ASSERT(strncmp(str->ptr() + str->length() - 5, "cast(", 5) == 0); Regards, Sergei

2 2

[Maria-developers] Probably in a bug in simple_pred()
by Alexander Barkov 04 Jun '15

04 Jun '15

Hi Sergey, I probably found a bug in simple_pred() in opt_sum.cc. In this piece of code: > case 3: > /* field BETWEEN const AND const */ > item= func_item->arguments()[0]->real_item(); > if (item->type() == Item::FIELD_ITEM) > { > args[0]= item; > for (int i= 1 ; i <= 2; i++) > { > item= func_item->arguments()[i]->real_item(); > if (!item->const_item()) > return 0; > args[i]= item; > if (check_item1_shorter_item2(args[0], args[1])) Should't it be if (check_item1_shorter_item2(args[0], args[i])) ? > return 0; > } > } If this is a bug, can you please help to make an SQL scrip which would be affected by this bug? Thanks.

2 3

[Maria-developers] Work on MariaDB management console snapin and WMI Provider
by Rolain Djeumen 04 Jun '15

04 Jun '15

Hello to All, I am new here , and i will like to work on the MariaDB management console snapin and WMI Provider. Who do i need to contact for that ? Thanks,

1 0

Re: [Maria-developers] [Commits] 9bb9bde: MDEV-6236 - [PATCH] mysql_tzinfo_to_sql may produce invalid SQL
by Sergei Golubchik 03 Jun '15

03 Jun '15

Hi, Sergey! On Jun 03, svoj(a)mariadb.org wrote: > revision-id: 9bb9bde42dbf6f936faadeaa34cde838bf6b63dd > parent(s): 54977bee6705b9f3ebe2e534245defdb18ac31be > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-03 14:30:09 +0400 > message: > > MDEV-6236 - [PATCH] mysql_tzinfo_to_sql may produce invalid SQL > > Factory timezone is supposed "For companies who don't want to put time > zone specification in their installation procedures. When users run > date, they'll get the message. Also useful for the "comp.sources" > version." > > This "message" is exposed as timezone abbreviation, which is supposed > to be short and thus may cause generated INSERT statements to fail. > > Do not attempt to load Factory timezone. Are you usure Factory is the only one with a message instead of a abbreviation? Perhaps it'd be safer to filter out all timezones with extra-long "abbreviations"? Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 54977be: MDEV-7207 - ALTER VIEW does not change ALGORITM
by Sergei Golubchik 03 Jun '15

03 Jun '15

Hi, Sergey! On Jun 03, svoj(a)mariadb.org wrote: > revision-id: 54977bee6705b9f3ebe2e534245defdb18ac31be > parent(s): 6bd76f8b7e086ab7cf0249a05277ff552e564554 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-03 13:59:58 +0400 > message: > > MDEV-7207 - ALTER VIEW does not change ALGORITM > > Fixed that ALTER VIEW ALGORITHM=UNDEFINED behaved as if algorithm was not > specified. ok to push, thanks! Regards, Sergei

1 0

[Maria-developers] which version MDEV-6991 GROUP_MIN_MAX optimization is erroneously applied in some cases
by Alexander Barkov 02 Jun '15

02 Jun '15

Hi Sergei, I suggest to move MDEV-6691 to 10.1: Reasons: 1. It will be easier to fix it on top of: MDEV-6991 GROUP_MIN_MAX optimization is erroneously applied in some cases So fixing before 10.0 is not desirable 2. The relevant code is different in 10.1. So it's not desirable to fix before 10.1. 3. The bug is not much important Thanks.

2 3

[Maria-developers] bugs MDEV-8167 and MDEV-6973
by Alexander Barkov 02 Jun '15

02 Jun '15

Hi Sergei, Can we move these bugs from 10.0 to 10.1: (MDEV-8167) XOR returns bad results for an indexed column (MDEV-6973) XOR aggregates argument collations ? I earlier sent you a joint patch that fixes both. It moves Item_func_xor from Item_bool_func2 to Item_bool_func. But the problem is that in 10.1 I had to move Arg_comparator from Item_bool_func2 to Item_bool_rowready_func2, to fix some other bugs in LIKE handling. So in 10.1 it's now Ok for Item_func_xor to stay in Item_bool_func2. and the patch will be different! The bugs are not quite important. So I suggest not to fix them in 10.0 to avoid merge problems and confusion about Item_func_xor jumping in the hierarchy. Thanks.

2 1

[Maria-developers] Patch for bug in libmysql
by Matthew Fagan 02 Jun '15

02 Jun '15

Hi, I'm new to this list. I have a bug and a patch for it for libmysql There is a bug in the file libmariadb.c (on line 1761 in latest source). The line is as follows: vio_wait_or_timeout(net->vio, FALSE, mysql->options.connect_timeout * 100) < 1) The bug is that FALSE should be TRUE. The bug means that the mysql_real_connect does not handle connection timeout correctly. It also means that the async connect call cannot be used with an edge-triggered event system (such as WSAEventSelect in Windows).

2 1

Re: [Maria-developers] [Commits] ab7640f: my_b_fill, inline my_b_* functions instead of hairy macros
by Sergey Vojtovich 02 Jun '15

02 Jun '15

Hi Sergei, ok to push. Should we always prefer "static inline" over macroses? Regards, Sergey On Tue, May 26, 2015 at 03:38:02PM +0200, serg(a)mariadb.org wrote: > revision-id: ab7640f4d30f5a160d3a1c77efe9d2c459a3bda6 > parent(s): d916a1dc7705ae29a916e7ec91f0c76bceab708c > committer: Sergei Golubchik > branch nick: maria > timestamp: 2015-05-26 15:28:23 +0200 > message: > > my_b_fill, inline my_b_* functions instead of hairy macros > > --- > include/my_sys.h | 141 ++++++++++++++++++++++++++++++++++++---------------- > include/thr_lock.h | 1 - > mysys/mf_iocache2.c | 55 -------------------- > sql/log.cc | 1 - > sql/wsrep_binlog.cc | 4 -- > 5 files changed, 97 insertions(+), 105 deletions(-) > > diff --git a/include/my_sys.h b/include/my_sys.h > index 3d930d3..8e0e34f 100644 > --- a/include/my_sys.h > +++ b/include/my_sys.h > @@ -19,6 +19,8 @@ > > #include "my_global.h" /* C_MODE_START, C_MODE_END */ > > +#include <m_string.h> > + > C_MODE_START > > #ifdef HAVE_AIOWAIT > @@ -507,52 +509,108 @@ typedef void (*my_error_reporter)(enum loglevel level, const char *format, ...) > > extern my_error_reporter my_charset_error_reporter; > > - /* defines for mf_iocache */ > +/* inline functions for mf_iocache */ > + > +extern int my_b_flush_io_cache(IO_CACHE *info, int need_append_buffer_lock); > +extern int _my_b_get(IO_CACHE *info); > +extern int _my_b_read(IO_CACHE *info,uchar *Buffer,size_t Count); > +extern int _my_b_write(IO_CACHE *info,const uchar *Buffer,size_t Count); > > - /* Test if buffer is inited */ > -#define my_b_clear(info) (info)->buffer=0 > -#define my_b_inited(info) (info)->buffer > +/* Test if buffer is inited */ > +static inline void my_b_clear(IO_CACHE *info) { info->buffer= 0; } > +static inline int my_b_inited(IO_CACHE *info) { return MY_TEST(info->buffer); } > #define my_b_EOF INT_MIN > > -#define my_b_read(info,Buffer,Count) \ > - ((info)->read_pos + (Count) <= (info)->read_end ?\ > - (memcpy(Buffer,(info)->read_pos,(size_t) (Count)), \ > - ((info)->read_pos+=(Count)), 0) : _my_b_read((info), (Buffer), (Count))) > - > -#define my_b_write(info,Buffer,Count) \ > - ((info)->write_pos + (Count) <=(info)->write_end ?\ > - (memcpy((info)->write_pos, (Buffer), (size_t)(Count)),\ > - ((info)->write_pos+=(Count)), 0) : _my_b_write((info), (Buffer), (Count))) > - > -#define my_b_get(info) \ > - ((info)->read_pos != (info)->read_end ?\ > - ((info)->read_pos++, (int) (uchar) (info)->read_pos[-1]) :\ > - _my_b_get(info)) > - > - /* my_b_write_byte dosn't have any err-check */ > -#define my_b_write_byte(info,chr) \ > - (((info)->write_pos < (info)->write_end) ?\ > - ((*(info)->write_pos++)=(chr)) :\ > - ((my_b_flush_io_cache(info, 1)), ((*(info)->write_pos++)=(chr)))) > - > -#define my_b_tell(info) ((info)->pos_in_file + \ > - (size_t) (*(info)->current_pos - (info)->request_pos)) > -#define my_b_write_tell(info) ((info)->pos_in_file + \ > - ((info)->write_pos - (info)->write_buffer)) > - > -#define my_b_get_buffer_start(info) (info)->request_pos > -#define my_b_get_bytes_in_buffer(info) (char*) (info)->read_end - \ > - (char*) my_b_get_buffer_start(info) > -#define my_b_get_pos_in_file(info) (info)->pos_in_file > - > -/* tell write offset in the SEQ_APPEND cache */ > +static inline int my_b_read(IO_CACHE *info, uchar *Buffer, size_t Count) > +{ > + if (info->read_pos + Count <= info->read_end) > + { > + memcpy(Buffer, info->read_pos, Count); > + info->read_pos+= Count; > + return 0; > + } > + return _my_b_read(info, Buffer, Count); > +} > + > +static inline int my_b_write(IO_CACHE *info, const uchar *Buffer, size_t Count) > +{ > + if (info->write_pos + Count <= info->write_end) > + { > + memcpy(info->write_pos, Buffer, Count); > + info->write_pos+= Count; > + return 0; > + } > + return _my_b_write(info, Buffer, Count); > +} > + > +static inline int my_b_get(IO_CACHE *info) > +{ > + if (info->read_pos != info->read_end) > + { > + info->read_pos++; > + return info->read_pos[-1]; > + } > + return _my_b_get(info); > +} > + > +/* my_b_write_byte dosn't have any err-check */ > +static inline void my_b_write_byte(IO_CACHE *info, uchar chr) > +{ > + if (info->write_pos >= info->write_end) > + my_b_flush_io_cache(info, 1); > + *info->write_pos++= chr; > +} > + > +/** > + Fill buffer of the cache. > + > + @note It assumes that you have already used all characters in the CACHE, > + independent of the read_pos value! > + > + @returns > + 0 On error or EOF (info->error = -1 on error) > + # Number of characters > +*/ > +static inline size_t my_b_fill(IO_CACHE *info) > +{ > + info->read_pos= info->read_end; > + return _my_b_read(info,0,0) ? 0 : info->read_end - info->read_pos; > +} > + > +static inline my_off_t my_b_tell(const IO_CACHE *info) > +{ > + return info->pos_in_file + (*info->current_pos - info->request_pos); > +} > + > +static inline my_off_t my_b_write_tell(const IO_CACHE *info) > +{ > + return info->pos_in_file + (info->write_pos - info->write_buffer); > +} > + > +static inline uchar* my_b_get_buffer_start(const IO_CACHE *info) > +{ > + return info->request_pos; > +} > + > +static inline size_t my_b_get_bytes_in_buffer(const IO_CACHE *info) > +{ > + return info->read_end - info->request_pos; > +} > + > +static inline my_off_t my_b_get_pos_in_file(const IO_CACHE *info) > +{ > + return info->pos_in_file; > +} > + > +static inline size_t my_b_bytes_in_cache(const IO_CACHE *info) > +{ > + return *info->current_end - *info->current_pos; > +} > + > int my_b_copy_to_file(IO_CACHE *cache, FILE *file); > my_off_t my_b_append_tell(IO_CACHE* info); > my_off_t my_b_safe_tell(IO_CACHE* info); /* picks the correct tell() */ > > -#define my_b_bytes_in_cache(info) (size_t) (*(info)->current_end - \ > - *(info)->current_pos) > - > typedef uint32 ha_checksum; > extern ulong my_crc_dbug_check; > > @@ -741,24 +799,19 @@ extern my_bool reinit_io_cache(IO_CACHE *info,enum cache_type type, > my_off_t seek_offset, my_bool use_async_io, > my_bool clear_cache); > extern void setup_io_cache(IO_CACHE* info); > -extern int _my_b_read(IO_CACHE *info,uchar *Buffer,size_t Count); > extern void init_io_cache_share(IO_CACHE *read_cache, IO_CACHE_SHARE *cshare, > IO_CACHE *write_cache, uint num_threads); > extern void remove_io_thread(IO_CACHE *info); > -extern int _my_b_get(IO_CACHE *info); > extern int _my_b_async_read(IO_CACHE *info,uchar *Buffer,size_t Count); > -extern int _my_b_write(IO_CACHE *info,const uchar *Buffer,size_t Count); > extern int my_b_append(IO_CACHE *info,const uchar *Buffer,size_t Count); > extern int my_b_safe_write(IO_CACHE *info,const uchar *Buffer,size_t Count); > > extern int my_block_write(IO_CACHE *info, const uchar *Buffer, > size_t Count, my_off_t pos); > -extern int my_b_flush_io_cache(IO_CACHE *info, int need_append_buffer_lock); > > #define flush_io_cache(info) my_b_flush_io_cache((info),1) > > extern int end_io_cache(IO_CACHE *info); > -extern size_t my_b_fill(IO_CACHE *info); > extern void my_b_seek(IO_CACHE *info,my_off_t pos); > extern size_t my_b_gets(IO_CACHE *info, char *to, size_t max_length); > extern my_off_t my_b_filelength(IO_CACHE *info); > diff --git a/include/thr_lock.h b/include/thr_lock.h > index 2561709..bc916b8 100644 > --- a/include/thr_lock.h > +++ b/include/thr_lock.h > @@ -166,7 +166,6 @@ void thr_set_lock_wait_callback(void (*before_wait)(void), > void (*after_wait)(void)); > > #ifdef WITH_WSREP > -#include <my_sys.h> > typedef my_bool (* wsrep_thd_is_brute_force_fun)(void *, my_bool); > typedef int (* wsrep_abort_thd_fun)(void *, void *, my_bool); > typedef int (* wsrep_on_fun)(void *); > diff --git a/mysys/mf_iocache2.c b/mysys/mf_iocache2.c > index 06dfc9f..a642e11 100644 > --- a/mysys/mf_iocache2.c > +++ b/mysys/mf_iocache2.c > @@ -61,7 +61,6 @@ my_b_copy_to_file(IO_CACHE *cache, FILE *file) > if (my_fwrite(file, cache->read_pos, bytes_in_cache, > MYF(MY_WME | MY_NABP)) == (size_t) -1) > DBUG_RETURN(1); > - cache->read_pos= cache->read_end; > } while ((bytes_in_cache= my_b_fill(cache))); > if(cache->error == -1) > DBUG_RETURN(1); > @@ -183,60 +182,6 @@ void my_b_seek(IO_CACHE *info,my_off_t pos) > > > /* > - Fill buffer of the cache. > - > - NOTES > - This assumes that you have already used all characters in the CACHE, > - independent of the read_pos value! > - > - RETURN > - 0 On error or EOF (info->error = -1 on error) > - # Number of characters > -*/ > - > - > -size_t my_b_fill(IO_CACHE *info) > -{ > - my_off_t pos_in_file=(info->pos_in_file+ > - (size_t) (info->read_end - info->buffer)); > - size_t diff_length, length, max_length; > - > - if (info->seek_not_done) > - { /* File touched, do seek */ > - if (mysql_file_seek(info->file, pos_in_file, MY_SEEK_SET, MYF(0)) == > - MY_FILEPOS_ERROR) > - { > - info->error= 0; > - return 0; > - } > - info->seek_not_done=0; > - } > - diff_length=(size_t) (pos_in_file & (IO_SIZE-1)); > - max_length=(info->read_length-diff_length); > - if (max_length >= (info->end_of_file - pos_in_file)) > - max_length= (size_t) (info->end_of_file - pos_in_file); > - > - if (!max_length) > - { > - info->error= 0; > - return 0; /* EOF */ > - } > - DBUG_EXECUTE_IF ("simulate_my_b_fill_error", > - {DBUG_SET("+d,simulate_file_read_error");}); > - if ((length= mysql_file_read(info->file, info->buffer, max_length, > - info->myflags)) == (size_t) -1) > - { > - info->error= -1; > - return 0; > - } > - info->read_pos=info->buffer; > - info->read_end=info->buffer+length; > - info->pos_in_file=pos_in_file; > - return length; > -} > - > - > -/* > Read a string ended by '\n' into a buffer of 'max_length' size. > Returns number of characters read, 0 on error. > last byte is set to '\0' > diff --git a/sql/log.cc b/sql/log.cc > index 826a4d1..5d71732 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -6694,7 +6694,6 @@ int MYSQL_BIN_LOG::write_cache(THD *thd, IO_CACHE *cache) > return ER_ERROR_ON_WRITE; > status_var_add(thd->status_var.binlog_bytes_written, length); > > - cache->read_pos=cache->read_end; // Mark buffer used up > } while ((length= my_b_fill(cache))); > > DBUG_ASSERT(carry == 0); > diff --git a/sql/wsrep_binlog.cc b/sql/wsrep_binlog.cc > index b71da70..3788a6c 100644 > --- a/sql/wsrep_binlog.cc > +++ b/sql/wsrep_binlog.cc > @@ -71,7 +71,6 @@ int wsrep_write_cache_buf(IO_CACHE *cache, uchar **buf, size_t *buf_len) > > memcpy(*buf + *buf_len, cache->read_pos, length); > *buf_len = total_length; > - cache->read_pos = cache->read_end; > } while ((cache->file >= 0) && (length = my_b_fill(cache))); > > if (reinit_io_cache(cache, WRITE_CACHE, saved_pos, 0, 0)) > @@ -200,7 +199,6 @@ static int wsrep_write_cache_once(wsrep_t* const wsrep, > > memcpy(buf + used, cache->read_pos, length); > used = total_length; > - cache->read_pos = cache->read_end; > } while ((cache->file >= 0) && (length = my_b_fill(cache))); > > if (used > 0) > @@ -270,7 +268,6 @@ static int wsrep_write_cache_inc(wsrep_t* const wsrep, > cache->read_pos, length))) > goto cleanup; > > - cache->read_pos = cache->read_end; > } while ((cache->file >= 0) && (length = my_b_fill(cache))); > > if (WSREP_OK == err) *len = total_length; > @@ -397,7 +394,6 @@ void wsrep_dump_rbr_direct(THD* thd, IO_CACHE* cache) > WSREP_ERROR("Failed to write file '%s'", filename); > goto cleanup; > } > - cache->read_pos= cache->read_end; > } while ((cache->file >= 0) && (bytes_in_cache= my_b_fill(cache))); > if(cache->error == -1) > { > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

Re: [Maria-developers] [Commits] 68b5057: always use my_b_pread() instead of mysql_file_pread()
by Sergey Vojtovich 02 Jun '15

02 Jun '15

Hi Sergei, ok to push. Regards, Sergey On Tue, May 26, 2015 at 03:38:11PM +0200, serg(a)mariadb.org wrote: > revision-id: 68b5057695165b88db38d5090f45ebe3a5149995 > parent(s): ab7640f4d30f5a160d3a1c77efe9d2c459a3bda6 > committer: Sergei Golubchik > branch nick: maria > timestamp: 2015-05-26 15:28:24 +0200 > message: > > always use my_b_pread() instead of mysql_file_pread() > > when working with IO_CACHE's, don't access IO_CACHE::file directly

1 0

[Maria-developers] MDEV-7502 Automatic provisioning of slave
by Martin Kaluznik 02 Jun '15

02 Jun '15

Provisioning has to be triggered by a command from slave, so my implementation started there. After performing initial checks, command handler starts IO and SQL threads, like during regular replication. I was trying to write separate code for provisioning IO thread. Main benefit would be clean code to start with – no backward compatibility / bug workaround code – old master will not be able to provide provisioning functionality anyway. On the other hand there would be too much duplicate code, and having to do future changes on two different places is not acceptable. I have decided to use regular replication IO thread as Kristian Nielsen suggested before and add special handling where necessary. Now when the IO thread requests binlog dump from master, it sends packet with following structure master position | binlog flags | server id | log name Master needs to receive information, that slaves IO thread is expecting provisioning data along with current binlog. I have created new BINLOG_PROVISIONING_MODE flag, but I believe that the name is incorrect and it doesn’t belong to binlog flags field. My question is, what is better for compatibility, change binlog flags to any flags and use it for sending information about provisioning request to master (as in current implementation) or add another byte to packet after log name containing provisioning mode flag and possibly more flags later. My current implementation progress can be seen at https://github.com/f4rnham/server/commits/MDEV-7502 diff of all commits at once https://github.com/MariaDB/server/compare/10.1...f4rnham:MDEV-7502 There is one more decision I am not sure about. What should result of provisioning on non-empty slave be? For example, slave contains table with same name in same database as master, but with different columns. Should slave detect already existing tables and fail with error, or silently drop everything what could cause conflicts (add optional FORCE part of provisioning start command?). Which one from these solutions should be default? Or maybe there is third one and better. Thanks for comments, Martin

2 1

[Maria-developers] [GSoC] MXS-2: Week 1
by sriram patil 01 Jun '15

01 Jun '15

Hi All, Please find links to my first blog posts of this GSoC season below. MaxScale Installation and Configuration <http://srirampatil.tumblr.com/post/120346691880/maxscale-installation-and-c…> MXS-2: Week 1 <http://srirampatil.tumblr.com/post/120349951405/mxs-2-week-1> Feel fre to point out mistakes, if any. Also, suggestions ans/or reviews are welcome! Thanks, Sriram

1 0

Re: [Maria-developers] [Commits] 52c3687: MDEV-8241: Debug build on Windows is broken: error LNK2019: unresolved external symbol pthread_detach referenced in function ma_checkpoint_init
by Sergei Golubchik 29 May '15

29 May '15

Hi, Sanja! On May 28, sanja(a)mariadb.com wrote: > revision-id: 52c36874e251c3ac5ff0f925c8133074166d481d > parent(s): 979c5049ef6fdd55c29fa5170a96c62d2a8a6823 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-05-28 07:59:57 +0200 > message: > > MDEV-8241: Debug build on Windows is broken: error LNK2019: unresolved external symbol pthread_detach referenced in function ma_checkpoint_init > > pthread_detach() replaced with pthread_detach_this_thread() > > pthread_detach_this_thread() definition fixed > > diff --git a/storage/maria/ma_checkpoint.c b/storage/maria/ma_checkpoint.c > index 2268ca6..36b729e 100644 > --- a/storage/maria/ma_checkpoint.c > +++ b/storage/maria/ma_checkpoint.c > @@ -340,7 +340,6 @@ int ma_checkpoint_init(ulong interval) > { > /* thread lives, will have to be killed */ > checkpoint_control.status= THREAD_RUNNING; > - pthread_detach(th); > } > } > DBUG_RETURN(res); > @@ -574,6 +573,8 @@ pthread_handler_t ma_checkpoint_background(void *arg) > sleeps= 1; > pages_to_flush_before_next_checkpoint= 0; > > + pthread_detach_this_thread(); > + > for(;;) /* iterations of checkpoints and dirty page flushing */ > { How does that help? What was wrong with pthread_detach()? Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 2b17bfc: MDEV-7484: Log Time not consistent with InnoDB errors nor with MySQL error log time format
by Sergei Golubchik 29 May '15

29 May '15

Hi, Jan! On May 22, Jan Lindström wrote: > revision-id: 2b17bfc6e14283aefeff87d959f3ecfc48af5c2c > parent(s): 8635c4b4e66712558330d9534491af0c9025e4dc > committer: Jan Lindström > branch nick: 10.1-encrypt > timestamp: 2015-05-22 08:24:59 +0300 > message: > > MDEV-7484: Log Time not consistent with InnoDB errors nor with MySQL error log time format Ok to push Regards, Sergei

1 0

[Maria-developers] Please review: MDEV-8222 "string_field LIKE int_const" returns a wrong result in case of UCS2
by Alexander Barkov 29 May '15

29 May '15

Hi Sergey, please review my patch for MDEV-8222. It also moves Arg_compatator from Item_bool_func2 to Item_bool_rowready_func2. Thanks.

1 0

Re: [Maria-developers] [Commits] 7819ddb: MDEV-8192 - THD::set_command() takes 0.05% in OLTP RO
by Sergei Golubchik 29 May '15

29 May '15

Hi, Sergey! On May 20, svoj(a)mariadb.org wrote: > revision-id: 7819ddb99be5dff26c8a2109ba70a54289a950ad > parent(s): fcb48ee418959ea2c05d528266bb2dcad5bce46b > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-05-20 16:55:20 +0400 > message: > > MDEV-8192 - THD::set_command() takes 0.05% in OLTP RO > > Moved THD::set_command() definition to sql_class.h so that it can be inlined. ok > Call PSI_server::set_thread_command() conditionally: it is cheaper than calling > dummy function when PFS is off (though it adds some overhead when PFS is on). I'm not sure about it. P_S will be used more and more over time, let's not intentionally penalize it. Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 6efaf34: MDEV-7011: MAX_STATEMENT_TIME has no effect in a procedure after a previous successful statement
by Sergei Golubchik 29 May '15

29 May '15

Hi, Sanja! On May 21, sanja(a)mariadb.com wrote: > revision-id: 6efaf349adec30115bdb94b9a9927f7c5103ed99 > parent(s): 3e55ef26d49a900782d2c2bb2c03470faed6ec9d > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-05-21 16:31:24 +0200 > message: > > MDEV-7011: MAX_STATEMENT_TIME has no effect in a procedure after a previous successful statement > > Do not reset timer inside stored procedures and functions. > > diff --git a/sql/sql_class.h b/sql/sql_class.h > index ca532ae..ebb76df 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -3966,6 +3966,8 @@ class THD :public Statement, > void reset_query_timer() > { > #ifndef EMBEDDED_LIBRARY > + if (spcont || in_sub_stmt) > + return; > if (!query_timer.expired) > thr_timer_end(&query_timer); > #endif I'd expect reset_query_timer() to use the same condition that set_query_timer() does: void set_query_timer() { if (!variables.max_statement_time || spcont || in_sub_stmt || slave_thread || query_timer.expired == 0) return; thr_timer_settime(&query_timer, variables.max_statement_time); } So, I'd think that reset_query_timer() could look like void reset_query_timer() { if (spcont || in_sub_stmt || slave_thread) return; if (!query_timer.expired) thr_timer_end(&query_timer); } Regards, Sergei

2 1

[Maria-developers] [GSoC] And it begins
by sriram patil 29 May '15

29 May '15

Hi All, Today is the official day (at least in my timezone) to start working on the GSoC'15 projects. I am totally excited! I will be working on MaxScale, MXS-2, for next 3 months. I will be posting a blog link every Monday starting from next week which will give an idea about what I did in the week previous to that and the tasks lined up for that week. I hope you all follow my blog. It will be great to get some inputs from the community as well. Also, all the best to all the other students who will be working with MariaDB this summer. Thanks, Sriram

2 2

Re: [Maria-developers] 8928497: MDEV-7067: Server outputs Galera (WSREP) information, even if Galera is disabled
by Sergei Golubchik 29 May '15

29 May '15

Hi, Nirbhay! On May 09, Nirbhay Choubey wrote: > revision-id: 8928497acbe3e17a105897f405d5e5138d57a249 > parent(s): 3674c363a7e77e534c3e0d28659dc614c53fbcbc > committer: Nirbhay Choubey > branch nick: 10.1-wsrep > timestamp: 2015-05-09 16:11:59 -0400 > message: > > MDEV-7067: Server outputs Galera (WSREP) information, even if Galera is disabled > > * mysqld_safe: Since wsrep_on variable is mandatory in 10.1, skip wsrep > position recovery if its OFF. > * mysqld: Remove "-wsrep" from server version > * mysqld: Remove wsrep patch version from @@version_comment > * mysqld: Introduce @@wsrep_patch_version Looks ok, I just didn't like the wsrep.cmake change. See below. > diff --git a/cmake/wsrep.cmake b/cmake/wsrep.cmake > index c37cf74..305ab27 100644 > --- a/cmake/wsrep.cmake > +++ b/cmake/wsrep.cmake > @@ -28,29 +28,22 @@ OPTION(WITH_WSREP "WSREP replication API (to use, e.g. Galera Replication librar > # Set the patch version > SET(WSREP_PATCH_VERSION "10") > > -# MariaDB addition: Revision number of the last revision merged from > -# codership branch visible in @@version_comment. > -# Branch : codership-mysql/5.6 > -SET(WSREP_PATCH_REVNO "4144") # Should be updated on every merge. > - > -# MariaDB addition: Revision number of the last revision merged from > -# Branch : lp:maria/maria-10.0-galera > -SET(WSREP_PATCH_REVNO2 "3919") # Should be updated on every merge. > - > -# MariaDB: Obtain patch revision number: > -# Update WSREP_PATCH_REVNO if WSREP_REV environment variable is set. > -IF (DEFINED ENV{WSREP_REV}) > - SET(WSREP_PATCH_REVNO $ENV{WSREP_REV}) > -ENDIF() > > -SET(WSREP_INTERFACE_VERSION 25) > +# Obtain wsrep API version > +EXECUTE_PROCESS( > + COMMAND sh -c "grep WSREP_INTERFACE_VERSION ${MySQL_SOURCE_DIR}/wsrep/wsrep_api.h | cut -d '\"' -f 2" > + OUTPUT_VARIABLE WSREP_API_VERSION > + RESULT_VARIABLE RESULT > +) I don't like calling grep|cut from cmake files. If you want to extract the version from wsrep_api.h, use cmake commands, e.g. FILE(STRINGS "${MySQL_SOURCE_DIR}/wsrep/wsrep_api.h" WSREP_API_H LIMIT_COUNT 1 REGEX "WSREP_INTERFACE_VERSION") STRING(REGEX MATCH .... > +STRING(REGEX REPLACE "(\r?\n)+$" "" WSREP_API_VERSION "${WSREP_API_VERSION}") > > -SET(WSREP_VERSION > - "${WSREP_INTERFACE_VERSION}.${WSREP_PATCH_VERSION}.r${WSREP_PATCH_REVNO}") > +SET(WSREP_VERSION "${WSREP_API_VERSION}.${WSREP_PATCH_VERSION}" > + CACHE STRING "WSREP version") make it INTERNAL, not STRING > SET(WSREP_PROC_INFO ${WITH_WSREP}) > > IF(WITH_WSREP) > - SET(COMPILATION_COMMENT "${COMPILATION_COMMENT}, wsrep_${WSREP_VERSION}") > + SET(WSREP_PATCH_VERSION "wsrep_${WSREP_VERSION}") > ENDIF() > > diff --git a/mysql-test/suite/sys_vars/r/sysvars_wsrep.result b/mysql-test/suite/sys_vars/r/sysvars_wsrep.result > index 9c392b1..1763078 100644 > --- a/mysql-test/suite/sys_vars/r/sysvars_wsrep.result > +++ b/mysql-test/suite/sys_vars/r/sysvars_wsrep.result > @@ -365,6 +365,20 @@ NUMERIC_BLOCK_SIZE NULL > ENUM_VALUE_LIST TOI,RSU > READ_ONLY NO > COMMAND_LINE_ARGUMENT OPTIONAL > +VARIABLE_NAME WSREP_PATCH_VERSION > +SESSION_VALUE NULL > +GLOBAL_VALUE wsrep_25.10 > +GLOBAL_VALUE_ORIGIN COMPILE-TIME > +DEFAULT_VALUE NULL > +VARIABLE_SCOPE GLOBAL > +VARIABLE_TYPE VARCHAR > +VARIABLE_COMMENT wsrep patch version > +NUMERIC_MIN_VALUE NULL > +NUMERIC_MAX_VALUE NULL > +NUMERIC_BLOCK_SIZE NULL > +ENUM_VALUE_LIST NULL > +READ_ONLY YES > +COMMAND_LINE_ARGUMENT NULL > VARIABLE_NAME WSREP_PROVIDER > SESSION_VALUE NULL > GLOBAL_VALUE none > diff --git a/mysql-test/suite/sys_vars/r/wsrep_patch_version_basic.result b/mysql-test/suite/sys_vars/r/wsrep_patch_version_basic.result > new file mode 100644 > index 0000000..80f29e6 > --- /dev/null > +++ b/mysql-test/suite/sys_vars/r/wsrep_patch_version_basic.result I've actually stopped creating sys_var tests for every new variable (even though I was the one who introduced sys_vars.all_vars test in the first place :). With the new I_S.SYSTEM_VARIABLES table we have all variables listed there anyway, with all properties. So there's no need to copy these boilerplate tests over and over. > diff --git a/mysql-test/suite/wsrep/suite.pm b/mysql-test/suite/wsrep/suite.pm > index 02dff24..33744a6 100644 > --- a/mysql-test/suite/wsrep/suite.pm > +++ b/mysql-test/suite/wsrep/suite.pm > @@ -27,6 +27,7 @@ push @::global_suppressions, > qr(WSREP: Could not open saved state file for reading: ), > qr(WSREP: option --wsrep-casual-reads is deprecated), > qr(WSREP: --wsrep-casual-reads=ON takes precedence over --wsrep-sync-wait=0), > + qr|WSREP: access file$gvwstate.dat$ failed$No such file or directory$|, why? > ); > > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 14d904b..49a639b 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -1902,13 +1902,15 @@ static void __cdecl kill_server(int sig_ptr) > } > #endif > > - if (WSREP_ON) > + /* Stop wsrep threads in case they are running. */ > wsrep_stop_replication(NULL); Does this print any messages? Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 6f03ea7: MDEV-8202 - st_select_lex::master_unit() takes 0.17% in OLTP RO
by Sergei Golubchik 28 May '15

28 May '15

Hi, Sergey! On May 21, svoj(a)mariadb.org wrote: > revision-id: 6f03ea78aa4a70b9a9f2f096f1d7a6c282ca9575 > parent(s): 79372cf58e426eb60acb9f0abf862a1f0fe0e4e3 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-05-21 19:35:49 +0400 > message: > > MDEV-8202 - st_select_lex::master_unit() takes 0.17% in OLTP RO > > To make st_select_lex::master_unit() inlinable: > - moved it's definition to sql_lex.h > - removed base class virtual master_unit() declaration since this method is > specific to st_select_lex ok to push Regards, Sergei

1 0

[Maria-developers] MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
by Alexander Barkov 28 May '15

28 May '15

Hello Sergey, Please review the next step for MDEV-7950. It does the following: - Adds specific implementations of virtual method get_mm_tree() into Item_func_like, Item_bool_rowready_func2, Item_func_spatial_rel - Gets rid of two virtual calls: select_optimize() and have_rev_func(). - Removes virtual methods Item_func::select_optimize(), Item_func::have_rev_func(), as well as enum Item_func::optimize_type, because they are not needed any more. Thanks.

2 1

Re: [Maria-developers] [Commits] 8a595a9: cleanup: LOAD DATA replication support in IO_CACHE
by Sergey Vojtovich 28 May '15

28 May '15

Hi Sergei, a few thoughts/questions inline. On Tue, May 26, 2015 at 03:37:36PM +0200, serg(a)mariadb.org wrote: > revision-id: 8a595a9825de286ad073f6f043db4c0dce88a2ea > parent(s): 3e55ef26d49a900782d2c2bb2c03470faed6ec9d > committer: Sergei Golubchik > branch nick: maria > timestamp: 2015-05-26 15:09:25 +0200 > message: > > cleanup: LOAD DATA replication support in IO_CACHE > > remove some 14-year old code that added support for > LOAD DATA replication to IO_CACHE: > * three callbacks, of which only two were actually used and that > were only needed for LOAD DATA replication but were > tested in every IO_CACHE instance > * an additional opaque void * argument in IO_CACHE, also only > used for LOAD DATA replication, but present everywhere > * the code to close IO_CACHE prematurely in LOAD DATA to have > these callbacks called in the correct order and a long > comment explaining what will happen if IO_CACHE is not > closed prematurely > * a variable to track whether IO_CACHE was closed prematurely > (to avoid double-closing it) > > this also fixes a bug with replication of LOAD DATA LOCAL > (IO_CACHE of READ_NET type was not calling these callbacks at all) I couldn't track this down. That is GET/my_b_get/pre_read was called in both cases as well as end_io_cache/pre_close. Could you elaborate? ...skip... > diff --git a/sql/sql_load.cc b/sql/sql_load.cc > index 18982c5..62b6e17 100644 > --- a/sql/sql_load.cc > +++ b/sql/sql_load.cc > @@ -1410,20 +1372,15 @@ READ_INFO::READ_INFO(File file_par, uint tot_length, CHARSET_INFO *cs, > } > else > { > - /* > - init_io_cache() will not initialize read_function member > - if the cache is READ_NET. So we work around the problem with a > - manual assignment > - */ > - need_end_io_cache = 1; > - > #ifndef EMBEDDED_LIBRARY > if (get_it_from_net) > cache.read_function = _my_b_net_read; > > if (mysql_bin_log.is_open()) > - cache.pre_read = cache.pre_close = > - (IO_CACHE_CALLBACK) log_loaded_block; > + { > + cache.real_read_function= cache.read_function; > + cache.read_function= log_loaded_block; > + } > #endif > } > } Will this compile with replication disabled? OTOH I guess it didn't compile even before: log_loaded_block() seem to be unavailable when replication is disabled. IOCACHE::read_function may change, e.g. on error in _my_b_async_read(). Shouldn't we handle this? > @@ -1432,8 +1389,7 @@ READ_INFO::READ_INFO(File file_par, uint tot_length, CHARSET_INFO *cs, > > READ_INFO::~READ_INFO() > { > - if (need_end_io_cache) > - ::end_io_cache(&cache); > + ::end_io_cache(&cache); > my_free(buffer); > List_iterator<XML_TAG> xmlit(taglist); > XML_TAG *t; Hmm... I may miss something, but shouldn't we call log_loaded_block() for last successfully read block? Thanks, Sergey

2 2

Re: [Maria-developers] [Commits] 1cd9319: myisam/aria: don't mess with IO_CACHE::file
by Sergei Golubchik 28 May '15

28 May '15

Hi, Sergey! On May 27, Sergey Vojtovich wrote: > Hi Sergei, > > I'm lost now. In Aria you did change very similar code and kept > init_io_cache(-1) while init_io_cache(info->dfile) was moved down > and replaced with init_io_cache(new_file). This sounds reasonable. > FWICS MyISAM has very same code. > > Isn't "temp" in fact info->dfile and not -1? Ok, the original problem. init_io_cache(-1) means that IO_CACHE uses a temporary file as a backing store. That is, when in-memory buffer is full, IO_CACHE creates a temporary file to keep the data. MyISAM in a couple of places was creating IO_CACHE with init_io_cache(-1) and later was replacing IO_CACHE::file. That is, IO_CACHE was created as if it was a temporary storage, but later changed to use a real file. Which is not a good idea now, as I (in a later changeset) detect temporary IO_CACHE's in init_io_cache. So, I've fixed MyISAM/Aria not to access internal IO_CACHE fields and create IO_CACHE to have the correct file descriptior from the start. Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 1cd9319: myisam/aria: don't mess with IO_CACHE::file
by Sergei Golubchik 27 May '15

27 May '15

Hi, Sergey! On May 27, Sergey Vojtovich wrote: > Hi Sergei, > > looks good. Ok to push, a few minor suggestions inline. > > > diff --git a/storage/maria/ma_check.c b/storage/maria/ma_check.c > > index 8c012d2..d50528e 100644 > > --- a/storage/maria/ma_check.c > > +++ b/storage/maria/ma_check.c > > @@ -4267,20 +4267,22 @@ int maria_repair_parallel(HA_CHECK *param, register MARIA_HA *info, > > > > if (!(sort_info.key_block= > > alloc_key_blocks(param, (uint) param->sort_key_blocks, > > - share->base.max_key_block_length)) || > > - init_io_cache(&param->read_cache, info->dfile.file, > > + share->base.max_key_block_length))) > > + goto err; > > + > > + if (init_io_cache(&param->read_cache, info->dfile.file, > > (uint) param->read_buffer_length, > > - READ_CACHE, share->pack.header_length, 1, MYF(MY_WME)) || > > - (!rep_quick && > > - (init_io_cache(&info->rec_cache, info->dfile.file, > > - (uint) param->write_buffer_length, > > - WRITE_CACHE, new_header_length, 1, > > - MYF(MY_WME | MY_WAIT_IF_FULL) & param->myf_rw) || > > - init_io_cache(&new_data_cache, -1, > > - (uint) param->write_buffer_length, > > - READ_CACHE, new_header_length, 1, > > - MYF(MY_WME | MY_DONT_CHECK_FILESIZE))))) > > + READ_CACHE, share->pack.header_length, 1, MYF(MY_WME))) > > goto err; > > + > > + if (!rep_quick) > > + { > > + if (init_io_cache(&new_data_cache, -1, > > + (uint) param->write_buffer_length, > > + READ_CACHE, new_header_length, 1, > > + MYF(MY_WME | MY_DONT_CHECK_FILESIZE))) > > + goto err; > > + } > Why not to move new_data_cache initialization down along with rec_cache? I've simply kept it where it was, the goal was to avoid io_cache.file=new_file_descriptor; for example, there was info->rec_cache.file=new_file; and I moved info->rec_cache initialization down where new_file was known and could be passed as an argument to init_io_cache(). There was no such problem with new_data_cache. Regards, Sergei

2 2

[Maria-developers] step7 (cleanup) MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
by Alexander Barkov 27 May '15

27 May '15

Hi Sergey, please review the next step for MDEV-7950. It breaks get_mm_parts() into a virtual method in Item, so - replaces one virtual call item->type() to another virtual call, item->get_mm_tree(). - and also removes this virtual call for functype(), which used to distinguish between COND_AND_FUNC and COND_OR_FUNC: >- if (((Item_cond*) cond)->functype() == Item_func::COND_AND_FUNC) The next patches will: 1. Split the code in Item::get_mm_tree() responsible for functions into methods in Item_func and its descendants: Item_func Item_func_like, to get rid of the select_optimize() call Item_func_between Item_func_in Item_equal 2. Remove virtual method Item_func::select_optimize(), as well as enum Item_func::optimize_type, because they won't be needed any more. Thanks.

2 3

[Maria-developers] Memory barrier problem in InnoDB/XtraDB mutex implementation causing hangs
by Kristian Nielsen 27 May '15

27 May '15

I have been debugging some test failures seen in Buildbot: https://mariadb.atlassian.net/browse/MDEV-7026 After some debugging, this is what I think is going on. I will refer to MariaDB 5.5 sources, though the problem is also in 10.0. InnoDB/XtraDB mutexes are taken in mutex_enter_func() and unlocked in mutex_exit_func(). Simplifying for clarity, here are the relevant parts of the logic: ENTER: # First spin wait, hoping to get the mutex without yielding. os_compare_and_swap_ulint(&mutex->waiters, 0, 1); if (os_atomic_test_and_set_byte(&mutex->lock_word, 1)) sync_array_wait_event() RELEASE: os_atomic_lock_release_byte(&mutex->lock_word); os_rmb; waiters = *(volatile *)&mutex->waiters; if (waiters) mutex_signal_object(mutex); The interesting point here is how to ensure that we do not lose the wakeup of a waiting thread when the holding thread releases the mutex. The idea is that the waiting thread first sets mutex->waiters=1. Then it checks if the mutex is free - if not, it goes to sleep. The thread that releases the lock first marks the mutex free. Then it checks if there are any waiters, and if so wakes them up. For this to work, we need to ensure that the setting of waiters=1 has completed before the load of lock_word can start - a full memory barrier. Likewise, we need the setting of lock_word to 0 to complete before the load of mutex->waiters - again a full memory barrier. Otherwise we can get something like this: ENTER RELEASE Start store to mutex->waiters Start store to lock_word Load mutex->lock_word Store to lock_word becomes visible Load mutex->waiters Store to waiters becomes visible Check lock word==1 Check waiters==0, skip wakeup Sleep (forever) Or this: ENTER RELEASE Start store to lock_word Start store to mutex->waiters Load mutex->waiters Store to waiters becomes visible Load mutex->lock_word Store to lock_word becomes visible Check lock word==1 Sleep (forever) Check waiters==0, skip wakeup (So both full barriers are needed, the one in ENTER to prevent the first example, the one in RELEASE to prevent the second). It looks like for x86, the full barriers were broken in MariaDB 5.5.40 and 10.0.13. os_compare_and_swap_ulint() is defined as either __sync_bool_compare_and_swap() or atomic_cas_ulong(), both of which implies a full barrier. So ENTER looks ok. os_atomic_lock_release_byte() however is now defined as __sync_lock_release(). This is _not_ a full barrier, just a release barrier. So the RELEASE case seems to be incorrect, allowing the second deadlock example to occur. Previously, RELEASE used os_atomic_test_and_set_byte(), which is __sync_lock_test_and_set(). This is defined in GCC docs as an "aquire" barrier only, which is also wrong, obviously. But it happens to generate an XCHG instruction on x86, which is a full barrier. This is probably why it works on x86 (and probably why it did not work on PowerPC). Here is a small test program, and the assembly from GCC 5.7 on amd64: void test(unsigned *p, unsigned *q) { while (__sync_lock_test_and_set(q, 1)) { } *p++; __sync_lock_release(q); } test: movl $1, %edx .L2: movl %edx, %eax xchgl (%rsi), %eax testl %eax, %eax jne .L2 movl $0, (%rsi) ret As can be seen, the __sync_lock_test_and_set() generates a full barrier, but the __sync_lock_release() generates nothing (as this is sufficient for release semantics for the rather strong memory order on x86). So the InnoDB mutexes look completely broken from MariaDB 5.5.40 and 10.0.13 :-( This is consistent with the observations in MDEV-7026. A thread is waiting for wakeup on a mutex that is already marked free. Apparently this is not a new problem. This comment in mutex_exit_func() makes me want to cry: /* A problem: we assume that mutex_reset_lock word is a memory barrier, that is when we read the waiters field next, the read must be serialized in memory after the reset. A speculative processor might perform the read first, which could leave a waiting thread hanging indefinitely. Our current solution call every second sync_arr_wake_threads_if_sema_free() to wake up possible hanging threads if they are missed in mutex_signal_object. */ So someone found the mutex implementation broken, but instead of fixing it, they implemented a horrible workaround to try and break any deadlocks that might occur. (The reason that the deadlocks are not broken in the MDEV-7026 seems to be that it occurs during startup, before starting the srv_error_monitor_thread() that does the sync_arr_wake_threads_if_sema_free()). Now, what is also interesting is that this seems to explain very nicely the many server lockups we have seen for users that upgrade to recent MariaDB versions. I was doing a very detailed investigation of one such case where the user was able to supply unusually detailed information. I found a thread that was stuck waiting on a mutex, but no other threads seemed to be holding that mutex. This is exactly what would be seen in case of a lost wakeup due to this issue. Furthermore, the sync_arr_wake_threads_if_sema_free() mechanism, as well as the normal InnoDB "long semaphore wait", was itself blocked on that same mutex wait, which explains why they were not effective for the user in that particular case. So it looks like we need to revert the incorrect patch and release new 5.5.40a and 10.0.15 releases ASAP, to solve these hangs for users. And do a proper patch for PowerPC with proper full memory barriers. (Incidentally, the os_rmb in RELEASE seems to also be part of some PowerPC patch, it also looks completely wrong). However, more importantly, this issue clearly shows some fundamental problems in our development procedures that we need to fix ASAP. Just look at it. We have users (and customers!) that suffer from strange hangs. This is their production systems that are down. Support and developers are unable to help them due to the difficulty of debugging rare race conditions in production systems. And all the time the bug was staring us in the face, easily reproducable in our own Buildbot! But we deliberately chose to ignore those failures. The only thing that matters is "when can you push", and getting more releases out. Never mind what kind of crap is then pushed out to users. I am deeply ashamed of treating our users like this. And extremely unhappy. I even wrote about this (the importance of not ignoring Buildbot failures) just a few weeks ago. This was completely ignored, I got zero replies, and the weekly calls continued with mindless juggling of Jira numbers and buggy releases. As a result, our users suffer. We need to stop this. Please? - Kristian.

4 16

Re: [Maria-developers] Rotating from encry
by Jonas Oreland 22 May '15

22 May '15

8043 I tried using a search engine and googled for "jira mariadb innodb encryption jonas" :) /Jonas On Thu, May 21, 2015 at 4:36 PM, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: > Hi, > > Can you give the MDEV, I do not seem to be able to find it. > > R: Jan > > On Thu, May 21, 2015 at 2:51 PM, Jonas Oreland <jonaso(a)google.com> wrote: > >> i made a JIRA entry with latest innodb encryption... >> >> /Jonas >> >> On Thu, May 21, 2015 at 12:30 PM, Jan Lindström < >> jan.lindstrom(a)mariadb.com> wrote: >> >>> Hi, >>> >>> On Thu, May 21, 2015 at 11:33 AM, Jonas Oreland <jonaso(a)google.com> >>> wrote: >>> >>>> Hi Jan, >>>> >>>> 1) rotating encrypted => unencrypted is definitely supported, >>>> in my latest version the TODO is removed...(hope you have a recent >>>> enough version) >>>> >>> >>> Based on this, I do not have the latest version. Where, I could get >>> that, it does not seem to be on branch? >>> >>>> >>>> >>>> 2) Thanks! for testcase, it did indeed reveal a bug with the "encrypted >>>> => unencrypted => encrypted" sequence >>>> >>>> attaching fix. >>>> >>>> Let me know it fixes your version of testcase (i've used a modified >>>> version since e.g we don't have per table settings...) >>>> >>> >>> Yes, this fix is good. Thanks. >>> >>> R: Jan >>> >> >> >

1 0

[Maria-developers] How do I read information_schema.global_status not using query.
by kyung nam Kim 22 May '15

22 May '15

Hi, I hava a trouble with getting a data from informaition_schema.global_status table not using query. I wrote below code and it's possible to get data from some table in performance_schema . ** reader.cc =============================================================================================== my_thread_init(); ... MEM_ROOT mem_field; READ_RECORD read_record_info; tables.init_one_table(STRING_WITH_LEN("performance_schema"), STRING_WITH_LEN("events_statements_current"), bool result = open_and_lock_tables(ctx->thd, &tables, FALSE, MYSQL_LOCK_IGNORE_TIMEOUT); if (result) { return false; } if(table = tables.table) /* if only table exists */ { init_sql_alloc(&mem_field, 1024, 0, MYF(0)); ... while (!(error= read_record_info.read_record(&read_record_info))) { char* value = get_field(&mem_field, table->field[0]); // It's OK. .... } } ... ==================================================================================== It's possible to read data from performance_schema tables using low-level function . But I tried to read data from information_schema.global_status table as same way(just changed the name of database and table) , an error occurred in open_and_lock_tables(). Is there a way to read data from information_schema.global_status not using query? Regards, Kim kyungnam.

1 0

Re: [Maria-developers] Rotating from encry
by Jonas Oreland 21 May '15

21 May '15

i made a JIRA entry with latest innodb encryption... /Jonas On Thu, May 21, 2015 at 12:30 PM, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: > Hi, > > On Thu, May 21, 2015 at 11:33 AM, Jonas Oreland <jonaso(a)google.com> wrote: > >> Hi Jan, >> >> 1) rotating encrypted => unencrypted is definitely supported, >> in my latest version the TODO is removed...(hope you have a recent enough >> version) >> > > Based on this, I do not have the latest version. Where, I could get that, > it does not seem to be on branch? > >> >> >> 2) Thanks! for testcase, it did indeed reveal a bug with the "encrypted >> => unencrypted => encrypted" sequence >> >> attaching fix. >> >> Let me know it fixes your version of testcase (i've used a modified >> version since e.g we don't have per table settings...) >> > > Yes, this fix is good. Thanks. > > R: Jan >

1 0

Re: [Maria-developers] Rotating from encry
by Jonas Oreland 21 May '15

21 May '15

Hi Jan, 1) rotating encrypted => unencrypted is definitely supported, in my latest version the TODO is removed...(hope you have a recent enough version) 2) Thanks! for testcase, it did indeed reveal a bug with the "encrypted => unencrypted => encrypted" sequence attaching fix. Let me know it fixes your version of testcase (i've used a modified version since e.g we don't have per table settings...) /Jonas On Fri, May 15, 2015 at 1:23 PM, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: > Hi, > > At fil0crypt.cc there is > > fil_crypt_needs_rotation(uint key_version, const key_state_t *key_state) > { > // TODO(jonaso): Add support for rotating encrypted => unencrypted > > if (key_version == 0 && key_state->key_version != 0) { > /* this is rotation unencrypted => encrypted > * ignore rotate_key_age */ > return true; > } > > Thus to me it is not clear is the support for rotating encrypted => > unencrypted really missing or not and furthermore, see attached test case > for this, > > encrypted + insert + grep : ok > encrypted => unencrypted + grep: ok > unencrypted => encrypted + grep: not ok > > R: Jan >

1 0