- developers - lists.mariadb.org

[Maria-developers] Issue with commit 0f43279c / MDEV-13936
by jerome brauge 12 Dec '17

12 Dec '17

Hi All, Since some time ago, my app crashed the server (10.2.-ext) in some case (repeatable) with the following stack trace : On Linux Centos 7: ./sql/mysqld(my_print_stacktrace+0x38)[0x7f463f49a093] ./sql/mysqld(handle_fatal_signal+0x3a3)[0x7f463ed2747a] /lib64/libpthread.so.0(+0xf370)[0x7f463dffd370] ./sql/mysqld(_ZN13Explain_query10get_selectEj+0x40)[0x7f463ebf4d06] ./sql/mysqld(_ZN13Explain_query8add_nodeEP12Explain_node+0x203)[0x7f463ebf4f15] ./sql/mysqld(_ZN4JOIN24save_explain_data_internEP13Explain_querybbbPKc+0x959)[0x7f463eaf3db9] ./sql/mysqld(_ZN4JOIN17save_explain_dataEP13Explain_querybbbb+0x14b)[0x7f463eabb963] ./sql/mysqld(_ZN4JOIN13build_explainEv+0xe2)[0x7f463eab3b70] ./sql/mysqld(_ZN4JOIN8optimizeEv+0xb6)[0x7f463eab3d2e] ./sql/mysqld(_ZN13st_select_lex31optimize_unflattened_subqueriesEb+0x263)[0x7f463ea57b5d] ./sql/mysqld(_ZN4JOIN31optimize_unflattened_subqueriesEv+0x24)[0x7f463ec1b746] ./sql/mysqld(_ZN4JOIN15optimize_stage2Ev+0x1cdb)[0x7f463eab7171] ./sql/mysqld(_ZN4JOIN14optimize_innerEv+0x14dd)[0x7f463eab542d] ./sql/mysqld(_ZN4JOIN8optimizeEv+0x7a)[0x7f463eab3cf2] ./sql/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x263)[0x7f463eabccd2] ./sql/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x16c)[0x7f463eab13e6] ./sql/mysqld(_Z21mysql_execute_commandP3THD+0x4b5b)[0x7f463ea76fb4] ./sql/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x80)[0x7f463eeaafb0] ./sql/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x1d1)[0x7f463eeaa549] ./sql/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1b5)[0x7f463eeaabf9] ./sql/mysqld(_ZN7sp_head7executeEP3THDb+0x754)[0x7f463eea4ff6] ./sql/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x971)[0x7f463eea74df] ./sql/mysqld(+0x645959)[0x7f463ea71959] ./sql/mysqld(_ZN12Sql_cmd_call7executeEP3THD+0x270)[0x7f463ea7240c] ./sql/mysqld(_Z21mysql_execute_commandP3THD+0x9f95)[0x7f463ea7c3ee] ./sql/mysqld(_ZN18Prepared_statement7executeEP6Stringb+0x508)[0x7f463ea9e468] ./sql/mysqld(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+0x200)[0x7f463ea9c6fa] ./sql/mysqld(+0x66e113)[0x7f463ea9a113] ./sql/mysqld(_Z19mysqld_stmt_executeP3THDPcj+0x97)[0x7f463ea99cd7] ./sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0xc9e)[0x7f463ea6e68d] ./sql/mysqld(_Z10do_commandP3THD+0x700)[0x7f463ea6d315] ./sql/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1ca)[0x7f463ebbe3d2] ./sql/mysqld(handle_one_connection+0x30)[0x7f463ebbe125] ./sql/mysqld(+0x1012852)[0x7f463f43e852] /lib64/libpthread.so.0(+0x7dc5)[0x7f463dff5dc5] /lib64/libc.so.6(clone+0x6d)[0x7f463c01673d] Or on Windows : Unhandled exception thrown: read access violation. Dynamic_array<Explain_select *>::at(...) returned 0xDB925000. > mysqld.exe!Explain_query::~Explain_query() Line 71 C++ mysqld.exe!Explain_query::`scalar deleting destructor'(unsigned int) C++ mysqld.exe!delete_explain_query(LEX * lex) Line 2397 C++ mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core(THD * thd, unsigned int * nextp, bool open_tables, sp_instr * instr) Line 3130 C++ mysqld.exe!sp_instr_stmt::execute(THD * thd, unsigned int * nextp) Line 3266 C++ mysqld.exe!sp_head::execute(THD * thd, bool merge_da_on_success) Line 1213 C++ mysqld.exe!sp_head::execute_procedure(THD * thd, List<Item> * args) Line 2072 C++ mysqld.exe!do_execute_sp(THD * thd, sp_head * sp) Line 2907 C++ mysqld.exe!Sql_cmd_call::execute(THD * thd) Line 3147 C++ mysqld.exe!mysql_execute_command(THD * thd) Line 6223 C++ mysqld.exe!Prepared_statement::execute(String * expanded_query, bool open_cursor) Line 4682 C++ mysqld.exe!Prepared_statement::execute_loop(String * expanded_query, bool open_cursor, unsigned char * packet, unsigned char * packet_end) Line 4111 C++ mysqld.exe!mysql_stmt_execute_common(THD * thd, unsigned long stmt_id, unsigned char * packet, unsigned char * packet_end, unsigned long cursor_flags, bool bulk_op, bool read_types) Line 3132 C++ mysqld.exe!mysqld_stmt_execute(THD * thd, char * packet_arg, unsigned int packet_length) Line 3028 C++ mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length, bool is_com_multi, bool is_next_command) Line 1763 C++ mysqld.exe!do_command(THD * thd) Line 1369 C++ mysqld.exe!threadpool_process_request(THD * thd) Line 366 C++ mysqld.exe!tp_callback(TP_connection * c) Line 192 C++ mysqld.exe!tp_callback(_TP_CALLBACK_INSTANCE * instance, void * context) Line 377 C++ mysqld.exe!work_callback(_TP_CALLBACK_INSTANCE * instance, void * context, _TP_WORK * work) Line 451 C++ The sql statement involved (in a stored procedure): Query (0x7f457df65bb8): INSERT INTO qnwtc( enrqnwtc, priqnwtc, lprqnwtc, cprqnwtc, nuiqnwtc) SELECT 'PRI' ,occgtpar ,intgtpar ,ch1gtpar , NAME_CONST('wpio_ident',110325)FROM gtpara WHERE padgtpar = 'PRIQNTAC' AND numgtets = NAME_CONST('wsysets',_utf8'IFR' COLLATE 'utf8_bin')AND etagtpar = 'A' Where gtpara is a view. If the stored procedure is called alone, it works fine. I've revert commit 0f43279c (MDEV-13936) and now my app works fine again. Regards.

1 0

Re: [Maria-developers] Mdev-10715 Initial Patch
by Sachin Setiya 11 Dec '17

11 Dec '17

Hi Andrei! Thanks for the review. On Thu, Oct 12, 2017 at 11:48 PM, <andrei.elkin(a)pp.inet.fi> wrote: > Howdy, Sachin! > > I've checked the patch mostly from the high-level design pov to gather > few suggestions. Find them on the top of diff hunks (hopefully the diff > remains applicable) below indented with '*review*>'. > > > > revision-id: daf9a67ea13c5c7309f85a104e7c07ec68f65e88 (mariadb-10.1.28-10-gdaf9a67) > parent(s): 5eb666ad37ff80e5f8dc714bb68fcb1e269b797a > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-10-10 17:45:21 +0530 > message: > > Mdev-10715 Initial Patch > *review*> The initial patch is a very good time to write down the work > intent and its key features. Done > Could you please do that and highlight how WSREP has really > become multi-source compatible. > > > --- > mysql-test/suite/galera/r/galera_gtid_slave.result | 18 +++ > .../r/galera_gtid_slave_sst_mysqldump.result | 92 ++++++++++++ > .../galera/r/galera_gtid_slave_sst_rsync.result | 92 ++++++++++++ > mysql-test/suite/galera/t/galera_gtid_slave.cnf | 18 +++ > mysql-test/suite/galera/t/galera_gtid_slave.test | 67 +++++++++ > .../galera/t/galera_gtid_slave_sst_mysqldump.cnf | 18 +++ > .../galera/t/galera_gtid_slave_sst_mysqldump.test | 156 +++++++++++++++++++++ > .../suite/galera/t/galera_gtid_slave_sst_rsync.cnf | 19 +++ > .../galera/t/galera_gtid_slave_sst_rsync.test | 156 +++++++++++++++++++++ > sql/log.cc | 37 ++++- > sql/wsrep_mysqld.cc | 9 ++ > 11 files changed, 681 insertions(+), 1 deletion(-) > > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave.result b/mysql-test/suite/galera/r/galera_gtid_slave.result > new file mode 100644 > index 0000000..7ab10c7 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave.result > @@ -0,0 +1,18 @@ > +START SLAVE; > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2 > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2,2-2-2 > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2,2-2-2,2-3-3 > +DROP TABLE t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > new file mode 100644 > index 0000000..8e8e2e0 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > @@ -0,0 +1,92 @@ > +START SLAVE; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +Shutting down server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-3-8,2-2-11 > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-3-8,2-2-11 > +DROP TABLE t2,t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > new file mode 100644 > index 0000000..9ab4601 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > @@ -0,0 +1,92 @@ > +START SLAVE; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +Shutting down server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +SELECT COUNT(*) from t1; > +COUNT(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +DROP TABLE t2,t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.cnf b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.test b/mysql-test/suite/galera/t/galera_gtid_slave.test > new file mode 100644 > index 0000000..3b3f6f3 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.test > @@ -0,0 +1,67 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/have_innodb.inc > + > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 1 FROM t1; > +--source include/wait_condition.inc > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t1; > +--source include/wait_condition.inc > + > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t1; > + > +# > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 1 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > new file mode 100644 > index 0000000..67aad21 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > @@ -0,0 +1,156 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t2; > +--source include/wait_condition.inc > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > + > +--connection node_2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--connection node_3 > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--connection node_2 > +Select * from t1 order by f1; > + > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +# > +#stop slave on node 2 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +#start slave > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > new file mode 100644 > index 0000000..2c82531 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > @@ -0,0 +1,19 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > +wsrep_sst_method=rsync > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > new file mode 100644 > index 0000000..67aad21 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > @@ -0,0 +1,156 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t2; > +--source include/wait_condition.inc > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > + > +--connection node_2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--connection node_3 > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--connection node_2 > +Select * from t1 order by f1; > + > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +# > +#stop slave on node 2 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +#start slave > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > > *review*> The better timing for the non-zero seq_no Gtid write deems to > inside THD::binlog_start_trans_and_stmt() invoked few instructions > earlier. This method is invoked once per *transaction* which > guarantees no duplication which the current chosen point seems to > concede. Perhaps you change its singature to pass the writer. Done as we discussed. this is added into binlog_statrt_trans_and_stmt() And I am checking for ha_info->is_started(). > Also I think we may write the non-zero seq_no Gtid generally, regardless > of WITH_WSREP (naturally to block doing that later at binlog commit > phase, also commented below). This commit will go into 10.1 so as we discussed , we can do this later in 10.3. > > diff --git a/sql/log.cc b/sql/log.cc > index a9f486d..53eeadf 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -5655,6 +5655,23 @@ int THD::binlog_write_table_map(TABLE *table, bool is_transactional, > DBUG_RETURN(error); > } > } > +#ifdef WITH_WSREP > + /* Get domain id only when gtid mode is set > + If this event is replicate through a master then , > + we will forward the same gtid another nodes > + */ > + if (wsrep_gtid_mode && this->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(this, this->variables.gtid_seq_no, > + this->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= this->variables.server_id; > + > + if ((error= writer.write(&gtid_event))) > + DBUG_RETURN(error); > + } > +#endif > if ((error= writer.write(&the_event))) > DBUG_RETURN(error); > > @@ -5826,7 +5843,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > DBUG_PRINT("enter", ("standalone: %d", standalone)); > > #ifdef WITH_WSREP > - if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && wsrep_gtid_mode) > + if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && wsrep_gtid_mode && !thd->variables.gtid_seq_no) > { > domain_id= wsrep_gtid_domain_id; > } else { > > *review*> I'm suggest we check the non-zero seq_no status in > MYSQL_BIN_LOG::write_gtid_event() > > *review*>diff --git a/sql/log.cc b/sql/log.cc > *review*>index 2fcb6f6dbf6..8229426a5bd 100644 > *review*>--- a/sql/log.cc > *review*>+++ b/sql/log.cc > *review*>@@ -5867,7 +5867,6 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > *review*> err= rpl_global_gtid_binlog_state.update_with_next_gtid(domain_id, > *review*> local_server_id, &gtid); > *review*> seq_no= gtid.seq_no; > *review*>- } > *review*> if (err) > *review*> DBUG_RETURN(true); > *review*> thd->last_commit_gtid= gtid; > *review*>@@ -5880,6 +5879,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > *review*> DBUG_ASSERT(this == &mysql_bin_log); > *review*> if (write_event(&gtid_event)) > *review*> DBUG_RETURN(true); > *review*>+ } > *review*> status_var_add(thd->status_var.binlog_bytes_written, gtid_event.data_written); > *review*> > *review*> DBUG_RETURN(false); > > *review*> so only the zero auto-evaluated seq_no Gtid would make to the > method you're trying to patch. And the hunk is unneeded in this > case. Like above we generalize WITH_WSREP away. > ^^^^^^^ I am not sure about this hunk , I think this was mistake. I have reverted this change. > @@ -6718,6 +6735,24 @@ int MYSQL_BIN_LOG::write_cache(THD *thd, IO_CACHE *cache) > mysql_mutex_assert_owner(&LOCK_log); > if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0)) > DBUG_RETURN(ER_ERROR_ON_WRITE); > + uchar *read_pos= cache->read_pos; > + > +#ifdef WITH_WSREP > + /* > + In the cache we have gtid event if , below condition is true, > + then we will simply increment the read position of cache by > + the length of Gtid_log_event. > + */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + uchar *head= read_pos; > + uint data_len= uint4korr(head + EVENT_LEN_OFFSET); > + uint event_type= (uchar)head[EVENT_TYPE_OFFSET]; > + if (event_type == GTID_LOG_EVENT) > + cache->read_pos+= data_len; > + } > +#endif > + > uint length= my_b_bytes_in_cache(cache), group, carry, hdr_offs; > long val; > ulong end_log_pos_inc= 0; // each event processed adds BINLOG_CHECKSUM_LEN 2 t > > *review*> The following block is fine *but* we shall explain > what is GTID_SUPPORT, whether it actually for us. > Well , there is no such thing as GTID_SUPPORT :( , we are using ifdef without even defining gtid_support ╭─sachin@sachin-desk ~/10715/server ‹bb-10.1-10715*› ╰─$ grep -Rnie 'GTID_SUPPORT' sql/wsrep_sst.cc:290:#ifdef GTID_SUPPORT sql/wsrep_sst.cc:292:#endif /* GTID_SUPPORT */ sql/wsrep_applier.cc:130:#ifdef GTID_SUPPORT sql/wsrep_applier.cc:141:#endif /* GTID_SUPPORT */ sql/wsrep_applier.cc:309:#ifdef GTID_SUPPORT sql/wsrep_applier.cc:311:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:50:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:53:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:256:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:274:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:413:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:415:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:1222:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:1229:#endif /* GTID_SUPPORT */ sql/sql_class.h:4085:#ifdef GTID_SUPPORT sql/sql_class.h:4087:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.h:261:#ifdef GTID_SUPPORT sql/wsrep_mysqld.h:263:#endif /* GTID_SUPPORT */ > diff --git a/sql/wsrep_mysqld.cc b/sql/wsrep_mysqld.cc > index 33689a9..65751f1 100644 > --- a/sql/wsrep_mysqld.cc > +++ b/sql/wsrep_mysqld.cc > @@ -1227,6 +1227,15 @@ int wsrep_to_buf_helper( > if (!ret && writer.write(&gtid_ev)) ret= 1; > } > #endif /* GTID_SUPPORT */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(thd, thd->variables.gtid_seq_no, > + thd->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= thd->variables.server_id; > + ret= writer.write(&gtid_event); > + } > > /* if there is prepare query, add event for it */ > if (!ret && thd->wsrep_TOI_pre_query) -- Regards Sachin Setiya Software Engineer at MariaDB

1 0

[Maria-developers] Mysterious files after build
by Michal Schorm 09 Dec '17

09 Dec '17

Hello, I'd like to know more about some files, that are left after MariaDB build in Fedora. I do not ship them, however I haven't found any explanation from previous maintainers. I gathered some clues and thoughts about them. Questions are: What are those files and what they does?; Is it safe to ship them?; Should I start to ship them? /usr/bin/mysql_embedded - you (MariaDB) ship this file in client package %{_datadir}/mariadb/binary-configure - this script creates the MySQL system tables and starts the server. %{_datadir}/mariadb/magic - FS files first-bytes recoginiton? %{_datadir}/mariadb/mysql.server %{_datadir}/%{pkg_name}/mysqld_multi.server - those shouldn't be needed on systemd OSs ? /usr/bin/mytop - you (mariaDB) do not ship this file -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

2 1

Re: [Maria-developers] b610986: MDEV-14593 human-readable XA RECOVER
by Sergei Golubchik 08 Dec '17

08 Dec '17

Hi, Alexey! Please, see few comments below. On Dec 07, Alexey Botchkov wrote: > revision-id: b610986c9c226e9b9cbe7c80ceaead1de0a55535 (mariadb-10.2.11-26-gb610986) > parent(s): 4d016e6ed2d32298977a66e125cbfcae39e23847 > committer: Alexey Botchkov > timestamp: 2017-12-07 23:49:12 +0400 > message: > > MDEV-14593 human-readable XA RECOVER. > > diff --git a/sql/handler.cc b/sql/handler.cc > index 7eed722..128721c 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -1978,6 +1978,97 @@ int ha_recover(HASH *commit_list) > } > > /** > + return the XID as it appears in the SQL function's arguments. > + So this string can be passed to XA START, XA PREPARE etc... > + > + @note > + the 'buf' has to have space for at least SQL_XIDSIZE bytes. > +*/ > + > + > +/* > + 'a'..'z' 'A'..'Z', '0'..'9' > + and '-' '_' ' ' symbols don't have to be > + converted. I'd say it's overcomplication, why not to convert everything to hex? > +*/ > + > +static const char xid_needs_conv[128]= > +{ > + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, > + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, > + 0,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1, > + 0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1, > + 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > + 0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,0, > + 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > + 0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1 > +}; > + > +uint get_sql_xid(XID *xid, char *buf) > +{ > + int tot_len= xid->gtrid_length + xid->bqual_length; > + int i; > + const char *orig_buf= buf; > + > + for (i=0; i<tot_len; i++) > + { > + uchar c= ((uchar *) xid->data)[i] > + if (c >= 128 || xid_needs_conv[c]) > + break; > + } > + > + if (i >= tot_len) > + { > + /* No need to convert characters to hexadecimals. */ > + *buf++= '\''; > + memcpy(buf, xid->data, xid->gtrid_length); > + buf+= xid->gtrid_length; > + *buf++= '\''; > + if (xid->bqual_length > 0 || xid->formatID != 1) > + { > + *buf++= ','; > + *buf++= '\''; > + memcpy(buf, xid->data+xid->gtrid_length, xid->bqual_length); > + buf+= xid->bqual_length; > + *buf++= '\''; > + } > + } > + else > + { > + *buf++= 'X'; > + *buf++= '\''; > + for (i= 0; i < xid->gtrid_length; i++) > + { > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] >> 4]; > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] & 0x0f]; > + } > + *buf++= '\''; > + if (xid->bqual_length > 0 || xid->formatID != 1) > + { > + *buf++= ','; > + *buf++= 'X'; > + *buf++= '\''; > + for (; i < tot_len; i++) > + { > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] >> 4]; > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] & 0x0f]; > + } > + *buf++= '\''; > + } > + } > + > + if (xid->formatID != 1) > + { > + *buf++= ','; > + buf+= my_longlong10_to_str_8bit(&my_charset_bin, buf, > + MY_INT64_NUM_DECIMAL_DIGITS, -10, xid->formatID); > + } > + > + return buf - orig_buf; > +} > + > + > +/** > return the list of XID's to a client, the same way SHOW commands do. > > @note > diff --git a/sql/handler.h b/sql/handler.h > index 486ba56..91e3168 100644 > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -628,6 +628,16 @@ struct xid_t { > }; > typedef struct xid_t XID; > > +/* > + The size of XID string representation in the form > + 'gtrid', 'bqual', formatID > + see xid_t::get_sql_string() for details. > +*/ > +#define SQL_XIDSIZE (XIDDATASIZE * 2 + 8 + MY_INT64_NUM_DECIMAL_DIGITS) > +/* The 'buf' has to have space for at least SQL_XIDSIZE bytes. */ > +uint get_sql_xid(XID *xid, char *buf); Not sure you need this as an extern function in the header. Why not to keep it static in handler.cc? > /* for recover() handlerton call */ > #define MIN_XID_LIST_SIZE 128 > #define MAX_XID_LIST_SIZE (1024*128) > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 17e6904..d0fa642 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -1705,6 +1705,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > %token WRITE_SYM /* SQL-2003-N */ > %token X509_SYM > %token XA_SYM > +%token XID_SYM > %token XML_SYM > %token XOR > %token YEAR_MONTH_SYM You forgot to add XID_SYM to the keyword_sp rule. Also I don't see why we should have CONVERT XID syntax if the output is incompatible. Perhaps it'd be better to have a different syntax? FORMAT=SQL? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Crash on view with union
by jerome brauge 08 Dec '17

08 Dec '17

Hi all, The following script crashs the server (built with last bb-10.2-ext sources). use test; CREATE or replace TABLE `gtfmt` ( `numgtfmt` char(10) COLLATE utf8_bin NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; create or replace view gtfmtl(numgtfmt) as select 'x' from gtfmt union select 'x' from gtfmt ; SELECT * FROM gtfmtl WHERE numgtfmt = NAME_CONST('wnumgtfmt',_utf8'QEDITIONS' COLLATE 'utf8_bin'); Stack : > mysqld.exe!my_sigabrt_handler(int sig) Line 488 C mysqld.exe!raise(int signum) Line 516 C++ mysqld.exe!abort() Line 64 C++ mysqld.exe!common_assert_to_stderr_direct(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 123 C++ mysqld.exe!common_assert_to_stderr<wchar_t>(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 142 C++ mysqld.exe!_wassert(const wchar_t * expression, const wchar_t * file_name, unsigned int line_number) Line 404 C++ mysqld.exe!Item_func::fix_fields(THD * thd, Item * * ref) Line 330 C++ mysqld.exe!Item_str_func::fix_fields(THD * thd, Item * * ref) Line 106 C++ mysqld.exe!Item_name_const::fix_fields(THD * thd, Item * * ref) Line 2186 C++ mysqld.exe!Item_func::fix_fields(THD * thd, Item * * ref) Line 361 C++ mysqld.exe!JOIN::optimize_inner() Line 1311 C++ mysqld.exe!JOIN::optimize() Line 1121 C++ mysqld.exe!st_select_lex_unit::optimize() Line 1344 C++ mysqld.exe!mysql_derived_optimize(THD * thd, LEX * lex, TABLE_LIST * derived) Line 887 C++ mysqld.exe!mysql_handle_single_derived(LEX * lex, TABLE_LIST * derived, unsigned int phases) Line 197 C++ mysqld.exe!JOIN::optimize_inner() Line 1356 C++ mysqld.exe!JOIN::optimize() Line 1121 C++ mysqld.exe!mysql_select(THD * thd, TABLE_LIST * tables, unsigned int wild_num, List<Item> & fields, Item * conds, unsigned int og_num, st_order * order, st_order * group, Item * having, st_order * proc_param, unsigned __int64 select_options, select_result * result, st_select_lex_unit * unit, st_select_lex * select_lex) Line 3758 C++ mysqld.exe!handle_select(THD * thd, LEX * lex, select_result * result, unsigned long setup_tables_done_option) Line 364 C++ mysqld.exe!execute_sqlcom_select(THD * thd, TABLE_LIST * all_tables) Line 6473 C++ mysqld.exe!mysql_execute_command(THD * thd) Line 3717 C++ mysqld.exe!mysql_parse(THD * thd, char * rawbuf, unsigned int length, Parser_state * parser_state, bool is_com_multi, bool is_next_command) Line 7935 C++ mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length, bool is_com_multi, bool is_next_command) Line 1820 C++ mysqld.exe!do_command(THD * thd) Line 1369 C++ mysqld.exe!threadpool_process_request(THD * thd) Line 366 C++ mysqld.exe!tp_callback(TP_connection * c) Line 192 C++ mysqld.exe!tp_callback(_TP_CALLBACK_INSTANCE * instance, void * context) Line 377 C++ mysqld.exe!work_callback(_TP_CALLBACK_INSTANCE * instance, void * context, _TP_WORK * work) Line 451 C++ Best regard, Jérôme.

1 0

[Maria-developers] -Wnon-virtual-dtor and scripts in
by Alexander Barkov 08 Dec '17

08 Dec '17

Hello Vlad, Are we still supposed to use these scripts in the BUILD directory: compile-pentium64-debug compile-pentium64-debug-all compile-pentium64-debug-max compile-pentium64-gcov compile-pentium64-gprof compile-pentium64-max compile-pentium64-valgrind-max I stopped using them long time ago. But these scripts set some extra warnings for gcc, so now it's easy to overlook some warnings. Monty is still using these scripts and gets those extra warnings. In particular, -Wnon-virtual-dtor is often a subject of warnings. Is there a way to enable the -Wnon-virtual-dtor for gcc in the default build, i.e. when just using "cmake" instead of BUILD scripts? Thanks!

2 2

Re: [Maria-developers] 78a1779: mysqldump fix for invisible column
by Sergei Golubchik 07 Dec '17

07 Dec '17

Hi, Sachin! Looks fine. Two comments: 1. optimization idea, see below 2. add a test case that actually restores from the dump (and checks that it was restored correctly). mysqldump.test has examples, search for $MYSQL. After that - please push! On Dec 07, sachin wrote: > revision-id: 78a1779f29ed8d2a96658583065270de8bdd14b4 (mariadb-10.3.2-61-g78a1779) > parent(s): e8c8300a1c7e7e2b99e98c5c6986df69943cb661 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-07 16:09:16 +0530 > message: > > mysqldump fix for invisible column > > Actually there are 2 issues in the case of invisible columns > > 1st `select fields from t1` will have more fields then `select * from t1`. > So instead of `select * from t1` we are using `select a,b,invisible from t1` > these fields are supplied from `select fields from t1`. > > 2nd We are using --complete-insert when we detect that this table is using > invisible columns. > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index a260065..818bc07 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -2971,6 +2979,12 @@ static uint get_table_structure(char *table, char *db, char *table_type, > DBUG_RETURN(0); > } > > + while ((row= mysql_fetch_row(result))) > + { > + if (strlen(row[SHOW_EXTRA]) && strstr(row[SHOW_EXTRA],"INVISIBLE")) > + complete_insert= 1; > + } > + mysql_data_seek(result, 0); > /* > If write_data is true, then we build up insert statements for > the table's data. Note: in subsequent lines of code, this test > @@ -3006,10 +3020,16 @@ static uint get_table_structure(char *table, char *db, char *table_type, > { > dynstr_append_checked(&insert_pat, ", "); > } > - init=1; > dynstr_append_checked(&insert_pat, > quote_name(row[SHOW_FIELDNAME], name_buff, 0)); > } > + if (init) > + { > + dynstr_append_checked(&select_field_names, ", "); > + } > + dynstr_append_checked(&select_field_names, > + quote_name(row[SHOW_FIELDNAME], name_buff, 0)); > + init=1; I think, you can do it with just one loop: while ((row= mysql_fetch_row(result))) { if (strlen(row[SHOW_EXTRA]) && strstr(row[SHOW_EXTRA],"INVISIBLE")) complete_insert= 1; if (init) dynstr_append_checked(&select_field_names, ", "); dynstr_append_checked(&select_field_names, quote_name(row[SHOW_FIELDNAME], name_buff, 0)); init=1; } if (complete_insert) dynstr_append_checked(&insert_pat, select_field_names->str); > } > num_fields= mysql_num_rows(result); > mysql_free_result(result); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 339dd3d: Invisible Column mysqldump bug fix
by Sergei Golubchik 07 Dec '17

07 Dec '17

Hi, Sachin! Summary: 1. Add a test case, it should include a mix of tables with and without invisible columns, and columns with strange names (reserved words, spaces). 2. Always use full SELECT syntax, even if the table has no invisible columns. On Dec 05, sachin wrote: > revision-id: 339dd3d7508a1e5c7c014b5a59ed99ff0ad96542 (mariadb-10.3.2-59-g339dd3d) > parent(s): 6ed3374c8a67ae64448783ab107d1a11cb87f40c > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-05 05:35:28 +0530 > message: > > Invisible Column mysqldump bug fix > > --- > client/mysqldump.c | 46 +++++++++++++++++++++++++++++++++++++++++----- > 1 file changed, 41 insertions(+), 5 deletions(-) > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index a260065..e59b203 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -115,7 +115,7 @@ static my_bool verbose= 0, opt_no_create_info= 0, opt_no_data= 0, opt_no_data_m > opt_events= 0, opt_comments_used= 0, > opt_alltspcs=0, opt_notspcs= 0, opt_logging, > opt_drop_trigger= 0 ; > -static my_bool insert_pat_inited= 0, debug_info_flag= 0, debug_check_flag= 0; > +static my_bool insert_pat_inited= 0, debug_info_flag= 0, debug_check_flag= 0, invisble_table= 0; typo, *invisible* > static ulong opt_max_allowed_packet, opt_net_buffer_length; > static MYSQL mysql_connection,*mysql=0; > static DYNAMIC_STRING insert_pat; > @@ -2726,7 +2726,7 @@ static uint get_table_structure(char *table, char *db, char *table_type, > complete_insert= 0; > if ((write_data= !(*ignore_flag & IGNORE_DATA))) > { > - complete_insert= opt_complete_insert; > + invisble_table= complete_insert= opt_complete_insert; > if (!insert_pat_inited) > { > insert_pat_inited= 1; > @@ -2971,6 +2971,12 @@ static uint get_table_structure(char *table, char *db, char *table_type, > DBUG_RETURN(0); > } > > + while ((row= mysql_fetch_row(result))) > + { > + if (strlen(row[SHOW_EXTRA]) && strstr(row[SHOW_EXTRA],"INVISIBLE")) > + invisble_table= complete_insert= opt_complete_insert= 1; I think, you need to set invisible_table flag per table, just as you do now, but you should not change complete_insert or opt_complete_insert. That is, if a database has many tables, some with invisible columns and some have only normal columns - tables with invisible columns should use complete_insert, while other tables should not. Basically, INSERT should contain column names if (invisible_table || opt_complete_insert) > + } > + mysql_data_seek(result, 0); > /* > If write_data is true, then we build up insert statements for > the table's data. Note: in subsequent lines of code, this test > @@ -3617,7 +3623,7 @@ static void dump_table(char *table, char *db) > { > char ignore_flag; > char buf[200], table_buff[NAME_LEN+3]; > - DYNAMIC_STRING query_string; > + DYNAMIC_STRING query_string, select_field_names; > char table_type[NAME_LEN]; > char *result_table, table_buff2[NAME_LEN*2+3], *opt_quoted_table; > int error= 0; > @@ -3687,7 +3693,25 @@ static void dump_table(char *table, char *db) > verbose_msg("-- Sending SELECT query...\n"); > > init_dynamic_string_checked(&query_string, "", 1024, 1024); > + init_dynamic_string_checked(&select_field_names, "", 1024, 1024); > > + if (invisble_table) > + { > + /* > + Copy field name from insert_pat into select_field_names because > + select * from table will leave hidden columns. > + We will search for "(" in insert_pat and will copy the whole data > + into select_field_names , and then we will search for ")" in > + select_field_names and decrese the lenght accordingly. > + */ > + char *ptr= insert_pat.str; > + while ( *ptr++ != '(' ){} > + dynstr_append_checked(&select_field_names, ptr); > + ptr= &select_field_names.str[select_field_names.length - 1]; > + while ( *ptr-- != ')') > + select_field_names.length--; > + select_field_names.str[select_field_names.length- 1]= 0; please, check how INSERT code is doing it, you need to quote column names. > + } > if (path) > { > char filename[FN_REFLEN], tmp_path[FN_REFLEN]; > @@ -3708,7 +3732,13 @@ static void dump_table(char *table, char *db) > > /* now build the query string */ > > - dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ * INTO OUTFILE '"); > + dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ "); > + if (invisble_table) > + dynstr_append_checked(&query_string, select_field_names.str); > + else > + dynstr_append_checked(&query_string, "*"); I don't think you need to bother, you can always have a complete column list here. It should not break anything. > + dynstr_append_checked(&query_string, " INTO OUTFILE '"); > + dynstr_free(&select_field_names); > dynstr_append_checked(&query_string, filename); > dynstr_append_checked(&query_string, "'"); > > @@ -3757,7 +3787,13 @@ static void dump_table(char *table, char *db) > "\n--\n-- Dumping data for table %s\n--\n", > fix_for_comment(result_table)); > > - dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ * FROM "); > + dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ "); > + if (invisble_table) > + dynstr_append_checked(&query_string, select_field_names.str); > + else > + dynstr_append_checked(&query_string, "*"); same > + dynstr_free(&select_field_names); > + dynstr_append_checked(&query_string, " FROM "); > dynstr_append_checked(&query_string, result_table); > > if (where) > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] 0f47171: MDEV-14005 Remove need for Partition Key to be part of Primary Key.
by Sergei Golubchik 06 Dec '17

06 Dec '17

Hi, Alexey! On Dec 04, Alexey Botchkov wrote: > revision-id: 0f4717118e135c178fc1b12065e33ac8a5e21d0b (mariadb-10.3.2-82-g0f47171) > parent(s): ddac2d7a1ef725361c7e83b819cb0bec72db024a > committer: Alexey Botchkov > timestamp: 2017-12-04 17:45:13 +0400 > message: > > MDEV-14005 Remove need for Partition Key to be part of Primary Key. > > Now we check all partitions for the unique keys if necessary > in ha_partition::write_row and ha_partition::update_row. > > diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc > index 6bebf5d..12a6287 100644 > --- a/sql/ha_partition.cc > +++ b/sql/ha_partition.cc > @@ -372,6 +372,7 @@ void ha_partition::init_handler_variables() > part_share= NULL; > m_new_partitions_share_refs.empty(); > m_part_ids_sorted_by_num_of_records= NULL; > + m_cu_errkey= (uint) -1; > > #ifdef DONT_HAVE_TO_BE_INITALIZED > m_start_key.flag= 0; > @@ -4088,6 +4089,79 @@ void ha_partition::try_semi_consistent_read(bool yes) > } > > > +int ha_partition::check_files_for_key(uchar *key, int n_key, > + int partition_to_skip, int *res) > +{ > +// uint num_subparts= part_info->num_subparts; > + for (uint n=0; n < m_part_info->num_parts; n++) > + { > + handler *f= m_file[n]; > + int ires; > + > + if ((int) n == partition_to_skip) > + continue; > + > + f->active_index= n_key; > + if ((*res= f->extra(HA_EXTRA_KEYREAD)) || > + (*res= f->ha_index_init(n_key, FALSE))) > + return 1; > + > + *res= f->index_read_map(m_part_info->table->record[0], > + key, HA_WHOLE_KEY, HA_READ_KEY_EXACT); use index_read_idx_map here > + ires= f->ha_index_end(); > + > + if (*res == HA_ERR_KEY_NOT_FOUND) > + continue; > + > + if (*res == 0) > + *res= ires ? ires : HA_ERR_FOUND_DUPP_KEY; > + > + m_last_part= n; > + m_cu_errkey= n_key; > + return 1; > + } > + > + *res= 0; > + return 0; > +} > + > + > +int ha_partition::check_uniques_insert(uchar *buf, > + int partition_to_skip, int *res) > +{ > + uint n_key; > + > + for (n_key=0; n_key < m_part_info->n_uniques_to_check; n_key++) > + { can you use key_copy() here? > + uchar *cbuf= m_part_info->unique_key_buf[0]; > + KEY *ckey= m_part_info->uniques_to_check[n_key]; > + uint n; > + > + for (n=0; n < ckey->user_defined_key_parts; n++) > + { > + const KEY_PART_INFO *cpart= ckey->key_part + n; > + uint maybe_null= MY_TEST(cpart->null_bit); > + Field *f= cpart->field; > + my_ptrdiff_t ofs= buf-table->record[0]; > + > + f->move_field_offset(ofs); > + if (maybe_null) > + cbuf[0]= f->is_null(); > + f->get_key_image(cbuf+maybe_null, cpart->length, Field::itRAW); > + cbuf+= cpart->store_length; > + f->move_field_offset(-ofs); > + } > + > + if (check_files_for_key(m_part_info->unique_key_buf[0], n_key, > + partition_to_skip, res)) > + return 1; > + } > + > + *res= 0; > + return 0; > +} > + > + > /**************************************************************************** > MODULE change record > ****************************************************************************/ > @@ -4199,6 +4273,11 @@ int ha_partition::write_row(uchar * buf) > error= HA_ERR_NOT_IN_LOCK_PARTITIONS; > goto exit; > } > + > + if (table->part_info->n_uniques_to_check && > + check_uniques_insert(buf, part_id, &error)) > + goto exit; This would _probably_ introduce the race condition I've mentioned in a comment to MDEV-14005. Say, you have a table with two partitions and a unique key. Now, two threads try to insert rows, the first thread inserts 1 into the first partition (there are more columns in a table and a partitioning key on those columns, but I omit these details for simplicity). And a second thread inserts 1 into the second partition. So, the first thread checks for 1 in the second partition, finds nothing. Second thread checks for 1 in the first partition, finds nothing. Then the first thread inserts 1 into the first partition, second thread inserts 1 into the second partition. Monty's idea was to insert optimistically - first you insert, then check for duplicates. Not ideal, in the above example you can have both threads getting a duplicate key error. There could be other, e.g. locking, solutions too. > m_last_part= part_id; > DBUG_PRINT("info", ("Insert in partition %d", part_id)); > start_part_bulk_insert(thd, part_id); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-13995 MAX(timestamp) returns a wrong result near DST change
by Alexander Barkov 04 Dec '17

04 Dec '17

Hi Sergei, Please review a patch for MDEV-13995. Thanks!

1 0