[Maria-developers] Need help with review/suggestions for MDEV-8075 (was Re: [Commits] c325a34: MDEV-8075 ...)

Hi Serg, Svoj, I need some help with MDEV-8075, as it involves code around transaction start/end which I am not familiar with. I hope one of you know enough about that part of the server to help me? I have a partial patch (below), but it still has some problems remaining and I am not sure how to solve it properly. I tried to base the patch on some corresponding MySQL code, which was only partially merged to MariaDB 10.1. The basic problem is to remember during a transaction whether DDL was used, and then at the end of the transaction in binlogging, to set a flag in the GTID event if DDL is used. The concrete problem is with this kind of SQL: BEGIN; INSERT INTO innodb_table VALUES (1); ALTER TABLE temporary_table ADD b INT; In this case the ALTER TABLE does not get the "ddl" mark in my patch. The reason is that during mysql_alter_table(), THD_TRANS::reset() gets called, which clears the flag DID_DDL that was set in mysql_execute_command(). Call stack: #0 THD_TRANS::reset #1 trans_commit_implicit #2 ha_enable_transaction #3 mysql_trans_commit_alter_copy_data #4 copy_data_between_tables #5 mysql_alter_table #6 Sql_cmd_alter_table::execute #7 mysql_execute_command (An earlier version of the patch looked directly at thd->lex->sql_command during binlogging to see if it was a DDL. That caused another problem: the INSERT INTO got wrongly marked as DDL, because it is implicitly committed by an ALTER TABLE statement). But the more general problem is that I am very unsure about the proper places to put in code to clear the required flags at the start of a transaction, and to copy relevant flags from "stmt" to "all" transaction. I tried using what I found in MySQL code, and to look for where the existing MariaDB code copies the existing THD_TRANS::modified_non_trans_table flag. But I am very unsure that I choose the right places, the existing code seems rather ad-hoc (I'm wondering if it is even correct). I also replaced some add-hoc code that clears THD_TRANS::modified_non_trans_table directly with calls to THD_TRANS::reset(); that seems more appropriate when more flags are affected, but it does more things (notably clearing no_2pc), so again I'm very unsure if it is correct... Any suggestions welcomed. The underlying bug makes optimistic parallel replication quite broken with temporary tables and statement-based binlogging in 10.1 (can probably even crash the server), so it is of some importance to get fixed.... - Kristian. ----------------------------------------------------------------------- revision-id: c325a34d93b931d1c927255c18df94e1560af6eb parent(s): 53382ac128d9571b5f779bf625567fc3ea0b475e committer: Kristian Nielsen branch nick: mariadb timestamp: 2015-05-15 14:56:54 +0200 message: MDEV-8075: DROP TEMPORARY TABLE not marked as ddl, causing optimistic parallel replication to fail CREATE/DROP TEMPORARY TABLE are not safe to optimistically replicate in parallel with other transactions, so they need to be marked as "ddl" in the binlog. This was already done for stand-alone CREATE/DROP TEMPORARY. But temporary tables can also be created and dropped inside a BEGIN...END transaction, and such transactions were not marked as ddl. Nor was the DROP TEMPORARY TABLE statement emitted implicitly when a client connection is closed. So this patch adds such ddl mark for the missing cases. --- .../include/binlog_parallel_replication_marks.test | 83 ++++++++++++++++++ .../r/binlog_parallel_replication_marks_row.result | 98 ++++++++++++++++++++++ ...inlog_parallel_replication_marks_stm_mix.result | 95 +++++++++++++++++++++ .../t/binlog_parallel_replication_marks_row.test | 3 + .../binlog_parallel_replication_marks_stm_mix.test | 3 + .../suite/rpl/r/rpl_parallel_optimistic.result | 27 ++++++ .../suite/rpl/t/rpl_parallel_optimistic.test | 25 ++++++ sql/handler.h | 14 ++++ sql/log_event.cc | 6 +- sql/sql_base.cc | 1 + sql/sql_class.h | 10 +++ sql/sql_insert.cc | 8 ++ sql/sql_parse.cc | 4 +- sql/sql_table.cc | 22 +++-- sql/transaction.cc | 25 +++--- 15 files changed, 400 insertions(+), 24 deletions(-) diff --git a/mysql-test/include/binlog_parallel_replication_marks.test b/mysql-test/include/binlog_parallel_replication_marks.test new file mode 100644 index 0000000..4775e30 --- /dev/null +++ b/mysql-test/include/binlog_parallel_replication_marks.test @@ -0,0 +1,83 @@ +# Test the markings on GTID events (ddl, waited, trans, +# @@skip_parallel_replication) that are used to control parallel +# replication on the slave. + +--source include/have_innodb.inc + +RESET MASTER; +--source include/wait_for_binlog_checkpoint.inc + +--let $stable_stamp= `SELECT UNIX_TIMESTAMP("2020-01-21 15:32:22")` +eval set timestamp=$stable_stamp; + +CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +--let $binlog_pos1=query_get_value(SHOW MASTER STATUS, Position, 1) +/* GTID */ INSERT INTO t1 VALUES (1,0); +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (2,0); +/* GTID */ ALTER TABLE t1 ADD c INT; +/* GTID */ INSERT INTO t1 VALUES (3,0,0); +/* GTID */ COMMIT; +/* GTID */ BEGIN; +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2; +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20); +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2; +/* GTID */ DROP TEMPORARY TABLE t2; +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0); +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ DELETE FROM t1 WHERE a=5; +/* GTID */ INSERT INTO t3 VALUES (7); +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3; +/* GTID */ UPDATE t1 SET c=1 WHERE a=7; +/* GTID */ DROP TEMPORARY TABLE t3; +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0); +/* GTID */ ALTER TABLE t4 ADD b INT; +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1); +/* GTID */ COMMIT; +connect (tmp_con,localhost,root,,); +eval set timestamp=$stable_stamp; +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0); +/* GTID */ BEGIN; +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0); +/* GTID */ COMMIT; +--let $before_drop_pos=query_get_value(SHOW MASTER STATUS, Position, 1) +disconnect tmp_con; +connection default; + +# We need to wait for the implicit DROP TEMPORARY TABLE to be logged after +# tmp_con disconnect, otherwise we get sporadic test failures. +--let $wait_condition= SELECT variable_value > $before_drop_pos FROM information_schema.global_status WHERE variable_name = 'binlog_snapshot_position' +--source include/wait_condition.inc + +--let $binlog_pos2=query_get_value(SHOW MASTER STATUS, Position, 1) + +--let $binlog_file= query_get_value(SHOW MASTER STATUS, File, 1) +FLUSH LOGS; + +--let $MYSQLD_DATADIR= `select @@datadir` +--let $file= $MYSQLTEST_VARDIR/tmp/binlog_parallel_replication_marks.out +--let OUTPUT_FILE=$file +exec $MYSQL_BINLOG --start_position=$binlog_pos1 --stop_position=$binlog_pos2 $MYSQLD_DATADIR/$binlog_file > $file; + +perl; +my $file= $ENV{'OUTPUT_FILE'}; +open F, "<", $file + or die "Unable to open file '$file': $!\n"; +while (<F>) { + s/GTID \d+-\d+-\d+/GTID #-#-#/; + s/end_log_pos \d+/end_log_pos #/; + s/table id \d+/table id #/; + s/mapped to number \d+/mapped to number #/; + print if /GTID|BEGIN|COMMIT|Table_map|Write_rows|Update_rows|Delete_rows|generated by server|40005 TEMPORARY/; +} +close F; +EOF + +DROP TABLE t1; diff --git a/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_row.result b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_row.result new file mode 100644 index 0000000..b9c8666 --- /dev/null +++ b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_row.result @@ -0,0 +1,98 @@ +RESET MASTER; +set timestamp=1579617142; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (1,0); +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (2,0); +/* GTID */ ALTER TABLE t1 ADD c INT; +/* GTID */ INSERT INTO t1 VALUES (3,0,0); +/* GTID */ COMMIT; +/* GTID */ BEGIN; +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2; +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20); +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2; +/* GTID */ DROP TEMPORARY TABLE t2; +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0); +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ DELETE FROM t1 WHERE a=5; +/* GTID */ INSERT INTO t3 VALUES (7); +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3; +/* GTID */ UPDATE t1 SET c=1 WHERE a=7; +/* GTID */ DROP TEMPORARY TABLE t3; +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0); +/* GTID */ ALTER TABLE t4 ADD b INT; +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1); +/* GTID */ COMMIT; +set timestamp=1579617142; +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0); +/* GTID */ BEGIN; +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0); +/* GTID */ COMMIT; +FLUSH LOGS; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ ALTER TABLE t1 ADD c INT +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Update_rows: table id # flags: STMT_END_F +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +DROP TEMPORARY TABLE IF EXISTS `test`.`t2` /* generated by server */ +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Delete_rows: table id # flags: STMT_END_F +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Update_rows: table id # flags: STMT_END_F +DROP TEMPORARY TABLE IF EXISTS `test`.`t3` /* generated by server */ +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `t5` +DROP TABLE t1; diff --git a/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_stm_mix.result b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_stm_mix.result new file mode 100644 index 0000000..c3dc00d --- /dev/null +++ b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_stm_mix.result @@ -0,0 +1,95 @@ +RESET MASTER; +set timestamp=1579617142; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (1,0); +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (2,0); +/* GTID */ ALTER TABLE t1 ADD c INT; +/* GTID */ INSERT INTO t1 VALUES (3,0,0); +/* GTID */ COMMIT; +/* GTID */ BEGIN; +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2; +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20); +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2; +/* GTID */ DROP TEMPORARY TABLE t2; +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0); +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ DELETE FROM t1 WHERE a=5; +/* GTID */ INSERT INTO t3 VALUES (7); +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3; +/* GTID */ UPDATE t1 SET c=1 WHERE a=7; +/* GTID */ DROP TEMPORARY TABLE t3; +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0); +/* GTID */ ALTER TABLE t4 ADD b INT; +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1); +/* GTID */ COMMIT; +set timestamp=1579617142; +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0); +/* GTID */ BEGIN; +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0); +/* GTID */ COMMIT; +FLUSH LOGS; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (1,0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (2,0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ ALTER TABLE t1 ADD c INT +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (3,0,0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2 +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20) +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2 +DROP TEMPORARY TABLE `t2` /* generated by server */ +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +/* GTID */ DELETE FROM t1 WHERE a=5 +/* GTID */ INSERT INTO t3 VALUES (7) +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3 +/* GTID */ UPDATE t1 SET c=1 WHERE a=7 +DROP TEMPORARY TABLE `t3` /* generated by server */ +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ ALTER TABLE t4 ADD b INT +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `t5` +DROP TABLE t1; diff --git a/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_row.test b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_row.test new file mode 100644 index 0000000..8289848 --- /dev/null +++ b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_row.test @@ -0,0 +1,3 @@ +--source include/have_log_bin.inc +--source include/have_binlog_format_row.inc +--source include/binlog_parallel_replication_marks.test diff --git a/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_stm_mix.test b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_stm_mix.test new file mode 100644 index 0000000..15042b3 --- /dev/null +++ b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_stm_mix.test @@ -0,0 +1,3 @@ +--source include/have_log_bin.inc +--source include/have_binlog_format_mixed_or_statement.inc +--source include/binlog_parallel_replication_marks.test diff --git a/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result b/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result index 7593b49..2da7830 100644 --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result @@ -455,6 +455,33 @@ a b include/stop_slave.inc SET GLOBAL debug_dbug= @old_debug; include/start_slave.inc +*** MDEV-8075: DROP TEMPORARY TABLE not marked as ddl, causing optimistic parallel replication to fail *** +include/stop_slave.inc +INSERT INTO t1 VALUES (40, 10); +CREATE TEMPORARY TABLE t1 (a INT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 VALUES (41); +BEGIN; +INSERT INTO t2 SELECT a, 20 FROM t1; +DROP TEMPORARY TABLE t1; +COMMIT; +INSERT INTO t1 VALUES (42, 10); +include/save_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +a b +40 10 +42 10 +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; +a b +41 20 +include/start_slave.inc +include/sync_with_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +a b +40 10 +42 10 +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; +a b +41 20 include/stop_slave.inc SET GLOBAL slave_parallel_mode=@old_parallel_mode; SET GLOBAL slave_parallel_threads=@old_parallel_threads; diff --git a/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test b/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test index e7d4f18..376c8d6 100644 --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test @@ -429,6 +429,31 @@ SET GLOBAL debug_dbug= @old_debug; --source include/start_slave.inc +--echo *** MDEV-8075: DROP TEMPORARY TABLE not marked as ddl, causing optimistic parallel replication to fail *** + +--connection server_2 +--source include/stop_slave.inc + +--connection server_1 +INSERT INTO t1 VALUES (40, 10); +CREATE TEMPORARY TABLE t1 (a INT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 VALUES (41); +BEGIN; +INSERT INTO t2 SELECT a, 20 FROM t1; +DROP TEMPORARY TABLE t1; +COMMIT; +INSERT INTO t1 VALUES (42, 10); +--source include/save_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; + +--connection server_2 +--source include/start_slave.inc +--source include/sync_with_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; + + # Clean up. --connection server_2 diff --git a/sql/handler.h b/sql/handler.h index 279bfaf..8faec7f 100644 --- a/sql/handler.h +++ b/sql/handler.h @@ -1444,16 +1444,30 @@ struct THD_TRANS static unsigned int const CREATED_TEMP_TABLE= 0x02; static unsigned int const DROPPED_TEMP_TABLE= 0x04; static unsigned int const DID_WAIT= 0x08; + static unsigned int const DID_DDL= 0x10; void mark_created_temp_table() { DBUG_PRINT("debug", ("mark_created_temp_table")); m_unsafe_rollback_flags|= CREATED_TEMP_TABLE; } + void mark_dropped_temp_table() + { + DBUG_PRINT("debug", ("mark_dropped_temp_table")); + m_unsafe_rollback_flags|= DROPPED_TEMP_TABLE; + } + bool has_created_dropped_temp_table() const { + return + (m_unsafe_rollback_flags & (CREATED_TEMP_TABLE|DROPPED_TEMP_TABLE)) != 0; + } void mark_trans_did_wait() { m_unsafe_rollback_flags|= DID_WAIT; } bool trans_did_wait() const { return (m_unsafe_rollback_flags & DID_WAIT) != 0; } + void mark_trans_did_ddl() { m_unsafe_rollback_flags|= DID_DDL; } + bool trans_did_ddl() const { + return (m_unsafe_rollback_flags & DID_DDL) != 0; + } }; diff --git a/sql/log_event.cc b/sql/log_event.cc index aa34996..98cc4b1 100644 --- a/sql/log_event.cc +++ b/sql/log_event.cc @@ -6405,8 +6405,10 @@ Gtid_log_event::Gtid_log_event(THD *thd_arg, uint64 seq_no_arg, if (thd_arg->transaction.stmt.trans_did_wait() || thd_arg->transaction.all.trans_did_wait()) flags2|= FL_WAITED; - if (sql_command_flags[thd->lex->sql_command] & - (CF_DISALLOW_IN_RO_TRANS | CF_AUTO_COMMIT_TRANS)) + if (thd_arg->transaction.stmt.trans_did_ddl() || + thd_arg->transaction.stmt.has_created_dropped_temp_table() || + thd_arg->transaction.all.trans_did_ddl() || + thd_arg->transaction.all.has_created_dropped_temp_table()) flags2|= FL_DDL; else if (is_transactional) flags2|= FL_TRANSACTIONAL; diff --git a/sql/sql_base.cc b/sql/sql_base.cc index c400598..6ef46ca 100644 --- a/sql/sql_base.cc +++ b/sql/sql_base.cc @@ -1253,6 +1253,7 @@ bool close_temporary_tables(THD *thd) thd->variables.character_set_client= cs_save; thd->get_stmt_da()->set_overwrite_status(true); + thd->transaction.stmt.mark_dropped_temp_table(); if ((error= (mysql_bin_log.write(&qinfo) || error))) { /* diff --git a/sql/sql_class.h b/sql/sql_class.h index a39c8cd..11eab4e 100644 --- a/sql/sql_class.h +++ b/sql/sql_class.h @@ -3953,6 +3953,16 @@ class THD :public Statement, { main_lex.restore_set_statement_var(); } + /* Copy relevant `stmt` transaction flags to `all` transaction. */ + void merge_unsafe_rollback_flags() + { + if (transaction.stmt.modified_non_trans_table) + transaction.all.modified_non_trans_table= TRUE; + transaction.all.m_unsafe_rollback_flags|= + (transaction.stmt.m_unsafe_rollback_flags & + (THD_TRANS::DID_WAIT | THD_TRANS::CREATED_TEMP_TABLE | + THD_TRANS::DROPPED_TEMP_TABLE | THD_TRANS::DID_DDL)); + } }; diff --git a/sql/sql_insert.cc b/sql/sql_insert.cc index 24d75d9..90e68d5 100644 --- a/sql/sql_insert.cc +++ b/sql/sql_insert.cc @@ -4226,6 +4226,14 @@ void select_create::store_values(List<Item> &values) bool select_create::send_eof() { + /* + The routine that writes the statement in the binary log + is in select_insert::send_eof(). For that reason, we + mark the flag at this point. + */ + if (create_info->options & HA_LEX_CREATE_TMP_TABLE) + thd->transaction.stmt.mark_created_temp_table(); + if (select_insert::send_eof()) { abort_result_set(); diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc index 5eca972..142596f 100644 --- a/sql/sql_parse.cc +++ b/sql/sql_parse.cc @@ -2798,6 +2798,7 @@ mysql_execute_command(THD *thd) goto error; } } + thd->transaction.stmt.mark_trans_did_ddl(); } #ifndef DBUG_OFF @@ -6781,8 +6782,7 @@ void THD::reset_for_next_command() if (!thd->in_multi_stmt_transaction_mode()) { thd->variables.option_bits&= ~OPTION_KEEP_LOG; - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); } DBUG_ASSERT(thd->security_ctx== &thd->main_security_ctx); thd->thread_specific_used= FALSE; diff --git a/sql/sql_table.cc b/sql/sql_table.cc index 1f96793..9abe012 100644 --- a/sql/sql_table.cc +++ b/sql/sql_table.cc @@ -2559,6 +2559,9 @@ int mysql_rm_table_no_locks(THD *thd, TABLE_LIST *tables, bool if_exists, if (non_trans_tmp_table_deleted || trans_tmp_table_deleted || non_tmp_table_deleted) { + if (non_trans_tmp_table_deleted || trans_tmp_table_deleted) + thd->transaction.stmt.mark_dropped_temp_table(); + query_cache_invalidate3(thd, tables, 0); if (!dont_log_query && mysql_bin_log.is_open()) { @@ -5014,6 +5017,9 @@ bool mysql_create_table(THD *thd, TABLE_LIST *create_table, if (thd->is_current_stmt_binlog_format_row() && create_info->tmp_table()) DBUG_RETURN(result); + if (create_info->options & HA_LEX_CREATE_TMP_TABLE) + thd->transaction.stmt.mark_created_temp_table(); + /* Write log if no error or if we already deleted a table */ if (!result || thd->log_current_statement) { @@ -5491,13 +5497,17 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table, DBUG_PRINT("info", ("res: %d tmp_table: %d create_info->table: %p", res, create_info->tmp_table(), local_create_info.table)); - if (!res && create_info->tmp_table() && local_create_info.table) + if (create_info->tmp_table()) { - /* - Remember that tmp table creation was logged so that we know if - we should log a delete of it. - */ - local_create_info.table->s->table_creation_was_logged= 1; + thd->transaction.stmt.mark_created_temp_table(); + if (!res && local_create_info.table) + { + /* + Remember that tmp table creation was logged so that we know if + we should log a delete of it. + */ + local_create_info.table->s->table_creation_was_logged= 1; + } } do_logging= TRUE; } diff --git a/sql/transaction.cc b/sql/transaction.cc index f84c4e5..fa6d338 100644 --- a/sql/transaction.cc +++ b/sql/transaction.cc @@ -154,8 +154,7 @@ bool trans_begin(THD *thd, uint flags) The following set should not be needed as the flag should always be 0 when we come here. We should at some point change this to an assert. */ - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->has_waiter= false; thd->waiting_on_group_commit= false; @@ -251,8 +250,7 @@ bool trans_commit(THD *thd) else (void) RUN_HOOK(transaction, after_commit, (thd, FALSE)); thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->lex->start_transaction_opt= 0; DBUG_RETURN(MY_TEST(res)); @@ -299,8 +297,7 @@ bool trans_commit_implicit(THD *thd) } thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); /* Upon implicit commit, reset the current transaction @@ -345,8 +342,7 @@ bool trans_rollback(THD *thd) thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); /* Reset the binlog transaction marker */ thd->variables.option_bits&= ~OPTION_GTID_BEGIN; - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->lex->start_transaction_opt= 0; DBUG_RETURN(MY_TEST(res)); @@ -390,8 +386,7 @@ bool trans_rollback_implicit(THD *thd) preserve backward compatibility. */ thd->variables.option_bits&= ~(OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= false; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); /* Rollback should clear transaction_rollback_request flag. */ DBUG_ASSERT(! thd->transaction_rollback_request); @@ -427,6 +422,8 @@ bool trans_commit_stmt(THD *thd) */ DBUG_ASSERT(! thd->in_sub_stmt); + thd->merge_unsafe_rollback_flags(); + if (thd->transaction.stmt.ha_list) { if (WSREP_ON) @@ -481,6 +478,8 @@ bool trans_rollback_stmt(THD *thd) */ DBUG_ASSERT(! thd->in_sub_stmt); + thd->merge_unsafe_rollback_flags(); + if (thd->transaction.stmt.ha_list) { if (WSREP_ON) @@ -913,8 +912,7 @@ bool trans_xa_commit(THD *thd) } thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->server_status&= ~(SERVER_STATUS_IN_TRANS | SERVER_STATUS_IN_TRANS_READONLY); DBUG_PRINT("info", ("clearing SERVER_STATUS_IN_TRANS")); @@ -969,8 +967,7 @@ bool trans_xa_rollback(THD *thd) res= xa_trans_force_rollback(thd); thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->server_status&= ~(SERVER_STATUS_IN_TRANS | SERVER_STATUS_IN_TRANS_READONLY); DBUG_PRINT("info", ("clearing SERVER_STATUS_IN_TRANS")); _______________________________________________ commits mailing list commits@mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits