- developers - lists.mariadb.org

[Maria-developers] Review for: MDEV-17399 Add support for JSON_TABLE, part #7
by Sergey Petrunia 30 Jul '20

30 Jul '20

Hi Alexey, On Thu, Jul 16, 2020 at 04:34:46PM +0400, Alexey Botchkov wrote: > > In the new patch i think i addressed most of your comments. > https://github.com/MariaDB/server/commit/d3a311a8e9508ef94dddb6b7f4d366337a… > ... > > The above doesn't do proper quoting, which is wrong. Testcase: > > > > select * > > from > > json_table('[{"co\\\\lor": "blue", "price": 50}]', > > '$[*]' columns( color varchar(100) path '$.co\\\\lor') > > ) as T; > > +-------+ > > | color | > > +-------+ > > | blue | > > +-------+ > > > > create view v2 as > > select * > > from > > json_table('[{"co\\\\lor": "blue", "price": 50}]', > > '$[*]' columns( color varchar(100) path '$.co\\\\lor') > > ) as T; > > select * from v2; Quoting is still an issue apparently: create view v1 as select * from json_table('[{"color": "blue", "price": 50}]', '$[*]' columns(color text path '$.nonexistent', seq for ordinality ) ) as `ALIAS NOT QUOTED`; select * from v1; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'NOT QUOTED' at line 1 create view v2 as select * from json_table('[{"color": "blue", "price": 50}, {"color": "red", "price": 100}]', '$[*]' columns(color text path '$.QUOTES " HERE "', seq for ordinality ) ) as T; select * from v2; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'HERE "", `seq` FOR ORDINALITY)) T' at line 1 Could you please go through the printout functions and make sure all printed literals are properly escaped/quoted? Printout code for other functions can be used as an example. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 1

Re: [Maria-developers] 2b527a4e858: MDEV-17729: Assertion `! is_set() || m_can_overwrite_status' failed in
by Sergei Golubchik 30 Jul '20

30 Jul '20

Hi, Rucha! On Jul 30, Rucha Deodhar wrote: > > > > > > - if ((level == Sql_condition::WARN_LEVEL_WARN) && > > > - really_abort_on_warning()) > > > + if ((killed == NOT_KILLED) && (level == Sql_condition::WARN_LEVEL_WARN) && > > > + really_abort_on_warning()) > > > > I don't understand this. I presume the assert that fails is in > > da->set_error_status(), called below as > > > > Yes, the assertion fails in set_error_status(). > > > if (!da->is_error()) > > { > > set_row_count_func(-1); > > da->set_error_status(sql_errno, msg, sqlstate, cond); > > } > > > > but if(!da->is_error()) should prevent set_error_status() from being > > called when it would need to overwrite the status. So how comes it is > > still called and triggers the assert? > > The status does not change to DA_ERROR yet, it changes in > set_error_status(), so !thd->is_error() becomes true and does not prevent > set_error_status() from being called. The reason we get assertion failure > in set_error_status() is because !is_set() is false (because status is > DA_OK) and m_can_overwrite_status is false too. I see. I also was confused by how can THD::raise_condition() be called after the status is already DA_OK? An error should not happen after DA_OK. But it turns out it was not just any error, it was push_warning() with the message "Query execution was interrupted. The query examined at least 4 rows, which exceeds LIMIT ROWS EXAMINED (2). The query result may be incomplete". This warning should not be converted to an error, obviously. It is converted, because thd->abort_on_warning is true. It's set to true in select_insert::prepare(). And the warning is pushed in handle_select(). Perhaps this warning should never have thd->abort_on_warning ? Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] 2b527a4e858: MDEV-17729: Assertion `! is_set() || m_can_overwrite_status' failed in
by Sergei Golubchik 30 Jul '20

30 Jul '20

Hi, Rucha! On Jul 30, Rucha Deodhar wrote: > revision-id: 2b527a4e858 (mariadb-10.1.43-218-g2b527a4e858) > parent(s): 57ec42bc321 > author: Rucha Deodhar <rucha.deodhar(a)mariadb.com> > committer: Rucha Deodhar <rucha.deodhar(a)mariadb.com> > timestamp: 2020-07-27 09:50:00 +0530 > message: > > MDEV-17729: Assertion `! is_set() || m_can_overwrite_status' failed in > Diagnostics_area::set_error_status > > Analysis: When strict mode is enabled, all warnings are converted to errors > even when there is no bad data and the query is already killed. > Fix: If the query is already killed because examined rows exceeded the limit, > the warning is not because of bad data and should not be converted to error. > > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 6897a26bda1..3591695a22d 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -1156,13 +1156,17 @@ Sql_condition* THD::raise_condition(uint sql_errno, > if (sqlstate == NULL) > sqlstate= mysql_errno_to_sqlstate(sql_errno); > > - if ((level == Sql_condition::WARN_LEVEL_WARN) && > - really_abort_on_warning()) > + if ((killed == NOT_KILLED) && (level == Sql_condition::WARN_LEVEL_WARN) && > + really_abort_on_warning()) I don't understand this. I presume the assert that fails is in da->set_error_status(), called below as if (!da->is_error()) { set_row_count_func(-1); da->set_error_status(sql_errno, msg, sqlstate, cond); } but if(!da->is_error()) should prevent set_error_status() from being called when it would need to overwrite the status. So how comes it is still called and triggers the assert? > { > - /* > - FIXME: > - push_warning and strict SQL_MODE case. > - */ > level= Sql_condition::WARN_LEVEL_ERROR; > set_killed(KILL_BAD_DATA); > } Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] bc73b455ba2: MDEV-15703 Crash in EXECUTE IMMEDIATE 'CREATE OR REPLACE TABLE t1 (a INT DEFAULT ?)' USING DEFAULT
by Sergei Golubchik 29 Jul '20

29 Jul '20

Hi, Oleksandr! On Jul 29, Oleksandr Byelkin wrote: > revision-id: bc73b455ba2 (mariadb-10.2.31-318-gbc73b455ba2) > parent(s): 7311586ca25 > author: Oleksandr Byelkin <sanja(a)mariadb.com> > committer: Oleksandr Byelkin <sanja(a)mariadb.com> > timestamp: 2020-07-17 13:47:26 +0200 > message: > > MDEV-15703 Crash in EXECUTE IMMEDIATE 'CREATE OR REPLACE TABLE t1 (a INT DEFAULT ?)' USING DEFAULT > > part 2: ... > diff --git a/sql/field.h b/sql/field.h > index d5e2cb25788..2755f5f687c 100644 > --- a/sql/field.h > +++ b/sql/field.h > @@ -51,6 +51,16 @@ enum enum_check_fields > CHECK_FIELD_ERROR_FOR_NULL > }; > > + > +enum ignore_value_reaction > +{ > + IGNORE_MEANS_ERROR, > + IGNORE_MEANS_DEFAULT, > + IGNORE_MEANS_FIELD_VALUE I thought you'll have IGNORE_MEANS_IGNORE :) > +}; > + > +ignore_value_reaction find_ignore_reaction(THD *thd); > + > /* > Common declarations for Field and Item > */ > diff --git a/sql/item.h b/sql/item.h > index 7338c8be47b..dabf237712b 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -1404,6 +1424,16 @@ class Item: public Value_source, > LOWEST_PRECEDENCE); > } > virtual void print(String *str, enum_query_type query_type); > + > + class Print: public String > + { > + public: > + Print(Item *item, enum_query_type type) > + { > + item->print(this, type); > + } no, I don't think this should be part of the bugfix. > + }; > + > void print_item_w_name(String *str, enum_query_type query_type); > void print_value(String *str); > > @@ -2077,6 +2107,15 @@ class Item: public Value_source, > { > marker &= ~EXTRACTION_MASK; > } > + > + virtual bool associate_with_target_field(THD *thd __attribute__((unused)), > + Item_field *field > + __attribute__((unused))) > + { > + DBUG_ASSERT(fixed); > + DBUG_ASSERT(fixed); you want to be double-sure? :) > + return false; > + } > }; > > > diff --git a/sql/item.cc b/sql/item.cc > index 382eb3ac47a..d11017f3ca6 100644 > --- a/sql/item.cc > +++ b/sql/item.cc > @@ -84,6 +85,15 @@ void item_init(void) > } > > > +void Item::raise_error_not_evaluable() > +{ > + Item::Print tmp(this, QT_ORDINARY); > + // TODO-10.5: add an error message to errmsg-utf8.txt > + my_printf_error(ER_UNKNOWN_ERROR, > + "'%s' is not allowed in this context", MYF(0), tmp.ptr()); I don't see any tests for this error > +} > + > + > void Item::push_note_converted_to_negative_complement(THD *thd) > { > push_warning(thd, Sql_condition::WARN_LEVEL_NOTE, ER_UNKNOWN_ERROR, > @@ -8945,6 +9022,14 @@ bool Item_default_value::fix_fields(THD *thd, Item **items) > return FALSE; > } > > + return tie_field(thd); > +} > + > + comment? what does tie_field do? > +bool Item_default_value::tie_field(THD *thd) > +{ > + Item *real_arg; > + Item_field *field_arg; > /* > DEFAULT() do not need table field so should not ask handler to bring > field value (mark column for read) > @@ -9131,8 +9241,54 @@ void Item_ignore_value::print(String *str, enum_query_type query_type) > str->append(STRING_WITH_LEN("ignore")); > } > > + > +bool Item_ignore_value::associate_with_target_field(THD *thd, > + Item_field *field) > +{ > + bitmap_set_bit(field->field->table->read_set, field->field->field_index); may be, remove it from the write_set instead? > + return Item_default_value::associate_with_target_field(thd, field); > +} > + > + > int Item_ignore_value::save_in_field(Field *field_arg, bool no_conversions) > { > + if (arg) > + { > + switch (find_ignore_reaction(field->table->in_use)) > + { > + case IGNORE_MEANS_DEFAULT: > + DBUG_ASSERT(0); // impossible now, but fully working code if needed Impossible? One cannot use IGNORE in INSERT? Not even in EXECUTE IMMEDIATE "INSERT ... " USING IGNORE? > + /* > + save default value > + Note: (((Field*)(arg()->real_item()) is Item_field* (checked in > + tie_field(). > + */ > + if ((((Item_field*)(arg->real_item()))->field->flags & > + NO_DEFAULT_VALUE_FLAG)) > + { > + my_error(ER_NO_DEFAULT_FOR_FIELD, MYF(0), > + ((Item_field*)(arg->real_item()))->field->field_name); > + return true; > + } > + calculate(); > + return Item_field::save_in_field(field_arg, no_conversions); > + case IGNORE_MEANS_FIELD_VALUE: > + // checked on tie_field > + DBUG_ASSERT(arg->real_item()->type() == FIELD_ITEM); > + /* > + save original field (Item::save_in_field is not applicable because > + result_field for the referenced field set to this temporary table). > + */ > + arg->save_val(field_arg); > + return false; > + default: > + ; // fall through to error > + } > + DBUG_ASSERT(0); //impossible > + my_error(ER_INVALID_DEFAULT_PARAM, MYF(0)); > + return true; > + } > + > return field_arg->save_in_field_ignore_value(context->error_processor == > &view_error_processor); > } > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 436f753557e..4dbda30708a 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -8340,6 +8341,9 @@ fill_record(THD *thd, TABLE *table, Field **ptr, List<Item> &values, > field->field_name, table->s->table_name.str); > } > > + if(evaluable && value->check_is_evaluable_expression_or_error()) > + goto err; this looks like the wrong way of doing it. whether an item is "evaluable" or not is metadata. It should be checked once before the statement execution > + > if (use_value) > value->save_val(field); > else > diff --git a/sql/sql_update.cc b/sql/sql_update.cc > index 65a828147ae..26ee427e3e7 100644 > --- a/sql/sql_update.cc > +++ b/sql/sql_update.cc > @@ -1796,6 +1796,9 @@ int multi_update::prepare(List<Item> &not_used_values, > while ((item= (Item_field *) field_it++)) > { > Item *value= value_it++; > + if (value->associate_with_target_field(thd, item)) > + DBUG_RETURN(1); what happens for single table updates? > + > uint offset= item->field->table->pos_in_table_list->shared; > fields_for_table[offset]->push_back(item, thd->mem_root); > values_for_table[offset]->push_back(value, thd->mem_root); > diff --git a/sql/unireg.cc b/sql/unireg.cc > index 083960523c1..a3bfc678e07 100644 > --- a/sql/unireg.cc > +++ b/sql/unireg.cc > @@ -1000,8 +1000,14 @@ static bool make_empty_rec(THD *thd, uchar *buff, uint table_options, > { > Item *expr= field->default_value->expr; > > - int res= !expr->fixed && // may be already fixed if ALTER TABLE > - expr->fix_fields(thd, &expr); > + int res= MY_TEST(!expr->fixed && // may be already fixed if ALTER TABLE > + expr->fix_fields(thd, &expr)); eh? what's the point of this change? > + if (!res && expr->check_is_evaluable_expression_or_error()) > + { > + error= 1; > + delete regfield; //To avoid memory leak > + goto err; > + } > if (!res) > res= expr->save_in_field(regfield, 1); > if (!res && (field->flags & BLOB_FLAG)) Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

[Maria-developers] Progress Report GSoC 2020 (20-26 Jule)
by Anton Mikhaylenko 28 Jul '20

28 Jul '20

Good day, everybody. In this post, I want to describe what I did this week, Jule 20-26. This week, I was refactoring. I added a flag for logging the number of threads and corrected the calculation of completed tasks. Also added comments to the code. Regards, Anton MIkhailenko

1 0

[Maria-developers] Weekly Report
by Kartik Soneji 27 Jul '20

27 Jul '20

Hi, This week - Resolved the abi_check issue with different compilers. Also added checks and updated documentation so that it does not happen again. - Started writing some tests for the lzma service. - Finished the first draft of the lzo service. - Worked on some refactors. - Discussed the idea of adding status variables to allow the storage engines to know which libraries are loaded. This will also allow the user to view the libraries that are loaded while the server is running. Next week - Finish writing tests for the lzma service. - Finalize the implementation of the service. - Start replicating the implementation for other libraries. - Discuss if we should change all to auto for the --use-compression= switch. That way, the server can throw an error if it is started with --use-compression=lzma but not if it is started normally. This will also allow for a configuration like --use-compression=lzma,auto so that the server will try to load all libraries but will not start unless lzma is loaded. Thanks, Kartik

1 0

[Maria-developers] 9b999e79a35: MDEV-23108: Point in time recovery of binary log fails when sql_mode=ORACLE
by sujatha 24 Jul '20

24 Jul '20

revision-id: 9b999e79a358de2c86ae44fe83cf6c62cff1d2ef (mariadb-10.3.21-180-g9b999e79a35) parent(s): 73aa31fbfd793bdb597bb19c52118ab4e637f4bc author: Sujatha committer: Sujatha timestamp: 2020-07-17 12:23:32 +0530 message: MDEV-23108: Point in time recovery of binary log fails when sql_mode=ORACLE Problem: ======== During point in time recovery of binary log syntax error is reported for BEGIN statement and recovery fails. Analysis: ========= In MariaDB 10.3 and later, setting the sql_mode system variable to Oracle allows the server to understand a subset of Oracle's PL/SQL language. When sql_mode=ORACLE is set, it switches the parser from the MariaDB parser to Oracle compatible parser. With this change 'BEGIN' is not considered as 'START TRANSACTION'. Hence the syntax error is reported. Fix: === Add a new option to 'mysqlbinlog' tool named 'sql_mode_oracle'. When 'sql_mode_oracle' option is specified 'BEGIN' statements will be replaced with 'START TRANSACTION' in the mysqlbinlog output. --- client/mysqlbinlog.cc | 7 +- .../compat/oracle/r/binlog_ptr_mysqlbinlog.result | 100 +++++++++++++++++ .../oracle/t/binlog_ptr_mysqlbinlog-master.opt | 1 + .../compat/oracle/t/binlog_ptr_mysqlbinlog.test | 120 +++++++++++++++++++++ sql/log_event.cc | 15 +-- sql/log_event.h | 1 + 6 files changed, 237 insertions(+), 7 deletions(-) diff --git a/client/mysqlbinlog.cc b/client/mysqlbinlog.cc index a732a6ab8c3..ad995629272 100644 --- a/client/mysqlbinlog.cc +++ b/client/mysqlbinlog.cc @@ -145,6 +145,7 @@ static MYSQL* mysql = NULL; static const char* dirname_for_local_load= 0; static bool opt_skip_annotate_row_events= 0; +static my_bool opt_sql_mode_oracle= 0; static my_bool opt_flashback; #ifdef WHEN_FLASHBACK_REVIEW_READY static my_bool opt_flashback_review; @@ -1045,7 +1046,8 @@ Exit_status process_event(PRINT_EVENT_INFO *print_event_info, Log_event *ev, DBUG_ENTER("process_event"); Exit_status retval= OK_CONTINUE; IO_CACHE *const head= &print_event_info->head_cache; - + /* Propogate sql_mode_oracle settings to event */ + ev->is_sql_mode_oracle= opt_sql_mode_oracle; /* Bypass flashback settings to event */ ev->is_flashback= opt_flashback; #ifdef WHEN_FLASHBACK_REVIEW_READY @@ -1704,6 +1706,9 @@ static struct my_option my_options[] = "use --base64-output=never", &short_form, &short_form, 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0}, + {"sql_mode_oracle", 0, "Converts BEGIN statement to 'ORALCE' " + "sql_mode compatible START TRANSACTION.", &opt_sql_mode_oracle, + &opt_sql_mode_oracle, 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0}, {"socket", 'S', "The socket file to use for connection.", &sock, &sock, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, diff --git a/mysql-test/suite/compat/oracle/r/binlog_ptr_mysqlbinlog.result b/mysql-test/suite/compat/oracle/r/binlog_ptr_mysqlbinlog.result new file mode 100644 index 00000000000..3080a4b69a0 --- /dev/null +++ b/mysql-test/suite/compat/oracle/r/binlog_ptr_mysqlbinlog.result @@ -0,0 +1,100 @@ +SET @@SQL_MODE = 'ORACLE'; +########################################################################## +# Test verifies Gtid_log_event/Xid_log_event specific print # +########################################################################## +CREATE TABLE tm (f INT) ENGINE=MYISAM; +INSERT INTO tm VALUES (10); +CREATE TABLE t(f INT) ENGINE=INNODB; +INSERT INTO t VALUES (10); +CREATE OR REPLACE PROCEDURE simpleproc (param1 OUT INT) AS +BEGIN +SELECT COUNT(*) INTO param1 FROM t; +END; +/ +CREATE FUNCTION f1 RETURN INT +AS +BEGIN +RETURN 10; +END; +/ +FLUSH LOGS; +########################################################################## +# Delete data from master so that it can be restored from binlog # +########################################################################## +DROP FUNCTION f1; +DROP PROCEDURE simpleproc; +DROP TABLE tm; +DROP TABLE t; +########################################################################## +# Post recovery using mysqlbinlog --sql-mode-oracle # +########################################################################## +SHOW TABLES; +Tables_in_test +t +tm +SELECT * FROM tm; +f +10 +SELECT * FROM t; +f +10 +SELECT f1(); +f1() +10 +CALL simpleproc(@a); +SELECT @a; +@a +1 +"***** Clean Up *****" +DROP TABLE t,tm; +DROP PROCEDURE simpleproc; +DROP FUNCTION f1; +RESET MASTER; +########################################################################## +# Test verifies Gtid_log_event/Xid_log_event/Qery_log_event # +# specific print along with flashback option # +########################################################################## +CREATE TABLE tm(f INT) ENGINE=MYISAM; +INSERT INTO tm VALUES (10); +INSERT INTO tm VALUES (20); +CREATE TABLE t(f INT) ENGINE=INNODB; +INSERT INTO t VALUES (10); +INSERT INTO t VALUES (20); +########################################################################## +# Initial data # +########################################################################## +SELECT * FROM tm; +f +10 +20 +SELECT * FROM t; +f +10 +20 +FLUSH LOGS; +DELETE FROM tm WHERE f=20; +DELETE FROM t WHERE f=20; +FLUSH LOGS; +########################################################################## +# Data after deletion # +########################################################################## +SELECT * FROM tm; +f +10 +SELECT * FROM t; +f +10 +FOUND 2 /START TRANSACTION/ in test_sql_mode.sql +########################################################################## +# Data after recovery using flashback, sql-mode-oracle # +########################################################################## +SELECT * FROM tm; +f +10 +20 +SELECT * FROM t; +f +10 +20 +"***** Clean Up *****" +DROP TABLE t,tm; diff --git a/mysql-test/suite/compat/oracle/t/binlog_ptr_mysqlbinlog-master.opt b/mysql-test/suite/compat/oracle/t/binlog_ptr_mysqlbinlog-master.opt new file mode 100644 index 00000000000..8f0cc182f51 --- /dev/null +++ b/mysql-test/suite/compat/oracle/t/binlog_ptr_mysqlbinlog-master.opt @@ -0,0 +1 @@ +--flashback diff --git a/mysql-test/suite/compat/oracle/t/binlog_ptr_mysqlbinlog.test b/mysql-test/suite/compat/oracle/t/binlog_ptr_mysqlbinlog.test new file mode 100644 index 00000000000..07a1b5c4fdb --- /dev/null +++ b/mysql-test/suite/compat/oracle/t/binlog_ptr_mysqlbinlog.test @@ -0,0 +1,120 @@ +# ==== Purpose ==== +# +# Test verifies that a binary log which was generated by setting +# 'sql_mod=ORACLE' can be succefully used for point in time recovery with +# following options. +# 1) mysqlbinlog --sql-mode-oracle +# 2) mysqlbinlog --flashback --sql-mode-oracle +# +# BEGIN statement is printed in three places +# 1) "Gtid_log_event::print" +# 2) "Xid_log_event::print" if flashback is enabled +# 3) "Query_log_event::print" if flashback is enabled and engine is +# non-transacional. +# +# Test verifies all these cases. +# +# ==== References ==== +# +# MDEV-23108: Point in time recovery of binary log fails when sql_mode=ORACLE +# +--source include/have_log_bin.inc +--source include/have_innodb.inc + +let $MYSQLD_DATADIR= `select @@datadir`; +SET @@SQL_MODE = 'ORACLE'; + +--echo ########################################################################## +--echo # Test verifies Gtid_log_event/Xid_log_event specific print # +--echo ########################################################################## +CREATE TABLE tm (f INT) ENGINE=MYISAM; +INSERT INTO tm VALUES (10); + +CREATE TABLE t(f INT) ENGINE=INNODB; +INSERT INTO t VALUES (10); + +DELIMITER /; +CREATE OR REPLACE PROCEDURE simpleproc (param1 OUT INT) AS + BEGIN + SELECT COUNT(*) INTO param1 FROM t; + END; +/ +CREATE FUNCTION f1 RETURN INT +AS +BEGIN + RETURN 10; +END; +/ +DELIMITER ;/ + +FLUSH LOGS; +--echo ########################################################################## +--echo # Delete data from master so that it can be restored from binlog # +--echo ########################################################################## +DROP FUNCTION f1; +DROP PROCEDURE simpleproc; +DROP TABLE tm; +DROP TABLE t; + +--exec $MYSQL_BINLOG --sql-mode-oracle $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/test.sql +--exec $MYSQL < $MYSQLTEST_VARDIR/tmp/test.sql + +--echo ########################################################################## +--echo # Post recovery using mysqlbinlog --sql-mode-oracle # +--echo ########################################################################## +SHOW TABLES; +SELECT * FROM tm; +SELECT * FROM t; +--horizontal_results +SELECT f1(); +CALL simpleproc(@a); +SELECT @a; + +--echo "***** Clean Up *****" +DROP TABLE t,tm; +DROP PROCEDURE simpleproc; +DROP FUNCTION f1; +--remove_file $MYSQLTEST_VARDIR/tmp/test.sql +RESET MASTER; + +--echo ########################################################################## +--echo # Test verifies Gtid_log_event/Xid_log_event/Qery_log_event # +--echo # specific print along with flashback option # +--echo ########################################################################## +CREATE TABLE tm(f INT) ENGINE=MYISAM; +INSERT INTO tm VALUES (10); +INSERT INTO tm VALUES (20); +CREATE TABLE t(f INT) ENGINE=INNODB; +INSERT INTO t VALUES (10); +INSERT INTO t VALUES (20); +--echo ########################################################################## +--echo # Initial data # +--echo ########################################################################## +SELECT * FROM tm; +SELECT * FROM t; +FLUSH LOGS; +DELETE FROM tm WHERE f=20; +DELETE FROM t WHERE f=20; +FLUSH LOGS; + +--echo ########################################################################## +--echo # Data after deletion # +--echo ########################################################################## +SELECT * FROM tm; +SELECT * FROM t; +--exec $MYSQL_BINLOG --flashback --sql-mode-oracle $MYSQLD_DATADIR/master-bin.000002 > $MYSQLTEST_VARDIR/tmp/test_sql_mode.sql + +--let SEARCH_FILE=$MYSQLTEST_VARDIR/tmp/test_sql_mode.sql +--let SEARCH_PATTERN=START TRANSACTION +--source include/search_pattern_in_file.inc +--exec $MYSQL < $MYSQLTEST_VARDIR/tmp/test_sql_mode.sql + +--echo ########################################################################## +--echo # Data after recovery using flashback, sql-mode-oracle # +--echo ########################################################################## +SELECT * FROM tm; +SELECT * FROM t; + +--echo "***** Clean Up *****" +DROP TABLE t,tm; +--remove_file $MYSQLTEST_VARDIR/tmp/test_sql_mode.sql diff --git a/sql/log_event.cc b/sql/log_event.cc index 57a39085e86..e06e6a1b1d6 100644 --- a/sql/log_event.cc +++ b/sql/log_event.cc @@ -5358,8 +5358,9 @@ bool Query_log_event::print(FILE* file, PRINT_EVENT_INFO* print_event_info) } else if (strcmp("COMMIT", query) == 0) { - if (my_b_write(&cache, (uchar*) "BEGIN", 5) || - my_b_printf(&cache, "\n%s\n", print_event_info->delimiter)) + if (my_b_printf(&cache, is_sql_mode_oracle ? + "START TRANSACTION\n%s\n" : "BEGIN\n%s\n" , + print_event_info->delimiter)) goto err; } } @@ -8254,9 +8255,10 @@ Gtid_log_event::print(FILE *file, PRINT_EVENT_INFO *print_event_info) goto err; } if (!(flags2 & FL_STANDALONE)) - if (my_b_printf(&cache, is_flashback ? "COMMIT\n%s\n" : "BEGIN\n%s\n", print_event_info->delimiter)) + if (my_b_printf(&cache, is_flashback ? "COMMIT\n%s\n" : + (is_sql_mode_oracle ? "START TRANSACTION\n%s\n" : + "BEGIN\n%s\n"), print_event_info->delimiter)) goto err; - return cache.flush_data(); err: return 1; @@ -8937,8 +8939,9 @@ bool Xid_log_event::print(FILE* file, PRINT_EVENT_INFO* print_event_info) my_b_printf(&cache, "\tXid = %s\n", buf)) goto err; } - if (my_b_printf(&cache, is_flashback ? "BEGIN%s\n" : "COMMIT%s\n", - print_event_info->delimiter)) + if (my_b_printf(&cache, is_flashback ? (is_sql_mode_oracle ? + "START TRANSACTION\n%s\n" : "BEGIN\n%s\n") : + "COMMIT\n%s\n", print_event_info->delimiter)) goto err; return cache.flush_data(); diff --git a/sql/log_event.h b/sql/log_event.h index 73809953d70..66e29579fbc 100644 --- a/sql/log_event.h +++ b/sql/log_event.h @@ -1270,6 +1270,7 @@ class Log_event /* The following code used for Flashback */ #ifdef MYSQL_CLIENT + my_bool is_sql_mode_oracle; my_bool is_flashback; my_bool need_flashback_review; String output_buf; // Storing the event output

3 7

Re: [Maria-developers] 0dd0bbe54e1: MDEV-14836: Assertion `m_status == DA_ERROR' failed in
by Sergei Golubchik 22 Jul '20

22 Jul '20

Hi, Rucha! On Jul 22, Rucha Deodhar wrote: > revision-id: 0dd0bbe54e1 (mariadb-10.2.31-307-g0dd0bbe54e1) > parent(s): 0994af43e58 > author: Rucha Deodhar <rucha.deodhar(a)mariadb.com> > committer: Rucha Deodhar <rucha.deodhar(a)mariadb.com> > timestamp: 2020-07-16 22:29:06 +0530 > message: > > MDEV-14836: Assertion `m_status == DA_ERROR' failed in > Diagnostics_area::sql_errno upon query from I_S with LIMIT ROWS EXAMINED > > Query is aborted because number of examined rows exceeds LIMIT EXAMINED ROWS. > Even though this is not a real error, open_normal_and_derived_tables() > returns true. As a result, thd->is_errno() is called and thus the assertion > failure. > > diff --git a/mysql-test/r/information_schema.result b/mysql-test/r/information_schema.result > index 2d404aff6bc..7644ff4e7a2 100644 > --- a/mysql-test/r/information_schema.result > +++ b/mysql-test/r/information_schema.result > @@ -2184,3 +2184,16 @@ SCHEMA_NAME > # > # End of 10.1 tests > # > +# > +# Start of 10.2 Test > +# > +# MDEV-14836: Assertion `m_status == DA_ERROR' failed in > +# Diagnostics_area::sql_errno upon query from I_S with LIMIT ROWS EXAMINED > +# > +SELECT * FROM INFORMATION_SCHEMA.`COLUMNS` LIMIT ROWS EXAMINED 10; > +TABLE_CATALOG TABLE_SCHEMA TABLE_NAME COLUMN_NAME ORDINAL_POSITION COLUMN_DEFAULT IS_NULLABLE DATA_TYPE CHARACTER_MAXIMUM_LENGTH CHARACTER_OCTET_LENGTH NUMERIC_PRECISION NUMERIC_SCALE DATETIME_PRECISION CHARACTER_SET_NAME COLLATION_NAME COLUMN_TYPE COLUMN_KEY EXTRA PRIVILEGES COLUMN_COMMENT IS_GENERATED GENERATION_EXPRESSION > +Warnings: > +Warning 1931 Query execution was interrupted. The query examined at least 666 rows, which exceeds LIMIT ROWS EXAMINED (10). The query result may be incomplete I don't understand why there are no rows in the output here? It has examined quite a lot of rows, I would expect a truncated result, but not a completely empty one. Could you check that out, please? > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 436f753557e..8f1c0ebab29 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -1590,6 +1590,12 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) > if (!(flags & MYSQL_OPEN_IGNORE_KILLED) && thd->killed) > { > thd->send_kill_message(); > + /* > + Query was only aborted, but this is not an actual error. > + So return false. > + */ > + if (thd->killed == ABORT_QUERY) > + DBUG_RETURN(FALSE); > DBUG_RETURN(TRUE); This looks somewhat risky. open_table() returns TRUE on error and FALSE on success. You here make it to return FALSE even if the table wasn't opened. I am not sure the caller will always be able to handle that. On the other hand, normally all tables are opened first, and then the server starts reading the data. It's only in I_S the server opens tables one by one. So get_all_tables() could be the only possibility for ABORT_QUERY to happen before open_table(). In that case it is safer to add a check for thd->killed into get_all_tables(). Could you try that approach instead? Also, note, don't compare thd->killed with ABORT_QUERY, use thd->killed_errno() method instead. > } > > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 18b7e92bca5..9f834ee118b 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -5622,6 +5622,16 @@ static int get_schema_column_record(THD *thd, TABLE_LIST *tables, > } > DBUG_RETURN(res); > } > + /* > + Query was aborted. This could be because of number of > + examined rows exceeded LIMIT ROWS EXAMINED. But don't > + issue warning here. It is done later, in handle_select(). > + */ > + if(thd->killed == ABORT_QUERY) > + { > + res= 0; > + DBUG_RETURN(res); > + } > > show_table= tables->table; > count= 0; > Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

[Maria-developers] 27b046d2e95: MDEV-14203: rpl.rpl_extra_col_master_myisam, rpl.rpl_slave_load_tmpdir_not_exist failed in buildbot with a warning
by sujatha 21 Jul '20

21 Jul '20

revision-id: 27b046d2e95add5ba87e231279d2a7774db5b0c7 (mariadb-10.2.31-317-g27b046d2e95) parent(s): a1e52e7f32cee7c204078db4c4beb88250d7e1f6 author: Sujatha committer: Sujatha timestamp: 2020-07-21 11:16:40 +0530 message: MDEV-14203: rpl.rpl_extra_col_master_myisam, rpl.rpl_slave_load_tmpdir_not_exist failed in buildbot with a warning Problem: ======= rpl.rpl_slave_load_tmpdir_not_exist 'stmt' w3 [ fail ] Found warnings/errors in server log file! Test ended at 2017-09-27 20:34:55 [Warning] Master is configured to log replication events with checksum, but will not send such events to slaves that cannot process them ^ Found warnings in /mnt/buildbot/build/mariadb-10.2.10/mysql-test/var/3/log/mysqld.1.err ok Analysis: ======== When slave tries to connect to master 'get_master_version_and_clock' function is invoked to perform elaborated slave-master handshake. During this process slave server queries master server, to know if it is checksum aware and at the same time master is notified about its CRC-awareness. The master's side instant value of @@global.binlog_checksum is stored in the dump thread's uservar area as well as cached locally to become known in consensus by master and slave. Post hand-shake slave requests master for binlog dump. It sends 'COM_BINLOG_DUMP'. This command is sent to master by 'cli_advanced_command' call. If there is some temporary network failure during this request_dump call, 'end_server' is invoked to close the current connection between master and slave. Upon connection close the dump thread on the master gets terminated and it clears the 'uservar' data it got through master-slave handshake. The 'COM_BINLOG_DUMP' command is sent once again without master-slave handshake. Since the checksum data is not available with new dump thread a warning gets reported. Fix: === Upon network write error donot attempt reconnect, proceed to master-slave handshake. This ensures that master is aware of slave's capability to use checksums. --- .../rpl/r/rpl_dump_request_retry_warning.result | 22 ++++++++ .../rpl/t/rpl_dump_request_retry_warning.test | 60 ++++++++++++++++++++++ sql-common/client.c | 2 + sql/net_serv.cc | 13 +++++ sql/slave.cc | 3 +- 5 files changed, 99 insertions(+), 1 deletion(-) diff --git a/mysql-test/suite/rpl/r/rpl_dump_request_retry_warning.result b/mysql-test/suite/rpl/r/rpl_dump_request_retry_warning.result new file mode 100644 index 00000000000..3a80d5b5f31 --- /dev/null +++ b/mysql-test/suite/rpl/r/rpl_dump_request_retry_warning.result @@ -0,0 +1,22 @@ +include/master-slave.inc +[connection master] +CREATE TABLE t1(a int); +INSERT INTO t1 VALUES(1); +connection slave; +SET @saved_dbug = @@GLOBAL.debug_dbug; +SET @@global.debug_dbug= 'd,simulate_error_on_packet_write'; +START SLAVE; +SET DEBUG_SYNC= 'now WAIT_FOR parked'; +SET @@GLOBAL.debug_dbug = @saved_dbug; +SET DEBUG_SYNC= 'now SIGNAL continue'; +SET DEBUG_SYNC= 'RESET'; +include/wait_for_slave_io_to_start.inc +include/wait_for_slave_sql_to_start.inc +connection master; +include/sync_slave_sql_with_master.inc +SELECT * FROM t1; +a +1 +connection master; +DROP TABLE t1; +include/rpl_end.inc diff --git a/mysql-test/suite/rpl/t/rpl_dump_request_retry_warning.test b/mysql-test/suite/rpl/t/rpl_dump_request_retry_warning.test new file mode 100644 index 00000000000..d9fb8258765 --- /dev/null +++ b/mysql-test/suite/rpl/t/rpl_dump_request_retry_warning.test @@ -0,0 +1,60 @@ +# ==== Purpose ==== +# +# Test verifies that, due to a temporary network error, if request dump +# command specific packet write operation fails then the write error gets +# handled appropriately. Further retry will be initiated with appropriate +# slave registration on master. This will ensure that master has all the +# details of slave and no warnings are reported on the master side. +# +# ==== Implementation ==== +# +# Steps: +# 0 - Skip the slave start. +# 1 - Enable debug simulation which will simulate packet write error during +# dump request command execution. +# 2 - Start the slave. Observe that slave is able to reconnect post +# temporary network write error. +# +# ==== References ==== +# +# MDEV-14203: rpl.rpl_extra_col_master_myisam, +# rpl.rpl_slave_load_tmpdir_not_exist failed in buildbot with a +# warning +# +# MDEV-13258: rpl.rpl_skip_replication, rpl.rpl_set_statement_default_master +# failed in buildbot +# + +--source include/have_debug.inc +--source include/have_debug_sync.inc +--let $rpl_skip_start_slave=1 +--source include/master-slave.inc + +# Do an insert on master +CREATE TABLE t1(a int); +INSERT INTO t1 VALUES(1); + +# Add a debug point and start the slave so that dump request fails. +connection slave; +SET @saved_dbug = @@GLOBAL.debug_dbug; +SET @@global.debug_dbug= 'd,simulate_error_on_packet_write'; + +START SLAVE; +SET DEBUG_SYNC= 'now WAIT_FOR parked'; +SET @@GLOBAL.debug_dbug = @saved_dbug; +SET DEBUG_SYNC= 'now SIGNAL continue'; +SET DEBUG_SYNC= 'RESET'; + +--source include/wait_for_slave_io_to_start.inc +--source include/wait_for_slave_sql_to_start.inc + +# Sync the slave and verify that slave has caught up with the master. +connection master; +--source include/sync_slave_sql_with_master.inc +SELECT * FROM t1; + +# Cleanup +connection master; +DROP TABLE t1; + +--source include/rpl_end.inc diff --git a/sql-common/client.c b/sql-common/client.c index 227759f4163..b66eee2a508 100644 --- a/sql-common/client.c +++ b/sql-common/client.c @@ -721,6 +721,8 @@ cli_advanced_command(MYSQL *mysql, enum enum_server_command command, set_mysql_error(mysql, CR_NET_PACKET_TOO_LARGE, unknown_sqlstate); goto end; } + if (net->last_errno == ER_NET_ERROR_ON_WRITE && command == COM_BINLOG_DUMP) + goto end; end_server(mysql); if (mysql_reconnect(mysql) || stmt_skip) goto end; diff --git a/sql/net_serv.cc b/sql/net_serv.cc index f39eaaadf46..e625b307343 100644 --- a/sql/net_serv.cc +++ b/sql/net_serv.cc @@ -45,6 +45,7 @@ #include <violite.h> #include <signal.h> #include "probes_mysql.h" +#include <debug_sync.h> #ifdef EMBEDDED_LIBRARY #undef MYSQL_SERVER @@ -486,6 +487,18 @@ net_write_command(NET *net,uchar command, DBUG_ENTER("net_write_command"); DBUG_PRINT("enter",("length: %lu", (ulong) len)); + DBUG_EXECUTE_IF("simulate_error_on_packet_write", + { + if (command == COM_BINLOG_DUMP) + { + net->last_errno = ER_NET_ERROR_ON_WRITE; + socket_errno = 2; + DBUG_ASSERT(!debug_sync_set_action( + (THD *)net->thd, + STRING_WITH_LEN("now SIGNAL parked WAIT_FOR continue"))); + DBUG_RETURN(true); + } + };); MYSQL_NET_WRITE_START(length); buff[4]=command; /* For first packet */ diff --git a/sql/slave.cc b/sql/slave.cc index 1bf83aa9652..87eacfcfd0a 100644 --- a/sql/slave.cc +++ b/sql/slave.cc @@ -3297,7 +3297,8 @@ static int request_dump(THD *thd, MYSQL* mysql, Master_info* mi, in the future, we should do a better error analysis, but for now we just fill up the error log :-) */ - if (mysql_errno(mysql) == ER_NET_READ_INTERRUPTED) + if (mysql_errno(mysql) == ER_NET_READ_INTERRUPTED || + mysql_errno(mysql) == ER_NET_ERROR_ON_WRITE) *suppress_warnings= TRUE; // Suppress reconnect warning else sql_print_error("Error on COM_BINLOG_DUMP: %d %s, will retry in %d secs",

1 0

[Maria-developers] Weekly Report
by Kartik Soneji 20 Jul '20

20 Jul '20

Hi, This week - Changed the build system to use our headers instead of the libraries’ headers. - I found that running make abi_update on gcc and compiling on clang (or vice versa) fails the abi_check. - During cmake documentation review, I discovered that the existing cmake configuration uses a lot of depricated commands. Even for this project, it might be easier and more stable to rewrite the relavent portions of the cmake files instead of trying to work around the depricated commands. - Removed the test cases and printf statements from the code. Next week - Resolve the abi_check issue with different compilers. - Write tests for the lzma service. - Start work on the next service. Thanks, Kartik

1 0