- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 3855: Bumping server version. (10.0.13-galera) in lp:~maria-captains/maria/maria-10.0-galera
by Roberto Spadim 30 Jun '14

30 Jun '14

=] nice nirbhay 2014-06-30 10:33 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: > Hi, > > On Mon, Jun 30, 2014 at 9:13 AM, Roberto Spadim <roberto(a)spadim.com.br> > wrote: >> >> this will be a production ready version? > > > 10.0.12-galera (released over weekend) is the first 10.0-galera stable > release. This 'patch version bump' is > for preparation towards the next release. > > Best, > Nirbhay > >> >> >> 2014-06-30 10:03 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: >> > At lp:~maria-captains/maria/maria-10.0-galera >> > >> > ------------------------------------------------------------ >> > revno: 3855 >> > revision-id: nirbhay(a)skysql.com-20140630130329-w8kiirtf3blli7l1 >> > parent: nirbhay(a)skysql.com-20140626161112-q1iqtoyq5owutvvb >> > committer: Nirbhay Choubey <nirbhay(a)skysql.com> >> > branch nick: maria-10.0-galera >> > timestamp: Mon 2014-06-30 09:03:29 -0400 >> > message: >> > Bumping server version. (10.0.13-galera) >> > >> > === modified file 'VERSION' >> > --- a/VERSION 2014-06-19 17:12:38 +0000 >> > +++ b/VERSION 2014-06-30 13:03:29 +0000 >> > @@ -4,5 +4,5 @@ >> > # >> > MYSQL_VERSION_MAJOR=10 >> > MYSQL_VERSION_MINOR=0 >> > -MYSQL_VERSION_PATCH=12 >> > +MYSQL_VERSION_PATCH=13 >> > MYSQL_VERSION_EXTRA= >> > >> > >> > _______________________________________________ >> > commits mailing list >> > commits(a)mariadb.org >> > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> _______________________________________________ >> commits mailing list >> commits(a)mariadb.org >> https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] [Commits] Rev 3855: Bumping server version. (10.0.13-galera) in lp:~maria-captains/maria/maria-10.0-galera
by Roberto Spadim 30 Jun '14

30 Jun '14

this will be a production ready version? 2014-06-30 10:03 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: > At lp:~maria-captains/maria/maria-10.0-galera > > ------------------------------------------------------------ > revno: 3855 > revision-id: nirbhay(a)skysql.com-20140630130329-w8kiirtf3blli7l1 > parent: nirbhay(a)skysql.com-20140626161112-q1iqtoyq5owutvvb > committer: Nirbhay Choubey <nirbhay(a)skysql.com> > branch nick: maria-10.0-galera > timestamp: Mon 2014-06-30 09:03:29 -0400 > message: > Bumping server version. (10.0.13-galera) > > === modified file 'VERSION' > --- a/VERSION 2014-06-19 17:12:38 +0000 > +++ b/VERSION 2014-06-30 13:03:29 +0000 > @@ -4,5 +4,5 @@ > # > MYSQL_VERSION_MAJOR=10 > MYSQL_VERSION_MINOR=0 > -MYSQL_VERSION_PATCH=12 > +MYSQL_VERSION_PATCH=13 > MYSQL_VERSION_EXTRA= > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Merge request: Cross-compile fixes (MySQL bug #61340)
by James Le Cuirot 30 Jun '14

30 Jun '14

Hello all, I posted a merge request to Launchpad back in April. A Launchpad glitch meant that it didn't get any attention but I was told on IRC to post about it here. The merge request: https://code.launchpad.net/~chewi/maria/mysql-fix-bug61340/+merge/217351 The commit diff: http://bazaar.launchpad.net/~chewi/maria/mysql-fix-bug61340/revision/4165 Given the nature of the fix, I can appreciate that it is difficult for other developers to test but I can tell you that I have just tried out a cross-built MariaDB under QEMU/proot and it seems to work. I have also seen posts from others stating that they made similar temporary adjustments to the CMake files with successful results. I am providing this contribution to the MariaDB Foundation under the MCA and also wish for it to be applied to MySQL. Regards, James

2 2

[Maria-developers] Connector Coding till 29th June
by Puneet Dewan 29 Jun '14

29 Jun '14

Hi Everyone, This week I fixed the null bitmap and types that were being sent to the server. So now I am able to successfully insert multiple values in the DB. Currently only Integer and String. This week 1.I will extend all the TypeParameter classes and ParameterWriter class so as to send the value in binary format. 2.Also try insert on different data types. 3. Write test cases Currently I am extending the PacketOutputStream.java class. Tomorrow I will have code review with Massimo. Regards Puneet.

1 0

[Maria-developers] TIME/DATETIME/TIMESTAMP data types with fractional precision
by Alexander Barkov 26 Jun '14

26 Jun '14

Hello, I propose to switch to MySQL-5.6 TIME/DATETIME/TIMESTAMP formats by default and deprecate MariaDB native temporal data types, together with pre-5.6 non-fractional types. The problem is that MariaDB native data types (and MySQL-5.5 non-fractional data types) do not work with row based replication in case of: TIME(N) -> TIME(M) There is no metadata telling about the column precision in the binary log file. It can only work when N=M. MySQL-5.6 uses separate type codes for the temporal data types: MYSQL_TYPE_TIME2 MYSQL_TYPE_DATETIME2 MYSQL_TYPE_TIMESTAMP2 and does put precision into metadata in the binary log. So it works for N != M. There is a JIRA task for this: MDEV-5377 Row-based replication of MariaDB temporal data types with FSP>0 into a different column type I'd propose to do it in 10.1.x, as 10.0.x is too late. Greetings.

2 1

Re: [Maria-developers] [Commits] Rev 4245: MDEV-6336: mysqldump --master-data does not work with GTID setups in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Sergey Vojtovich 25 Jun '14

25 Jun '14

Hi Kristian, MDEV-6344 looks fine. But I don't feel confident reviewing MDEV-6336. Regards, Sergey On Mon, Jun 16, 2014 at 11:38:04AM +0200, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4245 > revision-id: knielsen(a)knielsen-hq.org-20140616093803-gub60pda6t77ch3k > parent: igor(a)askmonty.org-20140610223256-nqs0kkp7i45qzyvx > committer: knielsen(a)knielsen-hq.org > branch nick: work-10.0 > timestamp: Mon 2014-06-16 11:38:03 +0200 > message: > MDEV-6336: mysqldump --master-data does not work with GTID setups > MDEV-6344: mysqldump issues FLUSH TABLES, which gets written into binlog and replicated > > Add a --gtid option (for compatibility, the original behaviour is preserved > when --gtid is not used). > > With --gtid, --master-data and --dump-slave output the GTID position (the > old-style file/offset position is still output, but commented out). Also, a > CHANGE MASTER TO master_use_gtid=slave_pos is output to ensure a provisioned > slave is configured in GTID, as requested. > > Without --gtid, the GTID position is still output, if available, but commented > out. > > Also fix MDEV-6344, to avoid FLUSH TABLES getting into the binlog. Otherwise a > mysqldump on a slave server will silently inject a GTID which does not exist > on the master, which is highly undesirable. > === modified file 'client/client_priv.h' > --- a/client/client_priv.h 2013-09-14 01:09:36 +0000 > +++ b/client/client_priv.h 2014-06-16 09:38:03 +0000 > @@ -92,6 +92,7 @@ enum options_client > OPT_REPORT_PROGRESS, > OPT_SKIP_ANNOTATE_ROWS_EVENTS, > OPT_SSL_CRL, OPT_SSL_CRLPATH, > + OPT_USE_GTID, > OPT_MAX_CLIENT_OPTION /* should be always the last */ > }; > > > === modified file 'client/mysqldump.c' > --- a/client/mysqldump.c 2014-05-09 10:35:11 +0000 > +++ b/client/mysqldump.c 2014-06-16 09:38:03 +0000 > @@ -134,6 +134,7 @@ static ulong opt_compatible_mode= 0; > #define MYSQL_OPT_SLAVE_DATA_COMMENTED_SQL 2 > static uint opt_mysql_port= 0, opt_master_data; > static uint opt_slave_data; > +static uint opt_use_gtid; > static uint my_end_arg; > static char * opt_mysql_unix_port=0; > static int first_error=0; > @@ -346,6 +347,13 @@ static struct my_option my_long_options[ > {"force", 'f', "Continue even if we get an SQL error.", > &ignore_errors, &ignore_errors, 0, GET_BOOL, NO_ARG, > 0, 0, 0, 0, 0, 0}, > + {"gtid", OPT_USE_GTID, "Used together with --master-data=1 or --dump-slave=1." > + "When enabled, the output from those options will set the GTID position " > + "instead of the binlog file and offset; the file/offset will appear only as " > + "a comment. When disabled, the GTID position will still appear in the " > + "output, but only commented.", > + &opt_use_gtid, &opt_use_gtid, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, > {"help", '?', "Display this help message and exit.", 0, 0, 0, GET_NO_ARG, > NO_ARG, 0, 0, 0, 0, 0, 0}, > {"hex-blob", OPT_HEXBLOB, "Dump binary strings (BINARY, " > @@ -1177,7 +1185,7 @@ check_consistent_binlog_pos(char *binlog > > if (mysql_query_with_error_report(mysql, &res, > "SHOW STATUS LIKE 'binlog_snapshot_%'")) > - return 1; > + return 0; > > found= 0; > while ((row= mysql_fetch_row(res))) > @@ -1200,6 +1208,90 @@ check_consistent_binlog_pos(char *binlog > return (found == 2); > } > > + > +/* > + Get the GTID position corresponding to a given old-style binlog position. > + This uses BINLOG_GTID_POS(). The advantage is that the GTID position can > + be obtained completely non-blocking in this way (without the need for > + FLUSH TABLES WITH READ LOCK), as the old-style position can be obtained > + with START TRANSACTION WITH CONSISTENT SNAPSHOT. > + > + Returns 0 if ok, non-zero if error. > +*/ > +static int > +get_binlog_gtid_pos(char *binlog_pos_file, char *binlog_pos_offset, > + char *out_gtid_pos) > +{ > + DYNAMIC_STRING query; > + MYSQL_RES *res; > + MYSQL_ROW row; > + int err; > + char file_buf[FN_REFLEN*2+1], offset_buf[LONGLONG_LEN*2+1]; > + size_t len_pos_file= strlen(binlog_pos_file); > + size_t len_pos_offset= strlen(binlog_pos_offset); > + > + if (len_pos_file >= FN_REFLEN || len_pos_offset > LONGLONG_LEN) > + return 0; > + mysql_real_escape_string(mysql, file_buf, binlog_pos_file, len_pos_file); > + mysql_real_escape_string(mysql, offset_buf, binlog_pos_offset, len_pos_offset); > + init_dynamic_string_checked(&query, "SELECT BINLOG_GTID_POS('", 256, 1024); > + dynstr_append_checked(&query, file_buf); > + dynstr_append_checked(&query, "', '"); > + dynstr_append_checked(&query, offset_buf); > + dynstr_append_checked(&query, "')"); > + > + err= mysql_query_with_error_report(mysql, &res, query.str); > + dynstr_free(&query); > + if (err) > + return err; > + > + err= 1; > + if ((row= mysql_fetch_row(res))) > + { > + strmake(out_gtid_pos, row[0], FN_REFLEN-1); > + err= 0; > + } > + mysql_free_result(res); > + > + return err; > +} > + > + > +/* > + Get the GTID position on a master or slave. > + The parameter MASTER is non-zero to get the position on a master > + (@@gtid_binlog_pos) or zero for a slave (@@gtid_slave_pos). > + > + This uses the @@gtid_binlog_pos or @@gtid_slave_pos, so requires FLUSH TABLES > + WITH READ LOCK or similar to be consistent. > + > + Returns 0 if ok, non-zero for error. > +*/ > +static int > +get_gtid_pos(char *out_gtid_pos, int master) > +{ > + MYSQL_RES *res; > + MYSQL_ROW row; > + int found; > + > + if (mysql_query_with_error_report(mysql, &res, > + (master ? > + "SELECT @@GLOBAL.gtid_binlog_pos" : > + "SELECT @@GLOBAL.gtid_slave_pos"))) > + return 1; > + > + found= 0; > + if ((row= mysql_fetch_row(res))) > + { > + strmake(out_gtid_pos, row[0], FN_REFLEN-1); > + found++; > + } > + mysql_free_result(res); > + > + return (found != 1); > +} > + > + > static char *my_case_str(const char *str, > uint str_len, > const char *token, > @@ -4799,12 +4891,14 @@ static int dump_selected_tables(char *db > } /* dump_selected_tables */ > > > -static int do_show_master_status(MYSQL *mysql_con, int consistent_binlog_pos) > +static int do_show_master_status(MYSQL *mysql_con, int consistent_binlog_pos, > + int have_mariadb_gtid, int use_gtid) > { > MYSQL_ROW row; > MYSQL_RES *UNINIT_VAR(master); > char binlog_pos_file[FN_REFLEN]; > char binlog_pos_offset[LONGLONG_LEN+1]; > + char gtid_pos[FN_REFLEN]; > char *file, *offset; > const char *comment_prefix= > (opt_master_data == MYSQL_OPT_MASTER_DATA_COMMENTED_SQL) ? "-- " : ""; > @@ -4815,6 +4909,9 @@ static int do_show_master_status(MYSQL * > return 1; > file= binlog_pos_file; > offset= binlog_pos_offset; > + if (have_mariadb_gtid && > + get_binlog_gtid_pos(binlog_pos_file, binlog_pos_offset, gtid_pos)) > + return 1; > } > else > { > @@ -4844,6 +4941,9 @@ static int do_show_master_status(MYSQL * > return 0; > } > } > + > + if (have_mariadb_gtid && get_gtid_pos(gtid_pos, 1)) > + return 1; > } > > /* SHOW MASTER STATUS reports file and position */ > @@ -4852,7 +4952,19 @@ static int do_show_master_status(MYSQL * > "recovery from\n--\n\n"); > fprintf(md_result_file, > "%sCHANGE MASTER TO MASTER_LOG_FILE='%s', MASTER_LOG_POS=%s;\n", > - comment_prefix, file, offset); > + (use_gtid ? "-- " : comment_prefix), file, offset); > + if (have_mariadb_gtid) > + { > + print_comment(md_result_file, 0, > + "\n--\n-- GTID to start replication from\n--\n\n"); > + if (use_gtid) > + fprintf(md_result_file, > + "%sCHANGE MASTER TO MASTER_USE_GTID=slave_pos;\n", > + comment_prefix); > + fprintf(md_result_file, > + "%sSET GLOBAL gtid_slave_pos='%s';\n", > + (!use_gtid ? "-- " : comment_prefix), gtid_pos); > + } > check_io(md_result_file); > > if (!consistent_binlog_pos) > @@ -4922,12 +5034,16 @@ static int add_slave_statements(void) > return(0); > } > > -static int do_show_slave_status(MYSQL *mysql_con) > +static int do_show_slave_status(MYSQL *mysql_con, int use_gtid, > + int have_mariadb_gtid) > { > MYSQL_RES *UNINIT_VAR(slave); > MYSQL_ROW row; > const char *comment_prefix= > (opt_slave_data == MYSQL_OPT_SLAVE_DATA_COMMENTED_SQL) ? "-- " : ""; > + const char *gtid_comment_prefix= (use_gtid ? comment_prefix : "-- "); > + const char *nogtid_comment_prefix= (!use_gtid ? comment_prefix : "-- "); > + int set_gtid_done= 0; > > if (mysql_query_with_error_report(mysql_con, &slave, > multi_source ? > @@ -4945,8 +5061,30 @@ static int do_show_slave_status(MYSQL *m > > while ((row= mysql_fetch_row(slave))) > { > + if (multi_source && !set_gtid_done) > + { > + char gtid_pos[FN_REFLEN]; > + if (have_mariadb_gtid && get_gtid_pos(gtid_pos, 0)) > + return 1; > + if (opt_comments) > + fprintf(md_result_file, "\n--\n-- Gtid position to start replication " > + "from\n--\n\n"); > + fprintf(md_result_file, "%sSET GLOBAL gtid_slave_pos='%s';\n", > + gtid_comment_prefix, gtid_pos); > + set_gtid_done= 1; > + } > if (row[9 + multi_source] && row[21 + multi_source]) > { > + if (use_gtid) > + { > + if (multi_source) > + fprintf(md_result_file, "%sCHANGE MASTER '%.80s' TO " > + "MASTER_USE_GTID=slave_pos;\n", gtid_comment_prefix, row[0]); > + else > + fprintf(md_result_file, "%sCHANGE MASTER TO " > + "MASTER_USE_GTID=slave_pos;\n", gtid_comment_prefix); > + } > + > /* SHOW MASTER STATUS reports file and position */ > if (opt_comments) > fprintf(md_result_file, > @@ -4955,9 +5093,9 @@ static int do_show_slave_status(MYSQL *m > > if (multi_source) > fprintf(md_result_file, "%sCHANGE MASTER '%.80s' TO ", > - comment_prefix, row[0]); > + nogtid_comment_prefix, row[0]); > else > - fprintf(md_result_file, "%sCHANGE MASTER TO ", comment_prefix); > + fprintf(md_result_file, "%sCHANGE MASTER TO ", nogtid_comment_prefix); > > if (opt_include_master_host_port) > { > @@ -5030,12 +5168,13 @@ static int do_flush_tables_read_lock(MYS > FLUSH TABLES is to lower the probability of a stage where both mysqldump > and most client connections are stalled. Of course, if a second long > update starts between the two FLUSHes, we have that bad stall. > + > + We use the LOCAL option, as we do not want the FLUSH TABLES replicated to > + other servers. > */ > return > - ( mysql_query_with_error_report(mysql_con, 0, > - ((opt_master_data != 0) ? > - "FLUSH /*!40101 LOCAL */ TABLES" : > - "FLUSH TABLES")) || > + ( mysql_query_with_error_report(mysql_con, 0, > + "FLUSH /*!40101 LOCAL */ TABLES") || > mysql_query_with_error_report(mysql_con, 0, > "FLUSH TABLES WITH READ LOCK") ); > } > @@ -5656,6 +5795,7 @@ int main(int argc, char **argv) > char bin_log_name[FN_REFLEN]; > int exit_code; > int consistent_binlog_pos= 0; > + int have_mariadb_gtid= 0; > MY_INIT(argv[0]); > > sf_leaking_memory=1; /* don't report memory leaks on early exits */ > @@ -5696,7 +5836,10 @@ int main(int argc, char **argv) > > /* Check if the server support multi source */ > if (mysql_get_server_version(mysql) >= 100000) > + { > multi_source= 2; > + have_mariadb_gtid= 1; > + } > > if (opt_slave_data && do_stop_slave_sql(mysql)) > goto err; > @@ -5743,9 +5886,11 @@ int main(int argc, char **argv) > /* Add 'STOP SLAVE to beginning of dump */ > if (opt_slave_apply && add_stop_slave()) > goto err; > - if (opt_master_data && do_show_master_status(mysql, consistent_binlog_pos)) > + if (opt_master_data && do_show_master_status(mysql, consistent_binlog_pos, > + have_mariadb_gtid, opt_use_gtid)) > goto err; > - if (opt_slave_data && do_show_slave_status(mysql)) > + if (opt_slave_data && do_show_slave_status(mysql, opt_use_gtid, > + have_mariadb_gtid)) > goto err; > if (opt_single_transaction && do_unlock_tables(mysql)) /* unlock but no commit! */ > goto err; > > === modified file 'mysql-test/r/rpl_mysqldump_slave.result' > --- a/mysql-test/r/rpl_mysqldump_slave.result 2014-02-26 14:28:07 +0000 > +++ b/mysql-test/r/rpl_mysqldump_slave.result 2014-06-16 09:38:03 +0000 > @@ -4,18 +4,59 @@ include/master-slave.inc > # New --dump-slave, --apply-slave-statements functionality > # > use test; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > STOP ALL SLAVES; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > START ALL SLAVES; > STOP ALL SLAVES; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_HOST='127.0.0.1', MASTER_PORT=MASTER_MYPORT, MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > START ALL SLAVES; > start slave; > Warnings: > Note 1254 Slave is already running > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > start slave; > Warnings: > Note 1254 Slave is already running > +*** Test mysqldump --dump-slave GTID functionality. > +SET gtid_seq_no = 1000; > +CREATE TABLE t1 (a INT PRIMARY KEY); > +DROP TABLE t1; > +CREATE TABLE t2 (a INT PRIMARY KEY); > +DROP TABLE t2; > + > +1. --dump-slave=1 > + > +SET GLOBAL gtid_slave_pos='0-1-1001'; > +CHANGE MASTER '' TO MASTER_USE_GTID=slave_pos; > +-- CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > + > +2. --dump-slave=2 > + > +-- SET GLOBAL gtid_slave_pos='0-1-1001'; > +-- CHANGE MASTER '' TO MASTER_USE_GTID=slave_pos; > +-- CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > +*** Test mysqldump --master-data GTID functionality. > + > +1. --master-data=1 > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +SET GLOBAL gtid_slave_pos='0-2-1003'; > + > +2. --master-data=2 > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +-- CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +-- SET GLOBAL gtid_slave_pos='0-2-1003'; > + > +3. --master-data --single-transaction > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +SET GLOBAL gtid_slave_pos='0-2-1003'; > include/rpl_end.inc > > === modified file 'mysql-test/t/rpl_mysqldump_slave.test' > --- a/mysql-test/t/rpl_mysqldump_slave.test 2014-02-26 14:28:07 +0000 > +++ b/mysql-test/t/rpl_mysqldump_slave.test 2014-06-16 09:38:03 +0000 > @@ -36,4 +36,53 @@ start slave; > --exec $MYSQL_DUMP_SLAVE --compact --dump-slave no_such_db > start slave; > > + > +--echo *** Test mysqldump --dump-slave GTID functionality. > + > +--connection master > +SET gtid_seq_no = 1000; > +CREATE TABLE t1 (a INT PRIMARY KEY); > +DROP TABLE t1; > +--sync_slave_with_master > + > +--connection slave > +# Inject a local transaction on the slave to check that this is not considered > +# for --dump-slave. > +CREATE TABLE t2 (a INT PRIMARY KEY); > +DROP TABLE t2; > + > +--echo > +--echo 1. --dump-slave=1 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --dump-slave=1 --gtid test > + > +--echo > +--echo 2. --dump-slave=2 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --dump-slave=2 --gtid test > + > + > +--echo *** Test mysqldump --master-data GTID functionality. > +--echo > +--echo 1. --master-data=1 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data=1 --gtid test > + > +--echo > +--echo 2. --master-data=2 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data=2 --gtid test > + > +--echo > +--echo 3. --master-data --single-transaction > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data --single-transaction --gtid test > + > + > + > --source include/rpl_end.inc > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

Re: [Maria-developers] [Commits] Rev 4259: MDEV-6120: When slave stops with error, error message should indicate the failing GTID in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Sergey Vojtovich 25 Jun '14

25 Jun '14

Hi Kristian, looks good. One minor question below. On Thu, Jun 19, 2014 at 02:50:44PM +0200, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4259 > revision-id: knielsen(a)knielsen-hq.org-20140619125043-acw2gx7rw90y3vqp > parent: svoj(a)mariadb.org-20140618141504-kqq6ws9gm93r2b2d > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: work-10.0 > timestamp: Thu 2014-06-19 14:50:43 +0200 > message: > MDEV-6120: When slave stops with error, error message should indicate the failing GTID > > If replication breaks in GTID mode, it is not trivial to determine the GTID of > the failing event group. This is a problem, as such GTID is needed eg. to > explicitly set @@gtid_slave_pos to skip to after that event group, or to > compare errors on different servers, etc. > > Fix by ensuring that relevant slave errors logged to the error log include the > GTID of the event group containing the problem event. ...skip... > === added file 'mysql-test/suite/rpl/t/rpl_gtid_errorlog.test' > --- a/mysql-test/suite/rpl/t/rpl_gtid_errorlog.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_gtid_errorlog.test 2014-06-19 12:50:43 +0000 > @@ -0,0 +1,94 @@ > +--source include/have_debug.inc > +--source include/master-slave.inc > + > +--echo *** Test MDEV-6120, output of current GTID when a replication error is logged to the errorlog *** > +--connection master > +CREATE TABLE t1(a INT PRIMARY KEY); > +--sync_slave_with_master > + > +--connection slave > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=slave_pos; > + > +--connection master > +INSERT INTO t1 VALUES (1); > +SET gtid_seq_no=100; > +INSERT INTO t1 VALUES (2); > +INSERT INTO t1 VALUES (3); > +INSERT INTO t1 VALUES (4); > +--save_master_pos > + > +--connection slave > +SET sql_log_bin=0; > +INSERT INTO t1 VALUES (2); > +SET sql_log_bin=1; > + > +START SLAVE; > +--let $slave_sql_errno=1062 > +--source include/wait_for_slave_sql_error.inc > + > +--source include/stop_slave.inc > +# Skip the problem event from the master. > +SET GLOBAL gtid_slave_pos= "0-1-100"; > +--source include/start_slave.inc > +--sync_with_master > + > +SELECT * FROM t1 ORDER BY a; > + > +--connection master > + > +SET @dbug_save= @@debug_dbug; > +SET debug_dbug= '+d,incident_database_resync_on_replace'; > +REPLACE INTO t1 VALUES (5); > +SET debug_dbug= @dbug_save; > +--save_master_pos > + > +--connection slave > +--let $slave_sql_errno=1590 > +--source include/wait_for_slave_sql_error.inc > +# ToDo no need to switch off GTID once MDEV-4937 is fixed > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=no; > +SET sql_slave_skip_counter=1; > +--source include/start_slave.inc > +--sync_with_master > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=slave_pos; > +--source include/start_slave.inc > + > +SELECT * FROM t1 ORDER BY a; > + > + > +# Check error log for correct messages. > +let $log_error_= `SELECT @@GLOBAL.log_error`; > +if(!$log_error_) > +{ > + # MySQL Server on windows is started with --console and thus > + # does not know the location of its .err log, use default location > + let $log_error_ = $MYSQLTEST_VARDIR/log/mysqld.2.err; > +} > +let LOG_ERROR=$log_error_; > + > +perl; > + use strict; > + use warnings; > + my $log_error= $ENV{'LOG_ERROR'} or die "LOG_ERROR not set"; > + open(FILE, "<", "$log_error") or die("Unable to open $log_error: $!\n"); > + my $found_dup= 0; > + my $found_incident= 0; > + while (<FILE>) { > + $found_dup= 1 > + if /Slave SQL: Error 'Duplicate entry .* on query\. .*Query: '.*', Gtid 0-1-100, Internal MariaDB error code:/ > + or /Slave SQL: Could not execute Write_rows.*table test.t1; Duplicate entry.*, Gtid 0-1-100, Internal MariaDB error/ > + ; > + $found_incident= 1 > + if /Slave SQL: The incident LOST_EVENTS occured on the master\. Message: <none>, Internal MariaDB error code: 1590/; > + } > + print "Duplicate key found: $found_dup; incident found: $found_incident\n"; > + close(FILE); > +EOF > + > +--connection master > +DROP TABLE t1; > + > +--source include/rpl_end.inc Why didn't you like include/search_pattern_in_file.inc? It should perfectly able to match your patterns. ...skip... Regards, Sergey

2 1

[Maria-developers] Review of MDEV-5730: enhance security using special compilation options
by Kristian Nielsen 24 Jun '14

24 Jun '14

> commit feab48657528b9bb40fb035d51bee28d93710c1e > Author: Sergei Golubchik <serg(a)mariadb.org> > Date: Mon Jun 16 21:39:09 2014 +0200 > > MDEV-5730 enhance security using special compilation options > > -Wl,-z,relro,-z,now > -pie > -fstack-protector --param=ssp-buffer-size=4 > -D_FORTIFY_SOURCE=2 The patch is ok to push with a clarified comment as requested below. I have checked the compiler/linker options added and commented some on them below. > diff --git a/CMakeLists.txt b/CMakeLists.txt > index a24f000..5b0d348 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > @@ -201,6 +201,20 @@ IF (WITH_ASAN) > ENDIF() > ENDIF() > > +OPTION(SECURITY_HARDENED "Use security-enhancing compiler features (stack protector, relro, etc)" ON) So these are on by default. This means that this patch will cause some performance impact. So having them on by default is a compromise between performance and security. You should probably include in the commit comment a discussion of why the security viewpoint was prefered over the performance viewpoint in this case (or a comment in the cmake file if you prefer). The proper way to handle a patch like this would be to measure the performance impact, eg. for single-threaded performance with some various simple sysbench loads. However, it is quite hard to do such measurements, as you need a setup that has low enough noise in results to detect sub-percentage differences of performance. I have had some success by locking processes to a single core and using `perf` measurements of cycles spent (rather than wall-clock time), but even so the results can be tricky to interpret. So it may be too much work for a small patch like this. (And even if the performance improvement can be measured, how can we measure the security impact to weight them against one another?) My personal opinion is that I do not buy into this "security" mindset much. I do not think these options improve security significantly in practise for most users. So while the performance impact is probably also not very significant, why accept it at all (by default)? But the opposite viewpoint exists and is valid as well. So I think this patch is ok if you add a discussion of why it was chosen to do it this way. (Even if the reason is "Most distros do it" - at least then it is documented what the reason is). Also, the option should mention the possible performance impact. Suggestion: "Use compiler options (stack protector, relro, etc) that may slightly enhance security of the resulting binary, at the possible cost of some small performance impact." > +IF(SECURITY_HARDENED) > + # security-enhancing flags > + MY_CHECK_AND_SET_COMPILER_FLAG("-pie -fPIC") -fPIC can have a significant impact on performance on some platforms. For example on 32-bit x86, it requires extra instructions at the head of every function, and it makes one less register available for code generation on a CPU already starved for registers. On 64-bit x86, the impact is much smaller, as the architecture was enhanced to better support -fPIC. However, I seem to recall that the server is anyway built with -fPIC, for libmysqld or something like that? In which case this has no extra impact. > + MY_CHECK_AND_SET_COMPILER_FLAG("-Wl,-z,relro,-z,now") This option seems to cause all relocations to happen at load time, rather than lazily on-demand. The impact should be higher memory usage and some extra CPU time for these relocations, many of which may never be needed. This option will have the biggest effect on the client programs probably, as server startup is already fairly heavy. Probably even for client programs the added cost will be not very significant, as even the client programs are usually called to do significant processing (eg. connect to a database and so on). I guess these options would mostly hurt small utility programs for shell scripts, like `mv` or so. > + MY_CHECK_AND_SET_COMPILER_FLAG("-fstack-protector --param=ssp-buffer-size=4") This also adds extra code and memory loads to each function, so it will have some impact to performance. It is probably the most significant impact of all the added options, though without detailed analysis this is only guesswork, of course. > + # sometimes _FORTIFY_SOURCE is predefined > + INCLUDE(CheckSymbolExists) > + CHECK_SYMBOL_EXISTS(_FORTIFY_SOURCE "" HAVE_FORTIFY_SOURCE) > + IF(NOT HAVE_FORTIFY_SOURCE) > + ADD_DEFINITIONS(-D_FORTIFY_SOURCE=2) > + ENDIF() > +ENDIF() FORTIFY_SOURCE also adds some runtime overhead, but it is not very much, it is an extra argument to memcpy() and friends about destination buffer size. Probably the way or code is written, there will be few places where this option makes any difference, so the performance impact does not seem likely to be very significant. Hope this helps, - Kristian.

3 2

[Maria-developers] [GSoC] Optimize mysql-test-runs - Setback
by Pablo Estrada 24 Jun '14

24 Jun '14

Hello Sergei, Elena and all, Today while working on the script, I found and fixed an issue: There is some faulty code code in my script that is in charge of collecting the statistics about whether a test failure was caught or not (here <https://github.com/pabloem/Kokiri/blob/master/basic_simulator.py#L393>). I looked into fixing it, and then I could see another *problem*: The *recall numbers* that I had collected previously were *too high*. The actual recall numbers, once we consider the test failures that are *not caught*, are disappointingly lower. I won't show you results yet, since I want to make sure that the code has been fixed, and I have accurate tests first. This is all for now. The strategy that I was using is a lot less effective than it seemed initially. I will send out a more detailed report with results, my opinion on the weak points of the strategy, and ideas, including a roadmap to try to improve results. Regards. All feedback is welcome. Pablo

3 18

[Maria-developers] Doubt in bin logging for CREATE VIEW
by sriram patil 24 Jun '14

24 Jun '14

Hi All, I was checking the behavior of bin logging in various methods. I found out CREATE VIEW differs a little bit than others in bin logging. The query is written into bin log if query is executed successfully. If there is any error the query is not logged. This behavior is observed across all the commands. In case of CREATE VIEW (method: mysql_create_view file: sql/sql_view.cc), while the bin logging is skipped for most of the errors by jumping to "err" label, it is not skipped in following code snippet (line number: 616) res= mysql_register_view(thd, view, mode); /* View TABLE_SHARE must be removed from the table definition cache in order to make ALTER VIEW work properly. Otherwise, we would not be able to detect meta-data changes after ALTER VIEW. */ if (!res) tdc_remove_table(thd, TDC_RT_REMOVE_ALL, view->db, view->table_name, false); if (mysql_bin_log.is_open()) { // this is where the log is written } If an error occurs in mysql_register_view, e.g. ER_OUT_OF_RESOURCES, ER_TABLE_EXISTS_ERROR, etc. the query is still logged. Is there a specific reason why it is logged even though an error occurs? Or is it a bug? Thanks & Regards, Sriram

4 9

Re: [Maria-developers] [Commits] 7d386bb: MDEV-4549 [PATCH] Clean up code working with ACL tables
by Sergey Vojtovich 23 Jun '14

23 Jun '14

Hi Sergei, ok to push. A few minor questions inline. On Mon, Jun 16, 2014 at 09:34:21AM +0200, serg(a)mariadb.org wrote: > revision-id: 7d386bbce216745ba28df728f45a0b796fcced98 > parent(s): f61f36b386e8d0c6448297bb84c92e8d9b82be6a > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-06-15 17:12:57 +0200 > message: > > MDEV-4549 [PATCH] Clean up code working with ACL tables > > enum values to index different ACL tables, instead of hard-coded numbers > (even different in diffent functions). Also factor out TABLE_LIST > initialization into a separate function. > > --- > mysql-test/r/not_embedded_server.result | 4 +- > mysql-test/r/sp-destruct.result | 3 - > .../suite/rpl/r/rpl_tmp_table_and_DDL.result | 22 +- > mysql-test/t/not_embedded_server.test | 2 +- > mysql-test/t/sp-destruct.test | 1 - > sql/sql_acl.cc | 533 +++++++++------------ > 6 files changed, 235 insertions(+), 330 deletions(-) > > diff --git a/mysql-test/r/not_embedded_server.result b/mysql-test/r/not_embedded_server.result > index 2295276..6dda855 100644 > --- a/mysql-test/r/not_embedded_server.result > +++ b/mysql-test/r/not_embedded_server.result > @@ -1,4 +1,4 @@ > -call mtr.add_suppression("Can't open and lock privilege tables: Table 'host' was not locked with LOCK TABLES"); > +call mtr.add_suppression("Can't open and lock privilege tables: Table 'roles_mapping' was not locked with LOCK TABLES"); > SHOW VARIABLES like 'slave_skip_errors'; > Variable_name Value > slave_skip_errors OFF Why did it change (here and in other tests)? I didn't notice any table order change in the code. ...skip... > diff --git a/mysql-test/r/sp-destruct.result b/mysql-test/r/sp-destruct.result > index 172e40c..fe68229 100644 > --- a/mysql-test/r/sp-destruct.result > +++ b/mysql-test/r/sp-destruct.result > @@ -128,11 +128,8 @@ CREATE FUNCTION f1() RETURNS INT RETURN 1; > RENAME TABLE mysql.procs_priv TO procs_priv_backup; > FLUSH TABLE mysql.procs_priv; > DROP FUNCTION f1; > -ERROR 42S02: Table 'mysql.procs_priv' doesn't exist > SHOW WARNINGS; > Level Code Message > -Error 1146 Table 'mysql.procs_priv' doesn't exist > -Warning 1405 Failed to revoke all privileges to dropped routine > # Restore the procs_priv table > RENAME TABLE procs_priv_backup TO mysql.procs_priv; > FLUSH TABLE mysql.procs_priv; Hmm... why? ...skip... > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 5e8e0e7..442d512 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -380,7 +380,7 @@ class ACL_PROXY_USER :public ACL_ACCESS > MYSQL_PROXIES_PRIV_PROXIED_USER, > MYSQL_PROXIES_PRIV_WITH_GRANT, > MYSQL_PROXIES_PRIV_GRANTOR, > - MYSQL_PROXIES_PRIV_TIMESTAMP } old_acl_proxy_users; > + MYSQL_PROXIES_PRIV_TIMESTAMP } proxy_table_fields; > public: > ACL_PROXY_USER () {}; > > @@ -760,6 +760,99 @@ static int traverse_role_graph_down(ACL_USER_BASE *, void *, > int (*) (ACL_USER_BASE *, ACL_ROLE *, void *)); > > /* > + Enumeration of ACL/GRANT tables in the mysql database > +*/ > +enum enum_acl_tables > +{ > + USER_TABLE, > + DB_TABLE, > + TABLES_PRIV_TABLE, > + COLUMNS_PRIV_TABLE, > +#define FIRST_OPTIONAL_TABLE HOST_TABLE > + HOST_TABLE, > + PROCS_PRIV_TABLE, > + PROXIES_PRIV_TABLE, > + ROLES_MAPPING_TABLE, > + TABLES_MAX // <== always the last > +}; > +// bits for init_priv_tables > +static const int Table_user= 1 << USER_TABLE; > +static const int Table_db= 1 << DB_TABLE; > +static const int Table_tables_priv= 1 << TABLES_PRIV_TABLE; > +static const int Table_columns_priv= 1 << COLUMNS_PRIV_TABLE; > +static const int Table_host= 1 << HOST_TABLE; > +static const int Table_procs_priv= 1 << PROCS_PRIV_TABLE; > +static const int Table_proxies_priv= 1 << PROXIES_PRIV_TABLE; > +static const int Table_roles_mapping= 1 << ROLES_MAPPING_TABLE; > + > +const LEX_STRING acl_table_names[]= // matches enum_acl_tables > +{ > + { C_STRING_WITH_LEN("user") }, > + { C_STRING_WITH_LEN("db") }, > + { C_STRING_WITH_LEN("tables_priv") }, > + { C_STRING_WITH_LEN("columns_priv") }, > + { C_STRING_WITH_LEN("host") }, > + { C_STRING_WITH_LEN("procs_priv") }, > + { C_STRING_WITH_LEN("proxies_priv") }, > + { C_STRING_WITH_LEN("roles_mapping") } > +}; > + > +/** > + Initialize a TABLE_LIST array to a set of acl/grant tables. > + All tables will be opened with the same lock type, either read or write. > + > + @retval !0 a pointer to the first initialized element in the TABLE_LIST > + @retval 0 replication filters matched. Abort operation, but return OK (!) > +*/ > +TABLE_LIST *init_priv_tables(THD *thd, TABLE_LIST *tables, > + enum thr_lock_type lock_type, int tables_to_open) Nice clean-up, I assume there was a reason not rewrite open_grant_tables() instead. > +{ > + int prev= -1; > + bzero(tables, sizeof(TABLE_LIST) * TABLES_MAX); > + for (int cur=0, mask=1; mask <= tables_to_open; cur++, mask <<= 1) > + { > + if ((tables_to_open & mask) == 0) > + continue; > + tables[cur].init_one_table(C_STRING_WITH_LEN("mysql"), > + acl_table_names[cur].str, > + acl_table_names[cur].length, > + acl_table_names[cur].str, lock_type); > + tables[cur].open_type= OT_BASE_ONLY; > + if (lock_type >= TL_WRITE_ALLOW_WRITE) > + tables[cur].updating= 1; > + if (cur >= FIRST_OPTIONAL_TABLE) > + tables[cur].open_strategy= TABLE_LIST::OPEN_IF_EXISTS; > + if (prev != -1) > + tables[cur].next_local= tables[cur].next_global= & tables[prev]; > + prev= cur; > + } > + > +#ifdef HAVE_REPLICATION > + if (lock_type >= TL_WRITE_ALLOW_WRITE && thd->slave_thread && !thd->spcont) > + { > + /* > + GRANT and REVOKE are applied the slave in/exclusion rules as they are > + some kind of updates to the mysql.% tables. > + */ > + Rpl_filter *rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > + if (rpl_filter->is_on() && !rpl_filter->tables_ok(0, tables)) > + return NULL; > + } > +#endif Was it a bug that mysql_grant_role() didn't have this code? open_grant_tables() does updating= 0 after tables_ok(). Is that needed? ...skip... > @@ -5783,8 +5796,9 @@ bool mysql_routine_grant(THD *thd, TABLE_LIST *table_list, bool is_proc, > } > } > > - if (replace_routine_table(thd, grant_name, tables[1].table, *Str, > - db_name, table_name, is_proc, rights, > + if (no_such_table(tables + PROCS_PRIV_TABLE) || > + replace_routine_table(thd, grant_name, tables[PROCS_PRIV_TABLE].table, > + *Str, db_name, table_name, is_proc, rights, > revoke_grant) != 0) > { > result= TRUE; Why no_such_table() here? ...skip... > @@ -6250,7 +6232,7 @@ bool mysql_grant(THD *thd, const char *db, List <LEX_USER> &list, > ulong db_rights= rights & DB_ACLS; > if (db_rights == rights) > { > - if (replace_db_table(tables[1].table, db, *Str, db_rights, > + if (replace_db_table(tables[DB_TABLE].table, db, *Str, db_rights, > revoke_grant)) > result= -1; > } Tabs here (and in a few hunks below). > @@ -6262,9 +6244,11 @@ bool mysql_grant(THD *thd, const char *db, List <LEX_USER> &list, > } > else if (is_proxy) > { > - if (replace_proxies_priv_table (thd, tables[1].table, Str, proxied_user, > - rights & GRANT_ACL ? TRUE : FALSE, > - revoke_grant)) > + if (no_such_table(tables + PROXIES_PRIV_TABLE) || > + replace_proxies_priv_table (thd, tables[PROXIES_PRIV_TABLE].table, > + Str, proxied_user, > + rights & GRANT_ACL ? TRUE : FALSE, > + revoke_grant)) > result= -1; > } > if (Str->is_role()) Why no_such_table() here? ...skip... Regards, Sergey

2 2

[Maria-developers] [GSoC] Week Five!
by sriram patil 23 Jun '14

23 Jun '14

Hi All, Following is the link to my blog post of fifth week into GSoC. http://goo.gl/x7dtcn Suggestions and/or review are welcome! Thanks, Sriram

1 0

[Maria-developers] Java Connector Coding till 21st June
by Puneet Dewan 21 Jun '14

21 Jun '14

Hi Everyone. Test case class(ServersidePreparedStatementTest.java) was written for checking the value of statement id. This week code review was done by Massimo and Georg. There pointers were really valuable as data was not being send in a proper format to the server for execute(com_stmt_execute) method : 1.NullBitMap was set incorrectly. 2.Creation of a new class extending PacketOuptutStream e.g integer of 4 bytes can be sent.Currently if an integer is sent it is sent as a byte. 3.Also I was expecting when we do a *prepare* for a statement like insert into XYZ(id) values(?) where id is of type Integer. Then server shall be returning integer as a type. This was a false assumption.It does not return the type. So now on the basis of value set for *id as example*,decide on the type to be sent from execute. So this week will fix the above things. Looking forward for suggestions. Regards Puneet.

1 0

Re: [Maria-developers] [Commits] 22f532c: MDEV-4260 Don't create frm files for temporary tables
by Sergey Vojtovich 20 Jun '14

20 Jun '14

Hi Sergei, ok to push. One note inline. On Fri, Jun 20, 2014 at 11:29:36AM +0200, serg(a)mariadb.org wrote: > revision-id: 22f532cfb2f40132e2f7b079a3aafb109af92c26 > parent(s): fb8818c1aff9ce67b74f965c019cad93addd811e > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-06-20 11:29:12 +0200 > message: > > MDEV-4260 Don't create frm files for temporary tables > > * Don't write frm for tmp tables > * pass frm image down to open_table_uncached, when possible > * don't use truncate-by-recreate for temp tables - cannot recreate > without frm, and delete_all_rows is faster anyway > * remove HTON_FLUSH_AFTER_RENAME (BDB-ism, unused for years) > > --- > mysql-test/r/temp_table_frm.result | 20 +++++++++++ > mysql-test/t/temp_table_frm.test | 13 ++++++++ > sql/handler.h | 2 +- > sql/sql_base.cc | 14 +++++++- > sql/sql_base.h | 3 +- > sql/sql_table.cc | 27 +++------------ > sql/sql_truncate.cc | 68 +------------------------------------- > sql/unireg.cc | 3 +- > 8 files changed, 56 insertions(+), 94 deletions(-) > ...skip... > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 0558066..7d69ccb 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -5561,7 +5563,17 @@ TABLE *open_table_uncached(THD *thd, handlerton *hton, > strend(saved_cache_key)+1, tmp_path); > share->db_plugin= ha_lock_engine(thd, hton); > > - if (open_table_def(thd, share, GTS_TABLE | GTS_USE_DISCOVERY)) > + /* > + Use the frm image, if possible, open the file otherwise. > + > + The image might be unavailable in ALTER TABLE, when the discovering > + engine took over the ownership (see TABLE::read_frm_image). > + */ > + int res= frm->str > + ? res= share->init_from_binary_frm_image(thd, false, frm->str, frm->length) Excessive "res="? Regards, Sergey

2 1

[Maria-developers] [GSoC] Week Four!
by sriram patil 19 Jun '14

19 Jun '14

Hi All, Sorry for a little delay in posting the blog link this week. Following is the blog link about my fourth week into GSoC. http://goo.gl/W54cvE Suggestions and/or reviews are welcome! Thanks, Sriram

4 4

Re: [Maria-developers] [Commits] Rev 4258: MDEV-6039 - WebScaleSQL patches in lp:maria/10.0
by Sergei Golubchik 18 Jun '14

18 Jun '14

Hi, Sergey! On Jun 18, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 4258 > revision-id: svoj(a)mariadb.org-20140618141504-kqq6ws9gm93r2b2d > parent: svoj(a)mariadb.org-20140618081243-jxs6yo9tx9cahdv6 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Wed 2014-06-18 18:15:04 +0400 > message: > MDEV-6039 - WebScaleSQL patches > > Stop spawning dummy threads on client library initialization > > Let's revert the fix for Bug#24507. To quote Monty from 2006: > > "After 1/2 a year, when all glibc versions are updated, we can delete > this code." > > Note: The upstream glibc bug was fixed in 2006. Ok to push Regards, Sergei

1 0

Re: [Maria-developers] [GSoC] self-tuning optimizer
by Sergei Golubchik 18 Jun '14

18 Jun '14

Hi, Anshu! Please, don't forget to cc: maria-developers@ when discussing development topics. On Jun 18, Anshu Avinash wrote: > > What I meant to say was we need N "linearly independent" equations to > solve for N variables and that might not be the case. Some of the > queries might not use any factor at all or use one factor more often. Yes, you're right, of course. > > Later we'll see what to do - perhaps we'll need to decrease the limit, > > or set it exactly to N (which means, the solution won't be "best > > fitting" anymore). But this will be not in this GSoC, not your project. > > I'm sure that in this project we won't have enough factors for memory > > usage to become an issue :) > > Yes, I'll start with a basic least squares fitting model without caring > much about the memory usage for now. Sounds good. Regards, Sergei

1 0

Re: [Maria-developers] [GSoC] self-tuning optimizer
by Sergei Golubchik 18 Jun '14

18 Jun '14

Hi, Anshu! On Jun 18, Anshu Avinash wrote: > Hi serg, > > Sorry that I had been out of touch for 1-2 days. I got caught up in some > other work. > > I have read through the comments you made on MDEV and I have just started > the implementation for the first approach, the approximate method. I have > pushed a prototype at: > https://github.com/igniting/server/commit/3bd4f5b6fd20d11575207ac36e8a3cc2c… > > Here is what I'm thinking, we add to the array of measurement data, after > each query and try to solve the system of equations because we don't know > when the system of equations would become solvable. Eh. To solve a system of linear equations with N unknowns, you need N equations. So, you wait until you get N equations and solve it. Few thoughts: 1. it might be that out of all factors that you have in your data structures (currently it's 2 + 2 * MAX_HA), only few will be actually used in queries (for example, only TIME_FOR_COMPARE and read_time for InnoDB). Then you only need two queries to solve that, not 2+2*MAX_HA. But it's an optimization, try to do the basic implementation without it first. 2. You normally store these data (for N queries) in the THD, right? It might be that the thread will disconnect before it'll do the required number of queries. To avoid throwing the useful data away, you can store one extra set of data globally and when some THD disconnects and it doesn't have enough data to solve the equation, you append this data to the global set and solve that as a whole. Again, this is an extension, only do it when the basic implementation works. > However this might be time consuming, trying to solve equations after > each query. Can we store the entire data (from all the queries) and > apply a best fitting model, instead of solving them exactly? This we > will do at disconnect. I don't have a good answer for that. It's a trade-off. Either you collect exactly N equations and solve them. Then you collect the next N equations (or you get another one and delete the first one) - and solve that. And so on. Or you collect many equations, more than N. And find a best fitting solution. This way you do less solving, but you'll need more memory. I think, for now you can do that - use a best fitting solution. We only have very few factors at the moment. Still, even now you'll need some limit to how many equations you'll collect. Later we'll see what to do - perhaps we'll need to decrease the limit, or set it exactly to N (which means, the solution won't be "best fitting" anymore). But this will be not in this GSoC, not your project. I'm sure that in this project we won't have enough factors for memory usage to become an issue :) > Should it be in ~THD() or somewhere else? Yes, ~THD is fine, just make sure you only do it for real connection THD's, where SELECT queries were issues and optimizer was run. Not for various internal THD's, like the one you create in cost_factors::init. For example, as you've noticed, I've mentioned in the last comment about using read_time() result and Nrows_estimated (the same applies to scan_time(), of course). So, when ha_read_time() is called, you need to remember (in your data set) the return value of read_time() and Nrows_estimated (that's the last argument of read_time()). This way, you only solve your equations if you have this read_time() and Nrows_estimated (or scan_time()) stored. If not - the optimizer was never run and you don't do anything. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] Rev 4257: Empty commit to check email setting in commit mails in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Roberto Spadim 18 Jun '14

18 Jun '14

working :) 2014-06-18 7:53 GMT-03:00 <knielsen(a)knielsen-hq.org>: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4257 > revision-id: knielsen(a)knielsen-hq.org-20140618105333-44ez01iye29sljur > parent: knielsen(a)knielsen-hq.org-20140618090308-frd0tuo8rycgyi32 > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: work-10.0 > timestamp: Wed 2014-06-18 12:53:33 +0200 > message: > Empty commit to check email setting in commit mails > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] [MDEV-4903] Regarding range support for multi-column IN predicates
by Sergey Petrunia 18 Jun '14

18 Jun '14

Hi, So, MySQL 5.7 has got support for range optimization for multi-column IN predicates: http://optimize-this.blogspot.ru/2013/12/range-access-now-in-in-predicate-n… We've also got a user issue about it: https://mariadb.atlassian.net/browse/MDEV-4903. I'm looking at their IN handling. I don't see anything apparently wrong with it. They don't process multi-column NOT IN, but one can say that they just have a limited scope. Relevant code in opt_range.cc for single-column IN: case Item_func::IN_FUNC: { Item_func_in *func=(Item_func_in*) cond_func; if (inv) { // complicated code to handle NOT IN } else { tree= get_mm_parts(param, cond_func, field, Item_func::EQ_FUNC, func->arguments()[1], cmp_type); if (tree) { Item **arg, **end; for (arg= func->arguments()+2, end= arg+func->argument_count()-2; arg < end ; arg++) { tree= tree_or(param, tree, get_mm_parts(param, cond_func, field, Item_func::EQ_FUNC, *arg, cmp_type)); } } } mysql-5.7 extends this approach to handle multi-column case (by invoking get_mm_parts() for each compared value, then doing tree_and(), and then tree_or()). Should we just go ahead and backport it? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] problem with update and myisam
by Roberto Spadim 18 Jun '14

18 Jun '14

hi guys, could anyone help me with https://mariadb.atlassian.net/browse/MDEV-6355 thanks -- Roberto Spadim

2 5

Re: [Maria-developers] [Commits] 0eb94ca: MDEV-6137 better help for SET/ENUM sysvars
by Sergey Vojtovich 17 Jun '14

17 Jun '14

Hi Sergei, ok to push. A few minor questions inline. On Mon, Jun 16, 2014 at 06:51:53PM +0200, serg(a)mariadb.org wrote: > revision-id: 0eb94ca810500977eb0336ee30b19e9c0b769bca > parent(s): f61f36b386e8d0c6448297bb84c92e8d9b82be6a > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-06-16 18:43:26 +0200 > message: > > MDEV-6137 better help for SET/ENUM sysvars > > Auto-generate the allowed list of values for enum/set/flagset options > in --help output. But don't do that when the help text already has them. > > Also, remove lists of values from help strings of various options, where > they were simply listed without any additional information. > > --- > mysys/my_getopt.c | 81 +++++++++++++++++++++++++++------ > plugin/server_audit/server_audit.c | 2 +- > sql/log.cc | 3 +- > sql/mysqld.cc | 8 ++-- > sql/sql_plugin.cc | 2 +- > sql/sys_vars.cc | 90 ++++++++----------------------------- > storage/maria/ha_maria.cc | 14 ++---- > storage/myisam/ha_myisam.cc | 7 ++- > storage/xtradb/handler/ha_innodb.cc | 35 +++++++-------- > 9 files changed, 116 insertions(+), 126 deletions(-) Hmmm... no test case affected? > > diff --git a/mysys/my_getopt.c b/mysys/my_getopt.c > index 2e97417..3662757 100644 > --- a/mysys/my_getopt.c > +++ b/mysys/my_getopt.c > @@ -1300,6 +1300,48 @@ static uint print_name(const struct my_option *optp) > return s - optp->name; > } > > +/** prints option comment with indentation and wrapping. > + > + The comment column starts at startpos, and has width of width > + Current cursor position is curpos, returns new cursor position > + > +Note: can print one character beyond width! The above line, is it properly indented? > +*/ > +static uint print_comment(const char *comment, > + int curpos, int startpos, int width) Probably make width local to this function, it doesn't seem to be used by the caller anyway? > +{ > + const char *end= strend(comment); > + int endpos= startpos + width; > + > + for (; curpos < startpos; curpos++) > + putchar(' '); > + > + if (*comment == '.' || *comment == ',') > + { > + putchar(*comment); > + comment++; > + curpos++; > + } > + > + while (end - comment > endpos - curpos) > + { > + const char *line_end; > + for (line_end= comment + endpos - curpos; > + line_end > comment && *line_end != ' '; > + line_end--); > + for (; comment < line_end; comment++) > + putchar(*comment); I would probably replace it with fwrite(), but fine with this form too. > + while (*comment == ' ') > + comment++; /* skip the space, as a newline will take it's place now */ > + putchar('\n'); > + for (curpos= 0; curpos < startpos; curpos++) > + putchar(' '); > + } > + printf("%s", comment); > + return curpos + (end - comment); > +} > + > + > /* > function: my_print_options > > @@ -1309,12 +1351,12 @@ static uint print_name(const struct my_option *optp) > void my_print_help(const struct my_option *options) > { > uint col, name_space= 22, comment_space= 57; > - const char *line_end; > const struct my_option *optp; > DBUG_ENTER("my_print_help"); > > for (optp= options; optp->name; optp++) > { > + const char *typelib_help= 0; > if (!optp->comment) > continue; > if (optp->id && optp->id < 256) > @@ -1359,25 +1401,36 @@ void my_print_help(const struct my_option *options) > col= 0; > } > } > - for (; col < name_space; col++) > - putchar(' '); > if (optp->comment && *optp->comment) Cleanup suggestion: optp->comment can't be NULL here. > { > - const char *comment= optp->comment, *end= strend(comment); > + col= print_comment(optp->comment, col, name_space, comment_space); > > - while ((uint) (end - comment) > comment_space) > + switch (optp->var_type & GET_TYPE_MASK) { > + case GET_ENUM: > + typelib_help= ". One of: "; > + break; > + case GET_SET: > + typelib_help= ". Any combination of: "; > + break; > + case GET_FLAGSET: > + typelib_help= ". Takes a comma-separated list of option=value pairs, " > + "where value is on or off, and options are: "; > + break; > + } > + if (typelib_help && > + strstr(optp->comment, optp->typelib->type_names[0]) == NULL) Should we handle case when optp->typelib == NULL || optp->typelib->type_names[0] == NULL? Probably not. I believe this condition is fragile: comment string may mention type_names[0], but not list possible values. OTOH since this condition is for very limited data set it should cause problems only in rare cases. > { > - for (line_end= comment + comment_space; *line_end != ' '; line_end--); > - for (; comment != line_end; comment++) > - putchar(*comment); > - comment++; /* skip the space, as a newline will take it's place now */ > - putchar('\n'); > - for (col= 0; col < name_space; col++) > - putchar(' '); > + int i; > + col= print_comment(typelib_help, col, name_space, comment_space); > + col= print_comment(optp->typelib->type_names[0], col, name_space, comment_space); > + for (i= 1; i < optp->typelib->count; i++) > + { > + col= print_comment(", ", col, name_space, comment_space); > + col= print_comment(optp->typelib->type_names[i], col, name_space, comment_space); > + } > } > - printf("%s", comment); > + putchar('\n'); > } > - putchar('\n'); Does this mean if comment is empty there won't be a new line? > if ((optp->var_type & GET_TYPE_MASK) == GET_BOOL) > { > if (optp->def_value != 0) ...skip... > @@ -2785,7 +2745,7 @@ static Sys_var_mybool Sys_master_verify_checksum( > > /* These names must match RPL_SKIP_XXX #defines in slave.h. */ > static const char *replicate_events_marked_for_skip_names[]= { > - "replicate", "filter_on_slave", "filter_on_master", 0 > + "REPLICATE", "FILTER_ON_SLAVE", "FILTER_ON_MASTER", 0 > }; Does this mean there is a convention that values must be in upper case? ...skip... Regards, Sergey

2 1

[Maria-developers] WebScaleSQL: Actually fix all the array bounds warnings
by Sergey Vojtovich 16 Jun '14

16 Jun '14

Hi Sergei and Alexey, during our recent discussion re WebScaleSQL patches we agreed to review $subj: https://github.com/webscalesql/webscalesql-5.6/commit/094187c4a20e8d0c02f17… All these warnings come from strings/decimal.c. I was able to reproduce them using gcc-4.8.1, cmake -DBUILD_CONFIG=mysql_release (basically needs -Wall -O3). I simplified this function to the following: gcc -Wall -O3 -c dec.c -o dec.o static const int powers10[2 + 1]= { 1, 10, 100 }; int remove_leading_zeroes(int decimals, int var) { decimals%= 2; while (var < powers10[decimals--]) /* no-op */; return decimals; } The only allowed value for decimals is 1 and var must be > 0. Similar assertion is enforced by original function. So far I see nothing wrong with this code. I believe we should skip this patch, but still we should probably do something about these warning. What's your take? Regards, Sergey

2 2

[Maria-developers] MariaDB 10.0.12 release prep
by Daniel Bartholomew 14 Jun '14

14 Jun '14

I've begun prep work for the MariaDB 10.0.12 release. Draft Release Notes and Changelog are here: - https://mariadb.com/kb/en/mariadb-10012-release-notes/ - https://mariadb.com/kb/en/mariadb-10012-changelog/ Please add/modify/etc... as needed. Current plan is to announce the release on Mon, 16 Jun 2014. Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

1 0

Re: [Maria-developers] Rev 3986: MDEV-5231: Per query variables from Percona Server
by Sergei Golubchik 14 Jun '14

14 Jun '14

Hi, Sanja! On May 22, sanja(a)askmonty.org wrote: > === added file 'mysql-test/suite/percona/percona_rpl_stm_per_query_variables_settings.test' > --- a/mysql-test/suite/percona/percona_rpl_stm_per_query_variables_settings.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/percona/percona_rpl_stm_per_query_variables_settings.test 2014-05-22 12:50:01 +0000 could you rename tests to have shorter names? rpl_set_statement looks good enough to me > @@ -0,0 +1,57 @@ > +--source include/master-slave.inc > +--source include/have_binlog_format_statement.inc > + > +--disable_warnings > +DROP TABLE IF EXISTS t1; > +DROP TABLE IF EXISTS t2; > +--enable_warnings > + > +call mtr.add_suppression("Unsafe statement written to the binary log*"); > +CREATE TABLE t1 (a bigint unsigned not null); > +CREATE TABLE t2 (a char(255) not null); okay, although I wouldn't bother creating *two* tables, I'd create one and hard-coded it into percona_rpl_set_statement_variable_test.inc (removing rpl_ssvt_table parameter) > + > +--echo > +--echo There are the following types of variables: > +--echo 1) variables that are NOT replicated correctly when using STATEMENT mode; > +--echo > + > +--let $rpl_ssvt_var_name=max_join_size > +--let $rpl_ssvt_var_value=2 > +--let $rpl_ssvt_table=t1 > +--source suite/percona/percona_rpl_set_statement_variable_test.inc > + > +--echo > +--echo 2) variables thar ARE replicated correctly > +--echo They must be replicated correctly with "SET STATEMENT" too. > +--echo > +--let $rpl_ssvt_var_name=auto_increment_increment > +--let $rpl_ssvt_var_value=10 > +--let $rpl_ssvt_table=t1 > +--source suite/percona/percona_rpl_set_statement_variable_test.inc > + > +--echo > +--echo 3) sql_mode which is replicated correctly exept NO_DIR_IN_CREATE value; > +--echo > +--let $rpl_ssvt_var_name=sql_mode > +--let $rpl_ssvt_var_value='ERROR_FOR_DIVISION_BY_ZERO' > +--let $rpl_ssvt_table=t2 > +--source suite/percona/percona_rpl_set_statement_variable_test.inc > +--let $rpl_ssvt_var_name=sql_mode > +--let $rpl_ssvt_var_value='NO_DIR_IN_CREATE' > +--let $rpl_ssvt_table=t2 > +--source suite/percona/percona_rpl_set_statement_variable_test.inc > + > +--echo > +--echo 4) variables that are not replicated at all: > +--echo default_storage_engine, storage_engine, max_heap_table_size > +--echo > +--let $rpl_ssvt_var_name=max_heap_table_size > +--let $rpl_ssvt_var_value=16384 > +--let $rpl_ssvt_table=t1 > +--source suite/percona/percona_rpl_set_statement_variable_test.inc > + > +connection master; > +DROP TABLE t1; > +DROP TABLE t2; > +sync_slave_with_master; > +source include/stop_slave.inc; > > === added file 'mysql-test/suite/percona/percona_statement_set.result' > --- a/mysql-test/suite/percona/percona_statement_set.result 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/percona/percona_statement_set.result 2014-05-22 12:50:01 +0000 reverse polish notation in percona someone loves. starwars too much seen he apparently. Please, rename to "set_statement.result" > @@ -0,0 +1,875 @@ > +'# SET STATEMENT ..... FOR .... TEST' > +DROP TABLE IF EXISTS t1; > +DROP FUNCTION IF EXISTS myProc; > +DROP PROCEDURE IF EXISTS p1; > +DROP PROCEDURE IF EXISTS p2; > +DROP PROCEDURE IF EXISTS p3; > +DROP PROCEDURE IF EXISTS p4; > +DROP PROCEDURE IF EXISTS p5; > +DROP TABLE IF EXISTS STATEMENT; > +'# Setup database' > +CREATE TABLE t1 (v1 INT, v2 INT); > +INSERT INTO t1 VALUES (1,2); > +INSERT INTO t1 VALUES (3,4); > +'' > +'#------------------ STATEMENT Test 1 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SET STATEMENT sort_buffer_size=150000 FOR SELECT * FROM t1; > +v1 v2 > +1 2 > +3 4 > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +'' > +'#------------------ STATEMENT Test 2 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION binlog_format=mixed; > +SET SESSION sort_buffer_size=100000; > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +SET STATEMENT sort_buffer_size=150000, binlog_format=row > +FOR SELECT * FROM t1; > +v1 v2 > +1 2 > +3 4 > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +'#------------------ STATEMENT Test 3 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION binlog_format=row; > +PREPARE stmt1 FROM 'SET STATEMENT binlog_format=row FOR SELECT * FROM t1'; > +'' > +'# Change variable setting' > +SET SESSION binlog_format=mixed; > +'' > +'# Pre-STATEMENT variable value' > +'' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +EXECUTE stmt1; > +v1 v2 > +1 2 > +3 4 > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +DEALLOCATE PREPARE stmt1; > +'#------------------ STATEMENT Test 4 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION myisam_sort_buffer_size=500000, myisam_repair_threads=1; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +SET STATEMENT myisam_sort_buffer_size=800000, > +myisam_repair_threads=2 FOR OPTIMIZE TABLE t1; how comes that is "make prepared statement" (see comment above) ? > +Table Op Msg_type Msg_text > +test.t1 optimize status OK > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' ... > === added file 'mysql-test/suite/percona/percona_statement_set.test' > --- a/mysql-test/suite/percona/percona_statement_set.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/percona/percona_statement_set.test 2014-05-22 12:50:01 +0000 > @@ -0,0 +1,847 @@ > +--echo '# SET STATEMENT ..... FOR .... TEST' many of the tests below are pretty useless. they don't test that the value is actually set. See below: > +############################ STATEMENT_SET ############################# > +# # > +# Testing working functionality of SET STATEMENT # > +# # > +# # > +# There is important documentation within # > +# # > +# # > +# Author: Joe Lukas # > +# Creation: # > +# 2009-08-02 Implement this test as part of # > +# WL#681 Per query variable settings # > +# # > +######################################################################## > + > +--disable_warnings > +DROP TABLE IF EXISTS t1; > +DROP FUNCTION IF EXISTS myProc; > +DROP PROCEDURE IF EXISTS p1; > +DROP PROCEDURE IF EXISTS p2; > +DROP PROCEDURE IF EXISTS p3; > +DROP PROCEDURE IF EXISTS p4; > +DROP PROCEDURE IF EXISTS p5; > +DROP TABLE IF EXISTS STATEMENT; > +--enable_warnings > +#################################################################### > +#Set up current database > +#################################################################### > +--echo '# Setup database' > +CREATE TABLE t1 (v1 INT, v2 INT); > +INSERT INTO t1 VALUES (1,2); > +INSERT INTO t1 VALUES (3,4); > +--echo '' > +--echo '#------------------ STATEMENT Test 1 -----------------------#' > +#################################################################### > +# Checks with variable value type ulong # > +#################################################################### > +--echo '# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +--echo '' > +--echo '# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +SET STATEMENT sort_buffer_size=150000 FOR SELECT * FROM t1; How do you know that SET STATEMENT did anything at all? > +--echo '' > +--echo '# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +--echo '' > +--echo '#------------------ STATEMENT Test 2 -----------------------#' > +#################################################################### > +# Checks for multiple set values inside STATEMENT ... FOR # > +#################################################################### > +--echo '# Initialize variables to known setting' > +SET SESSION binlog_format=mixed; > +SET SESSION sort_buffer_size=100000; > +--echo '# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +SET STATEMENT sort_buffer_size=150000, binlog_format=row > + FOR SELECT * FROM t1; > +--echo '# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > + > +--echo '' > +--echo '#------------------ STATEMENT Test 3 -----------------------#' > +#################################################################### > +# Check current variable value is stored in using stored # > +# statements. # > +#################################################################### > +--echo '# set initial variable value, make prepared statement > +SET SESSION binlog_format=row; > +PREPARE stmt1 FROM 'SET STATEMENT binlog_format=row FOR SELECT * FROM t1'; How will you know that SET STATEMENT did anything at all? (the same applies to most tests below, I'll stop repeating it) > +--echo '' > +--echo '# Change variable setting' > +SET SESSION binlog_format=mixed; > +--echo '' > +--echo '# Pre-STATEMENT variable value' > +--echo '' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +--echo '' > +EXECUTE stmt1; > +--echo '' > +--echo '# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > + > +--echo '' > +DEALLOCATE PREPARE stmt1; ... > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-02-11 13:21:48 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-05-22 12:50:01 +0000 > @@ -7067,3 +7067,5 @@ ER_IT_IS_A_VIEW 42S02 > eng "'%-.192s' is a view" > ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > +ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED > + eng "SET STATEMENT does not support using tables in the assignment variables expressions" why not? > > === modified file 'sql/sql_parse.cc' > --- a/sql/sql_parse.cc 2014-03-29 10:33:25 +0000 > +++ b/sql/sql_parse.cc 2014-05-22 12:50:01 +0000 > @@ -2430,6 +2432,33 @@ mysql_execute_command(THD *thd) > thd->get_binlog_format(&orig_binlog_format, > &orig_current_stmt_binlog_format); > > + if (lex->set_statement && !lex->var_list.is_empty()) Hmm? No separate list of variables, but a common lex->var_list? Then the following won't work: SET STATEMENT A=1 SET B=2 I remember in the WL and in GSoC we agreed that for SET STATEMENT A=2 SET A=2 (same variable in SET STATEMENT and SET), it's acceptable to restore the original value of A, basically ignoring SET. But I doubt that the same logic applies when SET STATEMENT and SET modify *different* variables. > + { > + MEM_ROOT *save_mem_root; > + /* > + We should be sure that variables backup and assignment made on > + runtime memory > + */ > + save_mem_root= thd->mem_root; > + thd->mem_root= thd->get_execution_mem_root(); Why? > + per_query_variables_backup= copy_system_variables(&thd->variables); > + if (!per_query_variables_backup || > + (res= sql_set_variables(thd, &lex->var_list))) > + { > + thd->mem_root= save_mem_root; > + /* > + We encountered some sort of error, but no message was sent. > + Send something semi-generic here since we don't know which > + assignment in the list caused the error. > + */ > + if (!thd->is_error()) > + my_error(ER_WRONG_ARGUMENTS, MYF(0), "SET"); > + goto error; > + } > + thd->mem_root= save_mem_root; > + } > + > /* > Force statement logging for DDL commands to allow us to update > privilege, system or statistic tables directly without the updates > @@ -5144,6 +5179,30 @@ finish: > > lex->unit.cleanup(); > > + if (lex->set_statement && !lex->var_list.is_empty()) { > + List_iterator_fast<set_var_base> it(thd->lex->var_list); > + set_var *var; > + > + free_system_variables(&thd->variables); > + thd->variables= *per_query_variables_backup; > + /* per_query_variables_backup was in a mem_root */ > + per_query_variables_backup= NULL; > + /* > + When variables are restored after "SET STATEMENT ... FOR ..." statement > + execution an update callback must be invoked for the system variables > + to save special logic if it is. set_var_base class does not contain > + refference to variable as it is just an interface class. But only > + system variables are allowed to be used in "SET STATEMENT ... FOR ..." > + statement, so cast from set_var_base* to set_var* can be used here. > + */ > + while ((var=(set_var *)it++)) > + { > + var->var->stmt_update(thd); > + } I don't think it'll work. Compare set_var::update() and set_var::stmt_update(). The former calls ::session_update(), while the latter does not. Normally, session_update() simply updates the value, performs the assignment. And it is, indeed, not needed here. but in some custom classes session_update() is more complex, and it's not safe to omit it. Please 1) see all non-trivial implementations of sys_var::session_update() 2) write SET SESSION tests for these variables 3) if they'll all work - please explain why 4) if they won't - I'm afraid, this whole approach needs to be redone, then > + > + thd->lex->set_statement= false; > + } > + > if (! thd->in_sub_stmt) > { > if (thd->killed != NOT_KILLED) > > === modified file 'sql/sql_plugin.cc' > --- a/sql/sql_plugin.cc 2014-03-26 21:25:38 +0000 > +++ b/sql/sql_plugin.cc 2014-05-22 12:50:01 +0000 > @@ -4088,3 +4088,48 @@ static void restore_ptr_backup(uint n, s > (backup++)->restore(); > } > > + > +/** > + Create deep copy of system_variables instance. > +*/ > +struct system_variables * > +copy_system_variables(const struct system_variables *src) What is it doing in sql_plugin.cc? > +{ > + struct system_variables *dst; > + > + DBUG_ASSERT(src); > + > + dst= (struct system_variables *) > + sql_alloc(sizeof(struct system_variables)); > + if (!dst) > + return NULL; > + *dst= *src; > + > + if (dst->dynamic_variables_ptr) > + { > + if (!(dst->dynamic_variables_ptr= > + (char *)my_malloc(dst->dynamic_variables_size, MYF(MY_WME | MY_FAE)))) > + return NULL; > + memcpy(dst->dynamic_variables_ptr, > + src->dynamic_variables_ptr, > + src->dynamic_variables_size); > + } > + > + mysql_mutex_lock(&LOCK_plugin); > + dst->table_plugin= > + intern_plugin_lock(NULL, src->table_plugin); I'm surprised that plugin locking actually works in here, (I believe it does, otherwise you'd noticed it). What about SET STATEMENT default_engine=MEMORY ? What about SET STATEMENT default_engine=MyISAM (where MyISAM is already default)? please, try both in debug build. > + mysql_mutex_unlock(&LOCK_plugin); > + > + return dst; > +} > + > +void free_system_variables(struct system_variables *v) > +{ > + DBUG_ASSERT(v); > + > + mysql_mutex_lock(&LOCK_plugin); > + intern_plugin_unlock(NULL, v->table_plugin); > + mysql_mutex_unlock(&LOCK_plugin); > + > + my_free(v->dynamic_variables_ptr); > +} > > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-03-26 21:25:38 +0000 > +++ b/sql/sql_yacc.yy 2014-05-22 12:50:01 +0000 > @@ -813,8 +813,10 @@ static void sp_create_assignment_lex(THD > lex->one_shot_set= 0; > lex->autocommit= 0; > /* get_ptr() is only correct with no lookahead. */ > - DBUG_ASSERT(no_lookahead); > + if (no_lookahead) > lex->sphead->m_tmp_query= lip->get_ptr(); > + else > + lex->sphead->m_tmp_query= lip->get_tok_end(); Why? > /* Inherit from outer lex. */ > lex->option_type= old_lex->option_type; > } > @@ -955,10 +957,10 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %parse-param { THD *thd } > %lex-param { THD *thd } > /* > - Currently there are 163 shift/reduce conflicts. > + Currently there are 164 shift/reduce conflicts. > We should not introduce new conflicts any more. > */ > -%expect 163 > +%expect 164 Why? > > /* > Comments for TOKENS. > @@ -14427,8 +14431,44 @@ set: > } > start_option_value_list > {} > + | SET STATEMENT_SYM > + { > + LEX *lex= Lex; > + mysql_init_select(lex); > + lex->sql_command= SQLCOM_SET_OPTION; > + /* Don't clear var_list in the case of recursive statement */ > + if (!lex->set_statement) > + lex->var_list.empty(); When var_list is not empty here (and set_statement not set)? In other words, when this statement will actually have any effect? > + lex->one_shot_set= 0; > + lex->autocommit= 0; > + lex->set_statement= true; > + sp_head *sp= lex->sphead; > + if (sp && !sp->is_invoked()) > + { > + sp->m_param_begin= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_start()); > + sp->m_param_end= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_end()); Why? > + } > + } > + set_stmt_option_value_following_option_type_list Eh? What does "set stmt option value following option type list" mean? Please, rename to set_stmt_option_value_list or even set_stmt_list > + { > + if (Lex->query_tables) > + { > + my_error(ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED, MYF(0)); > + MYSQL_YYABORT; > + } > + } > + FOR_SYM statement > + {} > ; > > +set_stmt_option_value_following_option_type_list: > + /* > + Only system variables can be used here. If this condition is changed > + please check careful code under lex->option_type == OPT_STATEMENT > + condition on wrong type casts. > + */ > + option_value_following_option_type > + | set_stmt_option_value_following_option_type_list ',' option_value_following_option_type > > // Start of option value list > start_option_value_list: Regards, Sergei

1 0