- developers - lists.mariadb.org

Re: [Maria-developers] MDEV-5756 CMake option to build without thread pool
by Sergei Golubchik 03 Jul '14

03 Jul '14

Hi, Holyfoot! On Jul 01, holyfoot(a)askmonty.org wrote: > revno: 4270 > revision-id: holyfoot(a)askmonty.org-20140701104158-4gj5ztnjqlr1alwa > parent: holyfoot(a)askmonty.org-20140630193024-56wpcvwwy24kuzt4 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-5756 > timestamp: Tue 2014-07-01 15:41:58 +0500 > message: > MDEV-5756 CMake option to build without thread pool. > Check if the threadpool is available on the system and set HAVE_POOL_OF_THREADS respectively. > === modified file 'sql/CMakeLists.txt' > --- a/sql/CMakeLists.txt 2014-05-30 11:24:25 +0000 > +++ b/sql/CMakeLists.txt 2014-07-01 10:41:58 +0000 > @@ -30,7 +30,15 @@ ${CMAKE_CURRENT_BINARY_DIR}/lex_hash.h > > SET_SOURCE_FILES_PROPERTIES(${GEN_SOURCES} PROPERTIES GENERATED 1) > > -ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER -DHAVE_POOL_OF_THREADS) > +ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER) > + > +IF (CMAKE_SYSTEM_NAME MATCHES "Linux" OR > + CMAKE_SYSTEM_NAME MATCHES "Windows" OR > + CMAKE_SYSTEM_NAME MATCHES "SunOS" OR > + HAVE_KQUEUE) > + ADD_DEFINITIONS(-DHAVE_POOL_OF_THREADS) > +ENDIF() > + > IF(SSL_DEFINES) > ADD_DEFINITIONS(${SSL_DEFINES}) > ENDIF() Did you test it on any of the platforms that that didn't support thread pool? If you would've done it, you'd noticed that your patch doesn't work. Because scheduler.h contains: #undef HAVE_POOL_OF_THREADS #if !defined(EMBEDDED_LIBRARY) && !defined(_AIX) #define HAVE_POOL_OF_THREADS 1 So, HAVE_POOL_OF_THREADS is only disabled in embedded and on AIX. The AIX part was added recently by Bar, as a quick fix to make MariaDB to compile on AIX. Your patch should remove the AIX test from scheduler.h. Actually, you should remove all #define and #undef HAVE_POOL_OF_THREADS from all files, only CMakeLists.txt should do it. Otherwise it's ok. I don't like that it explicitly checks for OS (Linux, Windows, SunOS) instead of checking for specific functions, libraries, or headers. But it's the same check that threadpool_unix.cc does, so it's ok. Regards, Sergei

1 0

[Maria-developers] [GSoC] Optimize mysql-test-runs - Analyzing some data
by Pablo Estrada 02 Jul '14

02 Jul '14

Hello Elena and all, I have spent a couple of days writing up some scripts to analyze the data, and try to find good candidates for metrics to improve recall, or some vices in the way the relevance is calculated for tests, such that they are not close enough in the priority queue to be spotted and run. Here's a writedown of some of the findings: *Of the failures that we miss, how many of them come from tests that have never failed before?* I added a small function in the simulator.py code, to keep diagnostics data of the test failures that we missed. I made a few charts regarding missed failures of tests that have never failed before (and, in the current model, we can't calculate a relevancy for them, because we have no data about them). In the next chart, we observe the percentage of missed failures that come from tests that had never failed before increases with time (as expected): By the way, this info comes from a standard run with 3000 learning cycles and 2000 prediction cycles. I feel that this is an important area. If we can figure a way to predict failures of tests that have not failed before, then we can predict subsequent failures of these tests, and thus improve recall significantly, specially over the long term. On the next section I analyze the option that I considered. *Is there a relationship between a test file being modified and the test failing any soon?* I figured that a good idea to catch failures of tests that had never failed before could be using the test file changes. If a test file is changed, I figured that the test itself would change behavior and might fail soon, and thus we could gather some data about that failure. I coded a small script that computes the number of test runs between a test file being edited, and the same test failing sometime in the future. The relationship, unfortunately is not very strong. I have attached the chart with the cummulative histogram of number of test runs that occurred between a test file being edited, and the next failure of that test. Unfortunately, the results were not too exciting. It turns out that tests don't tend to fail too soon after they have been changed (I guess this also makes sense). The following figures present the data (and a zoom in of the part of the graph we care most about), showing that failures are not very subsequent to test file editions. Not even 10% of failures occur in the first 100 runs. As we can see from the charts, there is not a very significant relationship between test file edition and subsequent failure. It seems this indicator won't be too useful. I am* still looking for ideas* on how to catch tests failing for the first time. They are all welcome : ) This is all the data that I collected so far. I am also trying to answer the following question: Failures of tests with non-zero relevance. How come we are missing those that we are missing? I have collected and charted some data, but I still haven't found anything that feels conclusive or revealing. I expect to write an email with that info soon as well, but I decided to send this out first. In conclusion: - I am thinking about how to predict first failures of a test. Any ideas? - I tested the relation between edition of a test result file and subsequent test failure. It was not a good predictor. Regards Pablo

1 0

[Maria-developers] [GSoC] Week Six!
by sriram patil 30 Jun '14

30 Jun '14

Hi all, Following is the blog link about my sixth week into GSoC. http://goo.gl/1dzgHe Suggestions and/or reviews are welcome! Thanks, Sriram

3 2

Re: [Maria-developers] [Commits] Rev 3855: Bumping server version. (10.0.13-galera) in lp:~maria-captains/maria/maria-10.0-galera
by Roberto Spadim 30 Jun '14

30 Jun '14

=] nice nirbhay 2014-06-30 10:33 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: > Hi, > > On Mon, Jun 30, 2014 at 9:13 AM, Roberto Spadim <roberto(a)spadim.com.br> > wrote: >> >> this will be a production ready version? > > > 10.0.12-galera (released over weekend) is the first 10.0-galera stable > release. This 'patch version bump' is > for preparation towards the next release. > > Best, > Nirbhay > >> >> >> 2014-06-30 10:03 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: >> > At lp:~maria-captains/maria/maria-10.0-galera >> > >> > ------------------------------------------------------------ >> > revno: 3855 >> > revision-id: nirbhay(a)skysql.com-20140630130329-w8kiirtf3blli7l1 >> > parent: nirbhay(a)skysql.com-20140626161112-q1iqtoyq5owutvvb >> > committer: Nirbhay Choubey <nirbhay(a)skysql.com> >> > branch nick: maria-10.0-galera >> > timestamp: Mon 2014-06-30 09:03:29 -0400 >> > message: >> > Bumping server version. (10.0.13-galera) >> > >> > === modified file 'VERSION' >> > --- a/VERSION 2014-06-19 17:12:38 +0000 >> > +++ b/VERSION 2014-06-30 13:03:29 +0000 >> > @@ -4,5 +4,5 @@ >> > # >> > MYSQL_VERSION_MAJOR=10 >> > MYSQL_VERSION_MINOR=0 >> > -MYSQL_VERSION_PATCH=12 >> > +MYSQL_VERSION_PATCH=13 >> > MYSQL_VERSION_EXTRA= >> > >> > >> > _______________________________________________ >> > commits mailing list >> > commits(a)mariadb.org >> > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> _______________________________________________ >> commits mailing list >> commits(a)mariadb.org >> https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] [Commits] Rev 3855: Bumping server version. (10.0.13-galera) in lp:~maria-captains/maria/maria-10.0-galera
by Roberto Spadim 30 Jun '14

30 Jun '14

this will be a production ready version? 2014-06-30 10:03 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: > At lp:~maria-captains/maria/maria-10.0-galera > > ------------------------------------------------------------ > revno: 3855 > revision-id: nirbhay(a)skysql.com-20140630130329-w8kiirtf3blli7l1 > parent: nirbhay(a)skysql.com-20140626161112-q1iqtoyq5owutvvb > committer: Nirbhay Choubey <nirbhay(a)skysql.com> > branch nick: maria-10.0-galera > timestamp: Mon 2014-06-30 09:03:29 -0400 > message: > Bumping server version. (10.0.13-galera) > > === modified file 'VERSION' > --- a/VERSION 2014-06-19 17:12:38 +0000 > +++ b/VERSION 2014-06-30 13:03:29 +0000 > @@ -4,5 +4,5 @@ > # > MYSQL_VERSION_MAJOR=10 > MYSQL_VERSION_MINOR=0 > -MYSQL_VERSION_PATCH=12 > +MYSQL_VERSION_PATCH=13 > MYSQL_VERSION_EXTRA= > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Merge request: Cross-compile fixes (MySQL bug #61340)
by James Le Cuirot 30 Jun '14

30 Jun '14

Hello all, I posted a merge request to Launchpad back in April. A Launchpad glitch meant that it didn't get any attention but I was told on IRC to post about it here. The merge request: https://code.launchpad.net/~chewi/maria/mysql-fix-bug61340/+merge/217351 The commit diff: http://bazaar.launchpad.net/~chewi/maria/mysql-fix-bug61340/revision/4165 Given the nature of the fix, I can appreciate that it is difficult for other developers to test but I can tell you that I have just tried out a cross-built MariaDB under QEMU/proot and it seems to work. I have also seen posts from others stating that they made similar temporary adjustments to the CMake files with successful results. I am providing this contribution to the MariaDB Foundation under the MCA and also wish for it to be applied to MySQL. Regards, James

2 2

[Maria-developers] Connector Coding till 29th June
by Puneet Dewan 29 Jun '14

29 Jun '14

Hi Everyone, This week I fixed the null bitmap and types that were being sent to the server. So now I am able to successfully insert multiple values in the DB. Currently only Integer and String. This week 1.I will extend all the TypeParameter classes and ParameterWriter class so as to send the value in binary format. 2.Also try insert on different data types. 3. Write test cases Currently I am extending the PacketOutputStream.java class. Tomorrow I will have code review with Massimo. Regards Puneet.

1 0

[Maria-developers] TIME/DATETIME/TIMESTAMP data types with fractional precision
by Alexander Barkov 26 Jun '14

26 Jun '14

Hello, I propose to switch to MySQL-5.6 TIME/DATETIME/TIMESTAMP formats by default and deprecate MariaDB native temporal data types, together with pre-5.6 non-fractional types. The problem is that MariaDB native data types (and MySQL-5.5 non-fractional data types) do not work with row based replication in case of: TIME(N) -> TIME(M) There is no metadata telling about the column precision in the binary log file. It can only work when N=M. MySQL-5.6 uses separate type codes for the temporal data types: MYSQL_TYPE_TIME2 MYSQL_TYPE_DATETIME2 MYSQL_TYPE_TIMESTAMP2 and does put precision into metadata in the binary log. So it works for N != M. There is a JIRA task for this: MDEV-5377 Row-based replication of MariaDB temporal data types with FSP>0 into a different column type I'd propose to do it in 10.1.x, as 10.0.x is too late. Greetings.

2 1

Re: [Maria-developers] [Commits] Rev 4245: MDEV-6336: mysqldump --master-data does not work with GTID setups in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Sergey Vojtovich 25 Jun '14

25 Jun '14

Hi Kristian, MDEV-6344 looks fine. But I don't feel confident reviewing MDEV-6336. Regards, Sergey On Mon, Jun 16, 2014 at 11:38:04AM +0200, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4245 > revision-id: knielsen(a)knielsen-hq.org-20140616093803-gub60pda6t77ch3k > parent: igor(a)askmonty.org-20140610223256-nqs0kkp7i45qzyvx > committer: knielsen(a)knielsen-hq.org > branch nick: work-10.0 > timestamp: Mon 2014-06-16 11:38:03 +0200 > message: > MDEV-6336: mysqldump --master-data does not work with GTID setups > MDEV-6344: mysqldump issues FLUSH TABLES, which gets written into binlog and replicated > > Add a --gtid option (for compatibility, the original behaviour is preserved > when --gtid is not used). > > With --gtid, --master-data and --dump-slave output the GTID position (the > old-style file/offset position is still output, but commented out). Also, a > CHANGE MASTER TO master_use_gtid=slave_pos is output to ensure a provisioned > slave is configured in GTID, as requested. > > Without --gtid, the GTID position is still output, if available, but commented > out. > > Also fix MDEV-6344, to avoid FLUSH TABLES getting into the binlog. Otherwise a > mysqldump on a slave server will silently inject a GTID which does not exist > on the master, which is highly undesirable. > === modified file 'client/client_priv.h' > --- a/client/client_priv.h 2013-09-14 01:09:36 +0000 > +++ b/client/client_priv.h 2014-06-16 09:38:03 +0000 > @@ -92,6 +92,7 @@ enum options_client > OPT_REPORT_PROGRESS, > OPT_SKIP_ANNOTATE_ROWS_EVENTS, > OPT_SSL_CRL, OPT_SSL_CRLPATH, > + OPT_USE_GTID, > OPT_MAX_CLIENT_OPTION /* should be always the last */ > }; > > > === modified file 'client/mysqldump.c' > --- a/client/mysqldump.c 2014-05-09 10:35:11 +0000 > +++ b/client/mysqldump.c 2014-06-16 09:38:03 +0000 > @@ -134,6 +134,7 @@ static ulong opt_compatible_mode= 0; > #define MYSQL_OPT_SLAVE_DATA_COMMENTED_SQL 2 > static uint opt_mysql_port= 0, opt_master_data; > static uint opt_slave_data; > +static uint opt_use_gtid; > static uint my_end_arg; > static char * opt_mysql_unix_port=0; > static int first_error=0; > @@ -346,6 +347,13 @@ static struct my_option my_long_options[ > {"force", 'f', "Continue even if we get an SQL error.", > &ignore_errors, &ignore_errors, 0, GET_BOOL, NO_ARG, > 0, 0, 0, 0, 0, 0}, > + {"gtid", OPT_USE_GTID, "Used together with --master-data=1 or --dump-slave=1." > + "When enabled, the output from those options will set the GTID position " > + "instead of the binlog file and offset; the file/offset will appear only as " > + "a comment. When disabled, the GTID position will still appear in the " > + "output, but only commented.", > + &opt_use_gtid, &opt_use_gtid, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, > {"help", '?', "Display this help message and exit.", 0, 0, 0, GET_NO_ARG, > NO_ARG, 0, 0, 0, 0, 0, 0}, > {"hex-blob", OPT_HEXBLOB, "Dump binary strings (BINARY, " > @@ -1177,7 +1185,7 @@ check_consistent_binlog_pos(char *binlog > > if (mysql_query_with_error_report(mysql, &res, > "SHOW STATUS LIKE 'binlog_snapshot_%'")) > - return 1; > + return 0; > > found= 0; > while ((row= mysql_fetch_row(res))) > @@ -1200,6 +1208,90 @@ check_consistent_binlog_pos(char *binlog > return (found == 2); > } > > + > +/* > + Get the GTID position corresponding to a given old-style binlog position. > + This uses BINLOG_GTID_POS(). The advantage is that the GTID position can > + be obtained completely non-blocking in this way (without the need for > + FLUSH TABLES WITH READ LOCK), as the old-style position can be obtained > + with START TRANSACTION WITH CONSISTENT SNAPSHOT. > + > + Returns 0 if ok, non-zero if error. > +*/ > +static int > +get_binlog_gtid_pos(char *binlog_pos_file, char *binlog_pos_offset, > + char *out_gtid_pos) > +{ > + DYNAMIC_STRING query; > + MYSQL_RES *res; > + MYSQL_ROW row; > + int err; > + char file_buf[FN_REFLEN*2+1], offset_buf[LONGLONG_LEN*2+1]; > + size_t len_pos_file= strlen(binlog_pos_file); > + size_t len_pos_offset= strlen(binlog_pos_offset); > + > + if (len_pos_file >= FN_REFLEN || len_pos_offset > LONGLONG_LEN) > + return 0; > + mysql_real_escape_string(mysql, file_buf, binlog_pos_file, len_pos_file); > + mysql_real_escape_string(mysql, offset_buf, binlog_pos_offset, len_pos_offset); > + init_dynamic_string_checked(&query, "SELECT BINLOG_GTID_POS('", 256, 1024); > + dynstr_append_checked(&query, file_buf); > + dynstr_append_checked(&query, "', '"); > + dynstr_append_checked(&query, offset_buf); > + dynstr_append_checked(&query, "')"); > + > + err= mysql_query_with_error_report(mysql, &res, query.str); > + dynstr_free(&query); > + if (err) > + return err; > + > + err= 1; > + if ((row= mysql_fetch_row(res))) > + { > + strmake(out_gtid_pos, row[0], FN_REFLEN-1); > + err= 0; > + } > + mysql_free_result(res); > + > + return err; > +} > + > + > +/* > + Get the GTID position on a master or slave. > + The parameter MASTER is non-zero to get the position on a master > + (@@gtid_binlog_pos) or zero for a slave (@@gtid_slave_pos). > + > + This uses the @@gtid_binlog_pos or @@gtid_slave_pos, so requires FLUSH TABLES > + WITH READ LOCK or similar to be consistent. > + > + Returns 0 if ok, non-zero for error. > +*/ > +static int > +get_gtid_pos(char *out_gtid_pos, int master) > +{ > + MYSQL_RES *res; > + MYSQL_ROW row; > + int found; > + > + if (mysql_query_with_error_report(mysql, &res, > + (master ? > + "SELECT @@GLOBAL.gtid_binlog_pos" : > + "SELECT @@GLOBAL.gtid_slave_pos"))) > + return 1; > + > + found= 0; > + if ((row= mysql_fetch_row(res))) > + { > + strmake(out_gtid_pos, row[0], FN_REFLEN-1); > + found++; > + } > + mysql_free_result(res); > + > + return (found != 1); > +} > + > + > static char *my_case_str(const char *str, > uint str_len, > const char *token, > @@ -4799,12 +4891,14 @@ static int dump_selected_tables(char *db > } /* dump_selected_tables */ > > > -static int do_show_master_status(MYSQL *mysql_con, int consistent_binlog_pos) > +static int do_show_master_status(MYSQL *mysql_con, int consistent_binlog_pos, > + int have_mariadb_gtid, int use_gtid) > { > MYSQL_ROW row; > MYSQL_RES *UNINIT_VAR(master); > char binlog_pos_file[FN_REFLEN]; > char binlog_pos_offset[LONGLONG_LEN+1]; > + char gtid_pos[FN_REFLEN]; > char *file, *offset; > const char *comment_prefix= > (opt_master_data == MYSQL_OPT_MASTER_DATA_COMMENTED_SQL) ? "-- " : ""; > @@ -4815,6 +4909,9 @@ static int do_show_master_status(MYSQL * > return 1; > file= binlog_pos_file; > offset= binlog_pos_offset; > + if (have_mariadb_gtid && > + get_binlog_gtid_pos(binlog_pos_file, binlog_pos_offset, gtid_pos)) > + return 1; > } > else > { > @@ -4844,6 +4941,9 @@ static int do_show_master_status(MYSQL * > return 0; > } > } > + > + if (have_mariadb_gtid && get_gtid_pos(gtid_pos, 1)) > + return 1; > } > > /* SHOW MASTER STATUS reports file and position */ > @@ -4852,7 +4952,19 @@ static int do_show_master_status(MYSQL * > "recovery from\n--\n\n"); > fprintf(md_result_file, > "%sCHANGE MASTER TO MASTER_LOG_FILE='%s', MASTER_LOG_POS=%s;\n", > - comment_prefix, file, offset); > + (use_gtid ? "-- " : comment_prefix), file, offset); > + if (have_mariadb_gtid) > + { > + print_comment(md_result_file, 0, > + "\n--\n-- GTID to start replication from\n--\n\n"); > + if (use_gtid) > + fprintf(md_result_file, > + "%sCHANGE MASTER TO MASTER_USE_GTID=slave_pos;\n", > + comment_prefix); > + fprintf(md_result_file, > + "%sSET GLOBAL gtid_slave_pos='%s';\n", > + (!use_gtid ? "-- " : comment_prefix), gtid_pos); > + } > check_io(md_result_file); > > if (!consistent_binlog_pos) > @@ -4922,12 +5034,16 @@ static int add_slave_statements(void) > return(0); > } > > -static int do_show_slave_status(MYSQL *mysql_con) > +static int do_show_slave_status(MYSQL *mysql_con, int use_gtid, > + int have_mariadb_gtid) > { > MYSQL_RES *UNINIT_VAR(slave); > MYSQL_ROW row; > const char *comment_prefix= > (opt_slave_data == MYSQL_OPT_SLAVE_DATA_COMMENTED_SQL) ? "-- " : ""; > + const char *gtid_comment_prefix= (use_gtid ? comment_prefix : "-- "); > + const char *nogtid_comment_prefix= (!use_gtid ? comment_prefix : "-- "); > + int set_gtid_done= 0; > > if (mysql_query_with_error_report(mysql_con, &slave, > multi_source ? > @@ -4945,8 +5061,30 @@ static int do_show_slave_status(MYSQL *m > > while ((row= mysql_fetch_row(slave))) > { > + if (multi_source && !set_gtid_done) > + { > + char gtid_pos[FN_REFLEN]; > + if (have_mariadb_gtid && get_gtid_pos(gtid_pos, 0)) > + return 1; > + if (opt_comments) > + fprintf(md_result_file, "\n--\n-- Gtid position to start replication " > + "from\n--\n\n"); > + fprintf(md_result_file, "%sSET GLOBAL gtid_slave_pos='%s';\n", > + gtid_comment_prefix, gtid_pos); > + set_gtid_done= 1; > + } > if (row[9 + multi_source] && row[21 + multi_source]) > { > + if (use_gtid) > + { > + if (multi_source) > + fprintf(md_result_file, "%sCHANGE MASTER '%.80s' TO " > + "MASTER_USE_GTID=slave_pos;\n", gtid_comment_prefix, row[0]); > + else > + fprintf(md_result_file, "%sCHANGE MASTER TO " > + "MASTER_USE_GTID=slave_pos;\n", gtid_comment_prefix); > + } > + > /* SHOW MASTER STATUS reports file and position */ > if (opt_comments) > fprintf(md_result_file, > @@ -4955,9 +5093,9 @@ static int do_show_slave_status(MYSQL *m > > if (multi_source) > fprintf(md_result_file, "%sCHANGE MASTER '%.80s' TO ", > - comment_prefix, row[0]); > + nogtid_comment_prefix, row[0]); > else > - fprintf(md_result_file, "%sCHANGE MASTER TO ", comment_prefix); > + fprintf(md_result_file, "%sCHANGE MASTER TO ", nogtid_comment_prefix); > > if (opt_include_master_host_port) > { > @@ -5030,12 +5168,13 @@ static int do_flush_tables_read_lock(MYS > FLUSH TABLES is to lower the probability of a stage where both mysqldump > and most client connections are stalled. Of course, if a second long > update starts between the two FLUSHes, we have that bad stall. > + > + We use the LOCAL option, as we do not want the FLUSH TABLES replicated to > + other servers. > */ > return > - ( mysql_query_with_error_report(mysql_con, 0, > - ((opt_master_data != 0) ? > - "FLUSH /*!40101 LOCAL */ TABLES" : > - "FLUSH TABLES")) || > + ( mysql_query_with_error_report(mysql_con, 0, > + "FLUSH /*!40101 LOCAL */ TABLES") || > mysql_query_with_error_report(mysql_con, 0, > "FLUSH TABLES WITH READ LOCK") ); > } > @@ -5656,6 +5795,7 @@ int main(int argc, char **argv) > char bin_log_name[FN_REFLEN]; > int exit_code; > int consistent_binlog_pos= 0; > + int have_mariadb_gtid= 0; > MY_INIT(argv[0]); > > sf_leaking_memory=1; /* don't report memory leaks on early exits */ > @@ -5696,7 +5836,10 @@ int main(int argc, char **argv) > > /* Check if the server support multi source */ > if (mysql_get_server_version(mysql) >= 100000) > + { > multi_source= 2; > + have_mariadb_gtid= 1; > + } > > if (opt_slave_data && do_stop_slave_sql(mysql)) > goto err; > @@ -5743,9 +5886,11 @@ int main(int argc, char **argv) > /* Add 'STOP SLAVE to beginning of dump */ > if (opt_slave_apply && add_stop_slave()) > goto err; > - if (opt_master_data && do_show_master_status(mysql, consistent_binlog_pos)) > + if (opt_master_data && do_show_master_status(mysql, consistent_binlog_pos, > + have_mariadb_gtid, opt_use_gtid)) > goto err; > - if (opt_slave_data && do_show_slave_status(mysql)) > + if (opt_slave_data && do_show_slave_status(mysql, opt_use_gtid, > + have_mariadb_gtid)) > goto err; > if (opt_single_transaction && do_unlock_tables(mysql)) /* unlock but no commit! */ > goto err; > > === modified file 'mysql-test/r/rpl_mysqldump_slave.result' > --- a/mysql-test/r/rpl_mysqldump_slave.result 2014-02-26 14:28:07 +0000 > +++ b/mysql-test/r/rpl_mysqldump_slave.result 2014-06-16 09:38:03 +0000 > @@ -4,18 +4,59 @@ include/master-slave.inc > # New --dump-slave, --apply-slave-statements functionality > # > use test; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > STOP ALL SLAVES; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > START ALL SLAVES; > STOP ALL SLAVES; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_HOST='127.0.0.1', MASTER_PORT=MASTER_MYPORT, MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > START ALL SLAVES; > start slave; > Warnings: > Note 1254 Slave is already running > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > start slave; > Warnings: > Note 1254 Slave is already running > +*** Test mysqldump --dump-slave GTID functionality. > +SET gtid_seq_no = 1000; > +CREATE TABLE t1 (a INT PRIMARY KEY); > +DROP TABLE t1; > +CREATE TABLE t2 (a INT PRIMARY KEY); > +DROP TABLE t2; > + > +1. --dump-slave=1 > + > +SET GLOBAL gtid_slave_pos='0-1-1001'; > +CHANGE MASTER '' TO MASTER_USE_GTID=slave_pos; > +-- CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > + > +2. --dump-slave=2 > + > +-- SET GLOBAL gtid_slave_pos='0-1-1001'; > +-- CHANGE MASTER '' TO MASTER_USE_GTID=slave_pos; > +-- CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > +*** Test mysqldump --master-data GTID functionality. > + > +1. --master-data=1 > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +SET GLOBAL gtid_slave_pos='0-2-1003'; > + > +2. --master-data=2 > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +-- CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +-- SET GLOBAL gtid_slave_pos='0-2-1003'; > + > +3. --master-data --single-transaction > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +SET GLOBAL gtid_slave_pos='0-2-1003'; > include/rpl_end.inc > > === modified file 'mysql-test/t/rpl_mysqldump_slave.test' > --- a/mysql-test/t/rpl_mysqldump_slave.test 2014-02-26 14:28:07 +0000 > +++ b/mysql-test/t/rpl_mysqldump_slave.test 2014-06-16 09:38:03 +0000 > @@ -36,4 +36,53 @@ start slave; > --exec $MYSQL_DUMP_SLAVE --compact --dump-slave no_such_db > start slave; > > + > +--echo *** Test mysqldump --dump-slave GTID functionality. > + > +--connection master > +SET gtid_seq_no = 1000; > +CREATE TABLE t1 (a INT PRIMARY KEY); > +DROP TABLE t1; > +--sync_slave_with_master > + > +--connection slave > +# Inject a local transaction on the slave to check that this is not considered > +# for --dump-slave. > +CREATE TABLE t2 (a INT PRIMARY KEY); > +DROP TABLE t2; > + > +--echo > +--echo 1. --dump-slave=1 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --dump-slave=1 --gtid test > + > +--echo > +--echo 2. --dump-slave=2 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --dump-slave=2 --gtid test > + > + > +--echo *** Test mysqldump --master-data GTID functionality. > +--echo > +--echo 1. --master-data=1 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data=1 --gtid test > + > +--echo > +--echo 2. --master-data=2 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data=2 --gtid test > + > +--echo > +--echo 3. --master-data --single-transaction > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data --single-transaction --gtid test > + > + > + > --source include/rpl_end.inc > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

Re: [Maria-developers] [Commits] Rev 4259: MDEV-6120: When slave stops with error, error message should indicate the failing GTID in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Sergey Vojtovich 25 Jun '14

25 Jun '14

Hi Kristian, looks good. One minor question below. On Thu, Jun 19, 2014 at 02:50:44PM +0200, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4259 > revision-id: knielsen(a)knielsen-hq.org-20140619125043-acw2gx7rw90y3vqp > parent: svoj(a)mariadb.org-20140618141504-kqq6ws9gm93r2b2d > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: work-10.0 > timestamp: Thu 2014-06-19 14:50:43 +0200 > message: > MDEV-6120: When slave stops with error, error message should indicate the failing GTID > > If replication breaks in GTID mode, it is not trivial to determine the GTID of > the failing event group. This is a problem, as such GTID is needed eg. to > explicitly set @@gtid_slave_pos to skip to after that event group, or to > compare errors on different servers, etc. > > Fix by ensuring that relevant slave errors logged to the error log include the > GTID of the event group containing the problem event. ...skip... > === added file 'mysql-test/suite/rpl/t/rpl_gtid_errorlog.test' > --- a/mysql-test/suite/rpl/t/rpl_gtid_errorlog.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_gtid_errorlog.test 2014-06-19 12:50:43 +0000 > @@ -0,0 +1,94 @@ > +--source include/have_debug.inc > +--source include/master-slave.inc > + > +--echo *** Test MDEV-6120, output of current GTID when a replication error is logged to the errorlog *** > +--connection master > +CREATE TABLE t1(a INT PRIMARY KEY); > +--sync_slave_with_master > + > +--connection slave > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=slave_pos; > + > +--connection master > +INSERT INTO t1 VALUES (1); > +SET gtid_seq_no=100; > +INSERT INTO t1 VALUES (2); > +INSERT INTO t1 VALUES (3); > +INSERT INTO t1 VALUES (4); > +--save_master_pos > + > +--connection slave > +SET sql_log_bin=0; > +INSERT INTO t1 VALUES (2); > +SET sql_log_bin=1; > + > +START SLAVE; > +--let $slave_sql_errno=1062 > +--source include/wait_for_slave_sql_error.inc > + > +--source include/stop_slave.inc > +# Skip the problem event from the master. > +SET GLOBAL gtid_slave_pos= "0-1-100"; > +--source include/start_slave.inc > +--sync_with_master > + > +SELECT * FROM t1 ORDER BY a; > + > +--connection master > + > +SET @dbug_save= @@debug_dbug; > +SET debug_dbug= '+d,incident_database_resync_on_replace'; > +REPLACE INTO t1 VALUES (5); > +SET debug_dbug= @dbug_save; > +--save_master_pos > + > +--connection slave > +--let $slave_sql_errno=1590 > +--source include/wait_for_slave_sql_error.inc > +# ToDo no need to switch off GTID once MDEV-4937 is fixed > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=no; > +SET sql_slave_skip_counter=1; > +--source include/start_slave.inc > +--sync_with_master > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=slave_pos; > +--source include/start_slave.inc > + > +SELECT * FROM t1 ORDER BY a; > + > + > +# Check error log for correct messages. > +let $log_error_= `SELECT @@GLOBAL.log_error`; > +if(!$log_error_) > +{ > + # MySQL Server on windows is started with --console and thus > + # does not know the location of its .err log, use default location > + let $log_error_ = $MYSQLTEST_VARDIR/log/mysqld.2.err; > +} > +let LOG_ERROR=$log_error_; > + > +perl; > + use strict; > + use warnings; > + my $log_error= $ENV{'LOG_ERROR'} or die "LOG_ERROR not set"; > + open(FILE, "<", "$log_error") or die("Unable to open $log_error: $!\n"); > + my $found_dup= 0; > + my $found_incident= 0; > + while (<FILE>) { > + $found_dup= 1 > + if /Slave SQL: Error 'Duplicate entry .* on query\. .*Query: '.*', Gtid 0-1-100, Internal MariaDB error code:/ > + or /Slave SQL: Could not execute Write_rows.*table test.t1; Duplicate entry.*, Gtid 0-1-100, Internal MariaDB error/ > + ; > + $found_incident= 1 > + if /Slave SQL: The incident LOST_EVENTS occured on the master\. Message: <none>, Internal MariaDB error code: 1590/; > + } > + print "Duplicate key found: $found_dup; incident found: $found_incident\n"; > + close(FILE); > +EOF > + > +--connection master > +DROP TABLE t1; > + > +--source include/rpl_end.inc Why didn't you like include/search_pattern_in_file.inc? It should perfectly able to match your patterns. ...skip... Regards, Sergey

2 1