August 2009 - developers - lists.mariadb.org

[Maria-developers] Updated (by Psergey): Add a mysqlbinlog option to change the used database (36)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add a mysqlbinlog option to change the used database CREATION DATE..: Fri, 07 Aug 2009, 14:57 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 36 (http://askmonty.org/worklog/?tid=36) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sun, 09 Aug 2009, 23:53)=-=- High-Level Specification modified. --- /tmp/wklog.36.old.13425 2009-08-09 23:53:54.000000000 +0300 +++ /tmp/wklog.36.new.13425 2009-08-09 23:53:54.000000000 +0300 @@ -1 +1,72 @@ +Context +------- +At the moment, the server has a replication slave option + + --replicate-rewrite-db="from->to" + +the option affects +- Table_map_log_event (all RBR events) +- Load_log_event (LOAD DATA) +- Query_log_event (SBR-based updates, with the usual assumption that the + statement refers to tables in current database, so that changing the current + database will make the statement to work on a table in a different database). + +What we could do +---------------- + +Option1: make mysqlbinlog accept --replicate-rewrite-db option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Make mysqlbinlog accept --replicate-rewrite-db options and process them to the +same extent as replication slave would process --replicate-rewrite-db option. + + +Option2: Add database-agnostic RBR events and --strip-db option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Right now RBR events require a databasename. It is not possible to have RBR +event stream that won't mention which database the events are for. When I +tried to use debugger and specify empty database name, attempt to apply the +binlog resulted in this error: + +090809 17:38:44 [ERROR] Slave SQL: Error 'Table '.tablename' doesn't exist' on +opening tables, + +We could do as follows: +- Make the server interpret empty database name in RBR event (i.e. in a + Table_map_log_event) as "use current database". Binlog slave thread + probably should not allow such events as it doesn't have a natural current + database. +- Add a mysqlbinlog --strip-db option that would + = not produce any "USE dbname" statements + = change databasename for all RBR events to be empty + +That way, mysqlbinlog output will be database-agnostic and apply to the +current database. +(this will have the usual limitations that we assume that all statements in +the binlog refer to the current database). + +Option3: Enhance database rewrite +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If there is a need to support database change for statements that use +dbname.tablename notation and are replicated as statements (i.e. are DDL +statements and/or DML statements that are binlogged as statements), +then that could be supported as follows: + +- Make the server's parser recognize special form of comments + + /* !database-alias(oldname,newname) */ + + and save the mapping somewhere + +- Put the hooks in table open and name resolution code to use the saved + mapping. + + +Once we've done the above, it will be easy to perform a complete, +no-compromise or restrictions database name change in binary log. + +It will be possible to do the rewrites either on the slave ( +--replicate-rewrite-db will work for all kinds of statements), or in +mysqlbinlog (adding a comment is easy and doesn't require use to parse the +statement). + -=-=(Psergey - Sun, 09 Aug 2009, 12:27)=-=- Dependency created: 39 now depends on 36 -=-=(Psergey - Fri, 07 Aug 2009, 14:57)=-=- Title modified. --- /tmp/wklog.36.old.14687 2009-08-07 14:57:49.000000000 +0300 +++ /tmp/wklog.36.new.14687 2009-08-07 14:57:49.000000000 +0300 @@ -1 +1 @@ -Add a mysqlbinlog option to change the database +Add a mysqlbinlog option to change the used database DESCRIPTION: Sometimes there is a need to take a binary log and apply it to a database with a different name than the original name of the database on binlog producer. If one is using statement-based replication, he can achieve this by grepping out "USE dbname" statements out of the output of mysqlbinlog(*). With row-based replication this is no longer possible, as database name is encoded within the the BINLOG '....' statement. This task is about adding an option to mysqlbinlog that would allow to change the names of used databases in both RBR and SBR events. (*) this implies that all statements refer to tables in the current database, doesn't catch updates made inside stored functions and so forth, but still works for a practially-important subset of cases. HIGH-LEVEL SPECIFICATION: Context ------- At the moment, the server has a replication slave option --replicate-rewrite-db="from->to" the option affects - Table_map_log_event (all RBR events) - Load_log_event (LOAD DATA) - Query_log_event (SBR-based updates, with the usual assumption that the statement refers to tables in current database, so that changing the current database will make the statement to work on a table in a different database). What we could do ---------------- Option1: make mysqlbinlog accept --replicate-rewrite-db option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Make mysqlbinlog accept --replicate-rewrite-db options and process them to the same extent as replication slave would process --replicate-rewrite-db option. Option2: Add database-agnostic RBR events and --strip-db option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Right now RBR events require a databasename. It is not possible to have RBR event stream that won't mention which database the events are for. When I tried to use debugger and specify empty database name, attempt to apply the binlog resulted in this error: 090809 17:38:44 [ERROR] Slave SQL: Error 'Table '.tablename' doesn't exist' on opening tables, We could do as follows: - Make the server interpret empty database name in RBR event (i.e. in a Table_map_log_event) as "use current database". Binlog slave thread probably should not allow such events as it doesn't have a natural current database. - Add a mysqlbinlog --strip-db option that would = not produce any "USE dbname" statements = change databasename for all RBR events to be empty That way, mysqlbinlog output will be database-agnostic and apply to the current database. (this will have the usual limitations that we assume that all statements in the binlog refer to the current database). Option3: Enhance database rewrite ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If there is a need to support database change for statements that use dbname.tablename notation and are replicated as statements (i.e. are DDL statements and/or DML statements that are binlogged as statements), then that could be supported as follows: - Make the server's parser recognize special form of comments /* !database-alias(oldname,newname) */ and save the mapping somewhere - Put the hooks in table open and name resolution code to use the saved mapping. Once we've done the above, it will be easy to perform a complete, no-compromise or restrictions database name change in binary log. It will be possible to do the rewrites either on the slave ( --replicate-rewrite-db will work for all kinds of statements), or in mysqlbinlog (adding a comment is easy and doesn't require use to parse the statement). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Add an option to mysqlbinlog to produce SQL script with fewer roundtrips (37)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add an option to mysqlbinlog to produce SQL script with fewer roundtrips CREATION DATE..: Fri, 07 Aug 2009, 17:14 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 37 (http://askmonty.org/worklog/?tid=37) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sun, 09 Aug 2009, 12:56)=-=- High-Level Specification modified. --- /tmp/wklog.37.old.22083 2009-08-09 12:56:36.000000000 +0300 +++ /tmp/wklog.37.new.22083 2009-08-09 12:56:36.000000000 +0300 @@ -11,3 +11,16 @@ if (my_b_tell(&cache) != 0) my_b_write(&cache,";;",2); +Note: mysqlbinlog already uses + + DELIMITER /*!*/; + +so that it can process "multi-statements" like + + CREATE PROCEDURE ... BEGIN stmt1; stmt2; ... END + +what remains to be done is to print the /*!*/; only when we're about to exceed +$args[combine-statements] bytes. In all other cases, delimit statements with +regular semicolon. + + -=-=(Psergey - Sun, 09 Aug 2009, 12:30)=-=- High Level Description modified. --- /tmp/wklog.37.old.21090 2009-08-09 12:30:26.000000000 +0300 +++ /tmp/wklog.37.new.21090 2009-08-09 12:30:26.000000000 +0300 @@ -1,6 +1,6 @@ SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot -of roundtrips, and they become a bottleneck. +of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: @@ -14,7 +14,7 @@ loading such sql script will require fewer roundtrips. -The behavior can be controlled using a command line option +The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 37 -=-=(Psergey - Fri, 07 Aug 2009, 17:16)=-=- High-Level Specification modified. --- /tmp/wklog.37.old.20454 2009-08-07 17:16:54.000000000 +0300 +++ /tmp/wklog.37.new.20454 2009-08-07 17:16:54.000000000 +0300 @@ -1 +1,13 @@ +Implementation overview: + +- At start, print "--delimiter=;;" +- Modify the start of each print functions as follows + + if (my_b_tell(&cache) - my_start_of_combine_statement) + + estimiated_size_of_log_event) > combine_statement_size) + my_b_write(&cache,";;",2); + +- And we should end mysqlbinlog with; + if (my_b_tell(&cache) != 0) + my_b_write(&cache,";;",2); DESCRIPTION: SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: +delimiter // binlog statement1; binlog statement2; binlog statement3; +// binlog statement4; loading such sql script will require fewer roundtrips. The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# Where the # is maximum allowed packet length. HIGH-LEVEL SPECIFICATION: Implementation overview: - At start, print "--delimiter=;;" - Modify the start of each print functions as follows if (my_b_tell(&cache) - my_start_of_combine_statement) + estimiated_size_of_log_event) > combine_statement_size) my_b_write(&cache,";;",2); - And we should end mysqlbinlog with; if (my_b_tell(&cache) != 0) my_b_write(&cache,";;",2); Note: mysqlbinlog already uses DELIMITER /*!*/; so that it can process "multi-statements" like CREATE PROCEDURE ... BEGIN stmt1; stmt2; ... END what remains to be done is to print the /*!*/; only when we're about to exceed $args[combine-statements] bytes. In all other cases, delimit statements with regular semicolon. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Add an option to mysqlbinlog to produce SQL script with fewer roundtrips (37)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add an option to mysqlbinlog to produce SQL script with fewer roundtrips CREATION DATE..: Fri, 07 Aug 2009, 17:14 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 37 (http://askmonty.org/worklog/?tid=37) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sun, 09 Aug 2009, 12:56)=-=- High-Level Specification modified. --- /tmp/wklog.37.old.22083 2009-08-09 12:56:36.000000000 +0300 +++ /tmp/wklog.37.new.22083 2009-08-09 12:56:36.000000000 +0300 @@ -11,3 +11,16 @@ if (my_b_tell(&cache) != 0) my_b_write(&cache,";;",2); +Note: mysqlbinlog already uses + + DELIMITER /*!*/; + +so that it can process "multi-statements" like + + CREATE PROCEDURE ... BEGIN stmt1; stmt2; ... END + +what remains to be done is to print the /*!*/; only when we're about to exceed +$args[combine-statements] bytes. In all other cases, delimit statements with +regular semicolon. + + -=-=(Psergey - Sun, 09 Aug 2009, 12:30)=-=- High Level Description modified. --- /tmp/wklog.37.old.21090 2009-08-09 12:30:26.000000000 +0300 +++ /tmp/wklog.37.new.21090 2009-08-09 12:30:26.000000000 +0300 @@ -1,6 +1,6 @@ SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot -of roundtrips, and they become a bottleneck. +of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: @@ -14,7 +14,7 @@ loading such sql script will require fewer roundtrips. -The behavior can be controlled using a command line option +The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 37 -=-=(Psergey - Fri, 07 Aug 2009, 17:16)=-=- High-Level Specification modified. --- /tmp/wklog.37.old.20454 2009-08-07 17:16:54.000000000 +0300 +++ /tmp/wklog.37.new.20454 2009-08-07 17:16:54.000000000 +0300 @@ -1 +1,13 @@ +Implementation overview: + +- At start, print "--delimiter=;;" +- Modify the start of each print functions as follows + + if (my_b_tell(&cache) - my_start_of_combine_statement) + + estimiated_size_of_log_event) > combine_statement_size) + my_b_write(&cache,";;",2); + +- And we should end mysqlbinlog with; + if (my_b_tell(&cache) != 0) + my_b_write(&cache,";;",2); DESCRIPTION: SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: +delimiter // binlog statement1; binlog statement2; binlog statement3; +// binlog statement4; loading such sql script will require fewer roundtrips. The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# Where the # is maximum allowed packet length. HIGH-LEVEL SPECIFICATION: Implementation overview: - At start, print "--delimiter=;;" - Modify the start of each print functions as follows if (my_b_tell(&cache) - my_start_of_combine_statement) + estimiated_size_of_log_event) > combine_statement_size) my_b_write(&cache,";;",2); - And we should end mysqlbinlog with; if (my_b_tell(&cache) != 0) my_b_write(&cache,";;",2); Note: mysqlbinlog already uses DELIMITER /*!*/; so that it can process "multi-statements" like CREATE PROCEDURE ... BEGIN stmt1; stmt2; ... END what remains to be done is to print the /*!*/; only when we're about to exceed $args[combine-statements] bytes. In all other cases, delimit statements with regular semicolon. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Add an option to mysqlbinlog to produce SQL script with fewer roundtrips (37)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add an option to mysqlbinlog to produce SQL script with fewer roundtrips CREATION DATE..: Fri, 07 Aug 2009, 17:14 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 37 (http://askmonty.org/worklog/?tid=37) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sun, 09 Aug 2009, 12:30)=-=- High Level Description modified. --- /tmp/wklog.37.old.21090 2009-08-09 12:30:26.000000000 +0300 +++ /tmp/wklog.37.new.21090 2009-08-09 12:30:26.000000000 +0300 @@ -1,6 +1,6 @@ SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot -of roundtrips, and they become a bottleneck. +of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: @@ -14,7 +14,7 @@ loading such sql script will require fewer roundtrips. -The behavior can be controlled using a command line option +The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 37 -=-=(Psergey - Fri, 07 Aug 2009, 17:16)=-=- High-Level Specification modified. --- /tmp/wklog.37.old.20454 2009-08-07 17:16:54.000000000 +0300 +++ /tmp/wklog.37.new.20454 2009-08-07 17:16:54.000000000 +0300 @@ -1 +1,13 @@ +Implementation overview: + +- At start, print "--delimiter=;;" +- Modify the start of each print functions as follows + + if (my_b_tell(&cache) - my_start_of_combine_statement) + + estimiated_size_of_log_event) > combine_statement_size) + my_b_write(&cache,";;",2); + +- And we should end mysqlbinlog with; + if (my_b_tell(&cache) != 0) + my_b_write(&cache,";;",2); DESCRIPTION: SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: +delimiter // binlog statement1; binlog statement2; binlog statement3; +// binlog statement4; loading such sql script will require fewer roundtrips. The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# Where the # is maximum allowed packet length. HIGH-LEVEL SPECIFICATION: Implementation overview: - At start, print "--delimiter=;;" - Modify the start of each print functions as follows if (my_b_tell(&cache) - my_start_of_combine_statement) + estimiated_size_of_log_event) > combine_statement_size) my_b_write(&cache,";;",2); - And we should end mysqlbinlog with; if (my_b_tell(&cache) != 0) my_b_write(&cache,";;",2); ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Add an option to mysqlbinlog to produce SQL script with fewer roundtrips (37)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add an option to mysqlbinlog to produce SQL script with fewer roundtrips CREATION DATE..: Fri, 07 Aug 2009, 17:14 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 37 (http://askmonty.org/worklog/?tid=37) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sun, 09 Aug 2009, 12:30)=-=- High Level Description modified. --- /tmp/wklog.37.old.21090 2009-08-09 12:30:26.000000000 +0300 +++ /tmp/wklog.37.new.21090 2009-08-09 12:30:26.000000000 +0300 @@ -1,6 +1,6 @@ SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot -of roundtrips, and they become a bottleneck. +of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: @@ -14,7 +14,7 @@ loading such sql script will require fewer roundtrips. -The behavior can be controlled using a command line option +The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 37 -=-=(Psergey - Fri, 07 Aug 2009, 17:16)=-=- High-Level Specification modified. --- /tmp/wklog.37.old.20454 2009-08-07 17:16:54.000000000 +0300 +++ /tmp/wklog.37.new.20454 2009-08-07 17:16:54.000000000 +0300 @@ -1 +1,13 @@ +Implementation overview: + +- At start, print "--delimiter=;;" +- Modify the start of each print functions as follows + + if (my_b_tell(&cache) - my_start_of_combine_statement) + + estimiated_size_of_log_event) > combine_statement_size) + my_b_write(&cache,";;",2); + +- And we should end mysqlbinlog with; + if (my_b_tell(&cache) != 0) + my_b_write(&cache,";;",2); DESCRIPTION: SQL scripts generated by mysqlbinlog can be slow to load because they have many small queries, hence applying the script against a remote server requires a lot of roundtrips, and the network roundtrips become the bottleneck. This bottleneck can be addressed by having mysqlbinlog combine multiple statements into one: +delimiter // binlog statement1; binlog statement2; binlog statement3; +// binlog statement4; loading such sql script will require fewer roundtrips. The behaviour can be controlled using a command line option mysqlbinlog --combine-statements=# Where the # is maximum allowed packet length. HIGH-LEVEL SPECIFICATION: Implementation overview: - At start, print "--delimiter=;;" - Modify the start of each print functions as follows if (my_b_tell(&cache) - my_start_of_combine_statement) + estimiated_size_of_log_event) > combine_statement_size) my_b_write(&cache,";;",2); - And we should end mysqlbinlog with; if (my_b_tell(&cache) != 0) my_b_write(&cache,";;",2); ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Replication tasks (39)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Replication tasks CREATION DATE..: Sun, 09 Aug 2009, 12:24 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Client-RawIdeaBin TASK ID........: 39 (http://askmonty.org/worklog/?tid=39) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: A combine task for all replication tasks. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Replication tasks (39)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Replication tasks CREATION DATE..: Sun, 09 Aug 2009, 12:24 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Client-RawIdeaBin TASK ID........: 39 (http://askmonty.org/worklog/?tid=39) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: A combine task for all replication tasks. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Make mysqlbinlog not to output unneeded statements when using --database (38)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Make mysqlbinlog not to output unneeded statements when using --database CREATION DATE..: Sat, 08 Aug 2009, 12:40 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 38 (http://askmonty.org/worklog/?tid=38) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Sun, 09 Aug 2009, 12:22)=-=- High-Level Specification modified. --- /tmp/wklog.38.old.20756 2009-08-09 12:22:52.000000000 +0300 +++ /tmp/wklog.38.new.20756 2009-08-09 12:22:52.000000000 +0300 @@ -1 +1,18 @@ +Monty's suggestions for fix: + +A way to fix this for 'most' cases are: + +If we do filtering (like mysqlbinlog --database='xxx') then: + +- In mysql_bin_log(), do a flush of the Log_event::cache() between + each statement. +- Log on statement. +- If the statement was ignored (we need a flag for this) and + there is something in the cache and the file position didn't change + (the cache didn't overflow), then reset the cache. + +Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single +database is used +- Could be fixed by having a flag to mark if something was printed + to the log since last commit. -=-=(Guest - Sun, 09 Aug 2009, 12:20)=-=- High Level Description modified. --- /tmp/wklog.38.old.20618 2009-08-09 12:20:16.000000000 +0300 +++ /tmp/wklog.38.new.20618 2009-08-09 12:20:16.000000000 +0300 @@ -1,10 +1,24 @@ -This comes from MySQL BUG#23890: +This comes from MySQL BUG#23890 and BUG#23894: when one runs - mysqlbinlog --database=bar N-bin.000003 + mysqlbinlog --database=bar binlog_file -will output all the COMMIT statements in the binary log even if it didn't print -any statements between the COMMITs (because all the statements that were there -were for the other databases). +the produced SQL may contain useless sequences of commands like: -The fix is trivial: in mysqlbinlog, check if we've printed anything after we've -printed the previous commit statement. +COMMIT; +COMMIT; +COMMIT; +... + +or + +SET INSERT_ID=val1; +SET INSERT_ID=val2; +SET INSERT_ID=val3; +... + +This happens because the statements between COMMIT (or SET) statements had no +effect on the specified database and so were filtered out. COMMIT and SET +statements themselves are not associated with any database and were left in. + +Presence of redundant COMMIT or SET statements makes binlog SQL script +unnecessarily big and it will take more client<->server roundtrips to apply it. -=-=(Guest - Sun, 09 Aug 2009, 12:19)=-=- Title modified. --- /tmp/wklog.38.old.20544 2009-08-09 12:19:22.000000000 +0300 +++ /tmp/wklog.38.new.20544 2009-08-09 12:19:22.000000000 +0300 @@ -1 +1 @@ -Make mysqlbinlog not to output unneeded COMMIT statements +Make mysqlbinlog not to output unneeded statements when using --database DESCRIPTION: This comes from MySQL BUG#23890 and BUG#23894: when one runs mysqlbinlog --database=bar binlog_file the produced SQL may contain useless sequences of commands like: COMMIT; COMMIT; COMMIT; ... or SET INSERT_ID=val1; SET INSERT_ID=val2; SET INSERT_ID=val3; ... This happens because the statements between COMMIT (or SET) statements had no effect on the specified database and so were filtered out. COMMIT and SET statements themselves are not associated with any database and were left in. Presence of redundant COMMIT or SET statements makes binlog SQL script unnecessarily big and it will take more client<->server roundtrips to apply it. HIGH-LEVEL SPECIFICATION: Monty's suggestions for fix: A way to fix this for 'most' cases are: If we do filtering (like mysqlbinlog --database='xxx') then: - In mysql_bin_log(), do a flush of the Log_event::cache() between each statement. - Log on statement. - If the statement was ignored (we need a flag for this) and there is something in the cache and the file position didn't change (the cache didn't overflow), then reset the cache. Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single database is used - Could be fixed by having a flag to mark if something was printed to the log since last commit. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Make mysqlbinlog not to output unneeded statements when using --database (38)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Make mysqlbinlog not to output unneeded statements when using --database CREATION DATE..: Sat, 08 Aug 2009, 12:40 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 38 (http://askmonty.org/worklog/?tid=38) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Sun, 09 Aug 2009, 12:22)=-=- High-Level Specification modified. --- /tmp/wklog.38.old.20756 2009-08-09 12:22:52.000000000 +0300 +++ /tmp/wklog.38.new.20756 2009-08-09 12:22:52.000000000 +0300 @@ -1 +1,18 @@ +Monty's suggestions for fix: + +A way to fix this for 'most' cases are: + +If we do filtering (like mysqlbinlog --database='xxx') then: + +- In mysql_bin_log(), do a flush of the Log_event::cache() between + each statement. +- Log on statement. +- If the statement was ignored (we need a flag for this) and + there is something in the cache and the file position didn't change + (the cache didn't overflow), then reset the cache. + +Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single +database is used +- Could be fixed by having a flag to mark if something was printed + to the log since last commit. -=-=(Guest - Sun, 09 Aug 2009, 12:20)=-=- High Level Description modified. --- /tmp/wklog.38.old.20618 2009-08-09 12:20:16.000000000 +0300 +++ /tmp/wklog.38.new.20618 2009-08-09 12:20:16.000000000 +0300 @@ -1,10 +1,24 @@ -This comes from MySQL BUG#23890: +This comes from MySQL BUG#23890 and BUG#23894: when one runs - mysqlbinlog --database=bar N-bin.000003 + mysqlbinlog --database=bar binlog_file -will output all the COMMIT statements in the binary log even if it didn't print -any statements between the COMMITs (because all the statements that were there -were for the other databases). +the produced SQL may contain useless sequences of commands like: -The fix is trivial: in mysqlbinlog, check if we've printed anything after we've -printed the previous commit statement. +COMMIT; +COMMIT; +COMMIT; +... + +or + +SET INSERT_ID=val1; +SET INSERT_ID=val2; +SET INSERT_ID=val3; +... + +This happens because the statements between COMMIT (or SET) statements had no +effect on the specified database and so were filtered out. COMMIT and SET +statements themselves are not associated with any database and were left in. + +Presence of redundant COMMIT or SET statements makes binlog SQL script +unnecessarily big and it will take more client<->server roundtrips to apply it. -=-=(Guest - Sun, 09 Aug 2009, 12:19)=-=- Title modified. --- /tmp/wklog.38.old.20544 2009-08-09 12:19:22.000000000 +0300 +++ /tmp/wklog.38.new.20544 2009-08-09 12:19:22.000000000 +0300 @@ -1 +1 @@ -Make mysqlbinlog not to output unneeded COMMIT statements +Make mysqlbinlog not to output unneeded statements when using --database DESCRIPTION: This comes from MySQL BUG#23890 and BUG#23894: when one runs mysqlbinlog --database=bar binlog_file the produced SQL may contain useless sequences of commands like: COMMIT; COMMIT; COMMIT; ... or SET INSERT_ID=val1; SET INSERT_ID=val2; SET INSERT_ID=val3; ... This happens because the statements between COMMIT (or SET) statements had no effect on the specified database and so were filtered out. COMMIT and SET statements themselves are not associated with any database and were left in. Presence of redundant COMMIT or SET statements makes binlog SQL script unnecessarily big and it will take more client<->server roundtrips to apply it. HIGH-LEVEL SPECIFICATION: Monty's suggestions for fix: A way to fix this for 'most' cases are: If we do filtering (like mysqlbinlog --database='xxx') then: - In mysql_bin_log(), do a flush of the Log_event::cache() between each statement. - Log on statement. - If the statement was ignored (we need a flag for this) and there is something in the cache and the file position didn't change (the cache didn't overflow), then reset the cache. Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single database is used - Could be fixed by having a flag to mark if something was printed to the log since last commit. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Make mysqlbinlog not to output unneeded statements when using --database (38)
by worklog-noreply＠askmonty.org 09 Aug '09

09 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Make mysqlbinlog not to output unneeded statements when using --database CREATION DATE..: Sat, 08 Aug 2009, 12:40 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 38 (http://askmonty.org/worklog/?tid=38) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Sun, 09 Aug 2009, 12:20)=-=- High Level Description modified. --- /tmp/wklog.38.old.20618 2009-08-09 12:20:16.000000000 +0300 +++ /tmp/wklog.38.new.20618 2009-08-09 12:20:16.000000000 +0300 @@ -1,10 +1,24 @@ -This comes from MySQL BUG#23890: +This comes from MySQL BUG#23890 and BUG#23894: when one runs - mysqlbinlog --database=bar N-bin.000003 + mysqlbinlog --database=bar binlog_file -will output all the COMMIT statements in the binary log even if it didn't print -any statements between the COMMITs (because all the statements that were there -were for the other databases). +the produced SQL may contain useless sequences of commands like: -The fix is trivial: in mysqlbinlog, check if we've printed anything after we've -printed the previous commit statement. +COMMIT; +COMMIT; +COMMIT; +... + +or + +SET INSERT_ID=val1; +SET INSERT_ID=val2; +SET INSERT_ID=val3; +... + +This happens because the statements between COMMIT (or SET) statements had no +effect on the specified database and so were filtered out. COMMIT and SET +statements themselves are not associated with any database and were left in. + +Presence of redundant COMMIT or SET statements makes binlog SQL script +unnecessarily big and it will take more client<->server roundtrips to apply it. -=-=(Guest - Sun, 09 Aug 2009, 12:19)=-=- Title modified. --- /tmp/wklog.38.old.20544 2009-08-09 12:19:22.000000000 +0300 +++ /tmp/wklog.38.new.20544 2009-08-09 12:19:22.000000000 +0300 @@ -1 +1 @@ -Make mysqlbinlog not to output unneeded COMMIT statements +Make mysqlbinlog not to output unneeded statements when using --database DESCRIPTION: This comes from MySQL BUG#23890 and BUG#23894: when one runs mysqlbinlog --database=bar binlog_file the produced SQL may contain useless sequences of commands like: COMMIT; COMMIT; COMMIT; ... or SET INSERT_ID=val1; SET INSERT_ID=val2; SET INSERT_ID=val3; ... This happens because the statements between COMMIT (or SET) statements had no effect on the specified database and so were filtered out. COMMIT and SET statements themselves are not associated with any database and were left in. Presence of redundant COMMIT or SET statements makes binlog SQL script unnecessarily big and it will take more client<->server roundtrips to apply it. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0