- developers - lists.mariadb.org

[Maria-developers] New (by Psergey): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Change BINLOG statement syntax to be human-readable (46)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Change BINLOG statement syntax to be human-readable CREATION DATE..: Sat, 15 Aug 2009, 23:42 SUPERVISOR.....: Monty IMPLEMENTOR....: Psergey COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 46 (http://askmonty.org/worklog/?tid=46) VERSION........: WorkLog-3.4 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sat, 15 Aug 2009, 23:43)=-=- High-Level Specification modified. --- /tmp/wklog.46.old.17742 2009-08-15 23:43:09.000000000 +0300 +++ /tmp/wklog.46.new.17742 2009-08-15 23:43:09.000000000 +0300 @@ -1 +1,28 @@ +Suggestion 1 +------------ +Original syntax suggestion by Kristian: + + BINLOG + WITH TIMESTAMP xxx SERVER_ID 1 MASTER_POS 415 FLAGS 0x0 + TABLE db1.table1 AS 1 COLUMNS (INT NOT NULL, BLOB, VARCHAR(100)) FLAGS 0x0 + TABLE db2.table2 AS 2 COLUMNS (CHAR(10)) FLAGS 0x0 + WRITE_ROW INTO db1.table1(1,3) VALUES (42, 'foobar'), (10, NULL) FLAGS 0x2 + UPDATE_ROW INTO db2.table2 (1) (1) VALUES FROM ('beforeval') TO ('toval'), + FROM ('a') TO ('b') FLAGS 0x0 + DELETE_ROW INTO db2.table2 (1) VALUES ('row_to_delete') FLAGS 0x0; + + This is basically a dump of what is stored in the events, and would be an + alternative to BINLOG 'gwWEShMBAA...'. + +Feedback and other suggestions +------------------------------ +* What is the need for WITH TIMESTAMP part? Can't one use a separate + SET TIMESTAMP statement? + +* mysqlbinlog --base64-output=DECODE-ROWS --verbose already produces something + that's close to readable SQL. Can we make it to be regular parseable SQL? + + This will be syntax that's familiar to our parser and to the users + - A stream of SQL statements will be slower to run than BINLOG statements + (due to locking, table open/close, etc). (TODO: is it really slower? we + haven't checked). DESCRIPTION: One of great things about mysqlbinlog was that its output was human-readable SQL, so it was possible to edit it manually or with help of scripts. With RBR events and BINLOG 'DpiGShMBAAAALQAAADcBAA...' statements this is no longer the case. This WL task is about making BINLOG statements to be human-readable (either as an option or by default HIGH-LEVEL SPECIFICATION: Suggestion 1 ------------ Original syntax suggestion by Kristian: BINLOG WITH TIMESTAMP xxx SERVER_ID 1 MASTER_POS 415 FLAGS 0x0 TABLE db1.table1 AS 1 COLUMNS (INT NOT NULL, BLOB, VARCHAR(100)) FLAGS 0x0 TABLE db2.table2 AS 2 COLUMNS (CHAR(10)) FLAGS 0x0 WRITE_ROW INTO db1.table1(1,3) VALUES (42, 'foobar'), (10, NULL) FLAGS 0x2 UPDATE_ROW INTO db2.table2 (1) (1) VALUES FROM ('beforeval') TO ('toval'), FROM ('a') TO ('b') FLAGS 0x0 DELETE_ROW INTO db2.table2 (1) VALUES ('row_to_delete') FLAGS 0x0; This is basically a dump of what is stored in the events, and would be an alternative to BINLOG 'gwWEShMBAA...'. Feedback and other suggestions ------------------------------ * What is the need for WITH TIMESTAMP part? Can't one use a separate SET TIMESTAMP statement? * mysqlbinlog --base64-output=DECODE-ROWS --verbose already produces something that's close to readable SQL. Can we make it to be regular parseable SQL? + This will be syntax that's familiar to our parser and to the users - A stream of SQL statements will be slower to run than BINLOG statements (due to locking, table open/close, etc). (TODO: is it really slower? we haven't checked). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Change BINLOG statement syntax to be human-readable (46)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Change BINLOG statement syntax to be human-readable CREATION DATE..: Sat, 15 Aug 2009, 23:42 SUPERVISOR.....: Monty IMPLEMENTOR....: Psergey COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 46 (http://askmonty.org/worklog/?tid=46) VERSION........: WorkLog-3.4 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sat, 15 Aug 2009, 23:43)=-=- High-Level Specification modified. --- /tmp/wklog.46.old.17742 2009-08-15 23:43:09.000000000 +0300 +++ /tmp/wklog.46.new.17742 2009-08-15 23:43:09.000000000 +0300 @@ -1 +1,28 @@ +Suggestion 1 +------------ +Original syntax suggestion by Kristian: + + BINLOG + WITH TIMESTAMP xxx SERVER_ID 1 MASTER_POS 415 FLAGS 0x0 + TABLE db1.table1 AS 1 COLUMNS (INT NOT NULL, BLOB, VARCHAR(100)) FLAGS 0x0 + TABLE db2.table2 AS 2 COLUMNS (CHAR(10)) FLAGS 0x0 + WRITE_ROW INTO db1.table1(1,3) VALUES (42, 'foobar'), (10, NULL) FLAGS 0x2 + UPDATE_ROW INTO db2.table2 (1) (1) VALUES FROM ('beforeval') TO ('toval'), + FROM ('a') TO ('b') FLAGS 0x0 + DELETE_ROW INTO db2.table2 (1) VALUES ('row_to_delete') FLAGS 0x0; + + This is basically a dump of what is stored in the events, and would be an + alternative to BINLOG 'gwWEShMBAA...'. + +Feedback and other suggestions +------------------------------ +* What is the need for WITH TIMESTAMP part? Can't one use a separate + SET TIMESTAMP statement? + +* mysqlbinlog --base64-output=DECODE-ROWS --verbose already produces something + that's close to readable SQL. Can we make it to be regular parseable SQL? + + This will be syntax that's familiar to our parser and to the users + - A stream of SQL statements will be slower to run than BINLOG statements + (due to locking, table open/close, etc). (TODO: is it really slower? we + haven't checked). DESCRIPTION: One of great things about mysqlbinlog was that its output was human-readable SQL, so it was possible to edit it manually or with help of scripts. With RBR events and BINLOG 'DpiGShMBAAAALQAAADcBAA...' statements this is no longer the case. This WL task is about making BINLOG statements to be human-readable (either as an option or by default HIGH-LEVEL SPECIFICATION: Suggestion 1 ------------ Original syntax suggestion by Kristian: BINLOG WITH TIMESTAMP xxx SERVER_ID 1 MASTER_POS 415 FLAGS 0x0 TABLE db1.table1 AS 1 COLUMNS (INT NOT NULL, BLOB, VARCHAR(100)) FLAGS 0x0 TABLE db2.table2 AS 2 COLUMNS (CHAR(10)) FLAGS 0x0 WRITE_ROW INTO db1.table1(1,3) VALUES (42, 'foobar'), (10, NULL) FLAGS 0x2 UPDATE_ROW INTO db2.table2 (1) (1) VALUES FROM ('beforeval') TO ('toval'), FROM ('a') TO ('b') FLAGS 0x0 DELETE_ROW INTO db2.table2 (1) VALUES ('row_to_delete') FLAGS 0x0; This is basically a dump of what is stored in the events, and would be an alternative to BINLOG 'gwWEShMBAA...'. Feedback and other suggestions ------------------------------ * What is the need for WITH TIMESTAMP part? Can't one use a separate SET TIMESTAMP statement? * mysqlbinlog --base64-output=DECODE-ROWS --verbose already produces something that's close to readable SQL. Can we make it to be regular parseable SQL? + This will be syntax that's familiar to our parser and to the users - A stream of SQL statements will be slower to run than BINLOG statements (due to locking, table open/close, etc). (TODO: is it really slower? we haven't checked). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Change BINLOG statement syntax to be human-readable (46)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Change BINLOG statement syntax to be human-readable CREATION DATE..: Sat, 15 Aug 2009, 23:42 SUPERVISOR.....: Monty IMPLEMENTOR....: Psergey COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 46 (http://askmonty.org/worklog/?tid=46) VERSION........: WorkLog-3.4 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: One of great things about mysqlbinlog was that its output was human-readable SQL, so it was possible to edit it manually or with help of scripts. With RBR events and BINLOG 'DpiGShMBAAAALQAAADcBAA...' statements this is no longer the case. This WL task is about making BINLOG statements to be human-readable (either as an option or by default ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Change BINLOG statement syntax to be human-readable (46)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Change BINLOG statement syntax to be human-readable CREATION DATE..: Sat, 15 Aug 2009, 23:42 SUPERVISOR.....: Monty IMPLEMENTOR....: Psergey COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 46 (http://askmonty.org/worklog/?tid=46) VERSION........: WorkLog-3.4 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: One of great things about mysqlbinlog was that its output was human-readable SQL, so it was possible to edit it manually or with help of scripts. With RBR events and BINLOG 'DpiGShMBAAAALQAAADcBAA...' statements this is no longer the case. This WL task is about making BINLOG statements to be human-readable (either as an option or by default ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Add a mysqlbinlog option to produce succint output (45)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add a mysqlbinlog option to produce succint output CREATION DATE..: Sat, 15 Aug 2009, 23:40 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 45 (http://askmonty.org/worklog/?tid=45) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sat, 15 Aug 2009, 23:40)=-=- Title modified. --- /tmp/wklog.45.old.17603 2009-08-15 23:40:38.000000000 +0300 +++ /tmp/wklog.45.new.17603 2009-08-15 23:40:38.000000000 +0300 @@ -1 +1 @@ -Add a mysqlbinlog option to produce siccint output +Add a mysqlbinlog option to produce succint output DESCRIPTION: Add a mysqlbinlog option to produce the most succinct output, without any comments or other statements that are not needed to apply binlog correctly. This will be different from --short-form option. That option causes mysqlbinlog not to print RBR events, i.e. the output is not supposed to be applied. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Add a mysqlbinlog option to produce succint output (45)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add a mysqlbinlog option to produce succint output CREATION DATE..: Sat, 15 Aug 2009, 23:40 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 45 (http://askmonty.org/worklog/?tid=45) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Sat, 15 Aug 2009, 23:40)=-=- Title modified. --- /tmp/wklog.45.old.17603 2009-08-15 23:40:38.000000000 +0300 +++ /tmp/wklog.45.new.17603 2009-08-15 23:40:38.000000000 +0300 @@ -1 +1 @@ -Add a mysqlbinlog option to produce siccint output +Add a mysqlbinlog option to produce succint output DESCRIPTION: Add a mysqlbinlog option to produce the most succinct output, without any comments or other statements that are not needed to apply binlog correctly. This will be different from --short-form option. That option causes mysqlbinlog not to print RBR events, i.e. the output is not supposed to be applied. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Add a mysqlbinlog option to produce siccint output (45)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add a mysqlbinlog option to produce siccint output CREATION DATE..: Sat, 15 Aug 2009, 23:40 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 45 (http://askmonty.org/worklog/?tid=45) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Add a mysqlbinlog option to produce the most succinct output, without any comments or other statements that are not needed to apply binlog correctly. This will be different from --short-form option. That option causes mysqlbinlog not to print RBR events, i.e. the output is not supposed to be applied. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Add a mysqlbinlog option to produce siccint output (45)
by worklog-noreply＠askmonty.org 15 Aug '09

15 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add a mysqlbinlog option to produce siccint output CREATION DATE..: Sat, 15 Aug 2009, 23:40 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 45 (http://askmonty.org/worklog/?tid=45) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Add a mysqlbinlog option to produce the most succinct output, without any comments or other statements that are not needed to apply binlog correctly. This will be different from --short-form option. That option causes mysqlbinlog not to print RBR events, i.e. the output is not supposed to be applied. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 2724: MWL#17: Table elimination in file:///home/psergey/dev/maria-5.1-table-elim-r10/
by Sergey Petrunya 15 Aug '09

15 Aug '09

At file:///home/psergey/dev/maria-5.1-table-elim-r10/ ------------------------------------------------------------ revno: 2724 revision-id: psergey(a)askmonty.org-20090815153912-q47vfp1j22ilmup2 parent: psergey(a)askmonty.org-20090815121442-706m9ujn8km4u4y1 committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.1-table-elim-r10 timestamp: Sat 2009-08-15 18:39:12 +0300 message: MWL#17: Table elimination - Review feedback, more variable renames === modified file 'sql/opt_table_elimination.cc' --- a/sql/opt_table_elimination.cc 2009-08-15 12:14:42 +0000 +++ b/sql/opt_table_elimination.cc 2009-08-15 15:39:12 +0000 @@ -114,7 +114,6 @@ MODULE_EXPRESSION, MODULE_MULTI_EQUALITY, MODULE_UNIQUE_KEY, - MODULE_TABLE, MODULE_OUTER_JOIN } type; /* Type of the object */ @@ -138,7 +137,7 @@ { public: Field_value *field; - Item *val; + Item *expression; /* Used during condition analysis only, similar to KEYUSE::level */ uint level; @@ -510,18 +509,18 @@ */ if (old->field == new_fields->field) { - if (!new_fields->val->const_item()) + if (!new_fields->expression->const_item()) { /* If the value matches, we can use the key reference. If not, we keep it until we have examined all new values */ - if (old->val->eq(new_fields->val, old->field->field->binary())) + if (old->expression->eq(new_fields->expression, old->field->field->binary())) { old->level= and_level; } } - else if (old->val->eq_by_collation(new_fields->val, + else if (old->expression->eq_by_collation(new_fields->expression, old->field->field->binary(), old->field->field->charset())) { @@ -633,7 +632,7 @@ /* Store possible eq field */ (*eq_dep)->type= Module_dep::MODULE_EXPRESSION; //psergey-todo; (*eq_dep)->field= get_field_value(te, field); - (*eq_dep)->val= *value; + (*eq_dep)->expression= *value; (*eq_dep)->level= and_level; (*eq_dep)++; } @@ -953,7 +952,7 @@ { deps_setter.expr_offset= eq_dep - te->equality_deps; eq_dep->unknown_args= 0; - eq_dep->val->walk(&Item::check_column_usage_processor, FALSE, + eq_dep->expression->walk(&Item::check_column_usage_processor, FALSE, (uchar*)&deps_setter); if (!eq_dep->unknown_args) { @@ -1283,7 +1282,7 @@ char buf[128]; String str(buf, sizeof(buf), &my_charset_bin); str.length(0); - eq_dep->val->print(&str, QT_ORDINARY); + eq_dep->expression->print(&str, QT_ORDINARY); fprintf(DBUG_FILE, " equality%d: %s -> %s.%s\n", eq_dep - te->equality_deps, str.c_ptr(),

1 0