[Maria-developers] Please check optimiser regression in MySQL 5.1.46
Hi Igor, Timour, Sergey, Can one of you please check this patch from MySQL 5.1.46? It is a commit to fix http://bugs.mysql.com/bug.php?id=51494 This patch introduces a regression: http://bugs.mysql.com/bug.php?id=53334 If I revert the patch, the regression disappears. And interestingly, the included test case does not fail even when the fix is reverted (go figure...) So it would be good if one of you could check the patch and check what is wrong with it, and if a different fix for Bug#51494 is needed. (needed to complete merge of MySQL 5.1.46). Thanks, - Kristian. ------------------------------------------------------------ revno: 3407.1.1 revision-id: sergey.glukhov@sun.com-20100319060102-57ykzjf4pc93avy1 parent: omer@mysql.com-20100318064207-l3ap0mpxt510b4n3 committer: Sergey Glukhov <Sergey.Glukhov@sun.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2010-03-19 10:01:02 +0400 message: Bug#51494 crash with join, explain and 'sounds like' operator The crash happens because of discrepancy between values of conts_tables and join->const_table_map(make_join_statisctics). Calculation of conts_tables used condition with HA_STATS_RECORDS_IS_EXACT flag check. Calculation of join->const_table_map does not use this flag check. In case of MERGE table without union with index the table does not become const table and thus join_read_const_table() is not called for the table. join->const_table_map supposes this table is const and later in make_join_select this table is used for making&calculation const condition. As table record buffer is not populated it leads to crash. The fix is adding a check if an engine supports HA_STATS_RECORDS_IS_EXACT flag before updating join->const_table_map. diff: === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2010-03-14 16:01:45 +0000 +++ sql/sql_select.cc 2010-03-19 06:01:02 +0000 @@ -2943,7 +2943,8 @@ s->quick=select->quick; s->needed_reg=select->needed_reg; select->quick=0; - if (records == 0 && s->table->reginfo.impossible_range) + if (records == 0 && s->table->reginfo.impossible_range && + (s->table->file->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT)) { /* Impossible WHERE or ON expression === modified file 'mysql-test/r/merge.result' --- mysql-test/r/merge.result 2010-03-03 10:49:03 +0000 +++ mysql-test/r/merge.result 2010-03-19 06:01:02 +0000 @@ -2286,4 +2286,16 @@ DROP TABLE m1; DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`; +# +# Bug#51494c rash with join, explain and 'sounds like' operator +# +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables +DROP TABLE t2, t1; End of 5.1 tests === modified file 'mysql-test/t/merge.test' --- mysql-test/t/merge.test 2010-03-03 10:49:03 +0000 +++ mysql-test/t/merge.test 2010-03-19 06:01:02 +0000 @@ -1690,4 +1690,19 @@ DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`; +--echo # +--echo # Bug#51494c rash with join, explain and 'sounds like' operator +--echo # + +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); + +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; + +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; + +DROP TABLE t2, t1; + --echo End of 5.1 tests
Hi! Kristian Nielsen wrote:
Hi Igor, Timour, Sergey,
Can one of you please check this patch from MySQL 5.1.46? It is a commit to fix http://bugs.mysql.com/bug.php?id=51494
This patch introduces a regression:
http://bugs.mysql.com/bug.php?id=53334
If I revert the patch, the regression disappears. And interestingly, the included test case does not fail even when the fix is reverted (go figure...)
So it would be good if one of you could check the patch and check what is wrong with it, and if a different fix for Bug#51494 is needed.
(needed to complete merge of MySQL 5.1.46).
Ok, I will take care of it. Regards, Igor.
Thanks,
- Kristian.
------------------------------------------------------------ revno: 3407.1.1 revision-id: sergey.glukhov@sun.com-20100319060102-57ykzjf4pc93avy1 parent: omer@mysql.com-20100318064207-l3ap0mpxt510b4n3 committer: Sergey Glukhov <Sergey.Glukhov@sun.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2010-03-19 10:01:02 +0400 message: Bug#51494 crash with join, explain and 'sounds like' operator The crash happens because of discrepancy between values of conts_tables and join->const_table_map(make_join_statisctics). Calculation of conts_tables used condition with HA_STATS_RECORDS_IS_EXACT flag check. Calculation of join->const_table_map does not use this flag check. In case of MERGE table without union with index the table does not become const table and thus join_read_const_table() is not called for the table. join->const_table_map supposes this table is const and later in make_join_select this table is used for making&calculation const condition. As table record buffer is not populated it leads to crash. The fix is adding a check if an engine supports HA_STATS_RECORDS_IS_EXACT flag before updating join->const_table_map. diff: === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2010-03-14 16:01:45 +0000 +++ sql/sql_select.cc 2010-03-19 06:01:02 +0000 @@ -2943,7 +2943,8 @@ s->quick=select->quick; s->needed_reg=select->needed_reg; select->quick=0; - if (records == 0 && s->table->reginfo.impossible_range) + if (records == 0 && s->table->reginfo.impossible_range && + (s->table->file->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT)) { /* Impossible WHERE or ON expression === modified file 'mysql-test/r/merge.result' --- mysql-test/r/merge.result 2010-03-03 10:49:03 +0000 +++ mysql-test/r/merge.result 2010-03-19 06:01:02 +0000 @@ -2286,4 +2286,16 @@ DROP TABLE m1; DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`; +# +# Bug#51494c rash with join, explain and 'sounds like' operator +# +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables +DROP TABLE t2, t1; End of 5.1 tests
=== modified file 'mysql-test/t/merge.test' --- mysql-test/t/merge.test 2010-03-03 10:49:03 +0000 +++ mysql-test/t/merge.test 2010-03-19 06:01:02 +0000 @@ -1690,4 +1690,19 @@ DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`;
+--echo # +--echo # Bug#51494c rash with join, explain and 'sounds like' operator +--echo # + +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); + +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; + +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; + +DROP TABLE t2, t1; + --echo End of 5.1 tests
Kristian, Below you'll find a patch for bug #53334 (against mysql-5.1.46). I investigated other latest fixes pushed by Sergey Glukhov. The fix for bugs ##51242/52336 looked suspicious for me. And it actually provides a work-around rather than fixes the following bug that causes both failures: In the function JOIN::exec there should be Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, used_tables, (table_map) 0); instead of Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, used_tables, used_tables); Regards, Igor. 3447 Igor Babaev 2010-05-02 Fixed bug #53334. The fix actually reverts the change introduced by the patch for bug 51494 The fact is that the patch for bug 52177 fixes bug 51194 as well. modified: mysql-test/r/innodb_mysql.result mysql-test/t/innodb_mysql.test sql/sql_select.cc === modified file 'mysql-test/r/innodb_mysql.result' --- a/mysql-test/r/innodb_mysql.result 2010-03-17 14:18:46 +0000 +++ b/mysql-test/r/innodb_mysql.result 2010-05-03 04:22:37 +0000 @@ -2350,4 +2350,34 @@ Null Index_type BTREE Comment DROP TABLE t1; +# +# Bug #53334: wrong result for outer join with impossible ON condition +# (see the same test case for MyISAM in join.test) +# +create table t1 (id int primary key); +create table t2 (id int); +insert into t1 values (75); +insert into t1 values (79); +insert into t1 values (78); +insert into t1 values (77); +replace into t1 values (76); +replace into t1 values (76); +insert into t1 values (104); +insert into t1 values (103); +insert into t1 values (102); +insert into t1 values (101); +insert into t1 values (105); +insert into t1 values (106); +insert into t1 values (107); +insert into t2 values (107),(75),(1000); +select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 +where t2.id=75 and t1.id is null; +id id +NULL 75 +explain select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 +where t2.id=75 and t1.id is null; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 const PRIMARY NULL NULL NULL 1 Impossible ON condition +1 SIMPLE t2 ALL NULL NULL NULL NULL 3 Using where +drop table t1,t2; End of 5.1 tests === modified file 'mysql-test/t/innodb_mysql.test' --- a/mysql-test/t/innodb_mysql.test 2010-03-17 14:18:46 +0000 +++ b/mysql-test/t/innodb_mysql.test 2010-05-03 04:22:37 +0000 @@ -589,4 +589,35 @@ ALTER TABLE t1 DROP INDEX k, ADD UNIQUE DROP TABLE t1; +--echo # +--echo # Bug #53334: wrong result for outer join with impossible ON condition +--echo # (see the same test case for MyISAM in join.test) +--echo # + +create table t1 (id int primary key); +create table t2 (id int); + +insert into t1 values (75); +insert into t1 values (79); +insert into t1 values (78); +insert into t1 values (77); +replace into t1 values (76); +replace into t1 values (76); +insert into t1 values (104); +insert into t1 values (103); +insert into t1 values (102); +insert into t1 values (101); +insert into t1 values (105); +insert into t1 values (106); +insert into t1 values (107); + +insert into t2 values (107),(75),(1000); + +select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 + where t2.id=75 and t1.id is null; +explain select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 + where t2.id=75 and t1.id is null; + +drop table t1,t2; + --echo End of 5.1 tests === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-04-12 10:12:20 +0000 +++ b/sql/sql_select.cc 2010-05-03 04:22:37 +0000 @@ -2968,8 +2968,7 @@ make_join_statistics(JOIN *join, TABLE_L s->quick=select->quick; s->needed_reg=select->needed_reg; select->quick=0; - if (records == 0 && s->table->reginfo.impossible_range && - (s->table->file->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT)) + if (records == 0 && s->table->reginfo.impossible_range) { /* Impossible WHERE or ON expression Kristian Nielsen wrote:
Hi Igor, Timour, Sergey,
Can one of you please check this patch from MySQL 5.1.46? It is a commit to fix http://bugs.mysql.com/bug.php?id=51494
This patch introduces a regression:
http://bugs.mysql.com/bug.php?id=53334
If I revert the patch, the regression disappears. And interestingly, the included test case does not fail even when the fix is reverted (go figure...)
So it would be good if one of you could check the patch and check what is wrong with it, and if a different fix for Bug#51494 is needed.
(needed to complete merge of MySQL 5.1.46).
Thanks,
- Kristian.
------------------------------------------------------------ revno: 3407.1.1 revision-id: sergey.glukhov@sun.com-20100319060102-57ykzjf4pc93avy1 parent: omer@mysql.com-20100318064207-l3ap0mpxt510b4n3 committer: Sergey Glukhov <Sergey.Glukhov@sun.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2010-03-19 10:01:02 +0400 message: Bug#51494 crash with join, explain and 'sounds like' operator The crash happens because of discrepancy between values of conts_tables and join->const_table_map(make_join_statisctics). Calculation of conts_tables used condition with HA_STATS_RECORDS_IS_EXACT flag check. Calculation of join->const_table_map does not use this flag check. In case of MERGE table without union with index the table does not become const table and thus join_read_const_table() is not called for the table. join->const_table_map supposes this table is const and later in make_join_select this table is used for making&calculation const condition. As table record buffer is not populated it leads to crash. The fix is adding a check if an engine supports HA_STATS_RECORDS_IS_EXACT flag before updating join->const_table_map. diff: === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2010-03-14 16:01:45 +0000 +++ sql/sql_select.cc 2010-03-19 06:01:02 +0000 @@ -2943,7 +2943,8 @@ s->quick=select->quick; s->needed_reg=select->needed_reg; select->quick=0; - if (records == 0 && s->table->reginfo.impossible_range) + if (records == 0 && s->table->reginfo.impossible_range && + (s->table->file->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT)) { /* Impossible WHERE or ON expression === modified file 'mysql-test/r/merge.result' --- mysql-test/r/merge.result 2010-03-03 10:49:03 +0000 +++ mysql-test/r/merge.result 2010-03-19 06:01:02 +0000 @@ -2286,4 +2286,16 @@ DROP TABLE m1; DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`; +# +# Bug#51494c rash with join, explain and 'sounds like' operator +# +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables +DROP TABLE t2, t1; End of 5.1 tests
=== modified file 'mysql-test/t/merge.test' --- mysql-test/t/merge.test 2010-03-03 10:49:03 +0000 +++ mysql-test/t/merge.test 2010-03-19 06:01:02 +0000 @@ -1690,4 +1690,19 @@ DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`;
+--echo # +--echo # Bug#51494c rash with join, explain and 'sounds like' operator +--echo # + +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); + +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; + +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; + +DROP TABLE t2, t1; + --echo End of 5.1 tests
Igor Babaev <igor@askmonty.org> writes:
Below you'll find a patch for bug #53334 (against mysql-5.1.46).
Thanks! I pushed it into 5.1-release (to be included in upcoming MariaDB 5.1.46).
I investigated other latest fixes pushed by Sergey Glukhov.
The fix for bugs ##51242/52336 looked suspicious for me. And it actually provides a work-around rather than fixes the following bug that causes both failures:
In the function JOIN::exec there should be Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, used_tables, (table_map) 0); instead of Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, used_tables, used_tables);
Very good that you could look into this. So if I understand you correctly, you suggest to revert Sergey Glukhov's fixes for bugs ##51242/52336, and instead apply your suggested fix above? I tried this (patch included below), reverting the code changes but leaving the new test cases in place. The test suite passes with this change, but with the following explain plan change (the change for Note 1003 seen in the patch is just reverting Sergey Glukhov's patches): CREATE TABLE t1 (f1 INT PRIMARY KEY, f2 INT, f3 INT); INSERT INTO t1 VALUES (2,7,9), (4,7,9), (6,2,9), (17,0,9); EXPLAIN EXTENDED SELECT table1.f1, table2.f2 FROM t1 AS table1 JOIN t1 AS table2 ON table1.f3 = table2.f3 WHERE table2.f1 = 2 GROUP BY table1.f1, table2.f2 HAVING (table2.f2 = 8 AND table1.f1 >= 6); id select_type table type possible_keys key key_len ref rows filtered Extra -1 SIMPLE NULL NULL NULL NULL NULL NULL NULL NULL Impossible HAVING noticed after reading const tables +1 SIMPLE table2 const PRIMARY PRIMARY 4 const 1 100.00 Using filesort +1 SIMPLE table1 ALL NULL NULL NULL NULL 4 100.00 Using where Please let me know if you think I should push this patch. - Kristian. ----------------------------------------------------------------------- === modified file 'mysql-test/r/having.result' --- mysql-test/r/having.result 2010-04-05 11:10:26 +0000 +++ mysql-test/r/having.result 2010-05-03 12:33:32 +0000 @@ -470,9 +470,10 @@ WHERE table2.f1 = 2 GROUP BY table1.f1, table2.f2 HAVING (table2.f2 = 8 AND table1.f1 >= 6); id select_type table type possible_keys key key_len ref rows filtered Extra -1 SIMPLE NULL NULL NULL NULL NULL NULL NULL NULL Impossible HAVING noticed after reading const tables +1 SIMPLE table2 const PRIMARY PRIMARY 4 const 1 100.00 Using filesort +1 SIMPLE table1 ALL NULL NULL NULL NULL 4 100.00 Using where Warnings: -Note 1003 select `test`.`table1`.`f1` AS `f1`,'7' AS `f2` from `test`.`t1` `table1` join `test`.`t1` `table2` where ((`test`.`table1`.`f3` = '9')) group by `test`.`table1`.`f1`,'7' having 0 +Note 1003 select `test`.`table1`.`f1` AS `f1`,'7' AS `f2` from `test`.`t1` `table1` join `test`.`t1` `table2` where ((`test`.`table1`.`f3` = '9')) group by `test`.`table1`.`f1`,'7' having (('7' = 8) and (`test`.`table1`.`f1` >= 6)) EXPLAIN EXTENDED SELECT table1.f1, table2.f2 FROM t1 AS table1 @@ -481,9 +482,10 @@ WHERE table2.f1 = 2 GROUP BY table1.f1, table2.f2 HAVING (table2.f2 = 8); id select_type table type possible_keys key key_len ref rows filtered Extra -1 SIMPLE NULL NULL NULL NULL NULL NULL NULL NULL Impossible HAVING noticed after reading const tables +1 SIMPLE table2 const PRIMARY PRIMARY 4 const 1 100.00 Using filesort +1 SIMPLE table1 ALL NULL NULL NULL NULL 4 100.00 Using where Warnings: -Note 1003 select `test`.`table1`.`f1` AS `f1`,'7' AS `f2` from `test`.`t1` `table1` join `test`.`t1` `table2` where ((`test`.`table1`.`f3` = '9')) group by `test`.`table1`.`f1`,'7' having 0 +Note 1003 select `test`.`table1`.`f1` AS `f1`,'7' AS `f2` from `test`.`t1` `table1` join `test`.`t1` `table2` where ((`test`.`table1`.`f3` = '9')) group by `test`.`table1`.`f1`,'7' having ('7' = 8) DROP TABLE t1; # # Bug#52336 Segfault / crash in 5.1 copy_fields (param=0x9872980) at sql_select.cc:15355 === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2010-05-03 08:44:39 +0000 +++ sql/sql_select.cc 2010-05-03 12:29:54 +0000 @@ -1122,31 +1122,6 @@ JOIN::optimize() { conds=new Item_int((longlong) 0,1); // Always false } - - /* - It's necessary to check const part of HAVING cond as - there is a chance that some cond parts may become - const items after make_join_statisctics(for example - when Item is a reference to cost table field from - outer join). - This check is performed only for those conditions - which do not use aggregate functions. In such case - temporary table may not be used and const condition - elements may be lost during further having - condition transformation in JOIN::exec. - */ - if (having && const_table_map) - { - having->update_used_tables(); - having= remove_eq_conds(thd, having, &having_value); - if (having_value == Item::COND_FALSE) - { - having= new Item_int((longlong) 0,1); - zero_result_cause= "Impossible HAVING noticed after reading const tables"; - DBUG_RETURN(0); - } - } - if (make_join_select(this, select, conds)) { zero_result_cause= @@ -2209,7 +2184,7 @@ JOIN::exec() Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, used_tables, - used_tables); + (table_map) 0); if (sort_table_cond) { if (!curr_table->select)
participants (2)
-
Igor Babaev -
Kristian Nielsen