Re: [Maria-developers] [Merge] lp:~paul-mccullagh/maria/pbxt-1.0.11 into lp:maria
Paul McCullagh <paul.mccullagh@primebase.org> writes:
All tests in the PBXT suite run through on Mac and Linux, except for one error under Linux, which is a bit weird (see below).
------------------
pbxt.select_safe [ fail ] Test ended at 2010-05-06 17:19:13
CURRENT_TEST: pbxt.select_safe mysqltest: At line 19: query 'select 1 from t1,t1 as t2,t1 as t3' failed: 1104: The SELECT would examine more than MAX_JOIN_SIZE rows; check your WHERE and use SET SQL_BIG_SELECTS=1 or SET SQL_MAX_JOIN_SIZE=# if the SELECT is okay
The result from queries just before the failure was: drop table if exists t1; SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=4, SQL_MAX_JOIN_SIZE=9; create table t1 (a int auto_increment primary key, b char(20)); insert into t1 values(1,"test"); SELECT SQL_BUFFER_RESULT * from t1; a b 1 test update t1 set b="a" whe
Ok, I looked into this. So this test does a three-way cartesian self-join on a table with 2 rows (a total of 8 rows). And it sets SQL_MAX_JOIN_SIZE=9. And sometimes (apparently timing dependent), this test fails because the optimiser estimates more than 9 rows in the join. So the problem here is that the optimiser estimate is sometimes too big for SQL_MAX_JOIN_SIZE=9. What I found is that when the test case fails, the function ha_pbxt::info() returns records==3 for the table. When the test case succeeds, it returns records==2. So it's pretty clear that the server will throw an error when records==3 and SQL_MAX_JOIN_SIZE=9. What remains to consider is why the returned records value differs between test runs. The testcase actually first inserts a row, then deletes it, then inserts two more rows. I'm speculating that PBXT has some background cleanup thread or similar that causes a race between freeing up space from the first row and allocating space for the two new rows? So Paul, do you think returning records = 2 or 3 at random from ha_pbxt::info() is expected or not? If expected, one way to fix the problem is to create a new table before running the test: create table t2 like t1; insert into t2 select * from t1; analyze table t2; # PBXT: required to get the correct COUNT(*) select 1 from t2 as t1,t2,t2 as t3; Then there is no possibility for the insert+delete to take up a slot and cause a failure (I was not able to repeat the failure with this change). Alternatively, we can just increate the SQL_MAX_JOIN_SIZE value. Or alternatively, you may decide that the test is meaningless for PBXT due to imprecise statistics, and just remove it from select_safe.test. Just let me know what you prefer, and I'll change it as needed. - Kristian.
Hi Kristian, Thanks for clearing this up. The problem is certainly as you say: it depends on the speed of a background thread. From this point of view the test does not make all that much sense for PBXT which returns inconsistent row count estimates. I think I will adjust the test to work in all cases with PBXT. On May 10, 2010, at 4:57 PM, Kristian Nielsen wrote:
Paul McCullagh <paul.mccullagh@primebase.org> writes:
All tests in the PBXT suite run through on Mac and Linux, except for one error under Linux, which is a bit weird (see below).
------------------
pbxt.select_safe [ fail ] Test ended at 2010-05-06 17:19:13
CURRENT_TEST: pbxt.select_safe mysqltest: At line 19: query 'select 1 from t1,t1 as t2,t1 as t3' failed: 1104: The SELECT would examine more than MAX_JOIN_SIZE rows; check your WHERE and use SET SQL_BIG_SELECTS=1 or SET SQL_MAX_JOIN_SIZE=# if the SELECT is okay
The result from queries just before the failure was: drop table if exists t1; SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=4, SQL_MAX_JOIN_SIZE=9; create table t1 (a int auto_increment primary key, b char(20)); insert into t1 values(1,"test"); SELECT SQL_BUFFER_RESULT * from t1; a b 1 test update t1 set b="a" whe
Ok, I looked into this.
So this test does a three-way cartesian self-join on a table with 2 rows (a total of 8 rows). And it sets SQL_MAX_JOIN_SIZE=9. And sometimes (apparently timing dependent), this test fails because the optimiser estimates more than 9 rows in the join. So the problem here is that the optimiser estimate is sometimes too big for SQL_MAX_JOIN_SIZE=9.
What I found is that when the test case fails, the function ha_pbxt::info() returns records==3 for the table. When the test case succeeds, it returns records==2. So it's pretty clear that the server will throw an error when records==3 and SQL_MAX_JOIN_SIZE=9.
What remains to consider is why the returned records value differs between test runs.
The testcase actually first inserts a row, then deletes it, then inserts two more rows. I'm speculating that PBXT has some background cleanup thread or similar that causes a race between freeing up space from the first row and allocating space for the two new rows?
So Paul, do you think returning records = 2 or 3 at random from ha_pbxt::info() is expected or not?
If expected, one way to fix the problem is to create a new table before running the test:
create table t2 like t1; insert into t2 select * from t1; analyze table t2; # PBXT: required to get the correct COUNT(*) select 1 from t2 as t1,t2,t2 as t3;
Then there is no possibility for the insert+delete to take up a slot and cause a failure (I was not able to repeat the failure with this change).
Alternatively, we can just increate the SQL_MAX_JOIN_SIZE value.
Or alternatively, you may decide that the test is meaningless for PBXT due to imprecise statistics, and just remove it from select_safe.test.
Just let me know what you prefer, and I'll change it as needed.
- Kristian.
-- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com
Paul McCullagh <paul.mccullagh@primebase.org> writes:
Thanks for clearing this up.
The problem is certainly as you say: it depends on the speed of a background thread.
From this point of view the test does not make all that much sense for PBXT which returns inconsistent row count estimates.
I think I will adjust the test to work in all cases with PBXT.
We are getting close to ready with MariaDB 5.1.46, so I would like to fix this issue. How about this patch, which seems to fix the issue? If you're ok with it, I will push it into the 5.1-release branch. - Kristian. ----------------------------------------------------------------------- === modified file 'mysql-test/suite/pbxt/r/select_safe.result' --- mysql-test/suite/pbxt/r/select_safe.result 2009-08-17 15:57:58 +0000 +++ mysql-test/suite/pbxt/r/select_safe.result 2010-05-14 18:19:40 +0000 @@ -1,4 +1,4 @@ -drop table if exists t1; +drop table if exists t1,t2; SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=4, SQL_MAX_JOIN_SIZE=9; create table t1 (a int auto_increment primary key, b char(20)); insert into t1 values(1,"test"); @@ -13,10 +13,12 @@ a b 1 test 2 test2 update t1 set b="a" where a=1; -analyze table t1; +create table t2 like t1; +insert into t2 select * from t1; +analyze table t2; Table Op Msg_type Msg_text -test.t1 analyze status OK -select 1 from t1,t1 as t2,t1 as t3; +test.t2 analyze status OK +select 1 from t2 as t1,t2,t2 as t3; 1 1 1 @@ -77,7 +79,7 @@ id select_type table type possible_keys 1 SIMPLE t1 ALL b NULL NULL NULL 21 1 SIMPLE t2 ref b b 21 test.t1.b 1 Using where SET MAX_SEEKS_FOR_KEY=DEFAULT; -drop table t1; +drop table t1, t2; create table t1 (a int); insert into t1 values (1),(2),(3),(4),(5); insert into t1 select * from t1; === modified file 'mysql-test/suite/pbxt/t/select_safe.test' --- mysql-test/suite/pbxt/t/select_safe.test 2009-04-02 10:03:14 +0000 +++ mysql-test/suite/pbxt/t/select_safe.test 2010-05-14 18:19:33 +0000 @@ -3,7 +3,7 @@ # --disable_warnings -drop table if exists t1; +drop table if exists t1,t2; --enable_warnings SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=4, SQL_MAX_JOIN_SIZE=9; @@ -15,8 +15,13 @@ delete from t1 where a=1; insert into t1 values(1,"test"),(2,"test2"); SELECT SQL_BUFFER_RESULT * from t1 order by a; # PBXT required for consistent result update t1 set b="a" where a=1; -analyze table t1; # PBXT: required to get the correct COUNT(*) -select 1 from t1,t1 as t2,t1 as t3; + +# Test MAX_JOIN_SIZE in a fresh table, as insert/delete combination +# makes PBXT statistics non-deterministic +create table t2 like t1; +insert into t2 select * from t1; +analyze table t2; # PBXT: required to get the correct COUNT(*) +select 1 from t2 as t1,t2,t2 as t3; # The following should give errors: --error 1175 @@ -64,7 +69,7 @@ set MAX_SEEKS_FOR_KEY=1; explain select STRAIGHT_JOIN * from t1,t1 as t2 where t1.b=t2.b; SET MAX_SEEKS_FOR_KEY=DEFAULT; -drop table t1; +drop table t1, t2; # BUG#8726 create table t1 (a int);
Hi Kristian, Please go ahead with the patch, its fine with me. On May 14, 2010, at 8:22 PM, Kristian Nielsen wrote:
Paul McCullagh <paul.mccullagh@primebase.org> writes:
Thanks for clearing this up.
The problem is certainly as you say: it depends on the speed of a background thread.
From this point of view the test does not make all that much sense for PBXT which returns inconsistent row count estimates.
I think I will adjust the test to work in all cases with PBXT.
We are getting close to ready with MariaDB 5.1.46, so I would like to fix this issue.
How about this patch, which seems to fix the issue? If you're ok with it, I will push it into the 5.1-release branch.
- Kristian.
----------------------------------------------------------------------- === modified file 'mysql-test/suite/pbxt/r/select_safe.result' --- mysql-test/suite/pbxt/r/select_safe.result 2009-08-17 15:57:58 +0000 +++ mysql-test/suite/pbxt/r/select_safe.result 2010-05-14 18:19:40 +0000 @@ -1,4 +1,4 @@ -drop table if exists t1; +drop table if exists t1,t2; SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=4, SQL_MAX_JOIN_SIZE=9; create table t1 (a int auto_increment primary key, b char(20)); insert into t1 values(1,"test"); @@ -13,10 +13,12 @@ a b 1 test 2 test2 update t1 set b="a" where a=1; -analyze table t1; +create table t2 like t1; +insert into t2 select * from t1; +analyze table t2; Table Op Msg_type Msg_text -test.t1 analyze status OK -select 1 from t1,t1 as t2,t1 as t3; +test.t2 analyze status OK +select 1 from t2 as t1,t2,t2 as t3; 1 1 1 @@ -77,7 +79,7 @@ id select_type table type possible_keys 1 SIMPLE t1 ALL b NULL NULL NULL 21 1 SIMPLE t2 ref b b 21 test.t1.b 1 Using where SET MAX_SEEKS_FOR_KEY=DEFAULT; -drop table t1; +drop table t1, t2; create table t1 (a int); insert into t1 values (1),(2),(3),(4),(5); insert into t1 select * from t1;
=== modified file 'mysql-test/suite/pbxt/t/select_safe.test' --- mysql-test/suite/pbxt/t/select_safe.test 2009-04-02 10:03:14 +0000 +++ mysql-test/suite/pbxt/t/select_safe.test 2010-05-14 18:19:33 +0000 @@ -3,7 +3,7 @@ #
--disable_warnings -drop table if exists t1; +drop table if exists t1,t2; --enable_warnings
SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=4, SQL_MAX_JOIN_SIZE=9; @@ -15,8 +15,13 @@ delete from t1 where a=1; insert into t1 values(1,"test"),(2,"test2"); SELECT SQL_BUFFER_RESULT * from t1 order by a; # PBXT required for consistent result update t1 set b="a" where a=1; -analyze table t1; # PBXT: required to get the correct COUNT(*) -select 1 from t1,t1 as t2,t1 as t3; + +# Test MAX_JOIN_SIZE in a fresh table, as insert/delete combination +# makes PBXT statistics non-deterministic +create table t2 like t1; +insert into t2 select * from t1; +analyze table t2; # PBXT: required to get the correct COUNT(*) +select 1 from t2 as t1,t2,t2 as t3;
# The following should give errors: --error 1175 @@ -64,7 +69,7 @@ set MAX_SEEKS_FOR_KEY=1; explain select STRAIGHT_JOIN * from t1,t1 as t2 where t1.b=t2.b; SET MAX_SEEKS_FOR_KEY=DEFAULT;
-drop table t1; +drop table t1, t2;
# BUG#8726 create table t1 (a int);
-- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com
participants (2)
-
Kristian Nielsen -
Paul McCullagh