[Maria-developers] mysql engines tests fail on mariadb 5.5.25
hello, when i run "./mysql-test-run.pl --suite=engines/funcs,engines/iuds --force --mysqld=--partition --nowarnings --max-test-fail=0", i see lots of test failures on mariadb-5.5.25 (and other distributions). either (1) i am not running the tests correctly, or (2) the test and/or test result files are broken, or (3) there are real problems being detected. i suspect most of the failures are caused by (2) above. anyway, here are the failures: Completed: Failed 37/323 tests, 88.54% were successful. Failing test(s): engines/funcs.an_calendar engines/funcs.datetime_function engines/funcs.db_alter_collate_ascii engines/funcs.db_alter_collate_utf8 engines/funcs.db_use_error engines/funcs.de_calendar_range engines/funcs.in_calendar_2_unique_constraints_duplicate_update engines/funcs.in_calendar_pk_constraint_duplicate_update engines/funcs.in_calendar_pk_constraint_error engines/funcs.in_calendar_pk_constraint_ignore engines/funcs.in_calendar_unique_constraint_duplicate_update engines/funcs.in_calendar_unique_constraint_error engines/funcs.in_calendar_unique_constraint_ignore engines/funcs.in_multicolumn_calendar_pk_constraint_duplicate_update engines/funcs.in_multicolumn_calendar_pk_constraint_error engines/funcs.in_multicolumn_calendar_pk_constraint_ignore engines/funcs.in_multicolumn_calendar_unique_constraint_duplicate_update engines/funcs.in_multicolumn_calendar_unique_constraint_error engines/funcs.in_multicolumn_calendar_unique_constraint_ignore engines/funcs.in_multicolumn_number_pk_constraint_duplicate_update engines/funcs.in_number_boundary_error engines/funcs.in_number_decimal_boundary_error engines/funcs.ld_all_number_string_calendar_types engines/funcs.se_join_left engines/funcs.se_join_left_outer engines/funcs.se_join_natural_left engines/funcs.se_join_natural_left_outer engines/funcs.sq_all engines/funcs.sq_error engines/funcs.up_calendar_range engines/iuds.delete_time engines/iuds.insert_decimal engines/iuds.insert_number engines/iuds.insert_time engines/iuds.type_bit_iuds engines/iuds.update_delete_number engines/iuds.update_time thanks rich prohaska
Hi Rich, On 6/25/2012 4:49 PM, Rich Prohaska wrote:
hello,
when i run "./mysql-test-run.pl --suite=engines/funcs,engines/iuds --force --mysqld=--partition --nowarnings --max-test-fail=0", i see lots of test failures on mariadb-5.5.25 (and other distributions). either (1) i am not running the tests correctly, or (2) the test and/or test result files are broken, or (3) there are real problems being detected. i suspect most of the failures are caused by (2) above.
You are right, the result files for this test suite have been taken "as is" from the MySQL test suite, and it causes the failures you mentioned, as there are some differences in the code, e.g. - MariaDB processes some comparisons with NULL in a more conventional way, - SQL states and error messages displayed by the tests can be different (like HY000 in MySQL vs 22007 in MariaDB); - MariaDB produces a warning when it truncates a value, while MySQL sometimes does it silently, - there is an extra collation in MariaDB, and some more. We do not use this test suite in our regular tests, so the mismatches slipped in unnoticed; but you are right, as long as it is included into our packages, it should be consistent with the code. Thank you for pointing it out. If you are looking for a test suite to test an engine, you might want to try out the engine independent suite that we have developed for this very purpose and are going to publish in the next couple of weeks. It is described in detail in the JIRA task https://mariadb.atlassian.net/browse/MDEV-11, and is based on recently developed extensions to MTR functionality, already released in MariaDB 5.5 (https://mariadb.atlassian.net/browse/MDEV-30). The idea is to provide a suite easily tunable for any new engine without having to copy over and modify test files. Currently it works for MyISAM, Aria, XtraDB, Archive, InnoDB, Memory, Merge, CSV, Federated. Regards, Elena Stepanova
anyway, here are the failures:
Completed: Failed 37/323 tests, 88.54% were successful.
Failing test(s): engines/funcs.an_calendar engines/funcs.datetime_function engines/funcs.db_alter_collate_ascii engines/funcs.db_alter_collate_utf8 engines/funcs.db_use_error engines/funcs.de_calendar_range engines/funcs.in_calendar_2_unique_constraints_duplicate_update engines/funcs.in_calendar_pk_constraint_duplicate_update engines/funcs.in_calendar_pk_constraint_error engines/funcs.in_calendar_pk_constraint_ignore engines/funcs.in_calendar_unique_constraint_duplicate_update engines/funcs.in_calendar_unique_constraint_error engines/funcs.in_calendar_unique_constraint_ignore engines/funcs.in_multicolumn_calendar_pk_constraint_duplicate_update engines/funcs.in_multicolumn_calendar_pk_constraint_error engines/funcs.in_multicolumn_calendar_pk_constraint_ignore engines/funcs.in_multicolumn_calendar_unique_constraint_duplicate_update engines/funcs.in_multicolumn_calendar_unique_constraint_error engines/funcs.in_multicolumn_calendar_unique_constraint_ignore engines/funcs.in_multicolumn_number_pk_constraint_duplicate_update engines/funcs.in_number_boundary_error engines/funcs.in_number_decimal_boundary_error engines/funcs.ld_all_number_string_calendar_types engines/funcs.se_join_left engines/funcs.se_join_left_outer engines/funcs.se_join_natural_left engines/funcs.se_join_natural_left_outer engines/funcs.sq_all engines/funcs.sq_error engines/funcs.up_calendar_range engines/iuds.delete_time engines/iuds.insert_decimal engines/iuds.insert_number engines/iuds.insert_time engines/iuds.type_bit_iuds engines/iuds.update_delete_number engines/iuds.update_time
thanks
rich prohaska
_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Hello Elena, Thanks for pointing out to us the engine independent tests. We would like to run this test with TokuDB as part of our test process, and look forward to its release. Rich Prohaska Tokutek On Tue, Jun 26, 2012 at 12:58 PM, Elena Stepanova <elenst@gmail.com> wrote:
Hi Rich,
On 6/25/2012 4:49 PM, Rich Prohaska wrote:
hello,
when i run "./mysql-test-run.pl --suite=engines/funcs,engines/iuds --force --mysqld=--partition --nowarnings --max-test-fail=0", i see lots of test failures on mariadb-5.5.25 (and other distributions). either (1) i am not running the tests correctly, or (2) the test and/or test result files are broken, or (3) there are real problems being detected. i suspect most of the failures are caused by (2) above.
You are right, the result files for this test suite have been taken "as is" from the MySQL test suite, and it causes the failures you mentioned, as there are some differences in the code, e.g.
- MariaDB processes some comparisons with NULL in a more conventional way, - SQL states and error messages displayed by the tests can be different (like HY000 in MySQL vs 22007 in MariaDB); - MariaDB produces a warning when it truncates a value, while MySQL sometimes does it silently, - there is an extra collation in MariaDB, and some more.
We do not use this test suite in our regular tests, so the mismatches slipped in unnoticed; but you are right, as long as it is included into our packages, it should be consistent with the code. Thank you for pointing it out.
If you are looking for a test suite to test an engine, you might want to try out the engine independent suite that we have developed for this very purpose and are going to publish in the next couple of weeks. It is described in detail in the JIRA task https://mariadb.atlassian.net/browse/MDEV-11, and is based on recently developed extensions to MTR functionality, already released in MariaDB 5.5 (https://mariadb.atlassian.net/browse/MDEV-30). The idea is to provide a suite easily tunable for any new engine without having to copy over and modify test files. Currently it works for MyISAM, Aria, XtraDB, Archive, InnoDB, Memory, Merge, CSV, Federated.
Regards, Elena Stepanova
anyway, here are the failures:
Completed: Failed 37/323 tests, 88.54% were successful.
Failing test(s): engines/funcs.an_calendar engines/funcs.datetime_function engines/funcs.db_alter_collate_ascii engines/funcs.db_alter_collate_utf8 engines/funcs.db_use_error engines/funcs.de_calendar_range engines/funcs.in_calendar_2_unique_constraints_duplicate_update engines/funcs.in_calendar_pk_constraint_duplicate_update engines/funcs.in_calendar_pk_constraint_error engines/funcs.in_calendar_pk_constraint_ignore engines/funcs.in_calendar_unique_constraint_duplicate_update engines/funcs.in_calendar_unique_constraint_error engines/funcs.in_calendar_unique_constraint_ignore engines/funcs.in_multicolumn_calendar_pk_constraint_duplicate_update engines/funcs.in_multicolumn_calendar_pk_constraint_error engines/funcs.in_multicolumn_calendar_pk_constraint_ignore engines/funcs.in_multicolumn_calendar_unique_constraint_duplicate_update engines/funcs.in_multicolumn_calendar_unique_constraint_error engines/funcs.in_multicolumn_calendar_unique_constraint_ignore engines/funcs.in_multicolumn_number_pk_constraint_duplicate_update engines/funcs.in_number_boundary_error engines/funcs.in_number_decimal_boundary_error engines/funcs.ld_all_number_string_calendar_types engines/funcs.se_join_left engines/funcs.se_join_left_outer engines/funcs.se_join_natural_left engines/funcs.se_join_natural_left_outer engines/funcs.sq_all engines/funcs.sq_error engines/funcs.up_calendar_range engines/iuds.delete_time engines/iuds.insert_decimal engines/iuds.insert_number engines/iuds.insert_time engines/iuds.type_bit_iuds engines/iuds.update_delete_number engines/iuds.update_time
thanks
rich prohaska
_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
participants (3)
-
Elena Stepanova -
Rich Prohaska -
Rich Prohaska