- developers - lists.mariadb.org

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2694)
by knielsen＠knielsen-hq.org 08 Apr '09

08 Apr '09

#At lp:maria 2694 knielsen(a)knielsen-hq.org 2009-04-08 [merge] Merge mariadb-5.1 modified: storage/maria/ma_blockrec.c === modified file 'storage/maria/ma_blockrec.c' --- a/storage/maria/ma_blockrec.c 2009-02-19 09:01:25 +0000 +++ b/storage/maria/ma_blockrec.c 2009-04-08 08:47:30 +0000 @@ -2874,7 +2874,7 @@ static my_bool write_block_record(MARIA_ if (cur_block->used & BLOCKUSED_TAIL) { DBUG_ASSERT(data_length < MAX_TAIL_SIZE(block_size)); - /* tail written to full tail page */ + /* tail written to tail page */ cur_block->used|= BLOCKUSED_USED; head_tail_block= cur_block; }

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2693)
by knielsen＠knielsen-hq.org 08 Apr '09

08 Apr '09

#At lp:maria 2693 knielsen(a)knielsen-hq.org 2009-04-08 Fix Valgrind errors seen in buildbot. Fix mysql-test-run.pl to not terminate early when warnings in error logs are detected during server shutdown. Instead, give a nice summary report at the end of the failures. Fix code to make 100% sure no failures will go undetected. Revert earlier wrong change. Fix race with port allocation semaphore file permissions. Adjust testsuite to copy with new PBXT engine now in the tree. The PBXT engine causes an extra table to appear in the INFORMATION_SCHEMA. This causes different output for a few test cases. added: mysql-test/include/have_pbxt.inc mysql-test/r/information_schema_all_engines.result mysql-test/t/information_schema_all_engines.test modified: dbug/dbug.c include/my_global.h mysql-test/extra/rpl_tests/rpl_auto_increment.test mysql-test/lib/mtr_report.pm mysql-test/lib/mtr_unique.pm mysql-test/mysql-test-run.pl mysql-test/r/information_schema.result mysql-test/r/information_schema_db.result mysql-test/r/innodb-autoinc.result mysql-test/r/mysqlshow.result mysql-test/suite/rpl/r/rpl_auto_increment.result mysql-test/t/information_schema.test mysql-test/t/information_schema_db.test mysql-test/t/innodb-autoinc.test mysql-test/t/mysqlshow.test mysql-test/valgrind.supp sql/mysqld.cc sql/set_var.cc sql/sql_select.cc sql/sql_string.h storage/archive/azio.c per-file messages: dbug/dbug.c If DbugParse() is called multiple times, the stack->keywords for the top stack frame could be overwritten without being freed, causing a memory leak reported by Valgrind. include/my_global.h Add useful macro for different values when Valgrind (HAVE_purify) and not. mysql-test/extra/rpl_tests/rpl_auto_increment.test Omit pbxt variables from show variables output. mysql-test/include/have_pbxt.inc Add facility to disable test if PBXT engine is not available. mysql-test/lib/mtr_report.pm Give a nice summary report at the end of tests of any warnings seen in logs during server shutdowns. mysql-test/lib/mtr_unique.pm Move chmod 777 to greatly reduce the risk of leaving the port semaphore file unaccessible bu other users. mysql-test/mysql-test-run.pl Don't abort in case of warnings detected, instead give a nice summary report. Fix code to make 100% sure no failures will go undetected. Revert earlier wrong change when master disconnects early. mysql-test/r/information_schema.result Omit PBXT INFORMATION_SCHEMA table from output. Move part of test to information_schema_all_engines. mysql-test/r/information_schema_all_engines.result New file for information_schema tests that depend on which engines are available. mysql-test/r/information_schema_db.result Move part of test to information_schema_all_engines. mysql-test/r/innodb-autoinc.result Omit pbxt variables from show variables output. mysql-test/r/mysqlshow.result Move part of test to information_schema_all_engines. mysql-test/suite/rpl/r/rpl_auto_increment.result Omit pbxt variables from show variables output. mysql-test/t/information_schema.test Omit PBXT INFORMATION_SCHEMA table from output. Move part of test to information_schema_all_engines. mysql-test/t/information_schema_all_engines.test New file for information_schema tests that depend on which engines are available. mysql-test/t/information_schema_db.test Move part of test to information_schema_all_engines. mysql-test/t/innodb-autoinc.test Omit pbxt variables from show variables output. mysql-test/t/mysqlshow.test Move part of test to information_schema_all_engines. mysql-test/valgrind.supp Add variant suppression (different system library versions). Add suppression for problem with inet_ntoa(). sql/mysqld.cc Fix missing DBUG_RETURN. Fix uninitialised thd->connect_utime, likely introduced by pool_of_threads. sql/set_var.cc Fix one-byte buffer overflow in several places. Fix unsafe use of String::c_ptr() of stack-allocated String buffer. sql/sql_select.cc Silence valgrind warning due to GCC bug. sql/sql_string.h Document potential problem with String::c_ptr() and String() constructor with caller-supplied buffer. storage/archive/azio.c Silence Valgrind false warning for libz. === modified file 'dbug/dbug.c' --- a/dbug/dbug.c 2009-03-22 12:16:09 +0000 +++ b/dbug/dbug.c 2009-04-08 16:55:26 +0000 @@ -506,6 +506,9 @@ int DbugParse(CODE_STATE *cs, const char rel= control[0] == '+' || control[0] == '-'; if ((!rel || (!stack->out_file && !stack->next))) { + /* If overwriting previous state, be sure to free old to avoid leak. */ + if (stack->out_file) + FreeState(cs, stack, 0); stack->flags= 0; stack->delay= 0; stack->maxdepth= 0; @@ -1648,10 +1651,12 @@ static void FreeState(CODE_STATE *cs, st FreeList(state->processes); if (!is_shared(state, p_functions)) FreeList(state->p_functions); - if (!is_shared(state, out_file)) + if (!is_shared(state, out_file) && + state->out_file != stderr && state->out_file != stdout) DBUGCloseFile(cs, state->out_file); (void) fflush(cs->stack->out_file); - if (state->prof_file) + if (state->prof_file && + state->prof_file != stderr && state->prof_file != stdout) DBUGCloseFile(cs, state->prof_file); if (free_state) free((void*) state); === modified file 'include/my_global.h' --- a/include/my_global.h 2009-02-19 09:01:25 +0000 +++ b/include/my_global.h 2009-04-08 16:55:26 +0000 @@ -569,6 +569,12 @@ int __void__; #define PURIFY_OR_LINT_INIT(var) #endif +#ifdef HAVE_purify +#define IF_PURIFY(A,B) (A) +#else +#define IF_PURIFY(A,B) (B) +#endif + #if !defined(HAVE_UINT) #undef HAVE_UINT #define HAVE_UINT === modified file 'mysql-test/extra/rpl_tests/rpl_auto_increment.test' --- a/mysql-test/extra/rpl_tests/rpl_auto_increment.test 2009-01-14 08:27:32 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_auto_increment.test 2009-04-08 16:55:26 +0000 @@ -34,7 +34,7 @@ connection master; drop table t1; set @@session.auto_increment_increment=100, @@session.auto_increment_offset=10; -show variables like "%auto_inc%"; +show variables like "auto_inc%"; eval create table t1 (a int not null auto_increment, primary key (a)) engine=$engine_type2; # Insert with 2 insert statements to get better testing of logging === added file 'mysql-test/include/have_pbxt.inc' --- a/mysql-test/include/have_pbxt.inc 1970-01-01 00:00:00 +0000 +++ b/mysql-test/include/have_pbxt.inc 2009-04-08 16:55:26 +0000 @@ -0,0 +1,4 @@ +disable_query_log; +--require r/true.require +select (support = 'YES' or support = 'DEFAULT') as `TRUE` from information_schema.engines where engine = 'pbxt'; +enable_query_log; === modified file 'mysql-test/lib/mtr_report.pm' --- a/mysql-test/lib/mtr_report.pm 2009-02-19 09:01:25 +0000 +++ b/mysql-test/lib/mtr_report.pm 2009-04-08 16:55:26 +0000 @@ -187,8 +187,10 @@ sub mtr_report_test ($) { } -sub mtr_report_stats ($) { +sub mtr_report_stats ($$$) { + my $fail= shift; my $tests= shift; + my $extra_warnings= shift; # ---------------------------------------------------------------------- # Find out how we where doing @@ -325,10 +327,27 @@ sub mtr_report_stats ($) { print "All $tot_tests tests were successful.\n\n"; } + if (@$extra_warnings) + { + print <<MSG; +Errors/warnings were found in logfiles during server shutdown after running the +following sequence(s) of tests: +MSG + print " $_\n" for @$extra_warnings; + } + if ( $tot_failed != 0 || $found_problems) { mtr_error("there were failing test cases"); } + elsif (@$extra_warnings) + { + mtr_error("There were errors/warnings in server logs after running test cases."); + } + elsif ($fail) + { + mtr_error("Test suite failure, see messages above for possible cause(s)."); + } } === modified file 'mysql-test/lib/mtr_unique.pm' --- a/mysql-test/lib/mtr_unique.pm 2009-02-19 09:01:25 +0000 +++ b/mysql-test/lib/mtr_unique.pm 2009-04-08 16:55:26 +0000 @@ -62,13 +62,14 @@ sub mtr_get_unique_id($$) { die 'lock file is a symbolic link'; } - chmod 0777, "$file.sem"; open SEM, ">", "$file.sem" or die "can't write to $file.sem"; + chmod 0777, "$file.sem"; flock SEM, LOCK_EX or die "can't lock $file.sem"; if(! -e $file) { open FILE, ">", $file or die "can't create $file"; close FILE; } + chmod 0777, $file; msg("HAVE THE LOCK"); @@ -76,7 +77,6 @@ sub mtr_get_unique_id($$) { die 'lock file is a symbolic link'; } - chmod 0777, $file; open FILE, "+<", $file or die "can't open $file"; #select undef,undef,undef,0.2; seek FILE, 0, 0; @@ -136,6 +136,7 @@ sub mtr_release_unique_id($) { } open SEM, ">", "$file.sem" or die "can't write to $file.sem"; + chmod 0777, "$file.sem"; flock SEM, LOCK_EX or die "can't lock $file.sem"; msg("HAVE THE LOCK"); @@ -148,6 +149,7 @@ sub mtr_release_unique_id($) { open FILE, ">", $file or die "can't create $file"; close FILE; } + chmod 0777, "$file.sem"; open FILE, "+<", $file or die "can't open $file"; #select undef,undef,undef,0.2; seek FILE, 0, 0; === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2009-04-03 08:09:55 +0000 +++ b/mysql-test/mysql-test-run.pl 2009-04-08 16:55:26 +0000 @@ -357,7 +357,8 @@ sub main { mtr_print_thick_line(); mtr_print_header(); - my ($completed, $fail)= run_test_server($server, $tests, $opt_parallel); + my ($fail, $completed, $extra_warnings)= + run_test_server($server, $tests, $opt_parallel); # Send Ctrl-C to any children still running kill("INT", keys(%children)); @@ -393,10 +394,6 @@ sub main { mtr_error("Not all tests completed"); } - if ($fail) { - mtr_error("Test suite failure."); - } - mtr_print_line(); if ( $opt_gcov ) { @@ -404,7 +401,7 @@ sub main { $opt_gcov_msg, $opt_gcov_err); } - mtr_report_stats($completed); + mtr_report_stats($fail, $completed, $extra_warnings); exit(0); } @@ -416,7 +413,8 @@ sub run_test_server ($$$) { my $num_saved_cores= 0; # Number of core files saved in vardir/log/ so far. my $num_saved_datadir= 0; # Number of datadirs saved in vardir/log/ so far. my $num_failed_test= 0; # Number of tests failed so far - my $test_failure= 0; + my $test_failure= 0; # Set true if test suite failed + my $extra_warnings= []; # Warnings found during server shutdowns # Scheduler variables my $max_ndb= $childs / 2; @@ -450,7 +448,7 @@ sub run_test_server ($$$) { $s->remove($sock); if (--$childs == 0){ $suite_timeout_proc->kill(); - return ($completed, $test_failure); + return ($test_failure, $completed, $extra_warnings); } next; } @@ -519,14 +517,14 @@ sub run_test_server ($$$) { # Test has failed, force is off $suite_timeout_proc->kill(); push(@$completed, $result); - return ($completed, 1); + return (1, $completed, $extra_warnings); } elsif ($opt_max_test_fail > 0 and $num_failed_test >= $opt_max_test_fail) { $suite_timeout_proc->kill(); mtr_report("Too many tests($num_failed_test) failed!", "Terminating..."); - return (undef, 1); + return (1, $completed, $extra_warnings); } $num_failed_test++; } @@ -580,13 +578,14 @@ sub run_test_server ($$$) { elsif ($line eq 'WARNINGS'){ my $fake_test= My::Test::read_test($sock); my $test_list= join (" ", @{$fake_test->{testnames}}); + push @$extra_warnings, $test_list; mtr_report("***Warnings generated in error logs during shutdown ". "after running tests: $test_list"); $test_failure= 1; if ( !$opt_force ) { # Test failure due to warnings, force is off $suite_timeout_proc->kill(); - return ($completed, 1); + return (1, $completed, $extra_warnings); } } else { mtr_error("Unknown response: '$line' from client"); @@ -666,7 +665,7 @@ sub run_test_server ($$$) { if ( ! $suite_timeout_proc->wait_one(0) ) { mtr_report("Test suite timeout! Terminating..."); - return (undef, 1); + return (1, $completed, $extra_warnings); } } } @@ -758,7 +757,9 @@ sub run_worker ($) { } } - die "Internal error: should not reach this place."; + stop_all_servers(); + + exit(1); } === modified file 'mysql-test/r/information_schema.result' --- a/mysql-test/r/information_schema.result 2009-02-15 10:58:34 +0000 +++ b/mysql-test/r/information_schema.result 2009-04-08 16:55:26 +0000 @@ -40,7 +40,8 @@ create view v1 (c) as SELECT table_name FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND -table_name<>'ndb_apply_status'; +table_name<>'ndb_apply_status' AND +NOT (table_schema = 'INFORMATION_SCHEMA' AND table_name LIKE 'PBXT_%'); select * from v1; c CHARACTER_SETS @@ -850,10 +851,6 @@ VIEWS TABLE_NAME select delete from mysql.user where user='mysqltest_4'; delete from mysql.db where user='mysqltest_4'; flush privileges; -SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; -table_schema count(*) -information_schema 28 -mysql 22 create table t1 (i int, j int); create trigger trg1 before insert on t1 for each row begin @@ -1224,92 +1221,6 @@ f1() DROP FUNCTION f1; DROP PROCEDURE p1; DROP USER mysql_bug20230@localhost; -SELECT t.table_name, c1.column_name -FROM information_schema.tables t -INNER JOIN -information_schema.columns c1 -ON t.table_schema = c1.table_schema AND -t.table_name = c1.table_name -WHERE t.table_schema = 'information_schema' AND -c1.ordinal_position = -( SELECT COALESCE(MIN(c2.ordinal_position),1) -FROM information_schema.columns c2 -WHERE c2.table_schema = t.table_schema AND -c2.table_name = t.table_name AND -c2.column_name LIKE '%SCHEMA%' - ); -table_name column_name -CHARACTER_SETS CHARACTER_SET_NAME -COLLATIONS COLLATION_NAME -COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME -COLUMNS TABLE_SCHEMA -COLUMN_PRIVILEGES TABLE_SCHEMA -ENGINES ENGINE -EVENTS EVENT_SCHEMA -FILES TABLE_SCHEMA -GLOBAL_STATUS VARIABLE_NAME -GLOBAL_VARIABLES VARIABLE_NAME -KEY_COLUMN_USAGE CONSTRAINT_SCHEMA -PARTITIONS TABLE_SCHEMA -PLUGINS PLUGIN_NAME -PROCESSLIST ID -PROFILING QUERY_ID -REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA -ROUTINES ROUTINE_SCHEMA -SCHEMATA SCHEMA_NAME -SCHEMA_PRIVILEGES TABLE_SCHEMA -SESSION_STATUS VARIABLE_NAME -SESSION_VARIABLES VARIABLE_NAME -STATISTICS TABLE_SCHEMA -TABLES TABLE_SCHEMA -TABLE_CONSTRAINTS CONSTRAINT_SCHEMA -TABLE_PRIVILEGES TABLE_SCHEMA -TRIGGERS TRIGGER_SCHEMA -USER_PRIVILEGES GRANTEE -VIEWS TABLE_SCHEMA -SELECT t.table_name, c1.column_name -FROM information_schema.tables t -INNER JOIN -information_schema.columns c1 -ON t.table_schema = c1.table_schema AND -t.table_name = c1.table_name -WHERE t.table_schema = 'information_schema' AND -c1.ordinal_position = -( SELECT COALESCE(MIN(c2.ordinal_position),1) -FROM information_schema.columns c2 -WHERE c2.table_schema = 'information_schema' AND -c2.table_name = t.table_name AND -c2.column_name LIKE '%SCHEMA%' - ); -table_name column_name -CHARACTER_SETS CHARACTER_SET_NAME -COLLATIONS COLLATION_NAME -COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME -COLUMNS TABLE_SCHEMA -COLUMN_PRIVILEGES TABLE_SCHEMA -ENGINES ENGINE -EVENTS EVENT_SCHEMA -FILES TABLE_SCHEMA -GLOBAL_STATUS VARIABLE_NAME -GLOBAL_VARIABLES VARIABLE_NAME -KEY_COLUMN_USAGE CONSTRAINT_SCHEMA -PARTITIONS TABLE_SCHEMA -PLUGINS PLUGIN_NAME -PROCESSLIST ID -PROFILING QUERY_ID -REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA -ROUTINES ROUTINE_SCHEMA -SCHEMATA SCHEMA_NAME -SCHEMA_PRIVILEGES TABLE_SCHEMA -SESSION_STATUS VARIABLE_NAME -SESSION_VARIABLES VARIABLE_NAME -STATISTICS TABLE_SCHEMA -TABLES TABLE_SCHEMA -TABLE_CONSTRAINTS CONSTRAINT_SCHEMA -TABLE_PRIVILEGES TABLE_SCHEMA -TRIGGERS TRIGGER_SCHEMA -USER_PRIVILEGES GRANTEE -VIEWS TABLE_SCHEMA SELECT MAX(table_name) FROM information_schema.tables WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test'); MAX(table_name) VIEWS @@ -1355,55 +1266,6 @@ table_name t1 t2 drop table t1,t2; -select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= -(select cast(table_name as char) from information_schema.tables -order by table_name limit 1) limit 1; -f1 -1 -select t.table_name, group_concat(t.table_schema, '.', t.table_name), -count(*) as num1 -from information_schema.tables t -inner join information_schema.columns c1 -on t.table_schema = c1.table_schema AND t.table_name = c1.table_name -where t.table_schema = 'information_schema' and -c1.ordinal_position = -(select isnull(c2.column_type) - -isnull(group_concat(c2.table_schema, '.', c2.table_name)) + -count(*) as num -from information_schema.columns c2 where -c2.table_schema='information_schema' and -(c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') -group by c2.column_type order by num limit 1) -group by t.table_name order by num1, t.table_name; -table_name group_concat(t.table_schema, '.', t.table_name) num1 -CHARACTER_SETS information_schema.CHARACTER_SETS 1 -COLLATIONS information_schema.COLLATIONS 1 -COLLATION_CHARACTER_SET_APPLICABILITY information_schema.COLLATION_CHARACTER_SET_APPLICABILITY 1 -COLUMNS information_schema.COLUMNS 1 -COLUMN_PRIVILEGES information_schema.COLUMN_PRIVILEGES 1 -ENGINES information_schema.ENGINES 1 -EVENTS information_schema.EVENTS 1 -FILES information_schema.FILES 1 -GLOBAL_STATUS information_schema.GLOBAL_STATUS 1 -GLOBAL_VARIABLES information_schema.GLOBAL_VARIABLES 1 -KEY_COLUMN_USAGE information_schema.KEY_COLUMN_USAGE 1 -PARTITIONS information_schema.PARTITIONS 1 -PLUGINS information_schema.PLUGINS 1 -PROCESSLIST information_schema.PROCESSLIST 1 -PROFILING information_schema.PROFILING 1 -REFERENTIAL_CONSTRAINTS information_schema.REFERENTIAL_CONSTRAINTS 1 -ROUTINES information_schema.ROUTINES 1 -SCHEMATA information_schema.SCHEMATA 1 -SCHEMA_PRIVILEGES information_schema.SCHEMA_PRIVILEGES 1 -SESSION_STATUS information_schema.SESSION_STATUS 1 -SESSION_VARIABLES information_schema.SESSION_VARIABLES 1 -STATISTICS information_schema.STATISTICS 1 -TABLES information_schema.TABLES 1 -TABLE_CONSTRAINTS information_schema.TABLE_CONSTRAINTS 1 -TABLE_PRIVILEGES information_schema.TABLE_PRIVILEGES 1 -TRIGGERS information_schema.TRIGGERS 1 -USER_PRIVILEGES information_schema.USER_PRIVILEGES 1 -VIEWS information_schema.VIEWS 1 create table t1(f1 int); create view v1 as select f1+1 as a from t1; create table t2 (f1 int, f2 int); === added file 'mysql-test/r/information_schema_all_engines.result' --- a/mysql-test/r/information_schema_all_engines.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/information_schema_all_engines.result 2009-04-08 16:55:26 +0000 @@ -0,0 +1,248 @@ +use INFORMATION_SCHEMA; +show tables; +Tables_in_information_schema +CHARACTER_SETS +COLLATIONS +COLLATION_CHARACTER_SET_APPLICABILITY +COLUMNS +COLUMN_PRIVILEGES +ENGINES +EVENTS +FILES +GLOBAL_STATUS +GLOBAL_VARIABLES +KEY_COLUMN_USAGE +PARTITIONS +PLUGINS +PROCESSLIST +PROFILING +REFERENTIAL_CONSTRAINTS +ROUTINES +SCHEMATA +SCHEMA_PRIVILEGES +SESSION_STATUS +SESSION_VARIABLES +STATISTICS +TABLES +TABLE_CONSTRAINTS +TABLE_PRIVILEGES +TRIGGERS +USER_PRIVILEGES +VIEWS +PBXT_STATISTICS +SELECT t.table_name, c1.column_name +FROM information_schema.tables t +INNER JOIN +information_schema.columns c1 +ON t.table_schema = c1.table_schema AND +t.table_name = c1.table_name +WHERE t.table_schema = 'information_schema' AND +c1.ordinal_position = +( SELECT COALESCE(MIN(c2.ordinal_position),1) +FROM information_schema.columns c2 +WHERE c2.table_schema = t.table_schema AND +c2.table_name = t.table_name AND +c2.column_name LIKE '%SCHEMA%' + ); +table_name column_name +CHARACTER_SETS CHARACTER_SET_NAME +COLLATIONS COLLATION_NAME +COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME +COLUMNS TABLE_SCHEMA +COLUMN_PRIVILEGES TABLE_SCHEMA +ENGINES ENGINE +EVENTS EVENT_SCHEMA +FILES TABLE_SCHEMA +GLOBAL_STATUS VARIABLE_NAME +GLOBAL_VARIABLES VARIABLE_NAME +KEY_COLUMN_USAGE CONSTRAINT_SCHEMA +PARTITIONS TABLE_SCHEMA +PLUGINS PLUGIN_NAME +PROCESSLIST ID +PROFILING QUERY_ID +REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA +ROUTINES ROUTINE_SCHEMA +SCHEMATA SCHEMA_NAME +SCHEMA_PRIVILEGES TABLE_SCHEMA +SESSION_STATUS VARIABLE_NAME +SESSION_VARIABLES VARIABLE_NAME +STATISTICS TABLE_SCHEMA +TABLES TABLE_SCHEMA +TABLE_CONSTRAINTS CONSTRAINT_SCHEMA +TABLE_PRIVILEGES TABLE_SCHEMA +TRIGGERS TRIGGER_SCHEMA +USER_PRIVILEGES GRANTEE +VIEWS TABLE_SCHEMA +PBXT_STATISTICS ID +SELECT t.table_name, c1.column_name +FROM information_schema.tables t +INNER JOIN +information_schema.columns c1 +ON t.table_schema = c1.table_schema AND +t.table_name = c1.table_name +WHERE t.table_schema = 'information_schema' AND +c1.ordinal_position = +( SELECT COALESCE(MIN(c2.ordinal_position),1) +FROM information_schema.columns c2 +WHERE c2.table_schema = 'information_schema' AND +c2.table_name = t.table_name AND +c2.column_name LIKE '%SCHEMA%' + ); +table_name column_name +CHARACTER_SETS CHARACTER_SET_NAME +COLLATIONS COLLATION_NAME +COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME +COLUMNS TABLE_SCHEMA +COLUMN_PRIVILEGES TABLE_SCHEMA +ENGINES ENGINE +EVENTS EVENT_SCHEMA +FILES TABLE_SCHEMA +GLOBAL_STATUS VARIABLE_NAME +GLOBAL_VARIABLES VARIABLE_NAME +KEY_COLUMN_USAGE CONSTRAINT_SCHEMA +PARTITIONS TABLE_SCHEMA +PLUGINS PLUGIN_NAME +PROCESSLIST ID +PROFILING QUERY_ID +REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA +ROUTINES ROUTINE_SCHEMA +SCHEMATA SCHEMA_NAME +SCHEMA_PRIVILEGES TABLE_SCHEMA +SESSION_STATUS VARIABLE_NAME +SESSION_VARIABLES VARIABLE_NAME +STATISTICS TABLE_SCHEMA +TABLES TABLE_SCHEMA +TABLE_CONSTRAINTS CONSTRAINT_SCHEMA +TABLE_PRIVILEGES TABLE_SCHEMA +TRIGGERS TRIGGER_SCHEMA +USER_PRIVILEGES GRANTEE +VIEWS TABLE_SCHEMA +PBXT_STATISTICS ID +select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= +(select cast(table_name as char) from information_schema.tables +order by table_name limit 1) limit 1; +f1 +1 +select t.table_name, group_concat(t.table_schema, '.', t.table_name), +count(*) as num1 +from information_schema.tables t +inner join information_schema.columns c1 +on t.table_schema = c1.table_schema AND t.table_name = c1.table_name +where t.table_schema = 'information_schema' and +c1.ordinal_position = +(select isnull(c2.column_type) - +isnull(group_concat(c2.table_schema, '.', c2.table_name)) + +count(*) as num +from information_schema.columns c2 where +c2.table_schema='information_schema' and +(c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') +group by c2.column_type order by num limit 1) +group by t.table_name order by num1, t.table_name; +table_name group_concat(t.table_schema, '.', t.table_name) num1 +CHARACTER_SETS information_schema.CHARACTER_SETS 1 +COLLATIONS information_schema.COLLATIONS 1 +COLLATION_CHARACTER_SET_APPLICABILITY information_schema.COLLATION_CHARACTER_SET_APPLICABILITY 1 +COLUMNS information_schema.COLUMNS 1 +COLUMN_PRIVILEGES information_schema.COLUMN_PRIVILEGES 1 +ENGINES information_schema.ENGINES 1 +EVENTS information_schema.EVENTS 1 +FILES information_schema.FILES 1 +GLOBAL_STATUS information_schema.GLOBAL_STATUS 1 +GLOBAL_VARIABLES information_schema.GLOBAL_VARIABLES 1 +KEY_COLUMN_USAGE information_schema.KEY_COLUMN_USAGE 1 +PARTITIONS information_schema.PARTITIONS 1 +PBXT_STATISTICS information_schema.PBXT_STATISTICS 1 +PLUGINS information_schema.PLUGINS 1 +PROCESSLIST information_schema.PROCESSLIST 1 +PROFILING information_schema.PROFILING 1 +REFERENTIAL_CONSTRAINTS information_schema.REFERENTIAL_CONSTRAINTS 1 +ROUTINES information_schema.ROUTINES 1 +SCHEMATA information_schema.SCHEMATA 1 +SCHEMA_PRIVILEGES information_schema.SCHEMA_PRIVILEGES 1 +SESSION_STATUS information_schema.SESSION_STATUS 1 +SESSION_VARIABLES information_schema.SESSION_VARIABLES 1 +STATISTICS information_schema.STATISTICS 1 +TABLES information_schema.TABLES 1 +TABLE_CONSTRAINTS information_schema.TABLE_CONSTRAINTS 1 +TABLE_PRIVILEGES information_schema.TABLE_PRIVILEGES 1 +TRIGGERS information_schema.TRIGGERS 1 +USER_PRIVILEGES information_schema.USER_PRIVILEGES 1 +VIEWS information_schema.VIEWS 1 +Database: information_schema ++---------------------------------------+ +| Tables | ++---------------------------------------+ +| CHARACTER_SETS | +| COLLATIONS | +| COLLATION_CHARACTER_SET_APPLICABILITY | +| COLUMNS | +| COLUMN_PRIVILEGES | +| ENGINES | +| EVENTS | +| FILES | +| GLOBAL_STATUS | +| GLOBAL_VARIABLES | +| KEY_COLUMN_USAGE | +| PARTITIONS | +| PLUGINS | +| PROCESSLIST | +| PROFILING | +| REFERENTIAL_CONSTRAINTS | +| ROUTINES | +| SCHEMATA | +| SCHEMA_PRIVILEGES | +| SESSION_STATUS | +| SESSION_VARIABLES | +| STATISTICS | +| TABLES | +| TABLE_CONSTRAINTS | +| TABLE_PRIVILEGES | +| TRIGGERS | +| USER_PRIVILEGES | +| VIEWS | +| PBXT_STATISTICS | ++---------------------------------------+ +Database: INFORMATION_SCHEMA ++---------------------------------------+ +| Tables | ++---------------------------------------+ +| CHARACTER_SETS | +| COLLATIONS | +| COLLATION_CHARACTER_SET_APPLICABILITY | +| COLUMNS | +| COLUMN_PRIVILEGES | +| ENGINES | +| EVENTS | +| FILES | +| GLOBAL_STATUS | +| GLOBAL_VARIABLES | +| KEY_COLUMN_USAGE | +| PARTITIONS | +| PLUGINS | +| PROCESSLIST | +| PROFILING | +| REFERENTIAL_CONSTRAINTS | +| ROUTINES | +| SCHEMATA | +| SCHEMA_PRIVILEGES | +| SESSION_STATUS | +| SESSION_VARIABLES | +| STATISTICS | +| TABLES | +| TABLE_CONSTRAINTS | +| TABLE_PRIVILEGES | +| TRIGGERS | +| USER_PRIVILEGES | +| VIEWS | +| PBXT_STATISTICS | ++---------------------------------------+ +Wildcard: inf_rmation_schema ++--------------------+ +| Databases | ++--------------------+ +| information_schema | ++--------------------+ +SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; +table_schema count(*) +information_schema 29 +mysql 22 === modified file 'mysql-test/r/information_schema_db.result' --- a/mysql-test/r/information_schema_db.result 2008-10-02 10:08:15 +0000 +++ b/mysql-test/r/information_schema_db.result 2009-04-08 16:55:26 +0000 @@ -2,37 +2,6 @@ drop table if exists t1,t2; drop view if exists v1,v2; drop function if exists f1; drop function if exists f2; -use INFORMATION_SCHEMA; -show tables; -Tables_in_information_schema -CHARACTER_SETS -COLLATIONS -COLLATION_CHARACTER_SET_APPLICABILITY -COLUMNS -COLUMN_PRIVILEGES -ENGINES -EVENTS -FILES -GLOBAL_STATUS -GLOBAL_VARIABLES -KEY_COLUMN_USAGE -PARTITIONS -PLUGINS -PROCESSLIST -PROFILING -REFERENTIAL_CONSTRAINTS -ROUTINES -SCHEMATA -SCHEMA_PRIVILEGES -SESSION_STATUS -SESSION_VARIABLES -STATISTICS -TABLES -TABLE_CONSTRAINTS -TABLE_PRIVILEGES -TRIGGERS -USER_PRIVILEGES -VIEWS show tables from INFORMATION_SCHEMA like 'T%'; Tables_in_information_schema (T%) TABLES === modified file 'mysql-test/r/innodb-autoinc.result' --- a/mysql-test/r/innodb-autoinc.result 2008-12-19 00:28:51 +0000 +++ b/mysql-test/r/innodb-autoinc.result 2009-04-08 16:55:26 +0000 @@ -197,7 +197,7 @@ c1 c2 5 9 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -230,7 +230,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -269,7 +269,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -282,7 +282,7 @@ SELECT * FROM t1; c1 -1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -315,7 +315,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -330,7 +330,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -370,7 +370,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -385,7 +385,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -419,7 +419,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -434,7 +434,7 @@ c1 1 9223372036854775794 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -452,7 +452,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -467,7 +467,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -485,7 +485,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -500,7 +500,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 5 auto_increment_offset 7 @@ -514,7 +514,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -533,7 +533,7 @@ c1 -9223372036854775806 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 3 auto_increment_offset 3 @@ -550,7 +550,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -568,7 +568,7 @@ SET @@SESSION.AUTO_INCREMENT_INCREMENT=1 Warnings: Warning 1292 Truncated incorrect auto_increment_increment value: '1152921504606846976' Warning 1292 Truncated incorrect auto_increment_offset value: '1152921504606846976' -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 65535 auto_increment_offset 65535 === modified file 'mysql-test/r/mysqlshow.result' --- a/mysql-test/r/mysqlshow.result 2007-04-27 20:45:01 +0000 +++ b/mysql-test/r/mysqlshow.result 2009-04-08 16:55:26 +0000 @@ -75,76 +75,4 @@ Database: test 2 rows in set. DROP TABLE t1, t2; -Database: information_schema -+---------------------------------------+ -| Tables | -+---------------------------------------+ -| CHARACTER_SETS | -| COLLATIONS | -| COLLATION_CHARACTER_SET_APPLICABILITY | -| COLUMNS | -| COLUMN_PRIVILEGES | -| ENGINES | -| EVENTS | -| FILES | -| GLOBAL_STATUS | -| GLOBAL_VARIABLES | -| KEY_COLUMN_USAGE | -| PARTITIONS | -| PLUGINS | -| PROCESSLIST | -| PROFILING | -| REFERENTIAL_CONSTRAINTS | -| ROUTINES | -| SCHEMATA | -| SCHEMA_PRIVILEGES | -| SESSION_STATUS | -| SESSION_VARIABLES | -| STATISTICS | -| TABLES | -| TABLE_CONSTRAINTS | -| TABLE_PRIVILEGES | -| TRIGGERS | -| USER_PRIVILEGES | -| VIEWS | -+---------------------------------------+ -Database: INFORMATION_SCHEMA -+---------------------------------------+ -| Tables | -+---------------------------------------+ -| CHARACTER_SETS | -| COLLATIONS | -| COLLATION_CHARACTER_SET_APPLICABILITY | -| COLUMNS | -| COLUMN_PRIVILEGES | -| ENGINES | -| EVENTS | -| FILES | -| GLOBAL_STATUS | -| GLOBAL_VARIABLES | -| KEY_COLUMN_USAGE | -| PARTITIONS | -| PLUGINS | -| PROCESSLIST | -| PROFILING | -| REFERENTIAL_CONSTRAINTS | -| ROUTINES | -| SCHEMATA | -| SCHEMA_PRIVILEGES | -| SESSION_STATUS | -| SESSION_VARIABLES | -| STATISTICS | -| TABLES | -| TABLE_CONSTRAINTS | -| TABLE_PRIVILEGES | -| TRIGGERS | -| USER_PRIVILEGES | -| VIEWS | -+---------------------------------------+ -Wildcard: inf_rmation_schema -+--------------------+ -| Databases | -+--------------------+ -| information_schema | -+--------------------+ End of 5.0 tests === modified file 'mysql-test/suite/rpl/r/rpl_auto_increment.result' --- a/mysql-test/suite/rpl/r/rpl_auto_increment.result 2009-01-14 08:27:32 +0000 +++ b/mysql-test/suite/rpl/r/rpl_auto_increment.result 2009-04-08 16:55:26 +0000 @@ -37,7 +37,7 @@ a b 32 6 drop table t1; set @@session.auto_increment_increment=100, @@session.auto_increment_offset=10; -show variables like "%auto_inc%"; +show variables like "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 === modified file 'mysql-test/t/information_schema.test' --- a/mysql-test/t/information_schema.test 2009-01-23 12:22:05 +0000 +++ b/mysql-test/t/information_schema.test 2009-04-08 16:55:26 +0000 @@ -41,7 +41,8 @@ create view v1 (c) as SELECT table_name FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND - table_name<>'ndb_apply_status'; + table_name<>'ndb_apply_status' AND + NOT (table_schema = 'INFORMATION_SCHEMA' AND table_name LIKE 'PBXT_%'); select * from v1; select c,table_name from v1 @@ -529,13 +530,6 @@ delete from mysql.db where user='mysqlte flush privileges; # -# Bug #9404 information_schema: Weird error messages -# with SELECT SUM() ... GROUP BY queries -# -SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; - - -# # TRIGGERS table test # create table t1 (i int, j int); @@ -898,39 +892,6 @@ DROP PROCEDURE p1; DROP USER mysql_bug20230@localhost; # -# Bug#18925: subqueries with MIN/MAX functions on INFORMARTION_SCHEMA -# - -SELECT t.table_name, c1.column_name - FROM information_schema.tables t - INNER JOIN - information_schema.columns c1 - ON t.table_schema = c1.table_schema AND - t.table_name = c1.table_name - WHERE t.table_schema = 'information_schema' AND - c1.ordinal_position = - ( SELECT COALESCE(MIN(c2.ordinal_position),1) - FROM information_schema.columns c2 - WHERE c2.table_schema = t.table_schema AND - c2.table_name = t.table_name AND - c2.column_name LIKE '%SCHEMA%' - ); -SELECT t.table_name, c1.column_name - FROM information_schema.tables t - INNER JOIN - information_schema.columns c1 - ON t.table_schema = c1.table_schema AND - t.table_name = c1.table_name - WHERE t.table_schema = 'information_schema' AND - c1.ordinal_position = - ( SELECT COALESCE(MIN(c2.ordinal_position),1) - FROM information_schema.columns c2 - WHERE c2.table_schema = 'information_schema' AND - c2.table_name = t.table_name AND - c2.column_name LIKE '%SCHEMA%' - ); - -# # Bug#21231: query with a simple non-correlated subquery over # INFORMARTION_SCHEMA.TABLES # @@ -1007,29 +968,6 @@ drop table t1,t2; # -# Bug#24630 Subselect query crashes mysqld -# -select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= -(select cast(table_name as char) from information_schema.tables - order by table_name limit 1) limit 1; - -select t.table_name, group_concat(t.table_schema, '.', t.table_name), - count(*) as num1 -from information_schema.tables t -inner join information_schema.columns c1 -on t.table_schema = c1.table_schema AND t.table_name = c1.table_name -where t.table_schema = 'information_schema' and - c1.ordinal_position = - (select isnull(c2.column_type) - - isnull(group_concat(c2.table_schema, '.', c2.table_name)) + - count(*) as num - from information_schema.columns c2 where - c2.table_schema='information_schema' and - (c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') - group by c2.column_type order by num limit 1) -group by t.table_name order by num1, t.table_name; - -# # Bug#28266 IS_UPDATABLE field on VIEWS table in I_S database is wrong # create table t1(f1 int); === added file 'mysql-test/t/information_schema_all_engines.test' --- a/mysql-test/t/information_schema_all_engines.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/information_schema_all_engines.test 2009-04-08 16:55:26 +0000 @@ -0,0 +1,79 @@ +# This file contains tests moved from information_schema.test and +# information_schema_db.test whose results depends on which engines are +# available (since these engines inject tables into INFORMATION_SCHEMA). + +--source include/not_embedded.inc +--source include/have_pbxt.inc + +use INFORMATION_SCHEMA; +--replace_result Tables_in_INFORMATION_SCHEMA Tables_in_information_schema +show tables; + +# +# Bug#18925: subqueries with MIN/MAX functions on INFORMARTION_SCHEMA +# + +SELECT t.table_name, c1.column_name + FROM information_schema.tables t + INNER JOIN + information_schema.columns c1 + ON t.table_schema = c1.table_schema AND + t.table_name = c1.table_name + WHERE t.table_schema = 'information_schema' AND + c1.ordinal_position = + ( SELECT COALESCE(MIN(c2.ordinal_position),1) + FROM information_schema.columns c2 + WHERE c2.table_schema = t.table_schema AND + c2.table_name = t.table_name AND + c2.column_name LIKE '%SCHEMA%' + ); +SELECT t.table_name, c1.column_name + FROM information_schema.tables t + INNER JOIN + information_schema.columns c1 + ON t.table_schema = c1.table_schema AND + t.table_name = c1.table_name + WHERE t.table_schema = 'information_schema' AND + c1.ordinal_position = + ( SELECT COALESCE(MIN(c2.ordinal_position),1) + FROM information_schema.columns c2 + WHERE c2.table_schema = 'information_schema' AND + c2.table_name = t.table_name AND + c2.column_name LIKE '%SCHEMA%' + ); + +# +# Bug#24630 Subselect query crashes mysqld +# +select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= +(select cast(table_name as char) from information_schema.tables + order by table_name limit 1) limit 1; + +select t.table_name, group_concat(t.table_schema, '.', t.table_name), + count(*) as num1 +from information_schema.tables t +inner join information_schema.columns c1 +on t.table_schema = c1.table_schema AND t.table_name = c1.table_name +where t.table_schema = 'information_schema' and + c1.ordinal_position = + (select isnull(c2.column_type) - + isnull(group_concat(c2.table_schema, '.', c2.table_name)) + + count(*) as num + from information_schema.columns c2 where + c2.table_schema='information_schema' and + (c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') + group by c2.column_type order by num limit 1) +group by t.table_name order by num1, t.table_name; + +# +# Bug #19147: mysqlshow INFORMATION_SCHEMA does not work +# +--exec $MYSQL_SHOW information_schema +--exec $MYSQL_SHOW INFORMATION_SCHEMA +--exec $MYSQL_SHOW inf_rmation_schema + +# +# Bug #9404 information_schema: Weird error messages +# with SELECT SUM() ... GROUP BY queries +# +SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; === modified file 'mysql-test/t/information_schema_db.test' --- a/mysql-test/t/information_schema_db.test 2008-10-02 09:37:07 +0000 +++ b/mysql-test/t/information_schema_db.test 2009-04-08 16:55:26 +0000 @@ -11,9 +11,6 @@ drop function if exists f1; drop function if exists f2; --enable_warnings -use INFORMATION_SCHEMA; ---replace_result Tables_in_INFORMATION_SCHEMA Tables_in_information_schema -show tables; --replace_result 'Tables_in_INFORMATION_SCHEMA (T%)' 'Tables_in_information_schema (T%)' show tables from INFORMATION_SCHEMA like 'T%'; create database `inf%`; === modified file 'mysql-test/t/innodb-autoinc.test' --- a/mysql-test/t/innodb-autoinc.test 2009-01-23 12:22:05 +0000 +++ b/mysql-test/t/innodb-autoinc.test 2009-04-08 16:55:26 +0000 @@ -156,7 +156,7 @@ DROP TABLE t1; # # Test changes to AUTOINC next value calculation SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES (NULL),(5),(NULL); @@ -173,7 +173,7 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(0); @@ -193,13 +193,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2), (NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -214,13 +214,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2); INSERT INTO t1 VALUES (NULL); INSERT INTO t1 VALUES (2); @@ -240,13 +240,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2),(NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -262,7 +262,7 @@ DROP TABLE t1; # Check for overflow handling when increment is > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -271,7 +271,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (9223372036854775794); #-- 2^63 - 14 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should just fit INSERT INTO t1 VALUES (NULL),(NULL),(NULL),(NULL),(NULL),(NULL); SELECT * FROM t1; @@ -281,7 +281,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offser are > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -290,7 +290,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't, it seems to be # a MySQL server bug. It wraps around to 0 for the last value. # See MySQL Bug# 39828 @@ -313,7 +313,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offset are odd numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -322,7 +322,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It fails with # a duplicate entry message because of a MySQL server bug, it wraps # around. See MySQL Bug# 39828, once MySQL fix the bug we can replace @@ -344,7 +344,7 @@ DROP TABLE t1; # and check for large -ve numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -355,7 +355,7 @@ INSERT INTO t1 VALUES(-92233720368547758 INSERT INTO t1 VALUES(-9223372036854775808); #-- -2^63 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (NULL),(NULL), (NULL); SELECT * FROM t1; DROP TABLE t1; @@ -364,7 +364,7 @@ DROP TABLE t1; # large numbers 2^60 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -373,7 +373,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551610); #-- 2^64 - 2 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1152921504606846976, @@SESSION.AUTO_INCREMENT_OFFSET=1152921504606846976; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It wraps around # and the autoinc values look bogus too. # See MySQL Bug# 39828, once MySQL fix the bug we can enable the error === modified file 'mysql-test/t/mysqlshow.test' --- a/mysql-test/t/mysqlshow.test 2006-07-22 03:29:25 +0000 +++ b/mysql-test/t/mysqlshow.test 2009-04-08 16:55:26 +0000 @@ -26,11 +26,4 @@ select "---- -v -v -t ------" as ""; --exec $MYSQL_SHOW test -v -v -t DROP TABLE t1, t2; -# -# Bug #19147: mysqlshow INFORMATION_SCHEMA does not work -# ---exec $MYSQL_SHOW information_schema ---exec $MYSQL_SHOW INFORMATION_SCHEMA ---exec $MYSQL_SHOW inf_rmation_schema - --echo End of 5.0 tests === modified file 'mysql-test/valgrind.supp' --- a/mysql-test/valgrind.supp 2008-11-21 14:21:50 +0000 +++ b/mysql-test/valgrind.supp 2009-04-08 16:55:26 +0000 @@ -379,7 +379,7 @@ } { - dlclose memory loss from plugin + dlclose memory loss from plugin variant 1 Memcheck:Leak fun:calloc fun:_dlerror_run @@ -388,6 +388,19 @@ } { + dlclose memory loss from plugin variant 2 + Memcheck:Leak + fun:malloc + fun:_dl_close_worker + fun:_dl_close + fun:_dl_catch_error + fun:_dlerror_run + fun:dlclose + fun:_Z15free_plugin_memP12st_plugin_dl + fun:_Z13plugin_dl_delPK19st_mysql_lex_string +} + +{ dlopen / ptread_cancel_init memory loss on Suse Linux 10.3 32/64 bit Memcheck:Leak fun:*alloc @@ -588,3 +601,19 @@ fun:dlopen* } +# +# In glibc (checked version 2.7), inet_ntoa allocates an 18-byte +# per-thread static buffer for the return value. That memory is freed +# at thread exit, however if called from the main thread, Valgrind +# does not see the free (test main.no-threads). +# +# Since inet_ntoa() does not allocate memory dynamically per-call, this +# suppression is safe. +# + +{ + inet_ntoa thread local storage + Memcheck:Leak + fun:malloc + fun:inet_ntoa +} === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2009-03-18 02:14:22 +0000 +++ b/sql/mysqld.cc 2009-04-08 16:55:26 +0000 @@ -4815,10 +4815,10 @@ static bool read_init_file(char *file_na DBUG_ENTER("read_init_file"); DBUG_PRINT("enter",("name: %s",file_name)); if (!(file=my_fopen(file_name,O_RDONLY,MYF(MY_WME)))) - return(1); + DBUG_RETURN(1); bootstrap(file); (void) my_fclose(file,MYF(MY_WME)); - return 0; + DBUG_RETURN(0); } @@ -4839,6 +4839,7 @@ void handle_connection_in_main_thread(TH safe_mutex_assert_owner(&LOCK_thread_count); thread_cache_size=0; // Safety threads.append(thd); + thd->connect_utime= my_micro_time(); (void) pthread_mutex_unlock(&LOCK_thread_count); handle_one_connection((void*) thd); } === modified file 'sql/set_var.cc' --- a/sql/set_var.cc 2009-03-12 22:27:35 +0000 +++ b/sql/set_var.cc 2009-04-08 16:55:26 +0000 @@ -1740,7 +1740,7 @@ bool sys_var::check_enum(THD *thd, set_v { char buff[STRING_BUFFER_USUAL_SIZE]; const char *value; - String str(buff, sizeof(buff), system_charset_info), *res; + String str(buff, sizeof(buff) - 1, system_charset_info), *res; if (var->value->result_type() == STRING_RESULT) { @@ -1777,7 +1777,7 @@ bool sys_var::check_set(THD *thd, set_va bool not_used; char buff[STRING_BUFFER_USUAL_SIZE], *error= 0; uint error_len= 0; - String str(buff, sizeof(buff), system_charset_info), *res; + String str(buff, sizeof(buff) - 1, system_charset_info), *res; if (var->value->result_type() == STRING_RESULT) { @@ -1795,7 +1795,7 @@ bool sys_var::check_set(THD *thd, set_va } var->save_result.ulong_value= ((ulong) - find_set(enum_names, res->c_ptr(), + find_set(enum_names, res->ptr(), res->length(), NULL, &error, &error_len, @@ -1942,7 +1942,7 @@ bool sys_var_thd_date_time_format::updat bool sys_var_thd_date_time_format::check(THD *thd, set_var *var) { char buff[STRING_BUFFER_USUAL_SIZE]; - String str(buff,sizeof(buff), system_charset_info), *res; + String str(buff,sizeof(buff) - 1, system_charset_info), *res; DATE_TIME_FORMAT *format; if (!(res=var->value->val_str(&str))) @@ -2047,7 +2047,7 @@ bool sys_var_collation::check(THD *thd, if (var->value->result_type() == STRING_RESULT) { char buff[STRING_BUFFER_USUAL_SIZE]; - String str(buff,sizeof(buff), system_charset_info), *res; + String str(buff,sizeof(buff) - 1, system_charset_info), *res; if (!(res=var->value->val_str(&str))) { my_error(ER_WRONG_VALUE_FOR_VAR, MYF(0), name, "NULL"); @@ -2082,7 +2082,7 @@ bool sys_var_character_set::check(THD *t if (var->value->result_type() == STRING_RESULT) { char buff[STRING_BUFFER_USUAL_SIZE]; - String str(buff,sizeof(buff), system_charset_info), *res; + String str(buff,sizeof(buff) - 1, system_charset_info), *res; if (!(res=var->value->val_str(&str))) { if (!nullable) @@ -3620,7 +3620,7 @@ bool sys_var_thd_storage_engine::check(T { char buff[STRING_BUFFER_USUAL_SIZE]; const char *value; - String str(buff, sizeof(buff), &my_charset_latin1), *res; + String str(buff, sizeof(buff) - 1, &my_charset_latin1), *res; var->save_result.plugin= NULL; if (var->value->result_type() == STRING_RESULT) @@ -3737,7 +3737,7 @@ sys_var_thd_sql_mode:: symbolic_mode_representation(THD *thd, ulonglong val, LEX_STRING *rep) { char buff[STRING_BUFFER_USUAL_SIZE*8]; - String tmp(buff, sizeof(buff), &my_charset_latin1); + String tmp(buff, sizeof(buff) - 1, &my_charset_latin1); tmp.length(0); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2009-02-19 09:01:25 +0000 +++ b/sql/sql_select.cc 2009-04-08 16:55:26 +0000 @@ -1994,8 +1994,17 @@ JOIN::exec() tmp_fields_list2, tmp_all_fields2, fields_list.elements, tmp_all_fields1)) DBUG_VOID_RETURN; - curr_join->tmp_fields_list2= tmp_fields_list2; - curr_join->tmp_all_fields2= tmp_all_fields2; +#ifdef HAVE_purify + /* + Some GCCs use memcpy() for struct assignment, even for x=x. + GCC bug 19410: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19410 + */ + if (curr_join != this) +#endif + { + curr_join->tmp_fields_list2= tmp_fields_list2; + curr_join->tmp_all_fields2= tmp_all_fields2; + } } curr_fields_list= &curr_join->tmp_fields_list2; curr_all_fields= &curr_join->tmp_all_fields2; === modified file 'sql/sql_string.h' --- a/sql/sql_string.h 2009-01-15 18:11:25 +0000 +++ b/sql/sql_string.h 2009-04-08 16:55:26 +0000 @@ -63,6 +63,11 @@ public: Ptr=(char*) str; str_length=(uint) strlen(str); Alloced_length=0; alloced=0; str_charset=cs; } + /* + NOTE: If one intend to use the c_ptr() method, the following two + contructors need the size of memory for STR to be at least LEN+1 (to make + room for zero termination). + */ String(const char *str,uint32 len, CHARSET_INFO *cs) { Ptr=(char*) str; str_length=len; Alloced_length=0; alloced=0; === modified file 'storage/archive/azio.c' --- a/storage/archive/azio.c 2007-12-12 13:16:59 +0000 +++ b/storage/archive/azio.c 2009-04-08 16:55:26 +0000 @@ -37,6 +37,40 @@ void putLong(File file, uLong x); uLong getLong(azio_stream *s); void read_header(azio_stream *s, unsigned char *buffer); +/* + Valgrind normally gives false alarms for zlib operations, in the form of + "conditional jump depends on uninitialised values" etc. The reason is + explained in the zlib FAQ (http://www.zlib.net/zlib_faq.html#faq36) + + "That is intentional for performance reasons, and the output of deflate + is not affected." + + Also discussed on a blog + (http://www.sirena.org.uk/log/2006/02/19/zlib-generating-valgrind-warnings/) + + "...loop unrolling in the zlib library causes the mentioned + “Conditional jump or move depends on uninitialised value(s)” + warnings. These are safe since the results of the comparison are + subsequently ignored..." + + "the results of the calculations are discarded by bounds checking done + after the loop exits" + + Fix by initializing the memory allocated by zlib when running under Valgrind. + + This fix is safe, since such memory is only used internally by zlib, so we + will not hide any bugs in mysql this way. +*/ +static void *az_allocator(void *dummy, uInt items, uInt size) +{ + return my_malloc((size_t)items*(size_t)size, IF_PURIFY(MY_ZEROFILL, MYF(0))); +} + +static void az_free(void *dummy, void *address) +{ + my_free(address, MYF(MY_ALLOW_ZERO_PTR)); +} + /* =========================================================================== Opens a gzip (.gz) file for reading or writing. The mode parameter is as in fopen ("rb" or "wb"). The file is given either by file descriptor @@ -52,8 +86,8 @@ int az_open (azio_stream *s, const char int level = Z_DEFAULT_COMPRESSION; /* compression level */ int strategy = Z_DEFAULT_STRATEGY; /* compression strategy */ - s->stream.zalloc = (alloc_func)0; - s->stream.zfree = (free_func)0; + s->stream.zalloc = az_allocator; + s->stream.zfree = az_free; s->stream.opaque = (voidpf)0; memset(s->inbuf, 0, AZ_BUFSIZE_READ); memset(s->outbuf, 0, AZ_BUFSIZE_WRITE);

1 0

Re: [Maria-developers] Working on microsecond patches
by Michael Widenius 08 Apr '09

08 Apr '09

Hi! >>>>> "Jeremiah" == Jeremiah Gowdy <basharteg(a)basharteg.com> writes: Vadim> Michael, Vadim> Thread-pool should be used very carefully. >> Vadim> We tested it and InnoDB hangs in sysbench benchmarks when count of Vadim> client connection > size of thread-pool. >> >> Yes, this a problem when you have more locks than threads. >> The innodb deadlock detector has to timeout the hanged items. >> Vadim> The problem is transaction state. Some connections may do internals Vadim> locks and after that could not enter to pool, because all slots are busy. Vadim> I expect you will have the same problem with Maria when it can fully Vadim> support transactions. >> >> That is one of the main reasons I added --extra-port to MariaDB >> This allows you to connect and check/kill things if you get a hang. >> >> So things are not perfect now, but at least a little better. >> >> In the future we have to also look at creating more pool-threads when >> all pool-threads gets locked by a handler. >> Jeremiah> This is a good solution IMO. You could base it on how long the next Jeremiah> queue item has been in the pool. So you determine a maximum latency Jeremiah> value that makes sense, and you grow the thread pool any time tasks are Jeremiah> waiting longer than that to execute, up to a certain number of threads Jeremiah> that would be your hard limit. Once you reach the hard limit, you Jeremiah> refuse to grow the pool further, and if no queries are being dequeued Jeremiah> after a long timeout, you could determine that things are deadlocked and Jeremiah> panic / start killing things, or let the user make that decision using Jeremiah> extra-port. Agree that this is a good way to do it. I will copy the above to a worklog item; Hope you don't mind.... http://askmonty.org/worklog/Server-RawIdeaBin/index.pl?tid=13 Regards, Monty

1 0

[Maria-developers] [patch 00/11] A number of fixes for buildbot-found problems
by knielsen＠knielsen-hq.org 08 Apr '09

08 Apr '09

Hi Monty, Sanja, all, I finally managed to get a valgrind test run to pass. I needed 11 patches to get there ... It would be really good to get a review of these so I can push, they are needed to get the Valgrind run to pass in buildbot. I think all of them are ok, but for at least some of them I would really like to get another pair of eyes before pushing as I'm not very familiar with the code they touch. Sanja, could you review first 2 patches? They are of mysql-test-run.pl, one of them is related to the previous mysqld.err fixes. Monty, could you review the remaining 9 patches? They are all fixes for Valgrind warnings. Especially the last 4 I am not 100% sure of are the best possible fix, I think you can help. Serg, I checked up on your comment on my earlier dbug.c patch (thanks!), found the problem you described and I think it is fixed in this version of the patch. Monty, I will commit as one commit before pushing, but due to large number of independent changes I found it convenient to split it up using quilt. - Kristian.

3 23

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2693)
by knielsen＠knielsen-hq.org 08 Apr '09

08 Apr '09

#At lp:maria 2693 knielsen(a)knielsen-hq.org 2009-04-08 Fix test suite following merge of PBXT The PBXT engine causes an extra table to appear in the INFORMATION_SCHEMA. This causes different output for a few test cases. Fix test suite to avoid any errors from this. added: mysql-test/include/have_pbxt.inc mysql-test/r/information_schema_all_engines.result mysql-test/t/information_schema_all_engines.test modified: mysql-test/extra/rpl_tests/rpl_auto_increment.test mysql-test/r/information_schema.result mysql-test/r/information_schema_db.result mysql-test/r/innodb-autoinc.result mysql-test/r/mysqlshow.result mysql-test/suite/rpl/r/rpl_auto_increment.result mysql-test/t/information_schema.test mysql-test/t/information_schema_db.test mysql-test/t/innodb-autoinc.test mysql-test/t/mysqlshow.test per-file messages: mysql-test/extra/rpl_tests/rpl_auto_increment.test Omit pbxt variables from show variables output. mysql-test/include/have_pbxt.inc Add facility to disable test if PBXT engine is not available. mysql-test/r/information_schema.result Omit PBXT INFORMATION_SCHEMA table from output. Move part of test to information_schema_all_engines. mysql-test/r/information_schema_all_engines.result New file for information_schema tests that depend on which engines are available. mysql-test/r/information_schema_db.result Move part of test to information_schema_all_engines. mysql-test/r/innodb-autoinc.result Omit pbxt variables from show variables output. mysql-test/r/mysqlshow.result Move part of test to information_schema_all_engines. mysql-test/suite/rpl/r/rpl_auto_increment.result Omit pbxt variables from show variables output. mysql-test/t/information_schema.test Omit PBXT INFORMATION_SCHEMA table from output. Move part of test to information_schema_all_engines. mysql-test/t/information_schema_all_engines.test New file for information_schema tests that depend on which engines are available. mysql-test/t/information_schema_db.test Omit PBXT INFORMATION_SCHEMA table from output. Move part of test to information_schema_all_engines. mysql-test/t/innodb-autoinc.test Omit pbxt variables from show variables output. mysql-test/t/mysqlshow.test Move part of test to information_schema_all_engines. === modified file 'mysql-test/extra/rpl_tests/rpl_auto_increment.test' --- a/mysql-test/extra/rpl_tests/rpl_auto_increment.test 2009-01-14 08:27:32 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_auto_increment.test 2009-04-08 13:40:20 +0000 @@ -34,7 +34,7 @@ connection master; drop table t1; set @@session.auto_increment_increment=100, @@session.auto_increment_offset=10; -show variables like "%auto_inc%"; +show variables like "auto_inc%"; eval create table t1 (a int not null auto_increment, primary key (a)) engine=$engine_type2; # Insert with 2 insert statements to get better testing of logging === added file 'mysql-test/include/have_pbxt.inc' --- a/mysql-test/include/have_pbxt.inc 1970-01-01 00:00:00 +0000 +++ b/mysql-test/include/have_pbxt.inc 2009-04-08 13:40:20 +0000 @@ -0,0 +1,4 @@ +disable_query_log; +--require r/true.require +select (support = 'YES' or support = 'DEFAULT') as `TRUE` from information_schema.engines where engine = 'pbxt'; +enable_query_log; === modified file 'mysql-test/r/information_schema.result' --- a/mysql-test/r/information_schema.result 2009-02-15 10:58:34 +0000 +++ b/mysql-test/r/information_schema.result 2009-04-08 13:40:20 +0000 @@ -40,7 +40,8 @@ create view v1 (c) as SELECT table_name FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND -table_name<>'ndb_apply_status'; +table_name<>'ndb_apply_status' AND +NOT (table_schema = 'INFORMATION_SCHEMA' AND table_name LIKE 'PBXT_%'); select * from v1; c CHARACTER_SETS @@ -850,7 +851,7 @@ VIEWS TABLE_NAME select delete from mysql.user where user='mysqltest_4'; delete from mysql.db where user='mysqltest_4'; flush privileges; -SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; +SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' AND NOT (table_schema = 'INFORMATION_SCHEMA' AND table_name LIKE 'PBXT_%') GROUP BY TABLE_SCHEMA; table_schema count(*) information_schema 28 mysql 22 @@ -1224,92 +1225,6 @@ f1() DROP FUNCTION f1; DROP PROCEDURE p1; DROP USER mysql_bug20230@localhost; -SELECT t.table_name, c1.column_name -FROM information_schema.tables t -INNER JOIN -information_schema.columns c1 -ON t.table_schema = c1.table_schema AND -t.table_name = c1.table_name -WHERE t.table_schema = 'information_schema' AND -c1.ordinal_position = -( SELECT COALESCE(MIN(c2.ordinal_position),1) -FROM information_schema.columns c2 -WHERE c2.table_schema = t.table_schema AND -c2.table_name = t.table_name AND -c2.column_name LIKE '%SCHEMA%' - ); -table_name column_name -CHARACTER_SETS CHARACTER_SET_NAME -COLLATIONS COLLATION_NAME -COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME -COLUMNS TABLE_SCHEMA -COLUMN_PRIVILEGES TABLE_SCHEMA -ENGINES ENGINE -EVENTS EVENT_SCHEMA -FILES TABLE_SCHEMA -GLOBAL_STATUS VARIABLE_NAME -GLOBAL_VARIABLES VARIABLE_NAME -KEY_COLUMN_USAGE CONSTRAINT_SCHEMA -PARTITIONS TABLE_SCHEMA -PLUGINS PLUGIN_NAME -PROCESSLIST ID -PROFILING QUERY_ID -REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA -ROUTINES ROUTINE_SCHEMA -SCHEMATA SCHEMA_NAME -SCHEMA_PRIVILEGES TABLE_SCHEMA -SESSION_STATUS VARIABLE_NAME -SESSION_VARIABLES VARIABLE_NAME -STATISTICS TABLE_SCHEMA -TABLES TABLE_SCHEMA -TABLE_CONSTRAINTS CONSTRAINT_SCHEMA -TABLE_PRIVILEGES TABLE_SCHEMA -TRIGGERS TRIGGER_SCHEMA -USER_PRIVILEGES GRANTEE -VIEWS TABLE_SCHEMA -SELECT t.table_name, c1.column_name -FROM information_schema.tables t -INNER JOIN -information_schema.columns c1 -ON t.table_schema = c1.table_schema AND -t.table_name = c1.table_name -WHERE t.table_schema = 'information_schema' AND -c1.ordinal_position = -( SELECT COALESCE(MIN(c2.ordinal_position),1) -FROM information_schema.columns c2 -WHERE c2.table_schema = 'information_schema' AND -c2.table_name = t.table_name AND -c2.column_name LIKE '%SCHEMA%' - ); -table_name column_name -CHARACTER_SETS CHARACTER_SET_NAME -COLLATIONS COLLATION_NAME -COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME -COLUMNS TABLE_SCHEMA -COLUMN_PRIVILEGES TABLE_SCHEMA -ENGINES ENGINE -EVENTS EVENT_SCHEMA -FILES TABLE_SCHEMA -GLOBAL_STATUS VARIABLE_NAME -GLOBAL_VARIABLES VARIABLE_NAME -KEY_COLUMN_USAGE CONSTRAINT_SCHEMA -PARTITIONS TABLE_SCHEMA -PLUGINS PLUGIN_NAME -PROCESSLIST ID -PROFILING QUERY_ID -REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA -ROUTINES ROUTINE_SCHEMA -SCHEMATA SCHEMA_NAME -SCHEMA_PRIVILEGES TABLE_SCHEMA -SESSION_STATUS VARIABLE_NAME -SESSION_VARIABLES VARIABLE_NAME -STATISTICS TABLE_SCHEMA -TABLES TABLE_SCHEMA -TABLE_CONSTRAINTS CONSTRAINT_SCHEMA -TABLE_PRIVILEGES TABLE_SCHEMA -TRIGGERS TRIGGER_SCHEMA -USER_PRIVILEGES GRANTEE -VIEWS TABLE_SCHEMA SELECT MAX(table_name) FROM information_schema.tables WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test'); MAX(table_name) VIEWS @@ -1355,55 +1270,6 @@ table_name t1 t2 drop table t1,t2; -select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= -(select cast(table_name as char) from information_schema.tables -order by table_name limit 1) limit 1; -f1 -1 -select t.table_name, group_concat(t.table_schema, '.', t.table_name), -count(*) as num1 -from information_schema.tables t -inner join information_schema.columns c1 -on t.table_schema = c1.table_schema AND t.table_name = c1.table_name -where t.table_schema = 'information_schema' and -c1.ordinal_position = -(select isnull(c2.column_type) - -isnull(group_concat(c2.table_schema, '.', c2.table_name)) + -count(*) as num -from information_schema.columns c2 where -c2.table_schema='information_schema' and -(c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') -group by c2.column_type order by num limit 1) -group by t.table_name order by num1, t.table_name; -table_name group_concat(t.table_schema, '.', t.table_name) num1 -CHARACTER_SETS information_schema.CHARACTER_SETS 1 -COLLATIONS information_schema.COLLATIONS 1 -COLLATION_CHARACTER_SET_APPLICABILITY information_schema.COLLATION_CHARACTER_SET_APPLICABILITY 1 -COLUMNS information_schema.COLUMNS 1 -COLUMN_PRIVILEGES information_schema.COLUMN_PRIVILEGES 1 -ENGINES information_schema.ENGINES 1 -EVENTS information_schema.EVENTS 1 -FILES information_schema.FILES 1 -GLOBAL_STATUS information_schema.GLOBAL_STATUS 1 -GLOBAL_VARIABLES information_schema.GLOBAL_VARIABLES 1 -KEY_COLUMN_USAGE information_schema.KEY_COLUMN_USAGE 1 -PARTITIONS information_schema.PARTITIONS 1 -PLUGINS information_schema.PLUGINS 1 -PROCESSLIST information_schema.PROCESSLIST 1 -PROFILING information_schema.PROFILING 1 -REFERENTIAL_CONSTRAINTS information_schema.REFERENTIAL_CONSTRAINTS 1 -ROUTINES information_schema.ROUTINES 1 -SCHEMATA information_schema.SCHEMATA 1 -SCHEMA_PRIVILEGES information_schema.SCHEMA_PRIVILEGES 1 -SESSION_STATUS information_schema.SESSION_STATUS 1 -SESSION_VARIABLES information_schema.SESSION_VARIABLES 1 -STATISTICS information_schema.STATISTICS 1 -TABLES information_schema.TABLES 1 -TABLE_CONSTRAINTS information_schema.TABLE_CONSTRAINTS 1 -TABLE_PRIVILEGES information_schema.TABLE_PRIVILEGES 1 -TRIGGERS information_schema.TRIGGERS 1 -USER_PRIVILEGES information_schema.USER_PRIVILEGES 1 -VIEWS information_schema.VIEWS 1 create table t1(f1 int); create view v1 as select f1+1 as a from t1; create table t2 (f1 int, f2 int); === added file 'mysql-test/r/information_schema_all_engines.result' --- a/mysql-test/r/information_schema_all_engines.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/information_schema_all_engines.result 2009-04-08 13:40:20 +0000 @@ -0,0 +1,244 @@ +use INFORMATION_SCHEMA; +show tables; +Tables_in_information_schema +CHARACTER_SETS +COLLATIONS +COLLATION_CHARACTER_SET_APPLICABILITY +COLUMNS +COLUMN_PRIVILEGES +ENGINES +EVENTS +FILES +GLOBAL_STATUS +GLOBAL_VARIABLES +KEY_COLUMN_USAGE +PARTITIONS +PLUGINS +PROCESSLIST +PROFILING +REFERENTIAL_CONSTRAINTS +ROUTINES +SCHEMATA +SCHEMA_PRIVILEGES +SESSION_STATUS +SESSION_VARIABLES +STATISTICS +TABLES +TABLE_CONSTRAINTS +TABLE_PRIVILEGES +TRIGGERS +USER_PRIVILEGES +VIEWS +PBXT_STATISTICS +SELECT t.table_name, c1.column_name +FROM information_schema.tables t +INNER JOIN +information_schema.columns c1 +ON t.table_schema = c1.table_schema AND +t.table_name = c1.table_name +WHERE t.table_schema = 'information_schema' AND +c1.ordinal_position = +( SELECT COALESCE(MIN(c2.ordinal_position),1) +FROM information_schema.columns c2 +WHERE c2.table_schema = t.table_schema AND +c2.table_name = t.table_name AND +c2.column_name LIKE '%SCHEMA%' + ); +table_name column_name +CHARACTER_SETS CHARACTER_SET_NAME +COLLATIONS COLLATION_NAME +COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME +COLUMNS TABLE_SCHEMA +COLUMN_PRIVILEGES TABLE_SCHEMA +ENGINES ENGINE +EVENTS EVENT_SCHEMA +FILES TABLE_SCHEMA +GLOBAL_STATUS VARIABLE_NAME +GLOBAL_VARIABLES VARIABLE_NAME +KEY_COLUMN_USAGE CONSTRAINT_SCHEMA +PARTITIONS TABLE_SCHEMA +PLUGINS PLUGIN_NAME +PROCESSLIST ID +PROFILING QUERY_ID +REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA +ROUTINES ROUTINE_SCHEMA +SCHEMATA SCHEMA_NAME +SCHEMA_PRIVILEGES TABLE_SCHEMA +SESSION_STATUS VARIABLE_NAME +SESSION_VARIABLES VARIABLE_NAME +STATISTICS TABLE_SCHEMA +TABLES TABLE_SCHEMA +TABLE_CONSTRAINTS CONSTRAINT_SCHEMA +TABLE_PRIVILEGES TABLE_SCHEMA +TRIGGERS TRIGGER_SCHEMA +USER_PRIVILEGES GRANTEE +VIEWS TABLE_SCHEMA +PBXT_STATISTICS ID +SELECT t.table_name, c1.column_name +FROM information_schema.tables t +INNER JOIN +information_schema.columns c1 +ON t.table_schema = c1.table_schema AND +t.table_name = c1.table_name +WHERE t.table_schema = 'information_schema' AND +c1.ordinal_position = +( SELECT COALESCE(MIN(c2.ordinal_position),1) +FROM information_schema.columns c2 +WHERE c2.table_schema = 'information_schema' AND +c2.table_name = t.table_name AND +c2.column_name LIKE '%SCHEMA%' + ); +table_name column_name +CHARACTER_SETS CHARACTER_SET_NAME +COLLATIONS COLLATION_NAME +COLLATION_CHARACTER_SET_APPLICABILITY COLLATION_NAME +COLUMNS TABLE_SCHEMA +COLUMN_PRIVILEGES TABLE_SCHEMA +ENGINES ENGINE +EVENTS EVENT_SCHEMA +FILES TABLE_SCHEMA +GLOBAL_STATUS VARIABLE_NAME +GLOBAL_VARIABLES VARIABLE_NAME +KEY_COLUMN_USAGE CONSTRAINT_SCHEMA +PARTITIONS TABLE_SCHEMA +PLUGINS PLUGIN_NAME +PROCESSLIST ID +PROFILING QUERY_ID +REFERENTIAL_CONSTRAINTS CONSTRAINT_SCHEMA +ROUTINES ROUTINE_SCHEMA +SCHEMATA SCHEMA_NAME +SCHEMA_PRIVILEGES TABLE_SCHEMA +SESSION_STATUS VARIABLE_NAME +SESSION_VARIABLES VARIABLE_NAME +STATISTICS TABLE_SCHEMA +TABLES TABLE_SCHEMA +TABLE_CONSTRAINTS CONSTRAINT_SCHEMA +TABLE_PRIVILEGES TABLE_SCHEMA +TRIGGERS TRIGGER_SCHEMA +USER_PRIVILEGES GRANTEE +VIEWS TABLE_SCHEMA +PBXT_STATISTICS ID +select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= +(select cast(table_name as char) from information_schema.tables +order by table_name limit 1) limit 1; +f1 +1 +select t.table_name, group_concat(t.table_schema, '.', t.table_name), +count(*) as num1 +from information_schema.tables t +inner join information_schema.columns c1 +on t.table_schema = c1.table_schema AND t.table_name = c1.table_name +where t.table_schema = 'information_schema' and +c1.ordinal_position = +(select isnull(c2.column_type) - +isnull(group_concat(c2.table_schema, '.', c2.table_name)) + +count(*) as num +from information_schema.columns c2 where +c2.table_schema='information_schema' and +(c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') +group by c2.column_type order by num limit 1) +group by t.table_name order by num1, t.table_name; +table_name group_concat(t.table_schema, '.', t.table_name) num1 +CHARACTER_SETS information_schema.CHARACTER_SETS 1 +COLLATIONS information_schema.COLLATIONS 1 +COLLATION_CHARACTER_SET_APPLICABILITY information_schema.COLLATION_CHARACTER_SET_APPLICABILITY 1 +COLUMNS information_schema.COLUMNS 1 +COLUMN_PRIVILEGES information_schema.COLUMN_PRIVILEGES 1 +ENGINES information_schema.ENGINES 1 +EVENTS information_schema.EVENTS 1 +FILES information_schema.FILES 1 +GLOBAL_STATUS information_schema.GLOBAL_STATUS 1 +GLOBAL_VARIABLES information_schema.GLOBAL_VARIABLES 1 +KEY_COLUMN_USAGE information_schema.KEY_COLUMN_USAGE 1 +PARTITIONS information_schema.PARTITIONS 1 +PBXT_STATISTICS information_schema.PBXT_STATISTICS 1 +PLUGINS information_schema.PLUGINS 1 +PROCESSLIST information_schema.PROCESSLIST 1 +PROFILING information_schema.PROFILING 1 +REFERENTIAL_CONSTRAINTS information_schema.REFERENTIAL_CONSTRAINTS 1 +ROUTINES information_schema.ROUTINES 1 +SCHEMATA information_schema.SCHEMATA 1 +SCHEMA_PRIVILEGES information_schema.SCHEMA_PRIVILEGES 1 +SESSION_STATUS information_schema.SESSION_STATUS 1 +SESSION_VARIABLES information_schema.SESSION_VARIABLES 1 +STATISTICS information_schema.STATISTICS 1 +TABLES information_schema.TABLES 1 +TABLE_CONSTRAINTS information_schema.TABLE_CONSTRAINTS 1 +TABLE_PRIVILEGES information_schema.TABLE_PRIVILEGES 1 +TRIGGERS information_schema.TRIGGERS 1 +USER_PRIVILEGES information_schema.USER_PRIVILEGES 1 +VIEWS information_schema.VIEWS 1 +Database: information_schema ++---------------------------------------+ +| Tables | ++---------------------------------------+ +| CHARACTER_SETS | +| COLLATIONS | +| COLLATION_CHARACTER_SET_APPLICABILITY | +| COLUMNS | +| COLUMN_PRIVILEGES | +| ENGINES | +| EVENTS | +| FILES | +| GLOBAL_STATUS | +| GLOBAL_VARIABLES | +| KEY_COLUMN_USAGE | +| PARTITIONS | +| PLUGINS | +| PROCESSLIST | +| PROFILING | +| REFERENTIAL_CONSTRAINTS | +| ROUTINES | +| SCHEMATA | +| SCHEMA_PRIVILEGES | +| SESSION_STATUS | +| SESSION_VARIABLES | +| STATISTICS | +| TABLES | +| TABLE_CONSTRAINTS | +| TABLE_PRIVILEGES | +| TRIGGERS | +| USER_PRIVILEGES | +| VIEWS | +| PBXT_STATISTICS | ++---------------------------------------+ +Database: INFORMATION_SCHEMA ++---------------------------------------+ +| Tables | ++---------------------------------------+ +| CHARACTER_SETS | +| COLLATIONS | +| COLLATION_CHARACTER_SET_APPLICABILITY | +| COLUMNS | +| COLUMN_PRIVILEGES | +| ENGINES | +| EVENTS | +| FILES | +| GLOBAL_STATUS | +| GLOBAL_VARIABLES | +| KEY_COLUMN_USAGE | +| PARTITIONS | +| PLUGINS | +| PROCESSLIST | +| PROFILING | +| REFERENTIAL_CONSTRAINTS | +| ROUTINES | +| SCHEMATA | +| SCHEMA_PRIVILEGES | +| SESSION_STATUS | +| SESSION_VARIABLES | +| STATISTICS | +| TABLES | +| TABLE_CONSTRAINTS | +| TABLE_PRIVILEGES | +| TRIGGERS | +| USER_PRIVILEGES | +| VIEWS | +| PBXT_STATISTICS | ++---------------------------------------+ +Wildcard: inf_rmation_schema ++--------------------+ +| Databases | ++--------------------+ +| information_schema | ++--------------------+ === modified file 'mysql-test/r/information_schema_db.result' --- a/mysql-test/r/information_schema_db.result 2008-10-02 10:08:15 +0000 +++ b/mysql-test/r/information_schema_db.result 2009-04-08 13:40:20 +0000 @@ -2,37 +2,6 @@ drop table if exists t1,t2; drop view if exists v1,v2; drop function if exists f1; drop function if exists f2; -use INFORMATION_SCHEMA; -show tables; -Tables_in_information_schema -CHARACTER_SETS -COLLATIONS -COLLATION_CHARACTER_SET_APPLICABILITY -COLUMNS -COLUMN_PRIVILEGES -ENGINES -EVENTS -FILES -GLOBAL_STATUS -GLOBAL_VARIABLES -KEY_COLUMN_USAGE -PARTITIONS -PLUGINS -PROCESSLIST -PROFILING -REFERENTIAL_CONSTRAINTS -ROUTINES -SCHEMATA -SCHEMA_PRIVILEGES -SESSION_STATUS -SESSION_VARIABLES -STATISTICS -TABLES -TABLE_CONSTRAINTS -TABLE_PRIVILEGES -TRIGGERS -USER_PRIVILEGES -VIEWS show tables from INFORMATION_SCHEMA like 'T%'; Tables_in_information_schema (T%) TABLES === modified file 'mysql-test/r/innodb-autoinc.result' --- a/mysql-test/r/innodb-autoinc.result 2008-12-19 00:28:51 +0000 +++ b/mysql-test/r/innodb-autoinc.result 2009-04-08 13:40:20 +0000 @@ -197,7 +197,7 @@ c1 c2 5 9 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -230,7 +230,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -269,7 +269,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -282,7 +282,7 @@ SELECT * FROM t1; c1 -1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -315,7 +315,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -330,7 +330,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -370,7 +370,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -385,7 +385,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -419,7 +419,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -434,7 +434,7 @@ c1 1 9223372036854775794 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -452,7 +452,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -467,7 +467,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -485,7 +485,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -500,7 +500,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 5 auto_increment_offset 7 @@ -514,7 +514,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -533,7 +533,7 @@ c1 -9223372036854775806 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 3 auto_increment_offset 3 @@ -550,7 +550,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -568,7 +568,7 @@ SET @@SESSION.AUTO_INCREMENT_INCREMENT=1 Warnings: Warning 1292 Truncated incorrect auto_increment_increment value: '1152921504606846976' Warning 1292 Truncated incorrect auto_increment_offset value: '1152921504606846976' -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 65535 auto_increment_offset 65535 === modified file 'mysql-test/r/mysqlshow.result' --- a/mysql-test/r/mysqlshow.result 2007-04-27 20:45:01 +0000 +++ b/mysql-test/r/mysqlshow.result 2009-04-08 13:40:20 +0000 @@ -75,76 +75,4 @@ Database: test 2 rows in set. DROP TABLE t1, t2; -Database: information_schema -+---------------------------------------+ -| Tables | -+---------------------------------------+ -| CHARACTER_SETS | -| COLLATIONS | -| COLLATION_CHARACTER_SET_APPLICABILITY | -| COLUMNS | -| COLUMN_PRIVILEGES | -| ENGINES | -| EVENTS | -| FILES | -| GLOBAL_STATUS | -| GLOBAL_VARIABLES | -| KEY_COLUMN_USAGE | -| PARTITIONS | -| PLUGINS | -| PROCESSLIST | -| PROFILING | -| REFERENTIAL_CONSTRAINTS | -| ROUTINES | -| SCHEMATA | -| SCHEMA_PRIVILEGES | -| SESSION_STATUS | -| SESSION_VARIABLES | -| STATISTICS | -| TABLES | -| TABLE_CONSTRAINTS | -| TABLE_PRIVILEGES | -| TRIGGERS | -| USER_PRIVILEGES | -| VIEWS | -+---------------------------------------+ -Database: INFORMATION_SCHEMA -+---------------------------------------+ -| Tables | -+---------------------------------------+ -| CHARACTER_SETS | -| COLLATIONS | -| COLLATION_CHARACTER_SET_APPLICABILITY | -| COLUMNS | -| COLUMN_PRIVILEGES | -| ENGINES | -| EVENTS | -| FILES | -| GLOBAL_STATUS | -| GLOBAL_VARIABLES | -| KEY_COLUMN_USAGE | -| PARTITIONS | -| PLUGINS | -| PROCESSLIST | -| PROFILING | -| REFERENTIAL_CONSTRAINTS | -| ROUTINES | -| SCHEMATA | -| SCHEMA_PRIVILEGES | -| SESSION_STATUS | -| SESSION_VARIABLES | -| STATISTICS | -| TABLES | -| TABLE_CONSTRAINTS | -| TABLE_PRIVILEGES | -| TRIGGERS | -| USER_PRIVILEGES | -| VIEWS | -+---------------------------------------+ -Wildcard: inf_rmation_schema -+--------------------+ -| Databases | -+--------------------+ -| information_schema | -+--------------------+ End of 5.0 tests === modified file 'mysql-test/suite/rpl/r/rpl_auto_increment.result' --- a/mysql-test/suite/rpl/r/rpl_auto_increment.result 2009-01-14 08:27:32 +0000 +++ b/mysql-test/suite/rpl/r/rpl_auto_increment.result 2009-04-08 13:40:20 +0000 @@ -37,7 +37,7 @@ a b 32 6 drop table t1; set @@session.auto_increment_increment=100, @@session.auto_increment_offset=10; -show variables like "%auto_inc%"; +show variables like "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 === modified file 'mysql-test/t/information_schema.test' --- a/mysql-test/t/information_schema.test 2009-01-23 12:22:05 +0000 +++ b/mysql-test/t/information_schema.test 2009-04-08 13:40:20 +0000 @@ -41,7 +41,8 @@ create view v1 (c) as SELECT table_name FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND - table_name<>'ndb_apply_status'; + table_name<>'ndb_apply_status' AND + NOT (table_schema = 'INFORMATION_SCHEMA' AND table_name LIKE 'PBXT_%'); select * from v1; select c,table_name from v1 @@ -532,7 +533,7 @@ flush privileges; # Bug #9404 information_schema: Weird error messages # with SELECT SUM() ... GROUP BY queries # -SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; +SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' AND NOT (table_schema = 'INFORMATION_SCHEMA' AND table_name LIKE 'PBXT_%') GROUP BY TABLE_SCHEMA; # @@ -898,39 +899,6 @@ DROP PROCEDURE p1; DROP USER mysql_bug20230@localhost; # -# Bug#18925: subqueries with MIN/MAX functions on INFORMARTION_SCHEMA -# - -SELECT t.table_name, c1.column_name - FROM information_schema.tables t - INNER JOIN - information_schema.columns c1 - ON t.table_schema = c1.table_schema AND - t.table_name = c1.table_name - WHERE t.table_schema = 'information_schema' AND - c1.ordinal_position = - ( SELECT COALESCE(MIN(c2.ordinal_position),1) - FROM information_schema.columns c2 - WHERE c2.table_schema = t.table_schema AND - c2.table_name = t.table_name AND - c2.column_name LIKE '%SCHEMA%' - ); -SELECT t.table_name, c1.column_name - FROM information_schema.tables t - INNER JOIN - information_schema.columns c1 - ON t.table_schema = c1.table_schema AND - t.table_name = c1.table_name - WHERE t.table_schema = 'information_schema' AND - c1.ordinal_position = - ( SELECT COALESCE(MIN(c2.ordinal_position),1) - FROM information_schema.columns c2 - WHERE c2.table_schema = 'information_schema' AND - c2.table_name = t.table_name AND - c2.column_name LIKE '%SCHEMA%' - ); - -# # Bug#21231: query with a simple non-correlated subquery over # INFORMARTION_SCHEMA.TABLES # @@ -1007,29 +975,6 @@ drop table t1,t2; # -# Bug#24630 Subselect query crashes mysqld -# -select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= -(select cast(table_name as char) from information_schema.tables - order by table_name limit 1) limit 1; - -select t.table_name, group_concat(t.table_schema, '.', t.table_name), - count(*) as num1 -from information_schema.tables t -inner join information_schema.columns c1 -on t.table_schema = c1.table_schema AND t.table_name = c1.table_name -where t.table_schema = 'information_schema' and - c1.ordinal_position = - (select isnull(c2.column_type) - - isnull(group_concat(c2.table_schema, '.', c2.table_name)) + - count(*) as num - from information_schema.columns c2 where - c2.table_schema='information_schema' and - (c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') - group by c2.column_type order by num limit 1) -group by t.table_name order by num1, t.table_name; - -# # Bug#28266 IS_UPDATABLE field on VIEWS table in I_S database is wrong # create table t1(f1 int); === added file 'mysql-test/t/information_schema_all_engines.test' --- a/mysql-test/t/information_schema_all_engines.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/information_schema_all_engines.test 2009-04-08 13:40:20 +0000 @@ -0,0 +1,73 @@ +# This file contains tests moved from information_schema.test and +# information_schema_db.test whose results depends on which engines are +# available (since these engines inject tables into INFORMATION_SCHEMA). + +--source include/not_embedded.inc +--source include/have_pbxt.inc + +use INFORMATION_SCHEMA; +--replace_result Tables_in_INFORMATION_SCHEMA Tables_in_information_schema +show tables; + +# +# Bug#18925: subqueries with MIN/MAX functions on INFORMARTION_SCHEMA +# + +SELECT t.table_name, c1.column_name + FROM information_schema.tables t + INNER JOIN + information_schema.columns c1 + ON t.table_schema = c1.table_schema AND + t.table_name = c1.table_name + WHERE t.table_schema = 'information_schema' AND + c1.ordinal_position = + ( SELECT COALESCE(MIN(c2.ordinal_position),1) + FROM information_schema.columns c2 + WHERE c2.table_schema = t.table_schema AND + c2.table_name = t.table_name AND + c2.column_name LIKE '%SCHEMA%' + ); +SELECT t.table_name, c1.column_name + FROM information_schema.tables t + INNER JOIN + information_schema.columns c1 + ON t.table_schema = c1.table_schema AND + t.table_name = c1.table_name + WHERE t.table_schema = 'information_schema' AND + c1.ordinal_position = + ( SELECT COALESCE(MIN(c2.ordinal_position),1) + FROM information_schema.columns c2 + WHERE c2.table_schema = 'information_schema' AND + c2.table_name = t.table_name AND + c2.column_name LIKE '%SCHEMA%' + ); + +# +# Bug#24630 Subselect query crashes mysqld +# +select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= +(select cast(table_name as char) from information_schema.tables + order by table_name limit 1) limit 1; + +select t.table_name, group_concat(t.table_schema, '.', t.table_name), + count(*) as num1 +from information_schema.tables t +inner join information_schema.columns c1 +on t.table_schema = c1.table_schema AND t.table_name = c1.table_name +where t.table_schema = 'information_schema' and + c1.ordinal_position = + (select isnull(c2.column_type) - + isnull(group_concat(c2.table_schema, '.', c2.table_name)) + + count(*) as num + from information_schema.columns c2 where + c2.table_schema='information_schema' and + (c2.column_type = 'varchar(7)' or c2.column_type = 'varchar(20)') + group by c2.column_type order by num limit 1) +group by t.table_name order by num1, t.table_name; + +# +# Bug #19147: mysqlshow INFORMATION_SCHEMA does not work +# +--exec $MYSQL_SHOW information_schema +--exec $MYSQL_SHOW INFORMATION_SCHEMA +--exec $MYSQL_SHOW inf_rmation_schema === modified file 'mysql-test/t/information_schema_db.test' --- a/mysql-test/t/information_schema_db.test 2008-10-02 09:37:07 +0000 +++ b/mysql-test/t/information_schema_db.test 2009-04-08 13:40:20 +0000 @@ -11,9 +11,6 @@ drop function if exists f1; drop function if exists f2; --enable_warnings -use INFORMATION_SCHEMA; ---replace_result Tables_in_INFORMATION_SCHEMA Tables_in_information_schema -show tables; --replace_result 'Tables_in_INFORMATION_SCHEMA (T%)' 'Tables_in_information_schema (T%)' show tables from INFORMATION_SCHEMA like 'T%'; create database `inf%`; === modified file 'mysql-test/t/innodb-autoinc.test' --- a/mysql-test/t/innodb-autoinc.test 2009-01-23 12:22:05 +0000 +++ b/mysql-test/t/innodb-autoinc.test 2009-04-08 13:40:20 +0000 @@ -156,7 +156,7 @@ DROP TABLE t1; # # Test changes to AUTOINC next value calculation SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES (NULL),(5),(NULL); @@ -173,7 +173,7 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(0); @@ -193,13 +193,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2), (NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -214,13 +214,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2); INSERT INTO t1 VALUES (NULL); INSERT INTO t1 VALUES (2); @@ -240,13 +240,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2),(NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -262,7 +262,7 @@ DROP TABLE t1; # Check for overflow handling when increment is > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -271,7 +271,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (9223372036854775794); #-- 2^63 - 14 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should just fit INSERT INTO t1 VALUES (NULL),(NULL),(NULL),(NULL),(NULL),(NULL); SELECT * FROM t1; @@ -281,7 +281,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offser are > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -290,7 +290,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't, it seems to be # a MySQL server bug. It wraps around to 0 for the last value. # See MySQL Bug# 39828 @@ -313,7 +313,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offset are odd numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -322,7 +322,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It fails with # a duplicate entry message because of a MySQL server bug, it wraps # around. See MySQL Bug# 39828, once MySQL fix the bug we can replace @@ -344,7 +344,7 @@ DROP TABLE t1; # and check for large -ve numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -355,7 +355,7 @@ INSERT INTO t1 VALUES(-92233720368547758 INSERT INTO t1 VALUES(-9223372036854775808); #-- -2^63 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (NULL),(NULL), (NULL); SELECT * FROM t1; DROP TABLE t1; @@ -364,7 +364,7 @@ DROP TABLE t1; # large numbers 2^60 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -373,7 +373,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551610); #-- 2^64 - 2 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1152921504606846976, @@SESSION.AUTO_INCREMENT_OFFSET=1152921504606846976; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It wraps around # and the autoinc values look bogus too. # See MySQL Bug# 39828, once MySQL fix the bug we can enable the error === modified file 'mysql-test/t/mysqlshow.test' --- a/mysql-test/t/mysqlshow.test 2006-07-22 03:29:25 +0000 +++ b/mysql-test/t/mysqlshow.test 2009-04-08 13:40:20 +0000 @@ -26,11 +26,4 @@ select "---- -v -v -t ------" as ""; --exec $MYSQL_SHOW test -v -v -t DROP TABLE t1, t2; -# -# Bug #19147: mysqlshow INFORMATION_SCHEMA does not work -# ---exec $MYSQL_SHOW information_schema ---exec $MYSQL_SHOW INFORMATION_SCHEMA ---exec $MYSQL_SHOW inf_rmation_schema - --echo End of 5.0 tests

1 0

[Maria-developers] mysql-test-run (was Re: BuildBot)
by Kristian Nielsen 08 Apr '09

08 Apr '09

Michael Widenius <michael.widenius(a)gmail.com> writes: > To go trough the things that are wrong with mysqltest: In my opinion, the single biggest problem with mysqltest is that it is a seperate program from the test suite driver mysql-test-run.pl. It's even written in a different language (C vs. Perl), and test cases themselves are written in a third language (the mysqltest language) + a mixture of SQL and even some shell scripts. That's _5_ different languages in a single application :-( This is a disaster for the structure of mysql-test-run. Such a large part of the code (and complexity) is just trying to handle communication between the various parts, which is really hard to do well due to different languages. The first step in any major improvement (in my opinion) should be to re-implement mysqltest in Perl inside mysql-test-run.pl. But that would be a project of several months, not something I'm eager to start just now ... > I like the philosophy behind it: Write a test in pure SQL and then > ensure that the generated result doesn't change. It's easy enough to > allow any developers to generate a test case. Yes. This is a very strong point for testing that correct results are produced by a query or similar. Keeping something like this is very important if thinking about extending or replacing the test framework. The weakness with mysqltest / mysql-test-run is not that it does this, it is that it does not provide additional/separate functinality for other kinds of tests that do not fit this simple framework well. > In particular, it's good that tests and results are different files as > it makes it easier to understand what are wrong when you get a failure. I am trying to understand what precisely you have in mind here... One big problem with other test suites I have seen is that if you want to debug a failure, you first have to spend a lot of time debugging and understanding the test case code just to understand what the problem actually is. Getting a diff of queries run and results produced like mysqltest does makes this step trivial (if the problem is wrong results from a query). Also the --record functionality for generating .result files can be really good to ease the production of test case and also make it more likely the developer will not be tempted to insufficiently check the results (everything is checked by default). It does have the big risk of not properly checking that the committed .result is in fact correct, something I have seen more than once. This does not really have to mean using separate files. I have in mind something like a Perl-based suite where I could do: query <<SQL, <<RESULT SELECT a, b FROM t1 JOIN t2 ON (a = b) ORDER BY a, b; SQL RECORDME RESULT and `mysql-test-run --record` would rewrite this as query <<SQL, <<RESULT SELECT a, b FROM t1 JOIN t2 ON (a = b) ORDER BY a, b; SQL a b 1 1 2 2 4 4 RESULT And in case of different result, it could output a diff of just the RESULT section of the failing query, along with the query. It could also give the correct line number in the .test file, avoiding the need to search the .test file for the query that produced the diff. I think that might make it even easier than separate result file. Something like that would of course allow putting arbitrary Perl code in-between running queries (with lots of useful modules available) where this is more appropriate. This would eliminate the need for the mysqltest language for new test cases, which is one less language again. And it would be quite easy to have a backwards-compatible emulation of the old .test and .result files (just a small wrapper that reads the .test and .result files and passes the contents to the query() function). > One benefit is that you can usually run a test by just piping it > trough the normal MySQL client, which simplifies testing and > debugging a lot! Yes, this should be kept for simple stuff, very important. - Kristian.

3 3

Re: [Maria-developers] Maria binary builds
by Kristian Nielsen 07 Apr '09

07 Apr '09

Hi Vadim, Just an update on what I have been doing with binaries and Buildbot. I have now set up Buildbot to build and test our trees on Launchpad after every push (similar to Pushbuild at MySQL/Sun if you are familiar with that): http://askmonty.org/buildbot/index.html For now we have only few (one) build slave. The plan of course is to add more, hopefully some of them volunteered from the community (hint :-). I would be happy to make the infrastructure available to you as well, ie. to set it up to also track your Launcpad trees and build on your build slaves with your scripts and options. But you may prefer to control the buildbot master yourself; just let me know if you need a copy of my Buildbot configuration files or if I can help with any questions in setting it up. I am currently working on getting all existing test failures visible in the Buildbot runs fixed (Valgrind errors mostly). After that (next week I hope) I plan to turn to working on getting Buildbot to also produce binaries. I will need to do some research in which platforms we need to build for (Debian/Ubuntu/CentOS/Suse/Generic? x86/amd64? Any others?), and which configure / build options to use. Any help you can provide here would be much appreciated. I have lots of experience with scripting this kind of thing, but less so with the details of how people actually use the binaries and what they need (I have mostly build from source myself even for production). I also plan to look at what official Debian/Ubuntu packages provide, and at the scripts from OurDelta. My vision is to have production and test of binaries fully integrated into our Buildbot infrastructure. I want it fully automated, so that binaries are made after every push, and so that problems with the binaries are dealt with as an integrated part of development just like any other problem. Ideally, releasing binaries would then just be a matter of picking the right files from the buildbot archives. It will take some work to get there of course, but at least that is the vision. >From what I have seen of Buildbot so far, I am hopeful that it will be sufficient for our needs. It seems to have all of the right ideas, and many of the necessary features are already there. Some stuff will be missing, and my plan is to spend the time to extend it as needed. So for concrete plans next week, these currently are: 1. Setup build of generic binaries (.tar.gz) for Linux x86/amd64. For this I need to understand better how to deal with external dependencies. Should I link statically with libc? My impression is not to do this as I think this causes big problems with any type of plugin, but I also seem to recall that existing generic packages do this. Alternatively, I need to understand which version of glibc to use. 2. Work on getting build slave machines set up for this. 3. Get a first version running on Buildbot that produces the binaries. After this I think will be Debian and Ubuntu binaries with apt-able repositories, and then RPMs. I think that's it for an update of current plans. Let me know if you have any questions/comments/suggestions. Especially any help on exact CFLAGS and ./configure arguments to use would be very helpful. - Kristian.

3 3

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (sanja:2723)
by sanja＠askmonty.org 03 Apr '09

03 Apr '09

#At lp:maria/5.2 2723 sanja(a)askmonty.org 2009-04-03 [merge] 5.1 -> 5.2 merge modified: mysys/my_malloc.c mysys/my_once.c mysys/my_realloc.c tests/fork_big2.pl === modified file 'mysys/my_malloc.c' --- a/mysys/my_malloc.c 2007-10-02 07:32:33 +0000 +++ b/mysys/my_malloc.c 2009-03-31 08:06:51 +0000 @@ -13,6 +13,9 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +/* my_global.h may define SAFEMALLOC (through my_config.h). */ +#include <my_global.h> + #ifdef SAFEMALLOC /* We don't need SAFEMALLOC here */ #undef SAFEMALLOC #endif === modified file 'mysys/my_once.c' --- a/mysys/my_once.c 2007-05-10 09:59:39 +0000 +++ b/mysys/my_once.c 2009-03-31 08:06:51 +0000 @@ -15,6 +15,9 @@ /* Not MT-SAFE */ +/* my_global.h may define SAFEMALLOC (through my_config.h). */ +#include <my_global.h> + #ifdef SAFEMALLOC /* We don't need SAFEMALLOC here */ #undef SAFEMALLOC #endif === modified file 'mysys/my_realloc.c' --- a/mysys/my_realloc.c 2008-04-28 16:24:05 +0000 +++ b/mysys/my_realloc.c 2009-03-31 08:06:51 +0000 @@ -13,6 +13,9 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +/* my_global.h may define SAFEMALLOC (through my_config.h). */ +#include <my_global.h> + #ifdef SAFEMALLOC /* We don't need SAFEMALLOC here */ #undef SAFEMALLOC #endif === modified file 'tests/fork_big2.pl' --- a/tests/fork_big2.pl 2006-02-12 21:26:30 +0000 +++ b/tests/fork_big2.pl 2009-04-03 13:23:08 +0000 @@ -16,21 +16,21 @@ package main; $opt_skip_create=$opt_skip_in=$opt_verbose=$opt_fast_insert= $opt_lock_tables=$opt_debug=$opt_skip_drop=$opt_fast=$opt_force=0; -$opt_thread_factor=1; -$opt_insert=1; -$opt_select=6;$opt_join=4; -$opt_select_count=$opt_join_count=0; -$opt_update=1;$opt_delete=0; -$opt_flush=$opt_check=$opt_repair=$opt_alter=0; -$opt_join_range=100; +$opt_thread_factor=1; +$opt_insert=1; +$opt_select=6;$opt_join=4; +$opt_select_count=$opt_join_count=0; +$opt_update=1;$opt_delete=0; +$opt_flush=$opt_check=$opt_repair=$opt_alter=0; +$opt_join_range=100; $opt_resize_interval=0; $opt_time=0; $opt_host=$opt_user=$opt_password=""; $opt_db="test"; $opt_verbose=$opt_debug=$opt_lock_tables=$opt_fast_insert=$opt_fast=$opt_skip_in=$opt_force=undef; # Ignore warnings from these -GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", - "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", - "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", +GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", + "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", + "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", "flush=i", "check=i", "repair=i", "alter=i", "resize-interval=i", "max-join_range=i", "time=i") || die "Aborted"; print "Test of multiple connections that test the following things:\n"; @@ -48,20 +48,20 @@ srand 100; # Make random numbers repea #### #### Start timeing and start test -#### - +#### + $opt_insert*=$opt_thread_factor; -$opt_select*=$opt_thread_factor; -$opt_join*=$opt_thread_factor; -$opt_select_count*=$opt_thread_factor; -$opt_join_count*=$opt_thread_factor; -$opt_update*=$opt_thread_factor; -$opt_delete*=$opt_thread_factor; - -if ($opt_time == 0 && $opt_insert == 0) -{ - $opt_insert=1; -} +$opt_select*=$opt_thread_factor; +$opt_join*=$opt_thread_factor; +$opt_select_count*=$opt_thread_factor; +$opt_join_count*=$opt_thread_factor; +$opt_update*=$opt_thread_factor; +$opt_delete*=$opt_thread_factor; + +if ($opt_time == 0 && $opt_insert == 0) +{ + $opt_insert=1; +} $start_time=new Benchmark; $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", @@ -100,71 +100,71 @@ $|= 1; # Autoflush #### #### Start the tests #### -if ($opt_time != 0) -{ - test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; +if ($opt_time != 0) +{ + test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; } for ($i=0 ; $i < $opt_insert ; $i ++) { test_insert() if (($pid=fork()) == 0); $work{$pid}="insert"; -} +} $threads=$i; -for ($i=0 ; $i < $opt_select ; $i ++) -{ - test_select() if (($pid=fork()) == 0); $work{$pid}="select"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_join ; $i ++) -{ - test_join() if (($pid=fork()) == 0); $work{$pid}="join"; -} -$threads+=$i; +for ($i=0 ; $i < $opt_select ; $i ++) +{ + test_select() if (($pid=fork()) == 0); $work{$pid}="select"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_join ; $i ++) +{ + test_join() if (($pid=fork()) == 0); $work{$pid}="join"; +} +$threads+=$i; for ($i=0 ; $i < $opt_select_count ; $i ++) { test_select_count() if (($pid=fork()) == 0); $work{$pid}="select_count"; } -$threads+=$i; -for ($i=0 ; $i < $opt_join_count ; $i ++) -{ - test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_update ; $i ++) -{ - test_update() if (($pid=fork()) == 0); $work{$pid}="update"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_delete ; $i ++) -{ - test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_flush ; $i ++) -{ - test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_check ; $i ++) -{ - test_check() if (($pid=fork()) == 0); $work{$pid}="check"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_repair ; $i ++) -{ - test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_alter ; $i ++) -{ - test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; -} -$threads+=$i; +$threads+=$i; +for ($i=0 ; $i < $opt_join_count ; $i ++) +{ + test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_update ; $i ++) +{ + test_update() if (($pid=fork()) == 0); $work{$pid}="update"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_delete ; $i ++) +{ + test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_flush ; $i ++) +{ + test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_check ; $i ++) +{ + test_check() if (($pid=fork()) == 0); $work{$pid}="check"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_repair ; $i ++) +{ + test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_alter ; $i ++) +{ + test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; +} +$threads+=$i; if ($opt_resize_interval != 0) { test_resize() if (($pid=fork()) == 0); $work{$pid}="resize"; $threads+=1; } - + print "Started $threads threads\n"; $errors=0; @@ -172,17 +172,17 @@ $running_insert_threads=$opt_insert; while (($pid=wait()) != -1) { $ret=$?/256; - print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; - if ($opt_time == 0) + print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; + if ($opt_time == 0) { if ($work{$pid} =~ /^insert/) { if (!--$running_insert_threads) - { - + { + # Time to stop other threads signal_abort(); - } + } } } $errors++ if ($ret != 0); @@ -214,17 +214,17 @@ print "Total time: " . exit(0); -# -# Sleep and then abort other threads -# - -sub test_abort -{ - sleep($opt_time); - signal_abort(); - exit(0); -} - +# +# Sleep and then abort other threads +# + +sub test_abort +{ + sleep($opt_time); + signal_abort(); + exit(0); +} + # # Insert records in the table @@ -363,58 +363,58 @@ sub test_join $dbh->disconnect; $dbh=0; print "Test_join: Executed $count joins\n"; exit(0); -} - -# -# select records -# Do continously joins between the first and second for range and count selected rows -# - -sub test_join_count -{ - my ($dbh, $i, $j, $count, $loop); - - $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", - $opt_user, $opt_password, - { PrintError => 0}) || die $DBI::errstr; - - $count_query=make_count_query($numtables); - $count=0; - $loop=9999; - $sum=0; - - srand(); - - $i=0; - while (($i++ % 10) || !test_if_abort($dbh)) - { - if ($loop++ >= 10) - { - $loop=0; - $row_counts=simple_query($dbh, $count_query); - } - for ($j=0 ; $j < $numtables-1 ; $j++) - { - my ($id1)= int rand $row_counts->[$j]; - my ($id2)= int rand $row_counts->[$j]; - if ($id1 > $id2) - { - my $id0=$id1; $id1=$id2; $id2=$id0; - if ($id2-$id1 > $opt_join_range) - { - $id2=$id1+$opt_join_range; - } - } - my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); - $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); - $sum+=$row->[0]; - $count++; - } - } - $dbh->disconnect; $dbh=0; - print "Test_join_count: Executed $count joins: total $sum rows\n"; - exit(0); -} +} + +# +# select records +# Do continously joins between the first and second for range and count selected rows +# + +sub test_join_count +{ + my ($dbh, $i, $j, $count, $loop); + + $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", + $opt_user, $opt_password, + { PrintError => 0}) || die $DBI::errstr; + + $count_query=make_count_query($numtables); + $count=0; + $loop=9999; + $sum=0; + + srand(); + + $i=0; + while (($i++ % 10) || !test_if_abort($dbh)) + { + if ($loop++ >= 10) + { + $loop=0; + $row_counts=simple_query($dbh, $count_query); + } + for ($j=0 ; $j < $numtables-1 ; $j++) + { + my ($id1)= int rand $row_counts->[$j]; + my ($id2)= int rand $row_counts->[$j]; + if ($id1 > $id2) + { + my $id0=$id1; $id1=$id2; $id2=$id0; + if ($id2-$id1 > $opt_join_range) + { + $id2=$id1+$opt_join_range; + } + } + my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); + $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); + $sum+=$row->[0]; + $count++; + } + } + $dbh->disconnect; $dbh=0; + print "Test_join_count: Executed $count joins: total $sum rows\n"; + exit(0); +} #

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (sanja:2690)
by sanja＠askmonty.org 03 Apr '09

03 Apr '09

#At lp:maria 2690 sanja(a)askmonty.org 2009-04-03 Fixed test written by windows to get it working under linux (line endings) modified: tests/fork_big2.pl per-file messages: tests/fork_big2.pl Fixed test written by windows to get it working under linux (line endings) === modified file 'tests/fork_big2.pl' --- a/tests/fork_big2.pl 2006-02-12 21:26:30 +0000 +++ b/tests/fork_big2.pl 2009-04-03 13:23:08 +0000 @@ -16,21 +16,21 @@ package main; $opt_skip_create=$opt_skip_in=$opt_verbose=$opt_fast_insert= $opt_lock_tables=$opt_debug=$opt_skip_drop=$opt_fast=$opt_force=0; -$opt_thread_factor=1; -$opt_insert=1; -$opt_select=6;$opt_join=4; -$opt_select_count=$opt_join_count=0; -$opt_update=1;$opt_delete=0; -$opt_flush=$opt_check=$opt_repair=$opt_alter=0; -$opt_join_range=100; +$opt_thread_factor=1; +$opt_insert=1; +$opt_select=6;$opt_join=4; +$opt_select_count=$opt_join_count=0; +$opt_update=1;$opt_delete=0; +$opt_flush=$opt_check=$opt_repair=$opt_alter=0; +$opt_join_range=100; $opt_resize_interval=0; $opt_time=0; $opt_host=$opt_user=$opt_password=""; $opt_db="test"; $opt_verbose=$opt_debug=$opt_lock_tables=$opt_fast_insert=$opt_fast=$opt_skip_in=$opt_force=undef; # Ignore warnings from these -GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", - "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", - "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", +GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", + "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", + "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", "flush=i", "check=i", "repair=i", "alter=i", "resize-interval=i", "max-join_range=i", "time=i") || die "Aborted"; print "Test of multiple connections that test the following things:\n"; @@ -48,20 +48,20 @@ srand 100; # Make random numbers repea #### #### Start timeing and start test -#### - +#### + $opt_insert*=$opt_thread_factor; -$opt_select*=$opt_thread_factor; -$opt_join*=$opt_thread_factor; -$opt_select_count*=$opt_thread_factor; -$opt_join_count*=$opt_thread_factor; -$opt_update*=$opt_thread_factor; -$opt_delete*=$opt_thread_factor; - -if ($opt_time == 0 && $opt_insert == 0) -{ - $opt_insert=1; -} +$opt_select*=$opt_thread_factor; +$opt_join*=$opt_thread_factor; +$opt_select_count*=$opt_thread_factor; +$opt_join_count*=$opt_thread_factor; +$opt_update*=$opt_thread_factor; +$opt_delete*=$opt_thread_factor; + +if ($opt_time == 0 && $opt_insert == 0) +{ + $opt_insert=1; +} $start_time=new Benchmark; $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", @@ -100,71 +100,71 @@ $|= 1; # Autoflush #### #### Start the tests #### -if ($opt_time != 0) -{ - test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; +if ($opt_time != 0) +{ + test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; } for ($i=0 ; $i < $opt_insert ; $i ++) { test_insert() if (($pid=fork()) == 0); $work{$pid}="insert"; -} +} $threads=$i; -for ($i=0 ; $i < $opt_select ; $i ++) -{ - test_select() if (($pid=fork()) == 0); $work{$pid}="select"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_join ; $i ++) -{ - test_join() if (($pid=fork()) == 0); $work{$pid}="join"; -} -$threads+=$i; +for ($i=0 ; $i < $opt_select ; $i ++) +{ + test_select() if (($pid=fork()) == 0); $work{$pid}="select"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_join ; $i ++) +{ + test_join() if (($pid=fork()) == 0); $work{$pid}="join"; +} +$threads+=$i; for ($i=0 ; $i < $opt_select_count ; $i ++) { test_select_count() if (($pid=fork()) == 0); $work{$pid}="select_count"; } -$threads+=$i; -for ($i=0 ; $i < $opt_join_count ; $i ++) -{ - test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_update ; $i ++) -{ - test_update() if (($pid=fork()) == 0); $work{$pid}="update"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_delete ; $i ++) -{ - test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_flush ; $i ++) -{ - test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_check ; $i ++) -{ - test_check() if (($pid=fork()) == 0); $work{$pid}="check"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_repair ; $i ++) -{ - test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_alter ; $i ++) -{ - test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; -} -$threads+=$i; +$threads+=$i; +for ($i=0 ; $i < $opt_join_count ; $i ++) +{ + test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_update ; $i ++) +{ + test_update() if (($pid=fork()) == 0); $work{$pid}="update"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_delete ; $i ++) +{ + test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_flush ; $i ++) +{ + test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_check ; $i ++) +{ + test_check() if (($pid=fork()) == 0); $work{$pid}="check"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_repair ; $i ++) +{ + test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_alter ; $i ++) +{ + test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; +} +$threads+=$i; if ($opt_resize_interval != 0) { test_resize() if (($pid=fork()) == 0); $work{$pid}="resize"; $threads+=1; } - + print "Started $threads threads\n"; $errors=0; @@ -172,17 +172,17 @@ $running_insert_threads=$opt_insert; while (($pid=wait()) != -1) { $ret=$?/256; - print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; - if ($opt_time == 0) + print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; + if ($opt_time == 0) { if ($work{$pid} =~ /^insert/) { if (!--$running_insert_threads) - { - + { + # Time to stop other threads signal_abort(); - } + } } } $errors++ if ($ret != 0); @@ -214,17 +214,17 @@ print "Total time: " . exit(0); -# -# Sleep and then abort other threads -# - -sub test_abort -{ - sleep($opt_time); - signal_abort(); - exit(0); -} - +# +# Sleep and then abort other threads +# + +sub test_abort +{ + sleep($opt_time); + signal_abort(); + exit(0); +} + # # Insert records in the table @@ -363,58 +363,58 @@ sub test_join $dbh->disconnect; $dbh=0; print "Test_join: Executed $count joins\n"; exit(0); -} - -# -# select records -# Do continously joins between the first and second for range and count selected rows -# - -sub test_join_count -{ - my ($dbh, $i, $j, $count, $loop); - - $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", - $opt_user, $opt_password, - { PrintError => 0}) || die $DBI::errstr; - - $count_query=make_count_query($numtables); - $count=0; - $loop=9999; - $sum=0; - - srand(); - - $i=0; - while (($i++ % 10) || !test_if_abort($dbh)) - { - if ($loop++ >= 10) - { - $loop=0; - $row_counts=simple_query($dbh, $count_query); - } - for ($j=0 ; $j < $numtables-1 ; $j++) - { - my ($id1)= int rand $row_counts->[$j]; - my ($id2)= int rand $row_counts->[$j]; - if ($id1 > $id2) - { - my $id0=$id1; $id1=$id2; $id2=$id0; - if ($id2-$id1 > $opt_join_range) - { - $id2=$id1+$opt_join_range; - } - } - my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); - $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); - $sum+=$row->[0]; - $count++; - } - } - $dbh->disconnect; $dbh=0; - print "Test_join_count: Executed $count joins: total $sum rows\n"; - exit(0); -} +} + +# +# select records +# Do continously joins between the first and second for range and count selected rows +# + +sub test_join_count +{ + my ($dbh, $i, $j, $count, $loop); + + $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", + $opt_user, $opt_password, + { PrintError => 0}) || die $DBI::errstr; + + $count_query=make_count_query($numtables); + $count=0; + $loop=9999; + $sum=0; + + srand(); + + $i=0; + while (($i++ % 10) || !test_if_abort($dbh)) + { + if ($loop++ >= 10) + { + $loop=0; + $row_counts=simple_query($dbh, $count_query); + } + for ($j=0 ; $j < $numtables-1 ; $j++) + { + my ($id1)= int rand $row_counts->[$j]; + my ($id2)= int rand $row_counts->[$j]; + if ($id1 > $id2) + { + my $id0=$id1; $id1=$id2; $id2=$id0; + if ($id2-$id1 > $opt_join_range) + { + $id2=$id1+$opt_join_range; + } + } + my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); + $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); + $sum+=$row->[0]; + $count++; + } + } + $dbh->disconnect; $dbh=0; + print "Test_join_count: Executed $count joins: total $sum rows\n"; + exit(0); +} #

1 0

[Maria-developers] what is the best way to get started with a dashboard?
by Mark Nielsen 01 Apr '09

01 Apr '09

Hey guys, Where and how would I put up code for a MySQL Dashboard project I would like to start on askmonty.org or the launchpad page we have for maria? The only place I saw I could start a project easily was on launchpad, so I made my own project on there. Is there a place I can setup a directory or page to start my own project relating to MySQL and Maria? A little but about the project .... I have been making scripts and dashboards for MySQL to help debug issues in MySQL for quite a few years. Now I want to bring it all together and open source it instead of letting it stay closed source at the companies I worked for where they *promised* they would open source the software ---- which never happend,. I have vacation coming up, so I plan on getting the skeleton started then. I already have a plan set out, I just need to do it. Thanks! Mark -- ---------------------------------- AIM: setthecleanser

2 1

[Maria-developers] Where do i start
by Rui Moreira 31 Mar '09

31 Mar '09

I there i am new to the mailing list and also new to the development of maria (have been using mysql for a while though) so the question is ... where do i start ? Thank you

2 1

Re: [Maria-developers] dutko-adam wants to join
by Kristian Nielsen 31 Mar '09

31 Mar '09

Maria developers <noreply(a)launchpad.net> writes: > Adam M Dutko (dutko-adam) wants to be a member of Maria developers > (maria-developers), but this is a moderated team, so that membership has > to be approved. You can approve, decline or leave it as proposed by > following the link below. > > https://launchpad.net/~maria-developers/+member/dutko-adam I tried approving this request, but I got an error from launchpad following the above link. On launchpad, the account is listed as Adam M Dutko (styluseater-merged) https://launchpad.net/~styluseater-merged and this also does not work in launchpad. Seems there is something wrong with the account on launchpad? If you still want to join maria-developers, I suggest trying to request membership again... - Kristian.

1 0

[Maria-developers] changes in virtual column patch
by Oleksandr "Sanja" Byelkin 30 Mar '09

30 Mar '09

Hi! Here is first version of my changes of virtual column patch. I also propose make following changes: 1) to make syntax even more compatible: - Add optional 'GENERATED ALWAYS' instead of removed mandatory 'VIRTUAL'. - Replase 'STORED' keyword with 'MATERIALIZED' and maybe make decision if nothing mention after field definition whether it will be stored depended on sql_mode (in oracle mode it will be stored) and add 'VIRTUAL' key word which force our current default (pure virtual column). 2) rename virtual_column_info class according MySQL codding style for classes (Virtual_column_info) What I've done: - VIRTUAL removed. - in error messages virtual replaced with computed. - test suite fixed according to above changes. - numeric error codes in --error in test suite replaced by sumbolic names - added forgotten Item::check_vcol_func_processor() for many items (which lead for several crashing bugs). - strange 'switch' which tests only top most item type replaced with tree walking so now there is no difference in function checks (and so error messages) depending on where prohibited function used in the expression. - because of above check in fix_fields_vcol_func() removed by #ifdef PARANOID usage (should be fixed in the final patch). - fixed bug in altering computed field put in partitioning function expression - Added test for subquery in computed column expression which tests subquery allowance and return ER_VIRTUAL_COLUMN_FUNCTION_IS_NOT_ALLOWED instead of previously tested syntax error. === modified file 'BUILD/SETUP.sh' --- BUILD/SETUP.sh 2009-03-22 12:16:09 +0000 +++ BUILD/SETUP.sh 2009-03-24 16:06:06 +0000 @@ -169,7 +169,7 @@ max_no_embedded_configs="$SSL_LIBRARY -- max_no_ndb_configs="$SSL_LIBRARY --with-plugins=max-no-ndb --with-embedded-server --with-libevent" max_configs="$SSL_LIBRARY --with-plugins=max --with-embedded-server -with-libevent" # Disable NDB in maria max builds -max_configs=$max_no_ndb_configs +#max_configs=$max_no_ndb_configs # # CPU and platform specific compilation flags. === modified file 'include/mysql_com.h' --- include/mysql_com.h 2008-10-10 15:28:41 +0000 +++ include/mysql_com.h 2009-03-24 10:47:20 +0000 @@ -67,7 +67,14 @@ enum enum_server_command COM_END }; - +/* The length of the header part for each virtual column in the .frm file. */ +#define FRM_VCOL_HEADER_SIZE 3 +/* + Maximum length of the expression statement defined for virtual columns. +*/ +#define VIRTUAL_COLUMN_EXPRESSION_MAXLEN 255 - FRM_VCOL_HEADER_SIZE +/* sql type field stored in .frm files for each virtual field. */ +#define MYSQL_TYPE_VIRTUAL 245 /* Length of random string sent by server on handshake; this is also length of obfuscated password, recieved from client === modified file 'sql/field.cc' --- sql/field.cc 2009-02-19 09:01:25 +0000 +++ sql/field.cc 2009-03-26 21:46:28 +0000 @@ -1312,7 +1312,8 @@ Field::Field(uchar *ptr_arg,uint32 lengt key_start(0), part_of_key(0), part_of_key_not_clustered(0), part_of_sortkey(0), unireg_check(unireg_check_arg), field_length(length_arg), null_bit(null_bit_arg), - is_created_from_null_item(FALSE) + is_created_from_null_item(FALSE), + vcol_info(0), stored_in_db(TRUE) { flags=null_ptr ? 0: NOT_NULL_FLAG; comment.str= (char*) ""; @@ -9495,6 +9496,8 @@ void Create_field::init_for_tmp_table(en ((decimals_arg & FIELDFLAG_MAX_DEC) << FIELDFLAG_DEC_SHIFT) | (maybe_null ? FIELDFLAG_MAYBE_NULL : 0) | (is_unsigned ? 0 : FIELDFLAG_DECIMAL)); + vcol_info= 0; + stored_in_db= TRUE; } @@ -9514,6 +9517,7 @@ void Create_field::init_for_tmp_table(en @param fld_interval_list Interval list (if any) @param fld_charset Field charset @param fld_geom_type Field geometry type (if any) + @param fld_vcol_info Virtual column data @retval FALSE on success @@ -9526,7 +9530,8 @@ bool Create_field::init(THD *thd, char * uint fld_type_modifier, Item *fld_default_value, Item *fld_on_update_value, LEX_STRING *fld_comment, char *fld_change, List<String> *fld_interval_list, - CHARSET_INFO *fld_charset, uint fld_geom_type) + CHARSET_INFO *fld_charset, uint fld_geom_type, + virtual_column_info *fld_vcol_info) { uint sign_len, allowed_type_modifier= 0; ulong max_field_charlength= MAX_FIELD_CHARLENGTH; @@ -9557,6 +9562,34 @@ bool Create_field::init(THD *thd, char * interval_list.empty(); comment= *fld_comment; + vcol_info= fld_vcol_info; + stored_in_db= TRUE; + + /* Initialize data for a virtual field */ + if ((uchar)fld_type == (uchar)MYSQL_TYPE_VIRTUAL) + { + DBUG_ASSERT(vcol_info); + DBUG_ASSERT(vcol_info->expr_item); + stored_in_db= vcol_info->get_field_stored(); + /* + Walk through the Item tree checking if all items are valid + to be part of the virtual column + */ + if (vcol_info->expr_item->walk(&Item::check_vcol_func_processor, 0, NULL)) + { + my_error(ER_VIRTUAL_COLUMN_FUNCTION_IS_NOT_ALLOWED, MYF(0), field_name); + DBUG_RETURN(TRUE); + } + + /* + Make a field created for the real type. + Note that "real" and virtual fields differ from each other + only by Field::vcol_info, which is always 0 for normal columns. + vcol_info is updated for fields later in procedure open_binary_frm. + */ + sql_type= fld_type= vcol_info->get_real_type(); + } + /* Set NO_DEFAULT_VALUE_FLAG if this field doesn't have a default value and it is NOT NULL, not an AUTO_INCREMENT field and not a TIMESTAMP. @@ -9847,7 +9880,7 @@ bool Create_field::init(THD *thd, char * } case MYSQL_TYPE_DECIMAL: DBUG_ASSERT(0); /* Was obsolete */ - } + } /* Remember the value of length */ char_length= length; @@ -9946,7 +9979,6 @@ uint pack_length_to_packflag(uint type) return 0; // This shouldn't happen } - Field *make_field(TABLE_SHARE *share, uchar *ptr, uint32 field_length, uchar *null_pos, uchar null_bit, uint pack_flag, @@ -10140,6 +10172,8 @@ Create_field::Create_field(Field *old_fi charset= old_field->charset(); // May be NULL ptr comment= old_field->comment; decimals= old_field->decimals(); + vcol_info= old_field->vcol_info; + stored_in_db= old_field->stored_in_db; /* Fix if the original table had 4 byte pointer blobs */ if (flags & BLOB_FLAG) === modified file 'sql/field.h' --- sql/field.h 2009-02-19 09:01:25 +0000 +++ sql/field.h 2009-03-26 21:35:01 +0000 @@ -45,6 +45,67 @@ inline uint get_set_pack_length(int elem return len > 4 ? 8 : len; } +class virtual_column_info: public Sql_alloc +{ +public: + Item *expr_item; + LEX_STRING expr_str; + Item *item_free_list; + virtual_column_info() + : expr_item(0), item_free_list(0), + field_type((enum enum_field_types)MYSQL_TYPE_VIRTUAL), + stored_in_db(FALSE), implicitly_stored_in_db(FALSE),data_inited(FALSE) + { + expr_str.str= NULL; + expr_str.length= 0; + }; + ~virtual_column_info() {} + enum_field_types get_real_type() + { + DBUG_ASSERT(data_inited); + return data_inited ? field_type : (enum enum_field_types)MYSQL_TYPE_VIRTUAL; + } + void set_field_type(enum_field_types fld_type) + { + /* Calling this function can only be done once. */ + DBUG_ASSERT(!data_inited); + data_inited= TRUE; + field_type= fld_type; + } + bool get_field_stored() + { + DBUG_ASSERT(data_inited); + return data_inited ? stored_in_db : TRUE; + } + void set_field_stored(bool stored) + { + stored_in_db= stored; + } + bool is_field_implicitly_stored() + { + return implicitly_stored_in_db; + } + void set_field_implicitly_stored() + { + implicitly_stored_in_db= TRUE; + } +private: + /* + The following data is only updated by the parser and read + when a Create_field object is created/initialized. + */ + enum_field_types field_type; /* Real field type*/ + /* Indication that the field is physically stored in the database*/ + my_bool stored_in_db; + /* Indication that the field used in partitioning expression */ + my_bool implicitly_stored_in_db; + /* + This flag is used to prevent other applications from + reading and using incorrect data. + */ + my_bool data_inited; +}; + class Field { Field(const Item &); /* Prevent use of these */ @@ -103,6 +164,15 @@ public: */ bool is_created_from_null_item; + /* Virtual column data */ + virtual_column_info *vcol_info; + /* + Indication that the field is phycically stored in tables + rather than just generated on SQL queries. + As of now, FALSE can only be set for generated-only virtual columns. + */ + bool stored_in_db; + Field(uchar *ptr_arg,uint32 length_arg,uchar *null_ptr_arg, uchar null_bit_arg, utype unireg_check_arg, const char *field_name_arg); @@ -2044,6 +2114,16 @@ public: uint8 row,col,sc_length,interval_id; // For rea_create_table uint offset,pack_flag; + + /* Virtual column expression statement */ + virtual_column_info *vcol_info; + /* + Indication that the field is phycically stored in tables + rather than just generated on SQL queries. + As of now, FALSE can only be set for generated-only virtual columns. + */ + bool stored_in_db; + Create_field() :after(0) {} Create_field(Field *field, Field *orig_field); /* Used to make a clone of this object for ALTER/CREATE TABLE */ @@ -2060,7 +2140,8 @@ public: char *decimals, uint type_modifier, Item *default_value, Item *on_update_value, LEX_STRING *comment, char *change, List<String> *interval_list, CHARSET_INFO *cs, - uint uint_geom_type); + uint uint_geom_type, + virtual_column_info *vcol_info); }; === modified file 'sql/filesort.cc' --- sql/filesort.cc 2009-02-19 09:01:25 +0000 +++ sql/filesort.cc 2009-03-24 10:47:20 +0000 @@ -563,6 +563,8 @@ static ha_rows find_all_keys(SORTPARAM * { if ((error= select->quick->get_next())) break; + if (!error) + update_virtual_fields_marked_for_write(sort_form); file->position(sort_form->record[0]); DBUG_EXECUTE_IF("debug_filesort", dbug_print_record(sort_form, TRUE);); } @@ -580,6 +582,8 @@ static ha_rows find_all_keys(SORTPARAM * else { error=file->rnd_next(sort_form->record[0]); + if (!error) + update_virtual_fields_marked_for_write(sort_form); if (!flag) { my_store_ptr(ref_pos,ref_length,record); // Position to row === modified file 'sql/ha_partition.cc' --- sql/ha_partition.cc 2009-02-19 09:01:25 +0000 +++ sql/ha_partition.cc 2009-03-24 10:47:20 +0000 @@ -2408,7 +2408,7 @@ int ha_partition::open(const char *name, DBUG_RETURN(1); m_start_key.length= 0; m_rec0= table->record[0]; - m_rec_length= table_share->reclength; + m_rec_length= table_share->stored_rec_length; alloc_len= m_tot_parts * (m_rec_length + PARTITION_BYTES_IN_POS); alloc_len+= table_share->max_key_length; if (!m_ordered_rec_buffer) === modified file 'sql/ha_partition.h' --- sql/ha_partition.h 2009-02-19 09:01:25 +0000 +++ sql/ha_partition.h 2009-03-24 10:47:20 +0000 @@ -245,6 +245,7 @@ public: DBUG_RETURN(0); } virtual void change_table_ptr(TABLE *table_arg, TABLE_SHARE *share); + bool check_if_supported_virtual_columns(void) { return TRUE;} virtual bool check_if_incompatible_data(HA_CREATE_INFO *create_info, uint table_changes); private: === modified file 'sql/handler.h' --- sql/handler.h 2009-02-19 09:01:25 +0000 +++ sql/handler.h 2009-03-24 10:47:20 +0000 @@ -1755,6 +1755,12 @@ public: LEX_STRING *engine_name() { return hton_name(ht); } + /* + This procedure defines if the storage engine supports virtual columns. + Default FALSE means "not supported". + */ + virtual bool check_if_supported_virtual_columns(void) { return FALSE;} + protected: /* Service methods for use by storage engines. */ void ha_statistic_increment(ulong SSV::*offset) const; === modified file 'sql/item.cc' --- sql/item.cc 2009-02-19 09:01:25 +0000 +++ sql/item.cc 2009-03-26 21:24:20 +0000 @@ -677,9 +677,26 @@ bool Item_field::register_field_in_read_ TABLE *table= (TABLE *) arg; if (field->table == table || !table) bitmap_set_bit(field->table->read_set, field->field_index); + if (field->vcol_info && field->vcol_info->expr_item) + return field->vcol_info->expr_item->walk(&Item::register_field_in_read_map, + 1, arg); return 0; } +/* + Mark field in bitmap supplied as *arg + +*/ + +bool Item_field::register_field_in_bitmap(uchar *arg) +{ + MY_BITMAP *bitmap= (MY_BITMAP *) arg; + DBUG_ASSERT(bitmap); + if (!bitmap) + return 1; + bitmap_set_bit(bitmap, field->field_index); + return 0; +} bool Item::check_cols(uint c) { @@ -4188,6 +4205,20 @@ error: return TRUE; } +/** + Marks virtual columns as implicitly stored +*/ + +bool Item_field::vcol_in_partition_func_processor(uchar *int_arg) +{ + DBUG_ASSERT(fixed); + if (field->vcol_info) + { + field->vcol_info->set_field_implicitly_stored(); + } + return TRUE; +} + Item *Item_field::safe_charset_converter(CHARSET_INFO *tocs) { === modified file 'sql/item.h' --- sql/item.h 2009-02-19 09:01:25 +0000 +++ sql/item.h 2009-03-26 20:59:23 +0000 @@ -889,6 +889,11 @@ public: virtual bool is_expensive_processor(uchar *arg) { return 0; } virtual bool register_field_in_read_map(uchar *arg) { return 0; } /* + The next function differs from the previous one that a bitmap to be updated + is passed as uchar *arg. + */ + virtual bool register_field_in_bitmap(uchar *arg) { return 0; } + /* Check if a partition function is allowed SYNOPSIS check_partition_func_processor() @@ -940,11 +945,28 @@ public: fields. */ virtual bool check_partition_func_processor(uchar *bool_arg) { return TRUE;} + virtual bool vcol_in_partition_func_processor(uchar *bool_arg) { return TRUE;} virtual bool subst_argument_checker(uchar **arg) - { + { if (*arg) - *arg= NULL; - return TRUE; + *arg= NULL; + return TRUE; + } + /* + Check if an expression/function is allowed for a virtual column + SYNOPSIS + check_vcol_func_processor() + int_arg is just ignored + RETURN VALUE + TRUE Function not accepted + FALSE Function accepted + */ + virtual bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); } virtual Item *equal_fields_propagator(uchar * arg) { return this; } @@ -1298,6 +1320,13 @@ public: { return value_item->send(protocol, str); } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_name_const::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; bool agg_item_collations(DTCollation &c, const char *name, @@ -1315,6 +1344,7 @@ public: virtual Item_num *neg()= 0; Item *safe_charset_converter(CHARSET_INFO *tocs); bool check_partition_func_processor(uchar *int_arg) { return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; #define NO_CACHED_FIELD_INDEX ((uint)(-1)) @@ -1474,7 +1504,10 @@ public: bool collect_item_field_processor(uchar * arg); bool find_item_in_field_list_processor(uchar *arg); bool register_field_in_read_map(uchar *arg); + bool register_field_in_bitmap(uchar *arg); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool vcol_in_partition_func_processor(uchar *int_arg); + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} void cleanup(); bool result_as_longlong() { @@ -1534,6 +1567,7 @@ public: Item *safe_charset_converter(CHARSET_INFO *tocs); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; class Item_null_result :public Item_null @@ -1547,7 +1581,14 @@ public: save_in_field(result_field, no_conversions); } bool check_partition_func_processor(uchar *int_arg) {return TRUE;} -}; + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_PRINT("info", (" Item name: %s", full_name())); + return TRUE; + } +}; /* Item represents one placeholder ('?') of prepared statement */ @@ -1688,6 +1729,7 @@ public: /** Item is a argument to a limit clause. */ bool limit_clause_param; void set_param_type_and_swap_value(Item_param *from); + }; @@ -1723,6 +1765,7 @@ public: { return (uint)(max_length - test(value < 0)); } bool eq(const Item *, bool binary_cmp) const; bool check_partition_func_processor(uchar *bool_arg) { return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -1741,6 +1784,7 @@ public: Item_num *neg (); uint decimal_precision() const { return max_length; } bool check_partition_func_processor(uchar *bool_arg) { return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -1782,6 +1826,7 @@ public: bool eq(const Item *, bool binary_cmp) const; void set_decimal_value(my_decimal *value_par); bool check_partition_func_processor(uchar *bool_arg) { return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -1939,6 +1984,7 @@ public: } virtual void print(String *str, enum_query_type query_type); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} /** Return TRUE if character-set-introducer was explicitly specified in the @@ -2006,6 +2052,13 @@ public: } bool check_partition_func_processor(uchar *int_arg) {return TRUE;} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_static_string_func::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -2017,6 +2070,14 @@ public: CHARSET_INFO *cs= NULL): Item_string(name_arg, length, cs) {} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_partition_func_safe_string::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_ASSERT(0); /* this is not possible */ + DBUG_RETURN(TRUE); + } }; @@ -2096,6 +2157,7 @@ public: bool eq(const Item *item, bool binary_cmp) const; virtual Item *safe_charset_converter(CHARSET_INFO *tocs); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -2126,6 +2188,7 @@ public: save_in_field(result_field, no_conversions); } void cleanup(); + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -2251,7 +2314,14 @@ public: if (ref && result_type() == ROW_RESULT) (*ref)->bring_value(); } - + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_ref::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_ASSERT(0); /* this is not possible */ + DBUG_RETURN(TRUE); + } }; @@ -2484,6 +2554,14 @@ public: table_map used_tables() const { return (table_map) 1L; } bool const_item() const { return 0; } bool is_null() { return null_value; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_sum::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_ASSERT(0); /* this is not possible */ + DBUG_RETURN(TRUE); + } }; @@ -2614,6 +2692,13 @@ public: return arg->walk(processor, walk_subquery, args) || (this->*processor)(args); } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_insert_value::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -2710,6 +2795,14 @@ private: BEFORE INSERT of BEFORE UPDATE trigger. */ bool read_only; + virtual bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_trigger_field::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_ASSERT(0); /* this is not possible */ + DBUG_RETURN(TRUE); + } }; @@ -2769,6 +2862,15 @@ public: { return this == item; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_cache::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_ASSERT(0); /* this is not possible */ + DBUG_RETURN(TRUE); + } + }; === modified file 'sql/item_func.cc' --- sql/item_func.cc 2009-02-19 09:01:25 +0000 +++ sql/item_func.cc 2009-03-24 10:47:20 +0000 @@ -3894,10 +3894,30 @@ bool Item_func_set_user_var::register_fi TABLE *table= (TABLE *) arg; if (result_field->table == table || !table) bitmap_set_bit(result_field->table->read_set, result_field->field_index); + if (result_field->vcol_info && result_field->vcol_info->expr_item) + return result_field->vcol_info-> + expr_item->walk(&Item::register_field_in_read_map, 1, arg); } return 0; } +/* + Mark field in bitmap supplied as *arg + +*/ + +bool Item_func_set_user_var::register_field_in_bitmap(uchar *arg) +{ + MY_BITMAP *bitmap = (MY_BITMAP *) arg; + DBUG_ASSERT(bitmap); + if (result_field) + { + if (!bitmap) + return 1; + bitmap_set_bit(bitmap, result_field->field_index); + } + return 0; +} /** Set value to user variable. === modified file 'sql/item_func.h' --- sql/item_func.h 2009-02-19 09:01:25 +0000 +++ sql/item_func.h 2009-03-24 20:19:28 +0000 @@ -339,6 +339,7 @@ public: void fix_length_and_dec(); bool fix_fields(THD *thd, Item **ref); longlong val_int() { DBUG_ASSERT(fixed == 1); return value; } + bool check_vcol_func_processor(uchar *int_arg) { return TRUE;} }; @@ -398,6 +399,7 @@ public: Item_func_additive_op(Item *a,Item *b) :Item_num_op(a,b) {} void result_precision(); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -433,6 +435,7 @@ public: my_decimal *decimal_op(my_decimal *); void result_precision(); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -465,6 +468,7 @@ public: } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -479,6 +483,7 @@ public: void result_precision(); void fix_length_and_dec(); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -495,6 +500,7 @@ public: void fix_num_length_and_dec(); uint decimal_precision() const { return args[0]->decimal_precision(); } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -508,6 +514,7 @@ public: const char *func_name() const { return "abs"; } void fix_length_and_dec(); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; // A class to handle logarithmic and trigonometric functions @@ -663,6 +670,7 @@ public: double real_op(); my_decimal *decimal_op(my_decimal *); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -675,6 +683,7 @@ public: double real_op(); my_decimal *decimal_op(my_decimal *); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; /* This handles round and truncate */ @@ -704,6 +713,13 @@ public: bool const_item() const { return 0; } void update_used_tables(); bool fix_fields(THD *thd, Item **ref); + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_rand::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } private: void seed_random (Item * val); }; @@ -986,6 +1002,13 @@ public: max_length= args[0]->max_length; } bool fix_fields(THD *thd, Item **ref); + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_last_insert_id::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -999,6 +1022,13 @@ public: const char *func_name() const { return "benchmark"; } void fix_length_and_dec() { max_length=1; maybe_null=0; } virtual void print(String *str, enum_query_type query_type); + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_benchmark::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -1014,6 +1044,13 @@ public: used_tables_cache|= RAND_TABLE_BIT; } longlong val_int(); + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_sleep::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -1263,6 +1300,13 @@ class Item_func_get_lock :public Item_in longlong val_int(); const char *func_name() const { return "get_lock"; } void fix_length_and_dec() { max_length=1; maybe_null=1;} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_get_lock::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; class Item_func_release_lock :public Item_int_func @@ -1273,6 +1317,13 @@ public: longlong val_int(); const char *func_name() const { return "release_lock"; } void fix_length_and_dec() { max_length=1; maybe_null=1;} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_release_lock::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; /* replication functions */ @@ -1286,6 +1337,13 @@ public: longlong val_int(); const char *func_name() const { return "master_pos_wait"; } void fix_length_and_dec() { max_length=21; maybe_null=1;} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_master_pos_wait::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -1356,6 +1414,7 @@ public: } void save_org_in_field(Field *field) { (void)save_in_field(field, 1, 0); } bool register_field_in_read_map(uchar *arg); + bool register_field_in_bitmap(uchar *arg); bool set_entry(THD *thd, bool create_if_not_exists); void cleanup(); }; @@ -1528,6 +1587,14 @@ public: bool fix_index(); void init_search(bool no_order); + bool check_vcol_func_processor(uchar *int_arg) + { + /* TODO: consider adding in support for the MATCH-based virtual columns */ + DBUG_ENTER("Item_func_match::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -1547,6 +1614,13 @@ public: longlong val_int(); const char *func_name() const { return "is_free_lock"; } void fix_length_and_dec() { decimals=0; max_length=1; maybe_null=1;} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_is_free_lock::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; class Item_func_is_used_lock :public Item_int_func @@ -1557,6 +1631,13 @@ public: longlong val_int(); const char *func_name() const { return "is_used_lock"; } void fix_length_and_dec() { decimals=0; max_length=10; maybe_null=1;} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_is_used_lock::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; /* For type casts */ @@ -1576,6 +1657,13 @@ public: longlong val_int(); const char *func_name() const { return "row_count"; } void fix_length_and_dec() { decimals= 0; maybe_null=0; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_row_count::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -1684,6 +1772,13 @@ public: { return sp_result_field; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_sp::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -1694,6 +1789,13 @@ public: longlong val_int(); const char *func_name() const { return "found_rows"; } void fix_length_and_dec() { decimals= 0; maybe_null=0; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_found_rows::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -1708,5 +1810,12 @@ public: void fix_length_and_dec() { max_length= 21; unsigned_flag=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_uuid_short::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; === modified file 'sql/item_row.h' --- sql/item_row.h 2008-02-22 10:30:33 +0000 +++ sql/item_row.h 2009-03-24 10:47:20 +0000 @@ -76,4 +76,5 @@ public: bool check_cols(uint c); bool null_inside() { return with_null; }; void bring_value(); -}; + bool check_vcol_func_processor(uchar *int_arg) {return FALSE; } + }; === modified file 'sql/item_strfunc.h' --- sql/item_strfunc.h 2009-02-19 09:01:25 +0000 +++ sql/item_strfunc.h 2009-03-24 10:47:20 +0000 @@ -335,6 +335,14 @@ public: String *val_str(String *); void fix_length_and_dec() { maybe_null=1; max_length = 13; } const char *func_name() const { return "encrypt"; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_encrypt::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } + }; #include "sql_crypt.h" @@ -372,6 +380,13 @@ public: call */ virtual const char *fully_qualified_func_name() const = 0; + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_sysconst::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -635,6 +650,13 @@ public: maybe_null=1; max_length=MAX_BLOB_WIDTH; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_load_file::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -831,5 +853,12 @@ public: } const char *func_name() const{ return "uuid"; } String *val_str(String *); + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_uuid::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; === modified file 'sql/item_subselect.h' --- sql/item_subselect.h 2008-02-22 10:30:33 +0000 +++ sql/item_subselect.h 2009-03-26 19:13:42 +0000 @@ -126,6 +126,13 @@ public: virtual void reset_value_registration() {} enum_parsing_place place() { return parsing_place; } bool walk(Item_processor processor, bool walk_subquery, uchar *arg); + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_sum::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } /** Get the SELECT_LEX structure associated with this Item. === modified file 'sql/item_sum.h' --- sql/item_sum.h 2008-12-09 19:43:10 +0000 +++ sql/item_sum.h 2009-03-25 21:24:46 +0000 @@ -384,6 +384,13 @@ public: Item *get_arg(int i) { return args[i]; } Item *set_arg(int i, THD *thd, Item *new_val); uint get_arg_count() { return arg_count; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_sum::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -664,6 +671,13 @@ public: } void fix_length_and_dec() {} enum Item_result result_type () const { return hybrid_type; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_avg_field::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -732,6 +746,13 @@ public: } void fix_length_and_dec() {} enum Item_result result_type () const { return hybrid_type; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_variance_field::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; === modified file 'sql/item_timefunc.h' --- sql/item_timefunc.h 2008-12-23 14:21:01 +0000 +++ sql/item_timefunc.h 2009-03-24 10:47:20 +0000 @@ -70,6 +70,7 @@ public: enum_monotonicity_info get_monotonicity_info() const; longlong val_int_endpoint(bool left_endp, bool *incl_endp); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -86,6 +87,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -111,6 +113,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -124,6 +127,7 @@ public: enum Item_result result_type () const { return STRING_RESULT; } void fix_length_and_dec(); bool check_partition_func_processor(uchar *int_arg) {return TRUE;} + bool check_vcol_func_processor(uchar *int_arg) {return FALSE;} }; @@ -140,6 +144,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -156,6 +161,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -172,6 +178,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -188,6 +195,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -204,6 +212,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -234,6 +243,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -252,6 +262,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -282,6 +293,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; class Item_func_dayname :public Item_func_weekday @@ -294,6 +306,7 @@ class Item_func_dayname :public Item_fun enum Item_result result_type () const { return STRING_RESULT; } void fix_length_and_dec(); bool check_partition_func_processor(uchar *int_arg) {return TRUE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -310,6 +323,18 @@ public: decimals=0; max_length=10*MY_CHARSET_BIN_MB_MAXLEN; } + bool check_vcol_func_processor(uchar *int_arg) + { + /* + TODO: Allow UNIX_TIMESTAMP called with an argument to be a part + of the expression for a virtual column + */ + DBUG_ENTER("Item_func_unix_timestamp::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } + }; @@ -325,6 +350,7 @@ public: max_length=10*MY_CHARSET_BIN_MB_MAXLEN; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -444,6 +470,14 @@ public: */ virtual void store_now_in_TIME(MYSQL_TIME *now_time)=0; bool result_as_longlong() { return TRUE; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_curtime::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } + }; @@ -480,6 +514,13 @@ public: void fix_length_and_dec(); bool get_date(MYSQL_TIME *res, uint fuzzy_date); virtual void store_now_in_TIME(MYSQL_TIME *now_time)=0; + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_curdate::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -520,6 +561,13 @@ public: void fix_length_and_dec(); bool get_date(MYSQL_TIME *res, uint fuzzy_date); virtual void store_now_in_TIME(MYSQL_TIME *now_time)=0; + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_func_now::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; @@ -577,6 +625,7 @@ public: const char *func_name() const { return "from_days"; } bool get_date(MYSQL_TIME *res, uint fuzzy_date); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -704,6 +753,7 @@ class Item_extract :public Item_int_func bool eq(const Item *item, bool binary_cmp) const; virtual void print(String *str, enum_query_type query_type); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; @@ -952,6 +1002,7 @@ public: maybe_null=1; } bool check_partition_func_processor(uchar *int_arg) {return FALSE;} + bool check_vcol_func_processor(uchar *int_arg) { return FALSE;} }; === modified file 'sql/item_xmlfunc.cc' --- sql/item_xmlfunc.cc 2009-01-31 21:22:44 +0000 +++ sql/item_xmlfunc.cc 2009-03-24 21:10:53 +0000 @@ -220,6 +220,14 @@ public: collation.collation= pxml->charset(); } const char *func_name() const { return "nodeset"; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_nodeset_func::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } + }; @@ -526,6 +534,13 @@ public: enum Type type() const { return XPATH_NODESET_CMP; }; const char *func_name() const { return "xpath_nodeset_to_const_comparator"; } bool is_bool_func() { return 1; } + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_nodeset_to_const_comparator::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } longlong val_int() { === modified file 'sql/item_xmlfunc.h' --- sql/item_xmlfunc.h 2007-11-21 12:00:09 +0000 +++ sql/item_xmlfunc.h 2009-03-24 10:47:20 +0000 @@ -40,6 +40,13 @@ public: } void fix_length_and_dec(); String *parse_xml(String *raw_xml, String *parsed_xml_buf); + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_xml_str_func::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_RETURN(TRUE); + } }; === modified file 'sql/lex.h' --- sql/lex.h 2008-04-28 16:24:05 +0000 +++ sql/lex.h 2009-03-24 15:40:47 +0000 @@ -388,6 +388,7 @@ static SYMBOL symbols[] = { { "PARSER", SYM(PARSER_SYM)}, { "PAGE", SYM(PAGE_SYM)}, { "PAGE_CHECKSUM", SYM(PAGE_CHECKSUM_SYM)}, + { "PARSE_VCOL_EXPR", SYM(PARSE_VCOL_EXPR_SYM)}, { "PARTIAL", SYM(PARTIAL)}, { "PARTITION", SYM(PARTITION_SYM)}, { "PARTITIONING", SYM(PARTITIONING_SYM)}, @@ -514,6 +515,7 @@ static SYMBOL symbols[] = { { "STATUS", SYM(STATUS_SYM)}, { "STOP", SYM(STOP_SYM)}, { "STORAGE", SYM(STORAGE_SYM)}, + { "STORED", SYM(STORED_SYM)}, { "STRAIGHT_JOIN", SYM(STRAIGHT_JOIN)}, { "STRING", SYM(STRING_SYM)}, { "SUBJECT", SYM(SUBJECT_SYM)}, === modified file 'sql/mysql_priv.h' --- sql/mysql_priv.h 2009-03-12 22:27:35 +0000 +++ sql/mysql_priv.h 2009-03-24 10:47:20 +0000 @@ -1312,6 +1312,8 @@ find_field_in_table(THD *thd, TABLE *tab bool allow_rowid, uint *cached_field_index_ptr); Field * find_field_in_table_sef(TABLE *table, const char *name); +int update_virtual_fields_marked_for_write(TABLE *table, + bool ignore_stored= TRUE); #endif /* MYSQL_SERVER */ @@ -1432,14 +1434,16 @@ bool add_field_to_list(THD *thd, LEX_STR LEX_STRING *comment, char *change, List<String> *interval_list, CHARSET_INFO *cs, - uint uint_geom_type); + uint uint_geom_type, + virtual_column_info *vcol_info); Create_field * new_create_field(THD *thd, char *field_name, enum_field_types type, char *length, char *decimals, uint type_modifier, Item *default_value, Item *on_update_value, LEX_STRING *comment, char *change, List<String> *interval_list, CHARSET_INFO *cs, - uint uint_geom_type); + uint uint_geom_type, + virtual_column_info *vcol_info); void store_position_for_column(const char *name); bool add_to_list(THD *thd, SQL_LIST &list,Item *group,bool asc); bool push_new_name_resolution_context(THD *thd, === modified file 'sql/procedure.h' --- sql/procedure.h 2008-03-21 15:48:28 +0000 +++ sql/procedure.h 2009-03-24 21:48:56 +0000 @@ -43,6 +43,14 @@ public: init_make_field(tmp_field,field_type()); } unsigned int size_of() { return sizeof(*this);} + bool check_vcol_func_processor(uchar *int_arg) + { + DBUG_ENTER("Item_sum::check_vcol_func_processor"); + DBUG_PRINT("info", + ("check_vcol_func_processor returns TRUE: unsupported function")); + DBUG_ASSERT(0); /* this is not possible */ + DBUG_RETURN(TRUE); + } }; class Item_proc_real :public Item_proc === modified file 'sql/records.cc' --- sql/records.cc 2008-07-17 18:26:55 +0000 +++ sql/records.cc 2009-03-24 10:47:20 +0000 @@ -323,6 +323,7 @@ static int rr_quick(READ_RECORD *info) break; } } + update_virtual_fields_marked_for_write(info->table); return tmp; } @@ -395,6 +396,8 @@ int rr_sequential(READ_RECORD *info) break; } } + if (!tmp) + update_virtual_fields_marked_for_write(info->table); return tmp; } === modified file 'sql/share/errmsg.txt' --- sql/share/errmsg.txt 2008-12-10 09:02:25 +0000 +++ sql/share/errmsg.txt 2009-03-25 21:24:46 +0000 @@ -6154,3 +6154,30 @@ WARN_PLUGIN_BUSY ER_VARIABLE_IS_READONLY eng "%s variable '%s' is read-only. Use SET %s to assign the value" + +ER_VCOL_BASED_ON_VCOL + eng "A computed column cannot be based on a computed column" + +ER_VIRTUAL_COLUMN_FUNCTION_IS_NOT_ALLOWED + eng "Function or expression is not allowed for column '%s'." + +ER_DATA_CONVERSION_ERROR_FOR_VIRTUAL_COLUMN + eng "Generated value for computed column '%s' cannot be converted to type '%s'." + +ER_PRIMARY_KEY_BASED_ON_VIRTUAL_COLUMN + eng "Primary key cannot be defined upon a computed column." + +ER_KEY_BASED_ON_GENERATED_VIRTUAL_COLUMN + eng "Key/Index cannot be defined on a non-stored computed column." + +ER_WRONG_FK_OPTION_FOR_VIRTUAL_COLUMN + eng "Cannot define foreign key with %s clause on a computed column." + +ER_WARNING_NON_DEFAULT_VALUE_FOR_VIRTUAL_COLUMN + eng "The value specified for computed column '%s' in table '%s' ignored." + +ER_UNSUPPORTED_ACTION_ON_VIRTUAL_COLUMN + eng "'%s' is not yet supported for computed columns." + +ER_CONST_EXPR_IN_VCOL + eng "Constant expression in computed column function is not allowed." === modified file 'sql/sp_head.cc' --- sql/sp_head.cc 2009-02-19 09:01:25 +0000 +++ sql/sp_head.cc 2009-03-24 10:47:20 +0000 @@ -815,6 +815,8 @@ sp_head::create_result_field(uint field_ m_return_field_def.interval, field_name ? field_name : (const char *) m_name.str); + field->vcol_info= m_return_field_def.vcol_info; + field->stored_in_db= m_return_field_def.stored_in_db; if (field) field->init(table); @@ -2194,7 +2196,8 @@ sp_head::fill_field_definition(THD *thd, &lex->interval_list, lex->charset ? lex->charset : thd->variables.collation_database, - lex->uint_geom_type)) + lex->uint_geom_type, + lex->vcol_info)) return TRUE; if (field_def->interval_list.elements) === modified file 'sql/sql_base.cc' --- sql/sql_base.cc 2009-02-19 09:01:25 +0000 +++ sql/sql_base.cc 2009-03-24 10:47:20 +0000 @@ -5875,6 +5875,23 @@ find_field_in_table(THD *thd, TABLE *tab if (field_ptr && *field_ptr) { + if ((*field_ptr)->vcol_info) + { + if (thd->mark_used_columns != MARK_COLUMNS_NONE) + { + Item *vcol_item= (*field_ptr)->vcol_info->expr_item; + DBUG_ASSERT(vcol_item); + vcol_item->walk(&Item::register_field_in_read_map, 1, (uchar *) 0); + /* + Set the virtual field for write here if + 1) this procedure is called for a read-only operation (SELECT), and + 2) the virtual column is not phycically stored in the table + */ + if (thd->mark_used_columns != MARK_COLUMNS_WRITE && + !(*field_ptr)->stored_in_db) + bitmap_set_bit((*field_ptr)->table->write_set, (*field_ptr)->field_index); + } + } *cached_field_index_ptr= field_ptr - table->field; field= *field_ptr; } @@ -7857,6 +7874,17 @@ insert_fields(THD *thd, Name_resolution_ { /* Mark fields as used to allow storage engine to optimze access */ bitmap_set_bit(field->table->read_set, field->field_index); + /* + Mark virtual fields for write and others that the virtual fields + depend on for read. + */ + if (field->vcol_info) + { + Item *vcol_item= field->vcol_info->expr_item; + DBUG_ASSERT(vcol_item); + vcol_item->walk(&Item::register_field_in_read_map, 1, (uchar *) 0); + bitmap_set_bit(field->table->write_set, field->field_index); + } if (table) { table->covering_keys.intersect(field->part_of_key); @@ -8068,7 +8096,10 @@ fill_record(THD * thd, List<Item> &field Item *value, *fld; Item_field *field; TABLE *table= 0; + List<TABLE> tbl_list; + bool abort_on_warning_saved= thd->abort_on_warning; DBUG_ENTER("fill_record"); + tbl_list.empty(); /* Reset the table->auto_increment_field_not_null as it is valid for @@ -8102,14 +8133,54 @@ fill_record(THD * thd, List<Item> &field table= rfield->table; if (rfield == table->next_number_field) table->auto_increment_field_not_null= TRUE; + if (rfield->vcol_info && + value->type() != Item::DEFAULT_VALUE_ITEM && + value->type() != Item::NULL_ITEM && + table->s->table_category != TABLE_CATEGORY_TEMPORARY) + { + thd->abort_on_warning= FALSE; + push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WARNING_NON_DEFAULT_VALUE_FOR_VIRTUAL_COLUMN, + ER(ER_WARNING_NON_DEFAULT_VALUE_FOR_VIRTUAL_COLUMN), + rfield->field_name, table->s->table_name.str); + thd->abort_on_warning= abort_on_warning_saved; + } if ((value->save_in_field(rfield, 0) < 0) && !ignore_errors) { my_message(ER_UNKNOWN_ERROR, ER(ER_UNKNOWN_ERROR), MYF(0)); goto err; } + tbl_list.push_back(table); } + /* Update virtual fields*/ + thd->abort_on_warning= FALSE; + if (tbl_list.head()) + { + List_iterator_fast<TABLE> t(tbl_list); + TABLE *prev_table= 0; + while ((table= t++)) + { + /* + Do simple optimization to prevent unnecessary re-generating + values for virtual fields + */ + if (table != prev_table) + { + prev_table= table; + if (table->vfield) + { + if (update_virtual_fields_marked_for_write(table, FALSE)) + { + goto err; + } + } + } + } + } + thd->abort_on_warning= abort_on_warning_saved; DBUG_RETURN(thd->is_error()); err: + thd->abort_on_warning= abort_on_warning_saved; if (table) table->auto_increment_field_not_null= FALSE; DBUG_RETURN(TRUE); @@ -8145,9 +8216,31 @@ fill_record_n_invoke_before_triggers(THD Table_triggers_list *triggers, enum trg_event_type event) { - return (fill_record(thd, fields, values, ignore_errors) || - (triggers && triggers->process_triggers(thd, event, - TRG_ACTION_BEFORE, TRUE))); + bool result; + result= (fill_record(thd, fields, values, ignore_errors) || + (triggers && triggers->process_triggers(thd, event, + TRG_ACTION_BEFORE, TRUE))); + /* + Re-calculate virtual fields to cater for cases when base columns are + updated by the triggers. + */ + if (!result && triggers) + { + TABLE *table= 0; + List_iterator_fast<Item> f(fields); + Item *fld; + Item_field *item_field; + if (fields.elements) + { + fld= (Item_field*)f++; + item_field= fld->filed_for_view_update(); + if (item_field && item_field->field && + (table= item_field->field->table) && + table->vfield) + result= update_virtual_fields_marked_for_write(table, FALSE); + } + } + return result; } @@ -8175,11 +8268,14 @@ bool fill_record(THD *thd, Field **ptr, List<Item> &values, bool ignore_errors) { List_iterator_fast<Item> v(values); + List<TABLE> tbl_list; Item *value; TABLE *table= 0; + bool abort_on_warning_saved= thd->abort_on_warning; DBUG_ENTER("fill_record"); Field *field; + tbl_list.empty(); /* Reset the table->auto_increment_field_not_null as it is valid for only one row. @@ -8199,12 +8295,52 @@ fill_record(THD *thd, Field **ptr, List< table= field->table; if (field == table->next_number_field) table->auto_increment_field_not_null= TRUE; + if (field->vcol_info && + value->type() != Item::DEFAULT_VALUE_ITEM && + value->type() != Item::NULL_ITEM && + table->s->table_category != TABLE_CATEGORY_TEMPORARY) + { + thd->abort_on_warning= FALSE; + push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WARNING_NON_DEFAULT_VALUE_FOR_VIRTUAL_COLUMN, + ER(ER_WARNING_NON_DEFAULT_VALUE_FOR_VIRTUAL_COLUMN), + field->field_name, table->s->table_name.str); + thd->abort_on_warning= abort_on_warning_saved; + } if (value->save_in_field(field, 0) < 0) goto err; + tbl_list.push_back(table); + } + /* Update virtual fields*/ + thd->abort_on_warning= FALSE; + if (tbl_list.head()) + { + List_iterator_fast<TABLE> t(tbl_list); + TABLE *prev_table= 0; + while ((table= t++)) + { + /* + Do simple optimization to prevent unnecessary re-generating + values for virtual fields + */ + if (table != prev_table) + { + prev_table= table; + if (table->vfield) + { + if (update_virtual_fields_marked_for_write(table, FALSE)) + { + goto err; + } + } + } + } } + thd->abort_on_warning= abort_on_warning_saved; DBUG_RETURN(thd->is_error()); err: + thd->abort_on_warning= abort_on_warning_saved; if (table) table->auto_increment_field_not_null= FALSE; DBUG_RETURN(TRUE); @@ -8240,9 +8376,22 @@ fill_record_n_invoke_before_triggers(THD Table_triggers_list *triggers, enum trg_event_type event) { - return (fill_record(thd, ptr, values, ignore_errors) || - (triggers && triggers->process_triggers(thd, event, - TRG_ACTION_BEFORE, TRUE))); + bool result; + result= (fill_record(thd, ptr, values, ignore_errors) || + (triggers && triggers->process_triggers(thd, event, + TRG_ACTION_BEFORE, TRUE))); + /* + Re-calculate virtual fields to cater for cases when base columns are + updated by the triggers. + */ + if (!result && triggers && *ptr) + { + TABLE *table= (*ptr)->table; + if (table->vfield) + result= update_virtual_fields_marked_for_write(table, FALSE); + } + return result; + } === modified file 'sql/sql_class.cc' --- sql/sql_class.cc 2009-03-12 22:27:35 +0000 +++ sql/sql_class.cc 2009-03-24 10:47:20 +0000 @@ -193,6 +193,57 @@ bool foreign_key_prefix(Key *a, Key *b) #endif } +/* + Check if the foreign key options are compatible with columns + on which the FK is created. + + RETURN + 0 Key valid + 1 Key invalid +*/ +bool Foreign_key::validate(List<Create_field> &table_fields) +{ + Create_field *sql_field; + Key_part_spec *column; + List_iterator<Key_part_spec> cols(columns); + List_iterator<Create_field> it(table_fields); + DBUG_ENTER("Foreign_key::validate"); + while ((column= cols++)) + { + it.rewind(); + while ((sql_field= it++) && + my_strcasecmp(system_charset_info, + column->field_name, + sql_field->field_name)) {} + if (!sql_field) + { + my_error(ER_KEY_COLUMN_DOES_NOT_EXITS, MYF(0), column->field_name); + DBUG_RETURN(TRUE); + } + if (type == Key::FOREIGN_KEY && sql_field->vcol_info) + { + if (delete_opt == FK_OPTION_SET_NULL) + { + my_error(ER_WRONG_FK_OPTION_FOR_VIRTUAL_COLUMN, MYF(0), + "ON DELETE SET NULL"); + DBUG_RETURN(TRUE); + } + if (update_opt == FK_OPTION_SET_NULL) + { + my_error(ER_WRONG_FK_OPTION_FOR_VIRTUAL_COLUMN, MYF(0), + "ON UPDATE SET NULL"); + DBUG_RETURN(TRUE); + } + if (update_opt == FK_OPTION_CASCADE) + { + my_error(ER_WRONG_FK_OPTION_FOR_VIRTUAL_COLUMN, MYF(0), + "ON UPDATE CASCADE"); + DBUG_RETURN(TRUE); + } + } + } + DBUG_RETURN(FALSE); +} /**************************************************************************** ** Thread specific functions === modified file 'sql/sql_class.h' --- sql/sql_class.h 2009-03-12 22:27:35 +0000 +++ sql/sql_class.h 2009-03-24 10:47:20 +0000 @@ -249,6 +249,8 @@ public: */ virtual Key *clone(MEM_ROOT *mem_root) const { return new (mem_root) Foreign_key(*this, mem_root); } + /* Used to validate foreign key options */ + bool validate(List<Create_field> &table_fields); }; typedef struct st_mysql_lock === modified file 'sql/sql_handler.cc' --- sql/sql_handler.cc 2008-02-19 12:58:08 +0000 +++ sql/sql_handler.cc 2009-03-24 10:47:20 +0000 @@ -636,6 +636,8 @@ retry: } goto ok; } + /* Generate values for virtual fields */ + update_virtual_fields_marked_for_write(table); if (cond && !cond->val_int()) continue; if (num_rows >= offset_limit_cnt) === modified file 'sql/sql_insert.cc' --- sql/sql_insert.cc 2009-02-19 09:01:25 +0000 +++ sql/sql_insert.cc 2009-03-24 10:47:21 +0000 @@ -279,6 +279,9 @@ static int check_insert_fields(THD *thd, table->timestamp_field->field_index); } } + /* Mark all virtual columns for write*/ + if (table->vfield) + table->mark_virtual_columns(); } // For the values we need select_priv #ifndef NO_EMBEDDED_ACCESS_CHECKS === modified file 'sql/sql_lex.cc' --- sql/sql_lex.cc 2009-02-15 10:58:34 +0000 +++ sql/sql_lex.cc 2009-03-24 10:47:21 +0000 @@ -340,6 +340,7 @@ void lex_start(THD *thd) lex->reset_query_tables_list(FALSE); lex->expr_allows_subselect= TRUE; lex->use_only_table_context= FALSE; + lex->parse_vcol_expr= FALSE; lex->name.str= 0; lex->name.length= 0; === modified file 'sql/sql_lex.h' --- sql/sql_lex.h 2009-01-15 18:11:25 +0000 +++ sql/sql_lex.h 2009-03-24 10:47:21 +0000 @@ -1521,6 +1521,7 @@ typedef struct st_lex : public Query_tab LEX_USER *grant_user; XID *xid; THD *thd; + virtual_column_info *vcol_info; /* maintain a list of used plugins for this LEX */ DYNAMIC_ARRAY plugins; @@ -1603,6 +1604,14 @@ typedef struct st_lex : public Query_tab syntax error back. */ bool expr_allows_subselect; + /* + A special command "PARSE_VCOL_EXPR" is defined for the parser + to translate an expression statement of a virtual column \ + (stored in the *.frm file as a string) into an Item object. + The following flag is used to prevent other applications to use + this command. + */ + bool parse_vcol_expr; thr_lock_type lock_option; enum SSL_type ssl_type; /* defined in violite.h */ === modified file 'sql/sql_parse.cc' --- sql/sql_parse.cc 2009-02-19 09:01:25 +0000 +++ sql/sql_parse.cc 2009-03-24 10:47:21 +0000 @@ -5907,7 +5907,8 @@ bool add_field_to_list(THD *thd, LEX_STR LEX_STRING *comment, char *change, List<String> *interval_list, CHARSET_INFO *cs, - uint uint_geom_type) + uint uint_geom_type, + virtual_column_info *vcol_info) { register Create_field *new_field; LEX *lex= thd->lex; @@ -5993,7 +5994,7 @@ bool add_field_to_list(THD *thd, LEX_STR if (!(new_field= new Create_field()) || new_field->init(thd, field_name->str, type, length, decimals, type_modifier, default_value, on_update_value, comment, change, - interval_list, cs, uint_geom_type)) + interval_list, cs, uint_geom_type, vcol_info)) DBUG_RETURN(1); lex->alter_info.create_list.push_back(new_field); === modified file 'sql/sql_partition.cc' --- sql/sql_partition.cc 2009-02-15 10:58:34 +0000 +++ sql/sql_partition.cc 2009-03-26 21:13:30 +0000 @@ -964,8 +964,9 @@ bool fix_fields_part_func(THD *thd, Item save_use_only_table_context= thd->lex->use_only_table_context; thd->lex->use_only_table_context= TRUE; - + error= func_expr->fix_fields(thd, (Item**)0); + func_expr->walk(&Item::vcol_in_partition_func_processor, 0, NULL); thd->lex->use_only_table_context= save_use_only_table_context; === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2009-02-19 09:01:25 +0000 +++ sql/sql_select.cc 2009-03-24 10:47:21 +0000 @@ -11717,6 +11717,7 @@ join_read_system(JOIN_TAB *tab) empty_record(table); // Make empty record return -1; } + update_virtual_fields_marked_for_write(table); store_record(table,record[1]); } else if (!table->status) // Only happens with left join @@ -11765,6 +11766,7 @@ join_read_const(JOIN_TAB *tab) return report_error(table, error); return -1; } + update_virtual_fields_marked_for_write(table); store_record(table,record[1]); } else if (!(table->status & ~STATUS_NULL_ROW)) // Only happens with left join @@ -11854,6 +11856,8 @@ join_read_always_key(JOIN_TAB *tab) return report_error(table, error); return -1; /* purecov: inspected */ } + if (!error) + update_virtual_fields_marked_for_write(table); return 0; } @@ -11881,6 +11885,7 @@ join_read_last_key(JOIN_TAB *tab) return report_error(table, error); return -1; /* purecov: inspected */ } + update_virtual_fields_marked_for_write(table); return 0; } @@ -11909,6 +11914,7 @@ join_read_next_same(READ_RECORD *info) table->status= STATUS_GARBAGE; return -1; } + update_virtual_fields_marked_for_write(table); return 0; } @@ -11922,12 +11928,14 @@ join_read_prev_same(READ_RECORD *info) if ((error=table->file->index_prev(table->record[0]))) return report_error(table, error); + update_virtual_fields_marked_for_write(table); if (key_cmp_if_same(table, tab->ref.key_buff, tab->ref.key, tab->ref.key_length)) { table->status=STATUS_NOT_FOUND; error= -1; } + update_virtual_fields_marked_for_write(table); return error; } @@ -11996,6 +12004,8 @@ join_read_first(JOIN_TAB *tab) report_error(table, error); return -1; } + if (not error) + update_virtual_fields_marked_for_write(tab->table); return 0; } @@ -12006,6 +12016,8 @@ join_read_next(READ_RECORD *info) int error; if ((error=info->file->index_next(info->record))) return report_error(info->table, error); + if (not error) + update_virtual_fields_marked_for_write(info->table); return 0; } @@ -12031,6 +12043,8 @@ join_read_last(JOIN_TAB *tab) table->file->ha_index_init(tab->index, 1); if ((error= tab->table->file->index_last(tab->table->record[0]))) return report_error(table, error); + if (not error) + update_virtual_fields_marked_for_write(tab->table); return 0; } @@ -12041,6 +12055,8 @@ join_read_prev(READ_RECORD *info) int error; if ((error= info->file->index_prev(info->record))) return report_error(info->table, error); + if (not error) + update_virtual_fields_marked_for_write(info->table); return 0; } @@ -12062,6 +12078,8 @@ join_ft_read_first(JOIN_TAB *tab) if ((error= table->file->ft_read(table->record[0]))) return report_error(table, error); + if (not error) + update_virtual_fields_marked_for_write(table); return 0; } @@ -12071,6 +12089,7 @@ join_ft_read_next(READ_RECORD *info) int error; if ((error= info->file->ft_read(info->table->record[0]))) return report_error(info->table, error); + update_virtual_fields_marked_for_write(info->table); return 0; } === modified file 'sql/sql_show.cc' --- sql/sql_show.cc 2009-02-19 09:01:25 +0000 +++ sql/sql_show.cc 2009-03-24 15:39:10 +0000 @@ -1175,6 +1175,17 @@ int store_create_info(THD *thd, TABLE_LI field->sql_type(type); packet->append(type.ptr(), type.length(), system_charset_info); + if (field->vcol_info) + { + packet->append(STRING_WITH_LEN(" AS (")); + packet->append(field->vcol_info->expr_str.str, + field->vcol_info->expr_str.length, + system_charset_info); + packet->append(STRING_WITH_LEN(")")); + if (field->stored_in_db) + packet->append(STRING_WITH_LEN(" STORED")); + } + if (field->has_charset() && !(thd->variables.sql_mode & (MODE_MYSQL323 | MODE_MYSQL40))) { @@ -1205,7 +1216,8 @@ int store_create_info(THD *thd, TABLE_LI packet->append(STRING_WITH_LEN(" NULL")); } - if (get_field_default_value(thd, table, field, &def_value, 1)) + if (!field->vcol_info && + get_field_default_value(thd, table, field, &def_value, 1)) { packet->append(STRING_WITH_LEN(" DEFAULT ")); packet->append(def_value.ptr(), def_value.length(), system_charset_info); @@ -3876,6 +3888,8 @@ static int get_schema_column_record(THD field->unireg_check != Field::TIMESTAMP_DN_FIELD) table->field[16]->store(STRING_WITH_LEN("on update CURRENT_TIMESTAMP"), cs); + if (field->vcol_info) + table->field[16]->store(STRING_WITH_LEN("VIRTUAL"), cs); table->field[18]->store(field->comment.str, field->comment.length, cs); if (schema_table_store_record(thd, table)) === modified file 'sql/sql_table.cc' --- sql/sql_table.cc 2009-02-19 09:01:25 +0000 +++ sql/sql_table.cc 2009-03-26 21:17:10 +0000 @@ -2166,7 +2166,8 @@ int prepare_create_field(Create_field *s (sql_field->decimals << FIELDFLAG_DEC_SHIFT)); break; } - if (!(sql_field->flags & NOT_NULL_FLAG)) + if (!(sql_field->flags & NOT_NULL_FLAG) || + (sql_field->vcol_info)) /* Make virtual columns allow NULL values */ sql_field->pack_flag|= FIELDFLAG_MAYBE_NULL; if (sql_field->flags & NO_DEFAULT_VALUE_FLAG) sql_field->pack_flag|= FIELDFLAG_NO_DEFAULT; @@ -2480,6 +2481,8 @@ mysql_prepare_create_table(THD *thd, HA_ null_fields--; sql_field->flags= dup_field->flags; sql_field->interval= dup_field->interval; + sql_field->vcol_info= dup_field->vcol_info; + sql_field->stored_in_db= dup_field->stored_in_db; it2.remove(); // Remove first (create) definition select_field_pos--; break; @@ -2512,7 +2515,23 @@ mysql_prepare_create_table(THD *thd, HA_ sql_field->offset= record_offset; if (MTYP_TYPENR(sql_field->unireg_check) == Field::NEXT_NUMBER) auto_increment++; - record_offset+= sql_field->pack_length; + /* + For now skip fields that are not physically stored in the database + (virtual fields) and update their offset later + (see the next loop). + */ + if (sql_field->stored_in_db) + record_offset+= sql_field->pack_length; + } + /* Update virtual fields' offset*/ + it.rewind(); + while ((sql_field=it++)) + { + if (!sql_field->stored_in_db) + { + sql_field->offset= record_offset; + record_offset+= sql_field->pack_length; + } } if (timestamps_with_niladic > 1) { @@ -2562,6 +2581,8 @@ mysql_prepare_create_table(THD *thd, HA_ if (key->type == Key::FOREIGN_KEY) { fk_key_count++; + if (((Foreign_key *)key)->validate(alter_info->create_list)) + DBUG_RETURN(TRUE); Foreign_key *fk_key= (Foreign_key*) key; if (fk_key->ref_columns.elements && fk_key->ref_columns.elements != fk_key->columns.elements) @@ -2848,6 +2869,17 @@ mysql_prepare_create_table(THD *thd, HA_ } } #endif + if (!sql_field->stored_in_db) + { + /* Key fields must always be physically stored. */ + my_error(ER_KEY_BASED_ON_GENERATED_VIRTUAL_COLUMN, MYF(0)); + DBUG_RETURN(TRUE); + } + if (key->type == Key::PRIMARY && sql_field->vcol_info) + { + my_error(ER_PRIMARY_KEY_BASED_ON_VIRTUAL_COLUMN, MYF(0)); + DBUG_RETURN(TRUE); + } if (!(sql_field->flags & NOT_NULL_FLAG)) { if (key->type == Key::PRIMARY) @@ -5357,6 +5389,18 @@ compare_tables(TABLE *table, DBUG_RETURN(0); } + /* + Check if the altered column is a stored virtual field. + TODO: Mark such a column with an alter flag only if + the expression functions are not equal. + */ + if (field->stored_in_db && field->vcol_info || + field->vcol_info && field->vcol_info->is_field_implicitly_stored()) + { + *need_copy_table= ALTER_TABLE_DATA_CHANGED; + DBUG_RETURN(0); + } + /* Don't pack rows in old tables if the user has requested this. */ if (create_info->row_type == ROW_TYPE_DYNAMIC || (tmp_new_field->flags & BLOB_FLAG) || @@ -5713,6 +5757,13 @@ mysql_prepare_alter_table(THD *thd, TABL if (def) { // Field is changed def->field=field; + if (field->stored_in_db != def->stored_in_db) + { + my_error(ER_UNSUPPORTED_ACTION_ON_VIRTUAL_COLUMN, + MYF(0), + "Changing the STORED status"); + goto err; + } if (!def->after) { new_create_list.push_back(def); @@ -5925,6 +5976,9 @@ mysql_prepare_alter_table(THD *thd, TABL Key *key; while ((key=key_it++)) // Add new keys { + if (key->type == Key::FOREIGN_KEY && + ((Foreign_key *)key)->validate(new_create_list)) + goto err; if (key->type != Key::FOREIGN_KEY) new_key_list.push_back(key); if (key->name && @@ -7327,6 +7381,12 @@ copy_data_between_tables(TABLE *from,TAB copy_ptr->do_copy(copy_ptr); } prev_insert_id= to->file->next_insert_id; + update_virtual_fields_marked_for_write(to, FALSE); + if (thd->is_error()) + { + error= 1; + break; + } error=to->file->ha_write_row(to->record[0]); to->auto_increment_field_not_null= FALSE; if (error) === modified file 'sql/sql_yacc.yy' --- sql/sql_yacc.yy 2009-02-19 09:01:25 +0000 +++ sql/sql_yacc.yy 2009-03-24 11:09:35 +0000 @@ -886,6 +886,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** %token PAGE_CHECKSUM_SYM %token PARAM_MARKER %token PARSER_SYM +%token PARSE_VCOL_EXPR_SYM %token PARTIAL /* SQL-2003-N */ %token PARTITIONING_SYM %token PARTITIONS_SYM @@ -1007,6 +1008,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** %token STD_SYM %token STOP_SYM %token STORAGE_SYM +%token STORED_SYM %token STRAIGHT_JOIN %token STRING_SYM %token SUBDATE_SYM @@ -1143,7 +1145,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** text_string opt_gconcat_separator %type <num> - type int_type real_type order_dir lock_option + type int_type real_type order_dir lock_option field_def udf_type if_exists opt_local opt_table_options table_options table_option opt_if_not_exists opt_no_write_to_binlog delete_option opt_temporary all_or_any opt_distinct @@ -1299,6 +1301,8 @@ bool my_yyoverflow(short **a, YYSTYPE ** init_key_options key_options key_opts key_opt key_using_alg server_def server_options_list server_option definer_opt no_definer definer + parse_vcol_expr vcol_opt_attribute vcol_opt_attribute_list + vcol_attribute END_OF_INPUT %type <NONE> call sp_proc_stmts sp_proc_stmts1 sp_proc_stmt @@ -1430,6 +1434,7 @@ statement: | lock | optimize | keycache + | parse_vcol_expr | partition_entry | preload | prepare @@ -4735,8 +4740,9 @@ field_spec: lex->default_value= lex->on_update_value= 0; lex->comment=null_lex_str; lex->charset=NULL; + lex->vcol_info= 0; } - type opt_attribute + field_def { LEX *lex=Lex; if (add_field_to_list(lex->thd, &$1, (enum enum_field_types) $3, @@ -4744,11 +4750,83 @@ field_spec: lex->default_value, lex->on_update_value, &lex->comment, lex->change,&lex->interval_list,lex->charset, - lex->uint_geom_type)) + lex->uint_geom_type, + lex->vcol_info)) MYSQL_YYABORT; } ; +field_def: + type opt_attribute {} + | type AS '(' virtual_column_func ')' vcol_opt_attribute + { + $$= (enum enum_field_types)MYSQL_TYPE_VIRTUAL; + Lex->vcol_info->set_field_type((enum enum_field_types) $1); + } + ; + +vcol_opt_attribute: + /* empty */ {} + | vcol_opt_attribute_list {} + ; + +vcol_opt_attribute_list: + vcol_opt_attribute_list vcol_attribute {} + | vcol_attribute + ; + +vcol_attribute: + UNIQUE_SYM + { + LEX *lex=Lex; + lex->type|= UNIQUE_FLAG; + lex->alter_info.flags|= ALTER_ADD_INDEX; + } + | UNIQUE_SYM KEY_SYM + { + LEX *lex=Lex; + lex->type|= UNIQUE_KEY_FLAG; + lex->alter_info.flags|= ALTER_ADD_INDEX; + } + | COMMENT_SYM TEXT_STRING_sys { Lex->comment= $2; } + | STORED_SYM + { + Lex->vcol_info->set_field_stored(TRUE); + } + ; + +parse_vcol_expr: + PARSE_VCOL_EXPR_SYM '(' virtual_column_func ')' + { + /* + "PARSE_VCOL_EXPR" can only be used by the SQL server + when reading a '*.frm' file. + Prevent the end user from invoking this command. + */ + if (!Lex->parse_vcol_expr) + { + my_message(ER_SYNTAX_ERROR, ER(ER_SYNTAX_ERROR), MYF(0)); + MYSQL_YYABORT; + } + } + ; + +virtual_column_func: + remember_name expr remember_end + { + Lex->vcol_info= new virtual_column_info(); + if (!Lex->vcol_info) + { + mem_alloc_error(sizeof(virtual_column_info)); + MYSQL_YYABORT; + } + uint expr_len= (uint)($3 - $1) - 1; + Lex->vcol_info->expr_str.str= (char* ) sql_memdup($1 + 1, expr_len); + Lex->vcol_info->expr_str.length= expr_len; + Lex->vcol_info->expr_item= $2; + } + ; + type: int_type opt_field_length field_options { $$=$1; } | real_type opt_precision field_options { $$=$1; } @@ -5836,8 +5914,9 @@ alter_list_item: lex->comment=null_lex_str; lex->charset= NULL; lex->alter_info.flags|= ALTER_CHANGE_COLUMN; + lex->vcol_info= 0; } - type opt_attribute + field_def { LEX *lex=Lex; if (add_field_to_list(lex->thd,&$3, @@ -5846,7 +5925,8 @@ alter_list_item: lex->default_value, lex->on_update_value, &lex->comment, $3.str, &lex->interval_list, lex->charset, - lex->uint_geom_type)) + lex->uint_geom_type, + lex->vcol_info)) MYSQL_YYABORT; } opt_place @@ -13274,6 +13354,7 @@ sf_tail: lex->length= lex->dec= NULL; lex->interval_list.empty(); lex->type= 0; + lex->vcol_info= 0; } type /* $11 */ { /* $12 */ === modified file 'sql/table.cc' --- sql/table.cc 2009-02-19 09:01:25 +0000 +++ sql/table.cc 2009-03-26 21:28:14 +0000 @@ -33,6 +33,9 @@ LEX_STRING GENERAL_LOG_NAME= {C_STRING_W /* SLOW_LOG name */ LEX_STRING SLOW_LOG_NAME= {C_STRING_WITH_LEN("slow_log")}; +/* Keyword for parsing virtual column functions */ +LEX_STRING parse_vcol_keyword= { C_STRING_WITH_LEN("PARSE_VCOL_EXPR ") }; + /* Functions defined in this file */ void open_table_error(TABLE_SHARE *share, int error, int db_errno, @@ -663,10 +666,11 @@ static int open_binary_frm(THD *thd, TAB uint interval_count, interval_parts, read_length, int_length; uint db_create_options, keys, key_parts, n_length; uint key_info_length, com_length, null_bit_pos; + uint vcol_screen_length; uint extra_rec_buf_length; uint i,j; bool use_hash; - char *keynames, *names, *comment_pos; + char *keynames, *names, *comment_pos, *vcol_screen_pos; uchar *record; uchar *disk_buff, *strpos, *null_flags, *null_pos; ulong pos, record_offset, *rec_per_key, rec_buff_length; @@ -836,6 +840,7 @@ static int open_binary_frm(THD *thd, TAB strpos+= (strmov(keynames, (char *) strpos) - keynames)+1; share->reclength = uint2korr((head+16)); + share->stored_rec_length= share->reclength; if (*(head+26) == 1) share->system= 1; /* one-record-database */ #ifdef HAVE_CRYPTED_FRM @@ -1040,24 +1045,28 @@ static int open_binary_frm(THD *thd, TAB int_length= uint2korr(head+274); share->null_fields= uint2korr(head+282); com_length= uint2korr(head+284); + vcol_screen_length= uint2korr(head+286); + share->vfields= 0; + share->stored_fields= share->fields; share->comment.length= (int) (head[46]); share->comment.str= strmake_root(&share->mem_root, (char*) head+47, share->comment.length); - DBUG_PRINT("info",("i_count: %d i_parts: %d index: %d n_length: %d int_length: %d com_length: %d", interval_count,interval_parts, share->keys,n_length,int_length, com_length)); - + DBUG_PRINT("info",("i_count: %d i_parts: %d index: %d n_length: %d int_length: %d com_length: %d vcol_screen_length: %d", interval_count,interval_parts, share->keys,n_length,int_length, com_length, vcol_screen_length)); if (!(field_ptr = (Field **) alloc_root(&share->mem_root, (uint) ((share->fields+1)*sizeof(Field*)+ interval_count*sizeof(TYPELIB)+ (share->fields+interval_parts+ keys+3)*sizeof(char *)+ - (n_length+int_length+com_length))))) + (n_length+int_length+com_length+ + vcol_screen_length))))) goto err; /* purecov: inspected */ share->field= field_ptr; read_length=(uint) (share->fields * field_pack_length + - pos+ (uint) (n_length+int_length+com_length)); + pos+ (uint) (n_length+int_length+com_length+ + vcol_screen_length)); if (read_string(file,(uchar**) &disk_buff,read_length)) goto err; /* purecov: inspected */ #ifdef HAVE_CRYPTED_FRM @@ -1078,7 +1087,11 @@ static int open_binary_frm(THD *thd, TAB memcpy((char*) names, strpos+(share->fields*field_pack_length), (uint) (n_length+int_length)); comment_pos= names+(n_length+int_length); - memcpy(comment_pos, disk_buff+read_length-com_length, com_length); + memcpy(comment_pos, disk_buff+read_length-com_length-vcol_screen_length, + com_length); + vcol_screen_pos= names+(n_length+int_length+com_length); + memcpy(vcol_screen_pos, disk_buff+read_length-vcol_screen_length, + vcol_screen_length); fix_type_pointers(&interval_array, &share->fieldnames, 1, &names); if (share->fieldnames.count != share->fields) @@ -1146,10 +1159,13 @@ static int open_binary_frm(THD *thd, TAB for (i=0 ; i < share->fields; i++, strpos+=field_pack_length, field_ptr++) { uint pack_flag, interval_nr, unireg_type, recpos, field_length; + uint vcol_info_length=0; enum_field_types field_type; CHARSET_INFO *charset=NULL; Field::geometry_type geom_type= Field::GEOM_GEOMETRY; LEX_STRING comment; + virtual_column_info *vcol_info= 0; + bool fld_stored_in_db= TRUE; if (new_frm_ver >= 3) { @@ -1184,6 +1200,18 @@ static int open_binary_frm(THD *thd, TAB goto err; } } + + if ((uchar)field_type == (uchar)MYSQL_TYPE_VIRTUAL) + { + DBUG_ASSERT(interval_nr); // Expect non-null expression + /* + The interval_id byte in the .frm file stores the length of the + expression statement for a virtual column. + */ + vcol_info_length= interval_nr; + interval_nr= 0; + } + if (!comment_length) { comment.str= (char*) ""; @@ -1195,6 +1223,33 @@ static int open_binary_frm(THD *thd, TAB comment.length= comment_length; comment_pos+= comment_length; } + + if (vcol_info_length) + { + /* + Get virtual column data stored in the .frm file as follows: + byte 1 = 1 (always 1 to allow for future extensions) + byte 2 = sql_type + byte 3 = flags (as of now, 0 - no flags, 1 - field is physically stored) + byte 4-... = virtual column expression (text data) + */ + vcol_info= new virtual_column_info(); + if ((uint)vcol_screen_pos[0] != 1) + { + error= 4; + goto err; + } + field_type= (enum_field_types) (uchar) vcol_screen_pos[1]; + fld_stored_in_db= (bool) (uint) vcol_screen_pos[2]; + vcol_info->expr_str.str= (char *)memdup_root(&share->mem_root, + vcol_screen_pos+ + (uint)FRM_VCOL_HEADER_SIZE, + vcol_info_length- + (uint)FRM_VCOL_HEADER_SIZE); + vcol_info->expr_str.length= vcol_info_length-(uint)FRM_VCOL_HEADER_SIZE; + vcol_screen_pos+= vcol_info_length; + share->vfields++; + } } else { @@ -1285,6 +1340,8 @@ static int open_binary_frm(THD *thd, TAB reg_field->field_index= i; reg_field->comment=comment; + reg_field->vcol_info= vcol_info; + reg_field->stored_in_db= fld_stored_in_db; if (field_type == MYSQL_TYPE_BIT && !f_bit_as_char(pack_flag)) { if ((null_bit_pos+= field_length & 7) > 7) @@ -1309,8 +1366,17 @@ static int open_binary_frm(THD *thd, TAB if (use_hash) (void) my_hash_insert(&share->name_hash, (uchar*) field_ptr); // never fail + if (!reg_field->stored_in_db) + { + share->stored_fields--; + if (share->stored_rec_length>=recpos) + share->stored_rec_length= recpos-1; + } } *field_ptr=0; // End marker + /* Sanity checks: */ + DBUG_ASSERT(share->fields>=share->stored_fields); + DBUG_ASSERT(share->reclength>=share->stored_rec_length); /* Fix key->name and key_part->field */ if (key_parts) @@ -1597,6 +1663,268 @@ static int open_binary_frm(THD *thd, TAB DBUG_RETURN(error); } /* open_binary_frm */ +/* + Clear flag GET_FIXED_FIELDS_FLAG in all fields of the table. + This routine is used for error handling purposes. + + SYNOPSIS + clear_field_flag() + table TABLE object for which virtual columns are set-up + + RETURN VALUE + NONE +*/ +static void clear_field_flag(TABLE *table) +{ + Field **ptr; + DBUG_ENTER("clear_field_flag"); + + for (ptr= table->field; *ptr; ptr++) + (*ptr)->flags&= (~GET_FIXED_FIELDS_FLAG); + DBUG_VOID_RETURN; +} + +/* + The function uses the feature in fix_fields where the flag + GET_FIXED_FIELDS_FLAG is set for all fields in the item tree. + This field must always be reset before returning from the function + since it is used for other purposes as well. + + SYNOPSIS + fix_fields_vcol_func() + thd The thread object + vcol_info The item tree reference of the virtual columnfunction + table The table object + field_name The name of the processed field + + RETURN VALUE + TRUE An error occurred, something was wrong with the + function. + FALSE Ok, a partition field array was created +*/ + +bool fix_fields_vcol_func(THD *thd, + virtual_column_info *vcol_info, + TABLE *table, + const char *field_name) +{ + Item* func_expr= vcol_info->expr_item; + uint dir_length, home_dir_length; + bool result= TRUE; + TABLE_LIST tables; + TABLE_LIST *save_table_list, *save_first_table, *save_last_table; + int error; + Name_resolution_context *context; + const char *save_where; + char* db_name; + char db_name_string[FN_REFLEN]; + bool save_use_only_table_context; + Field **ptr, *field; + enum_mark_columns save_mark_used_columns= thd->mark_used_columns; + DBUG_ASSERT(func_expr); + DBUG_ENTER("fix_fields_vcol_func"); + + /* + Set-up the TABLE_LIST object to be a list with a single table + Set the object to zero to create NULL pointers and set alias + and real name to table name and get database name from file name. + */ + + bzero((void*)&tables, sizeof(TABLE_LIST)); + tables.alias= tables.table_name= (char*) table->s->table_name.str; + tables.table= table; + tables.next_local= 0; + tables.next_name_resolution_table= 0; + strmov(db_name_string, table->s->normalized_path.str); + dir_length= dirname_length(db_name_string); + db_name_string[dir_length - 1]= 0; + home_dir_length= dirname_length(db_name_string); + db_name= &db_name_string[home_dir_length]; + tables.db= db_name; + + thd->mark_used_columns= MARK_COLUMNS_NONE; + + context= thd->lex->current_context(); + table->map= 1; //To ensure correct calculation of const item + table->get_fields_in_item_tree= TRUE; + save_table_list= context->table_list; + save_first_table= context->first_name_resolution_table; + save_last_table= context->last_name_resolution_table; + context->table_list= &tables; + context->first_name_resolution_table= &tables; + context->last_name_resolution_table= NULL; + func_expr->walk(&Item::change_context_processor, 0, (uchar*) context); + save_where= thd->where; + thd->where= "virtual column function"; + + /* Save the context before fixing the fields*/ + save_use_only_table_context= thd->lex->use_only_table_context; + thd->lex->use_only_table_context= TRUE; + /* Fix fields referenced to by the virtual column function */ + error= func_expr->fix_fields(thd, (Item**)0); + /* Restore the original context*/ + thd->lex->use_only_table_context= save_use_only_table_context; + context->table_list= save_table_list; + context->first_name_resolution_table= save_first_table; + context->last_name_resolution_table= save_last_table; + + if (unlikely(error)) + { + DBUG_PRINT("info", ("Field in virtual column function not part of table")); + clear_field_flag(table); + goto end; + } + thd->where= save_where; +#ifdef PARANOID + /* + Walk through the Item tree checking if all items are valid + to be part of the virtual column + */ + error= func_expr->walk(&Item::check_vcol_func_processor, 0, NULL); + if (error) + { + my_error(ER_VIRTUAL_COLUMN_FUNCTION_IS_NOT_ALLOWED, MYF(0), field_name); + clear_field_flag(table); + goto end; + } +#endif + if (unlikely(func_expr->const_item())) + { + my_error(ER_CONST_EXPR_IN_VCOL, MYF(0)); + clear_field_flag(table); + goto end; + } + /* Ensure that this virtual column is not based on another virtual field. */ + ptr= table->field; + while ((field= *(ptr++))) + { + if ((field->flags & GET_FIXED_FIELDS_FLAG) && + (field->vcol_info)) + { + my_error(ER_VCOL_BASED_ON_VCOL, MYF(0)); + clear_field_flag(table); + goto end; + } + } + /* + Cleanup the fields marked with flag GET_FIXED_FIELDS_FLAG + when calling fix_fields. + */ + clear_field_flag(table); + result= FALSE; + +end: + table->get_fields_in_item_tree= FALSE; + thd->mark_used_columns= save_mark_used_columns; + table->map= 0; //Restore old value + DBUG_RETURN(result); +} + +/* + Unpack the definition of a virtual column + + SYNOPSIS + unpack_vcol_info_from_frm() + thd Thread handler + table Table with the checked field + field Pointer to Field object + error_reported updated flag for the caller that no other error + messages are to be generated. + + RETURN VALUES + TRUE Failure + FALSE Success +*/ +bool unpack_vcol_info_from_frm(THD *thd, + TABLE *table, + Field *field, + LEX_STRING *vcol_expr, + bool *error_reported) +{ + DBUG_ENTER("unpack_vcol_info_from_frm"); + DBUG_ASSERT(vcol_expr); + + /* + Step 1: Construct a statement for the parser. + The parsed string needs to take the following format: + "PARSE_VCOL_EXPR (<expr_string_from_frm>)" + */ + char *vcol_expr_str; + int str_len= 0; + CHARSET_INFO *old_character_set_client; + + if (!(vcol_expr_str= (char*) alloc_root(&table->mem_root, + vcol_expr->length + + parse_vcol_keyword.length + 3))) + { + DBUG_RETURN(TRUE); + } + memcpy(vcol_expr_str, + (char*) parse_vcol_keyword.str, + parse_vcol_keyword.length); + str_len= parse_vcol_keyword.length; + memcpy(vcol_expr_str + str_len, "(", 1); + str_len++; + memcpy(vcol_expr_str + str_len, + (char*) vcol_expr->str, + vcol_expr->length); + str_len+= vcol_expr->length; + memcpy(vcol_expr_str + str_len, ")", 1); + str_len++; + memcpy(vcol_expr_str + str_len, "\0", 1); + str_len++; + Parser_state parser_state(thd, vcol_expr_str, str_len); + + /* + Step 2: Setup thd for parsing. + */ + Query_arena *backup_stmt_arena_ptr= thd->stmt_arena; + Query_arena backup_arena; + Query_arena vcol_arena(&table->mem_root, Query_arena::INITIALIZED); + thd->set_n_backup_active_arena(&vcol_arena, &backup_arena); + thd->stmt_arena= &vcol_arena; + + thd->lex->parse_vcol_expr= TRUE; + old_character_set_client= thd->variables.character_set_client; + + /* + Step 3: Use the parser to build an Item object from. + */ + if (parse_sql(thd, &parser_state, NULL)) + { + goto parse_err; + } + /* From now on use vcol_info generated by the parser. */ + field->vcol_info= thd->lex->vcol_info; + + /* Validate the Item tree. */ + if (fix_fields_vcol_func(thd, + field->vcol_info, + table, + field->field_name)) + { + *error_reported= TRUE; + field->vcol_info= 0; + goto parse_err; + } + thd->stmt_arena= backup_stmt_arena_ptr; + thd->restore_active_arena(&vcol_arena, &backup_arena); + field->vcol_info->item_free_list= vcol_arena.free_list; + + DBUG_RETURN(FALSE); + +parse_err: + thd->lex->parse_vcol_expr= FALSE; + thd->free_items(); + thd->stmt_arena= backup_stmt_arena_ptr; + thd->restore_active_arena(&vcol_arena, &backup_arena); + thd->variables.character_set_client= old_character_set_client; + DBUG_RETURN(TRUE); +} + +/* + Read data from a binary .frm file from MySQL 3.23 - 5.0 into TABLE_SHARE +*/ /* Open a table based on a TABLE_SHARE @@ -1631,7 +1959,7 @@ int open_table_from_share(THD *thd, TABL uint records, i, bitmap_size; bool error_reported= FALSE; uchar *record, *bitmaps; - Field **field_ptr; + Field **field_ptr, **vfield_ptr; DBUG_ENTER("open_table_from_share"); DBUG_PRINT("enter",("name: '%s.%s' form: 0x%lx", share->db.str, share->table_name.str, (long) outparam)); @@ -1784,6 +2112,34 @@ int open_table_from_share(THD *thd, TABL } } + /* + Process virtual columns, if any. + */ + if (!(vfield_ptr = (Field **) alloc_root(&outparam->mem_root, + (uint) ((share->vfields+1)* + sizeof(Field*))))) + goto err; + + outparam->vfield= vfield_ptr; + + for (field_ptr= outparam->field; *field_ptr; field_ptr++) + { + if ((*field_ptr)->vcol_info) + { + if (unpack_vcol_info_from_frm(thd, + outparam, + *field_ptr, + &(*field_ptr)->vcol_info->expr_str, + &error_reported)) + { + error= 4; // in case no error is reported + goto err; + } + *(vfield_ptr++)= *field_ptr; + } + } + *vfield_ptr= 0; // End marker + #ifdef WITH_PARTITION_STORAGE_ENGINE if (share->partition_info_len && outparam->file) { @@ -1851,6 +2207,20 @@ partititon_err: } #endif + /* Check virtual columns against table's storage engine. */ + if (share->vfields && + ((outparam->file && + !outparam->file->check_if_supported_virtual_columns()) || + (!outparam->file && share->db_type() && + share->db_type()->db_type == DB_TYPE_CSV_DB))) // Workaround for CSV + { + my_error(ER_UNSUPPORTED_ACTION_ON_VIRTUAL_COLUMN, + MYF(0), + "Specified storage engine"); + error_reported= TRUE; + goto err; + } + /* Allocate bitmaps */ bitmap_size= share->column_bitmap_size; @@ -1965,7 +2335,11 @@ int closefrm(register TABLE *table, bool if (table->field) { for (Field **ptr=table->field ; *ptr ; ptr++) + { + if ((*ptr)->vcol_info) + free_items((*ptr)->vcol_info->item_free_list); delete *ptr; + } table->field= 0; } delete table->file; @@ -3934,6 +4308,25 @@ const char *Field_iterator_table::name() return (*ptr)->field_name; } +Item *create_table_vcol_field(THD *thd, Item **field_ref, + const char *fld_name, + const char *tbl_name) +{ + Item *field= *field_ref; + DBUG_ENTER("create_table_vcol_field"); + DBUG_ASSERT(field); + + if (!field->fixed) + { + if (field->fix_fields(thd, field_ref)) + { + DBUG_RETURN(0); + } + field= *field_ref; + } + field->set_name(fld_name, strlen(fld_name), system_charset_info); + DBUG_RETURN(field); +} Item *Field_iterator_table::create_item(THD *thd) { @@ -4388,7 +4781,14 @@ void st_table::mark_columns_used_by_inde KEY_PART_INFO *key_part_end= (key_part + key_info[index].key_parts); for (;key_part != key_part_end; key_part++) + { bitmap_set_bit(bitmap, key_part->fieldnr-1); + if (key_part->field->vcol_info && + key_part->field->vcol_info->expr_item) + key_part->field->vcol_info-> + expr_item->walk(&Item::register_field_in_bitmap, + 1, (uchar *) bitmap); + } } @@ -4515,6 +4915,8 @@ void st_table::mark_columns_needed_for_u file->column_bitmaps_signal(); } } + /* Mark all virtual columns as writable */ + mark_virtual_columns(); DBUG_VOID_RETURN; } @@ -4541,8 +4943,42 @@ void st_table::mark_columns_needed_for_i } if (found_next_number_field) mark_auto_increment_column(); + /* Mark all virtual columns as writable */ + mark_virtual_columns(); } +/* + @brief Update the write and read table bitmap to allow + using procedure save_in_field for all virtual columns + in the table. + + @return void + + @detail + Each virtual field is set in the write column map. + All fields that the virtual columns are based on are set in the + read bitmap. +*/ + +void st_table::mark_virtual_columns(void) +{ + Field **vfield_ptr, *tmp_vfield; + bool bitmap_updated= FALSE; + + for (vfield_ptr= vfield; *vfield_ptr; vfield_ptr++) + { + tmp_vfield= *vfield_ptr; + DBUG_ASSERT(tmp_vfield->vcol_info && tmp_vfield->vcol_info->expr_item); + tmp_vfield->vcol_info->expr_item->walk(&Item::register_field_in_read_map, + 1, (uchar *) 0); + bitmap_set_bit(read_set, tmp_vfield->field_index); + bitmap_set_bit(write_set, tmp_vfield->field_index); + // TODO: consider updating column maps for index + bitmap_updated= TRUE; + } + if (bitmap_updated) + file->column_bitmaps_signal(); +} /** @brief Check if this is part of a MERGE table with attached children. @@ -4797,6 +5233,54 @@ size_t max_row_length(TABLE *table, cons return length; } +/* + Calculate data for each virtual field marked for write in the + corresponding column map. + + SYNOPSIS + update_virtual_fields_marked_for_write() + table The TABLE object + ignore_stored Indication whether physically stored virtual + fields do not need updating. + This value is false when during INSERT and UPDATE + and true in all other cases. + + RETURN + 0 - Success + >0 - Error occurred during the generation/calculation of a virtual field value + +*/ + +int update_virtual_fields_marked_for_write(TABLE *table, + bool ignore_stored) +{ + DBUG_ENTER("update_virtual_fields_marked_for_write"); + Field **vfield_ptr, *vfield; + int error= 0; + if (!table || !table->vfield) + DBUG_RETURN(0); + + /* Iterate over virtual fields in the table */ + for (vfield_ptr= table->vfield; *vfield_ptr; vfield_ptr++) + { + vfield= (*vfield_ptr); + DBUG_ASSERT(vfield->vcol_info && vfield->vcol_info->expr_item); + /* Only update those fields that are marked in the write_set bitmap */ + if (bitmap_is_set(table->write_set, vfield->field_index) && + (not (ignore_stored && vfield->stored_in_db))) + { + /* Generate the actual value of the virtual fields */ + error= vfield->vcol_info->expr_item->save_in_field(vfield, 0); + DBUG_PRINT("info", ("field '%s' - updated", vfield->field_name)); + } + else + { + DBUG_PRINT("info", ("field '%s' - skipped", vfield->field_name)); + } + } + DBUG_RETURN(0); +} + /***************************************************************************** ** Instansiate templates *****************************************************************************/ === modified file 'sql/table.h' --- sql/table.h 2009-02-19 09:01:25 +0000 +++ sql/table.h 2009-03-24 10:47:21 +0000 @@ -352,6 +352,8 @@ typedef struct st_table_share ulong version, mysql_version; ulong timestamp_offset; /* Set to offset+1 of record */ ulong reclength; /* Recordlength */ + ulong stored_rec_length; /* Stored record length + (no generated-only virtual fields) */ plugin_ref db_plugin; /* storage engine plugin */ inline handlerton *db_type() const /* table_type for handler */ @@ -370,6 +372,8 @@ typedef struct st_table_share uint key_block_size; /* create key_block_size, if used */ uint null_bytes, last_null_bit_pos; uint fields; /* Number of fields */ + uint stored_fields; /* Number of stored fields + (i.e. without generated-only ones) */ uint rec_buff_length; /* Size of table->record[] buffer */ uint keys, key_parts; uint max_key_length, max_unique_length, total_key_length; @@ -391,6 +395,7 @@ typedef struct st_table_share uint error, open_errno, errarg; /* error from open_table_def() */ uint column_bitmap_size; uchar frm_version; + uint vfields; /* Number of virtual fields */ bool null_field_first; bool system; /* Set if system table (one record) */ bool crypted; /* If .frm file is crypted */ @@ -655,6 +660,7 @@ struct st_table { Field *next_number_field; /* Set if next_number is activated */ Field *found_next_number_field; /* Set on open */ Field_timestamp *timestamp_field; + Field **vfield; /* Pointer to virtual fields*/ /* Table's triggers, 0 if there are no of them */ Table_triggers_list *triggers; @@ -811,6 +817,7 @@ struct st_table { void mark_columns_needed_for_update(void); void mark_columns_needed_for_delete(void); void mark_columns_needed_for_insert(void); + void mark_virtual_columns(void); inline void column_bitmaps_set(MY_BITMAP *read_set_arg, MY_BITMAP *write_set_arg) { === modified file 'sql/unireg.cc' --- sql/unireg.cc 2009-02-19 09:01:25 +0000 +++ sql/unireg.cc 2009-03-24 10:47:21 +0000 @@ -584,7 +584,7 @@ static bool pack_header(uchar *forminfo, { uint length,int_count,int_length,no_empty, int_parts; uint time_stamp_pos,null_fields; - ulong reclength, totlength, n_length, com_length; + ulong reclength, totlength, n_length, com_length, vcol_info_length; DBUG_ENTER("pack_header"); if (create_fields.elements > MAX_FIELDS) @@ -595,8 +595,8 @@ static bool pack_header(uchar *forminfo, totlength= 0L; reclength= data_offset; - no_empty=int_count=int_parts=int_length=time_stamp_pos=null_fields= - com_length=0; + no_empty=int_count=int_parts=int_length=time_stamp_pos=null_fields=0; + com_length=vcol_info_length=0; n_length=2L; /* Check fields */ @@ -625,6 +625,27 @@ static bool pack_header(uchar *forminfo, ER_UNKNOWN_ERROR, ER(ER_UNKNOWN_ERROR), buff); field->comment.length= tmp_len; } + if (field->vcol_info) + { + tmp_len= system_charset_info->cset->charpos(system_charset_info, + field->vcol_info->expr_str.str, + field->vcol_info->expr_str.str + + field->vcol_info->expr_str.length, + VIRTUAL_COLUMN_EXPRESSION_MAXLEN); + + if (tmp_len < field->vcol_info->expr_str.length) + { + my_error(ER_WRONG_STRING_LENGTH, MYF(0), + field->vcol_info->expr_str.str,"VIRTUAL COLUMN EXPRESSION", + (uint) VIRTUAL_COLUMN_EXPRESSION_MAXLEN); + DBUG_RETURN(1); + } + /* + Sum up the length of the expression string and mandatory header bytes + to the total length. + */ + vcol_info_length+= field->vcol_info->expr_str.length+(uint)FRM_VCOL_HEADER_SIZE; + } totlength+= field->length; com_length+= field->comment.length; @@ -644,8 +665,6 @@ static bool pack_header(uchar *forminfo, !time_stamp_pos) time_stamp_pos= (uint) field->offset+ (uint) data_offset + 1; length=field->pack_length; - /* Ensure we don't have any bugs when generating offsets */ - DBUG_ASSERT(reclength == field->offset + data_offset); if ((uint) field->offset+ (uint) data_offset+ length > reclength) reclength=(uint) (field->offset+ data_offset + length); n_length+= (ulong) strlen(field->field_name)+1; @@ -712,7 +731,8 @@ static bool pack_header(uchar *forminfo, /* Hack to avoid bugs with small static rows in MySQL */ reclength=max(file->min_record_length(table_options),reclength); if (info_length+(ulong) create_fields.elements*FCOMP+288+ - n_length+int_length+com_length > 65535L || int_count > 255) + n_length+int_length+com_length+vcol_info_length > 65535L || + int_count > 255) { my_message(ER_TOO_MANY_FIELDS, ER(ER_TOO_MANY_FIELDS), MYF(0)); DBUG_RETURN(1); @@ -720,7 +740,7 @@ static bool pack_header(uchar *forminfo, bzero((char*)forminfo,288); length=(info_length+create_fields.elements*FCOMP+288+n_length+int_length+ - com_length); + com_length+vcol_info_length); int2store(forminfo,length); forminfo[256] = (uint8) screens; int2store(forminfo+258,create_fields.elements); @@ -737,7 +757,8 @@ static bool pack_header(uchar *forminfo, int2store(forminfo+280,22); /* Rows needed */ int2store(forminfo+282,null_fields); int2store(forminfo+284,com_length); - /* Up to forminfo+288 is free to use for additional information */ + int2store(forminfo+286,vcol_info_length); + /* forminfo+288 is free to use for additional information */ DBUG_RETURN(0); } /* pack_header */ @@ -776,7 +797,7 @@ static bool pack_fields(File file, List< ulong data_offset) { reg2 uint i; - uint int_count, comment_length=0; + uint int_count, comment_length, vcol_info_length=0; uchar buff[MAX_FIELD_WIDTH]; Create_field *field; DBUG_ENTER("pack_fields"); @@ -789,6 +810,7 @@ static bool pack_fields(File file, List< while ((field=it++)) { uint recpos; + uint cur_vcol_expr_len= 0; buff[0]= (uchar) field->row; buff[1]= (uchar) field->col; buff[2]= (uchar) field->sc_length; @@ -811,6 +833,17 @@ static bool pack_fields(File file, List< buff[14]= (uchar) field->charset->number; else buff[14]= 0; // Numerical + if (field->vcol_info) + { + /* + Use the interval_id place in the .frm file to store the length of + virtual field's data. + */ + buff[12]= cur_vcol_expr_len= field->vcol_info->expr_str.length + + (uint)FRM_VCOL_HEADER_SIZE; + vcol_info_length+= cur_vcol_expr_len+(uint)FRM_VCOL_HEADER_SIZE; + buff[13]= (uchar) MYSQL_TYPE_VIRTUAL; + } int2store(buff+15, field->comment.length); comment_length+= field->comment.length; set_if_bigger(int_count,field->interval_id); @@ -905,6 +938,35 @@ static bool pack_fields(File file, List< DBUG_RETURN(1); } } + if (vcol_info_length) + { + it.rewind(); + int_count=0; + while ((field=it++)) + { + /* + Pack each virtual field as follows: + byte 1 = 1 (always 1 to allow for future extensions) + byte 2 = sql_type + byte 3 = flags (as of now, 0 - no flags, 1 - field is physically stored) + byte 4-... = virtual column expression (text data) + */ + if (field->vcol_info && field->vcol_info->expr_str.length) + { + Item *item= field->vcol_info->expr_item; + buff[0]= (uchar)1; + buff[1]= (uchar) field->sql_type; + buff[2]= (uchar) field->stored_in_db; + if (my_write(file, buff, 3, MYF_RW)) + DBUG_RETURN(1); + if (my_write(file, + (uchar*) field->vcol_info->expr_str.str, + field->vcol_info->expr_str.length, + MYF_RW)) + DBUG_RETURN(1); + } + } + } DBUG_RETURN(0); } === modified file 'storage/innobase/handler/ha_innodb.cc' --- storage/innobase/handler/ha_innodb.cc 2009-02-19 09:01:25 +0000 +++ storage/innobase/handler/ha_innodb.cc 2009-03-24 10:47:21 +0000 @@ -2437,12 +2437,12 @@ ha_innobase::open( } /* Create buffers for packing the fields of a record. Why - table->reclength did not work here? Obviously, because char + table->stored_rec_length did not work here? Obviously, because char fields when packed actually became 1 byte longer, when we also stored the string length as the first byte. */ upd_and_key_val_buff_len = - table->s->reclength + table->s->max_key_length + table->s->stored_rec_length + table->s->max_key_length + MAX_REF_PARTS * 3; if (!(uchar*) my_multi_malloc(MYF(MY_WME), &upd_buff, upd_and_key_val_buff_len, @@ -2517,7 +2517,7 @@ retry: prebuilt = row_create_prebuilt(ib_table); - prebuilt->mysql_row_len = table->s->reclength; + prebuilt->mysql_row_len = table->s->stored_rec_length;; prebuilt->default_rec = table->s->default_values; ut_ad(prebuilt->default_rec); @@ -3220,11 +3220,11 @@ build_template( dict_index_t* clust_index; mysql_row_templ_t* templ; Field* field; - ulint n_fields; + ulint n_fields, n_stored_fields; ulint n_requested_fields = 0; ibool fetch_all_in_key = FALSE; ibool fetch_primary_key_cols = FALSE; - ulint i; + ulint i, sql_idx, innodb_idx=0; /* byte offset of the end of last requested column */ ulint mysql_prefix_len = 0; @@ -3285,11 +3285,12 @@ build_template( } n_fields = (ulint)table->s->fields; /* number of columns */ + n_stored_fields= (ulint)table->s->stored_fields; /* number of stored columns */ if (!prebuilt->mysql_template) { prebuilt->mysql_template = (mysql_row_templ_t*) mem_alloc_noninline( - n_fields * sizeof(mysql_row_templ_t)); + n_stored_fields * sizeof(mysql_row_templ_t)); } prebuilt->template_type = templ_type; @@ -3299,15 +3300,17 @@ build_template( /* Note that in InnoDB, i is the column number. MySQL calls columns 'fields'. */ - for (i = 0; i < n_fields; i++) { + for (sql_idx = 0; sql_idx < n_fields; sql_idx++) { templ = prebuilt->mysql_template + n_requested_fields; - field = table->field[i]; + field = table->field[sql_idx]; + if (!field->stored_in_db) + goto skip_field; if (UNIV_LIKELY(templ_type == ROW_MYSQL_REC_FIELDS)) { /* Decide which columns we should fetch and which we can skip. */ register const ibool index_contains_field = - dict_index_contains_col_or_prefix(index, i); + dict_index_contains_col_or_prefix(index, innodb_idx); if (!index_contains_field && prebuilt->read_just_key) { /* If this is a 'key read', we do not need @@ -3322,8 +3325,8 @@ build_template( goto include_field; } - if (bitmap_is_set(table->read_set, i) || - bitmap_is_set(table->write_set, i)) { + if (bitmap_is_set(table->read_set, sql_idx) || + bitmap_is_set(table->write_set, sql_idx)) { /* This field is needed in the query */ goto include_field; @@ -3331,7 +3334,7 @@ build_template( if (fetch_primary_key_cols && dict_table_col_in_clustered_key( - index->table, i)) { + index->table, innodb_idx)) { /* This field is needed in the query */ goto include_field; @@ -3344,14 +3347,14 @@ build_template( include_field: n_requested_fields++; - templ->col_no = i; + templ->col_no = innodb_idx; if (index == clust_index) { templ->rec_field_no = dict_col_get_clust_pos_noninline( - &index->table->cols[i], index); + &index->table->cols[innodb_idx], index); } else { templ->rec_field_no = dict_index_get_nth_col_pos( - index, i); + index, innodb_idx); } if (templ->rec_field_no == ULINT_UNDEFINED) { @@ -3377,7 +3380,7 @@ include_field: mysql_prefix_len = templ->mysql_col_offset + templ->mysql_col_len; } - templ->type = index->table->cols[i].mtype; + templ->type = index->table->cols[innodb_idx].mtype; templ->mysql_type = (ulint)field->type(); if (templ->mysql_type == DATA_MYSQL_TRUE_VARCHAR) { @@ -3386,16 +3389,18 @@ include_field: } templ->charset = dtype_get_charset_coll_noninline( - index->table->cols[i].prtype); - templ->mbminlen = index->table->cols[i].mbminlen; - templ->mbmaxlen = index->table->cols[i].mbmaxlen; - templ->is_unsigned = index->table->cols[i].prtype + index->table->cols[innodb_idx].prtype); + templ->mbminlen = index->table->cols[innodb_idx].mbminlen; + templ->mbmaxlen = index->table->cols[innodb_idx].mbmaxlen; + templ->is_unsigned = index->table->cols[innodb_idx].prtype & DATA_UNSIGNED; if (templ->type == DATA_BLOB) { prebuilt->templ_contains_blob = TRUE; } skip_field: - ; + if (field->stored_in_db) { + innodb_idx++; + } } prebuilt->n_template = n_requested_fields; @@ -3848,7 +3853,7 @@ calc_row_difference( ulint n_changed = 0; dfield_t dfield; dict_index_t* clust_index; - uint i; + uint sql_idx, innodb_idx= 0; n_fields = table->s->fields; clust_index = dict_table_get_first_index_noninline(prebuilt->table); @@ -3856,8 +3861,10 @@ calc_row_difference( /* We use upd_buff to convert changed fields */ buf = (byte*) upd_buff; - for (i = 0; i < n_fields; i++) { - field = table->field[i]; + for (sql_idx = 0; sql_idx < n_fields; sql_idx++) { + field = table->field[sql_idx]; + if (!field->stored_in_db) + continue; o_ptr = (byte*) old_row + get_field_offset(table, field); n_ptr = (byte*) new_row + get_field_offset(table, field); @@ -3875,7 +3882,7 @@ calc_row_difference( field_mysql_type = field->type(); - col_type = prebuilt->table->cols[i].mtype; + col_type = prebuilt->table->cols[innodb_idx].mtype; switch (col_type) { @@ -3930,7 +3937,7 @@ calc_row_difference( /* Let us use a dummy dfield to make the conversion from the MySQL column format to the InnoDB format */ - dict_col_copy_type_noninline(prebuilt->table->cols + i, + dict_col_copy_type_noninline(prebuilt->table->cols + innodb_idx, &dfield.type); if (n_len != UNIV_SQL_NULL) { @@ -3951,9 +3958,11 @@ calc_row_difference( ufield->exp = NULL; ufield->field_no = dict_col_get_clust_pos_noninline( - &prebuilt->table->cols[i], clust_index); + &prebuilt->table->cols[innodb_idx], clust_index); n_changed++; } + if (field->stored_in_db) + innodb_idx++; } uvect->n_fields = n_changed; @@ -4939,7 +4948,7 @@ create_table_def( /* We pass 0 as the space id, and determine at a lower level the space id where to store the table */ - table = dict_mem_table_create(table_name, 0, n_cols, flags); + table = dict_mem_table_create(table_name, 0, form->s->stored_fields, flags); if (path_of_temp_table) { table->dir_path_of_temp_table = @@ -4948,6 +4957,8 @@ create_table_def( for (i = 0; i < n_cols; i++) { field = form->field[i]; + if (!field->stored_in_db) + continue; col_type = get_innobase_type_from_mysql_type(&unsigned_type, field); @@ -5236,7 +5247,7 @@ ha_innobase::create( } #endif - if (form->s->fields > 1000) { + if (form->s->stored_fields > 1000) { /* The limit probably should be REC_MAX_N_FIELDS - 3 = 1020, but we play safe here */ @@ -5746,10 +5757,10 @@ ha_innobase::records_in_range( KEY* key; dict_index_t* index; uchar* key_val_buff2 = (uchar*) my_malloc( - table->s->reclength + table->s->stored_rec_length + table->s->max_key_length + 100, MYF(MY_FAE)); - ulint buff2_len = table->s->reclength + ulint buff2_len = table->s->stored_rec_length + table->s->max_key_length + 100; dtuple_t* range_start; dtuple_t* range_end; === modified file 'storage/innobase/handler/ha_innodb.h' --- storage/innobase/handler/ha_innodb.h 2008-12-14 20:59:50 +0000 +++ storage/innobase/handler/ha_innodb.h 2009-03-24 10:47:21 +0000 @@ -199,6 +199,7 @@ class ha_innobase: public handler int cmp_ref(const uchar *ref1, const uchar *ref2); bool check_if_incompatible_data(HA_CREATE_INFO *info, uint table_changes); + bool check_if_supported_virtual_columns(void) { return TRUE;} }; /* Some accessor functions which the InnoDB plugin needs, but which === modified file 'storage/myisam/ha_myisam.cc' --- storage/myisam/ha_myisam.cc 2008-10-10 15:28:41 +0000 +++ storage/myisam/ha_myisam.cc 2009-03-24 10:47:21 +0000 @@ -230,7 +230,7 @@ int table2myisam(TABLE *table_arg, MI_KE record= table_arg->record[0]; recpos= 0; recinfo_pos= recinfo; - while (recpos < (uint) share->reclength) + while (recpos < (uint) share->stored_rec_length) { Field **field, *found= 0; minpos= share->reclength; === modified file 'storage/myisam/ha_myisam.h' --- storage/myisam/ha_myisam.h 2008-06-28 12:45:15 +0000 +++ storage/myisam/ha_myisam.h 2009-03-24 10:47:22 +0000 @@ -132,6 +132,7 @@ class ha_myisam: public handler int assign_to_keycache(THD* thd, HA_CHECK_OPT* check_opt); int preload_keys(THD* thd, HA_CHECK_OPT* check_opt); bool check_if_incompatible_data(HA_CREATE_INFO *info, uint table_changes); + bool check_if_supported_virtual_columns(void) { return TRUE;} #ifdef HAVE_REPLICATION int dump(THD* thd, int fd); int net_read_dump(NET* net);

1 2

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (sanja:2689)
by sanja＠askmonty.org 25 Mar '09

25 Mar '09

#At lp:maria 2689 sanja(a)askmonty.org 2009-03-25 Bugs found after Item types check fixed. modified: sql/item.h per-file messages: sql/item.h Unused VIEW_FIXER_ITEM removed (appeared as result of fix with then was redone). Item_insert_value::type() (which checked in Item_insert_value::eq() but was skipped in original patch) added. === modified file 'sql/item.h' --- a/sql/item.h 2009-02-19 09:01:25 +0000 +++ b/sql/item.h 2009-03-25 10:04:29 +0000 @@ -481,8 +481,7 @@ public: FIELD_VARIANCE_ITEM, INSERT_VALUE_ITEM, SUBSELECT_ITEM, ROW_ITEM, CACHE_ITEM, TYPE_HOLDER, PARAM_ITEM, TRIGGER_FIELD_ITEM, DECIMAL_ITEM, - XPATH_NODESET, XPATH_NODESET_CMP, - VIEW_FIXER_ITEM}; + XPATH_NODESET, XPATH_NODESET_CMP}; enum cond_result { COND_UNDEF,COND_OK,COND_TRUE,COND_FALSE }; @@ -2603,7 +2602,8 @@ public: { return Item_field::save_in_field(field_arg, no_conversions); } - /* + enum Type type() const { return INSERT_VALUE_ITEM; } + /* We use RAND_TABLE_BIT to prevent Item_insert_value from being treated as a constant and precalculated before execution */

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2688)
by Michael Widenius 22 Mar '09

22 Mar '09

#At lp:maria based on revid:monty@askmonty.org-20090322121609-r1sdxnlfydqsc7io 2688 Michael Widenius 2009-03-22 [merge] Merge with trunk modified: mysql-test/lib/My/Test.pm mysql-test/lib/mtr_cases.pm mysql-test/mysql-test-run.pl mysql-test/t/multi_update2.test storage/maria/ma_loghandler.c === modified file 'mysql-test/lib/My/Test.pm' --- a/mysql-test/lib/My/Test.pm 2008-11-14 10:49:12 +0000 +++ b/mysql-test/lib/My/Test.pm 2009-03-20 14:18:22 +0000 @@ -9,6 +9,7 @@ package My::Test; use strict; use warnings; use Carp; +use Storable(); sub new { @@ -30,18 +31,6 @@ sub key { } -sub _encode { - my ($value)= @_; - $value =~ s/([|\\\x{0a}\x{0d}])/sprintf('\%02X', ord($1))/eg; - return $value; -} - -sub _decode { - my ($value)= @_; - $value =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/ge; - return $value; -} - sub is_failed { my ($self)= @_; my $result= $self->{result}; @@ -58,66 +47,22 @@ sub write_test { # Give the test a unique key before serializing it $test->{key}= "$test" unless defined $test->{key}; - print $sock $header, "\n"; - while ((my ($key, $value)) = each(%$test)) { - print $sock $key, "= "; - if (ref $value eq "ARRAY") { - print $sock "[", _encode(join(", ", @$value)), "]"; - } else { - print $sock _encode($value); - } - print $sock "\n"; - } - print $sock "\n"; + my $serialized= Storable::freeze($test); + $serialized =~ s/([\x0d\x0a\\])/sprintf("\\%02x", ord($1))/eg; + print $sock $header, "\n", $serialized, "\n"; } sub read_test { my ($sock)= @_; - my $test= My::Test->new(); - # Read the : separated key value pairs until a - # single newline on it's own line - my $line; - while (defined($line= <$sock>)) { - # List is terminated by newline on it's own - if ($line eq "\n") { - # Correctly terminated reply - # print "Got newline\n"; - last; - } - chomp($line); - - # Split key/value on the first "=" - my ($key, $value)= split("= ", $line, 2); - - if ($value =~ /^\[(.*)\]/){ - my @values= split(", ", _decode($1)); - push(@{$test->{$key}}, @values); - } - else - { - $test->{$key}= _decode($value); - } - } + my $serialized= <$sock>; + chomp($serialized); + $serialized =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/eg; + my $test= Storable::thaw($serialized); + die "wrong class (hack attempt?)" + unless ref($test) eq 'My::Test'; return $test; } -sub print_test { - my ($self)= @_; - - print "[", $self->{name}, "]", "\n"; - while ((my ($key, $value)) = each(%$self)) { - print " ", $key, "= "; - if (ref $value eq "ARRAY") { - print "[", join(", ", @$value), "]"; - } else { - print $value; - } - print "\n"; - } - print "\n"; -} - - 1; === modified file 'mysql-test/lib/mtr_cases.pm' --- a/mysql-test/lib/mtr_cases.pm 2009-02-19 09:01:25 +0000 +++ b/mysql-test/lib/mtr_cases.pm 2009-03-20 14:39:37 +0000 @@ -799,15 +799,6 @@ sub collect_one_test_case { push(@{$tinfo->{'master_opt'}}, @$suite_opts); push(@{$tinfo->{'slave_opt'}}, @$suite_opts); - #----------------------------------------------------------------------- - # Check for test specific config file - #----------------------------------------------------------------------- - my $test_cnf_file= "$testdir/$tname.cnf"; - if ( -f $test_cnf_file) { - # Specifies the configuration file to use for this test - $tinfo->{'template_path'}= $test_cnf_file; - } - # ---------------------------------------------------------------------- # Check for test specific config file # ---------------------------------------------------------------------- === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2009-03-12 22:27:35 +0000 +++ b/mysql-test/mysql-test-run.pl 2009-03-20 14:18:22 +0000 @@ -357,7 +357,7 @@ sub main { mtr_print_thick_line(); mtr_print_header(); - my $completed= run_test_server($server, $tests, $opt_parallel); + my ($completed, $fail)= run_test_server($server, $tests, $opt_parallel); # Send Ctrl-C to any children still running kill("INT", keys(%children)); @@ -393,6 +393,10 @@ sub main { mtr_error("Not all tests completed"); } + if ($fail) { + mtr_error("Test suite failure."); + } + mtr_print_line(); if ( $opt_gcov ) { @@ -412,6 +416,7 @@ sub run_test_server ($$$) { my $num_saved_cores= 0; # Number of core files saved in vardir/log/ so far. my $num_saved_datadir= 0; # Number of datadirs saved in vardir/log/ so far. my $num_failed_test= 0; # Number of tests failed so far + my $test_failure= 0; # Scheduler variables my $max_ndb= $childs / 2; @@ -445,7 +450,7 @@ sub run_test_server ($$$) { $s->remove($sock); if (--$childs == 0){ $suite_timeout_proc->kill(); - return $completed; + return ($completed, $test_failure); } next; } @@ -509,18 +514,19 @@ sub run_test_server ($$$) { } $num_saved_datadir++; + $test_failure= 1; if ( !$opt_force ) { # Test has failed, force is off $suite_timeout_proc->kill(); push(@$completed, $result); - return $completed; + return ($completed, 1); } elsif ($opt_max_test_fail > 0 and $num_failed_test >= $opt_max_test_fail) { $suite_timeout_proc->kill(); mtr_report("Too many tests($num_failed_test) failed!", "Terminating..."); - return undef; + return (undef, 1); } $num_failed_test++; } @@ -571,7 +577,18 @@ sub run_test_server ($$$) { elsif ($line eq 'START'){ ; # Send first test } - else { + elsif ($line eq 'WARNINGS'){ + my $fake_test= My::Test::read_test($sock); + my $test_list= join (" ", @{$fake_test->{testnames}}); + mtr_report("***Warnings generated in error logs during shutdown ". + "after running tests: $test_list"); + $test_failure= 1; + if ( !$opt_force ) { + # Test failure due to warnings, force is off + $suite_timeout_proc->kill(); + return ($completed, 1); + } + } else { mtr_error("Unknown response: '$line' from client"); } @@ -649,7 +666,7 @@ sub run_test_server ($$$) { if ( ! $suite_timeout_proc->wait_one(0) ) { mtr_report("Test suite timeout! Terminating..."); - return undef; + return (undef, 1); } } } @@ -717,7 +734,7 @@ sub run_worker ($) { delete($test->{'comment'}); delete($test->{'logfile'}); - run_testcase($test); + run_testcase($test, $server); #$test->{result}= 'MTR_RES_PASSED'; # Send it back, now with results set #$test->print_test(); @@ -725,6 +742,15 @@ sub run_worker ($) { } elsif ($line eq 'BYE'){ mtr_report("Server said BYE"); + # We need to gracefully shut down the servers to see any + # Valgrind memory leak errors etc. since last server restart. + if ($opt_warnings) { + stop_servers(all_servers()); + if(check_warnings_post_shutdown($server)) { + # Warnings appeared in log file(s) during final server shutdown. + exit(1); + } + } exit(0); } else { @@ -732,9 +758,7 @@ sub run_worker ($) { } } - stop_all_servers(); - - exit(1); + die "Internal error: should not reach this place."; } @@ -3109,6 +3133,7 @@ sub run_on_all($$) sub mark_log { my ($log, $tinfo)= @_; my $log_msg= "CURRENT_TEST: $tinfo->{name}\n"; + pre_write_errorlog($log, $tinfo->{name}); mtr_tofile($log, $log_msg); } @@ -3181,8 +3206,8 @@ my %old_env; # > 0 failure # -sub run_testcase ($) { - my $tinfo= shift; +sub run_testcase ($$) { + my ($tinfo, $server_socket)= @_; mtr_verbose("Running test:", $tinfo->{name}); @@ -3197,7 +3222,12 @@ sub run_testcase ($) { { my @restart= servers_need_restart($tinfo); if ( @restart != 0) { - stop_servers($tinfo, @restart ); + # Remember that we restarted for this test case (count restarts) + $tinfo->{'restarted'}= 1; + stop_servers(@restart ); + if ($opt_warnings) { + check_warnings_post_shutdown($server_socket); + } } if ( started(all_servers()) == 0 ) @@ -3336,7 +3366,18 @@ sub run_testcase ($) { { if ($check_res == 1) { # Test case had sideeffects, not fatal error, just continue - stop_all_servers(); + if ($opt_warnings) { + # Checking error logs for warnings, so need to stop server + # gracefully so that memory leaks etc. can be properly detected. + stop_servers(all_servers()); + check_warnings_post_shutdown($server_socket); + # Even if we got warnings here, we should not fail this + # particular test, as the warnings may be caused by an earlier + # test. + } else { + # Not checking warnings, so can do a hard shutdown. + stop_all_servers(); + } mtr_report("Resuming tests...\n"); } else { @@ -3468,6 +3509,64 @@ sub run_testcase ($) { } +# We want to preserve the error log between server restarts, as it may contain +# valuable debugging information even if there is no test failure recorded. +sub _preserve_error_log_names { + my ($mysqld)= @_; + my $error_log_file= $mysqld->value('log-error'); + my $error_log_dir= dirname($error_log_file); + my $save_name= $error_log_dir ."/../". $mysqld->name() .".error.log"; + return ($error_log_file, $save_name); +} + +sub preserve_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($error_log_file, $save_name); + # Ignore any errors, as it's just a best-effort to keep the log if possible. +} + +sub restore_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($save_name, $error_log_file); +} + +# Keep track of last position in mysqld error log where we scanned for +# warnings, so we can attribute any warnings found to the correct test +# suite or server restart. +my $last_warning_position= { }; + +# Called just before a mysqld server is started or a testcase is run, +# to keep track of which tests have been run since last restart, and +# of when the error log is reset. +# +# Second argument $test_name is test name, or undef for server restart. +sub pre_write_errorlog { + my ($error_log, $test_name)= @_; + + if (! -e $error_log) { + # If the error log is moved away, reset the warning parse position. + delete $last_warning_position->{$error_log}; + } + + if (defined($test_name)) { + $last_warning_position->{$error_log}{test_names}= [] + unless exists($last_warning_position->{$error_log}{test_names}); + push @{$last_warning_position->{$error_log}{test_names}}, $test_name; + } else { + # Server restart, so clear the list of tests run since last restart. + # (except the last one (if any), which is the test about to be run). + if (defined($last_warning_position->{$error_log}{test_names}) && + @{$last_warning_position->{$error_log}{test_names}}) { + $last_warning_position->{$error_log}{test_names}= + [$last_warning_position->{$error_log}{test_names}[-1]]; + } else { + $last_warning_position->{$error_log}{test_names}= []; + } + } +} + # # Perform a rough examination of the servers # error log and write all lines that look @@ -3479,18 +3578,14 @@ sub extract_warning_lines ($) { # Open the servers .err log file and read all lines # belonging to current tets into @lines my $Ferr = IO::File->new($error_log) - or mtr_error("Could not open file '$error_log' for reading: $!"); + or return []; + my $last_pos= $last_warning_position->{$error_log}{seek_pos}; + $Ferr->seek($last_pos, 0) if defined($last_pos); + # If the seek fails, we will parse the whole error log, at least we will not + # miss any warnings. - my @lines; - while ( my $line = <$Ferr> ) - { - if ( $line =~ /^CURRENT_TEST:/ ) - { - # Throw away lines from previous tests - @lines = (); - } - push(@lines, $line); - } + my @lines= <$Ferr>; + $last_warning_position->{$error_log}{seek_pos}= $Ferr->tell(); $Ferr = undef; # Close error log file # mysql_client_test.test sends a COM_DEBUG packet to the server @@ -3537,20 +3632,47 @@ sub extract_warning_lines ($) { qr/Attempting backtrace/, qr/Assertion .* failed/, ); + # These are taken from the include/mtr_warnings.sql global suppression + # list. They occur delayed, so they can be parsed during shutdown rather + # than during the per-test check. + # + # ToDo: having the warning suppressions inside the mysqld we are trying to + # check is in any case horrible. We should change it to all be done here + # within the Perl code, which is both simpler, easier, faster, and more + # robust. We could still have individual test cases put in suppressions by + # parsing statically or by writing dynamically to a CSV table read by the + # Perl code. + my @antipatterns = + ( + qr/InnoDB: Error: in ALTER TABLE `test`.`t[12]`/, + qr/InnoDB: Error: table `test`.`t[12]` does not exist in the InnoDB internal/, + ); - foreach my $line ( @lines ) + my $match_count= 0; + LINE: foreach my $line ( @lines ) { - foreach my $pat ( @patterns ) + PAT: foreach my $pat ( @patterns ) { if ( $line =~ /$pat/ ) { + foreach my $apat (@antipatterns) + { + next LINE if $line =~ $apat; + } print $Fwarn $line; - last; + ++$match_count; + last PAT; } } } $Fwarn = undef; # Close file + if ($match_count > 0 && + defined($last_warning_position->{$error_log}{test_names})) { + return $last_warning_position->{$error_log}{test_names}; + } else { + return []; + } } @@ -3717,6 +3839,22 @@ sub check_warnings ($) { mtr_error("INTERNAL_ERROR: check_warnings"); } +# Check for warnings generated during shutdown of a mysqld server. +# If any, report them to master server, and return true; else just return false. +sub check_warnings_post_shutdown { + my ($server_socket)= @_; + my $testname_hash= { }; + foreach my $mysqld ( mysqlds()) + { + my $testlist= extract_warning_lines($mysqld->value('log-error')); + $testname_hash->{$_}= 1 for @$testlist; + } + my @warning_tests= keys(%$testname_hash); + if (@warning_tests) { + my $fake_test= My::Test->new(testnames => \@warning_tests); + $fake_test->write_test($server_socket, 'WARNINGS'); + } +} # # Loop through our list of processes and look for and entry @@ -3814,6 +3952,7 @@ sub clean_datadir { foreach my $mysqld ( mysqlds() ) { my $mysqld_dir= dirname($mysqld->value('datadir')); + preserve_error_log($mysqld); if (-d $mysqld_dir ) { mtr_verbose(" - removing '$mysqld_dir'"); rmtree($mysqld_dir); @@ -4142,6 +4281,7 @@ sub mysqld_start ($$) { if ( defined $exe ) { + pre_write_errorlog($output); $mysqld->{'proc'}= My::SafeProcess->new ( name => $mysqld->name(), @@ -4371,10 +4511,7 @@ sub get_extra_opts { sub stop_servers($$) { - my ($tinfo, @servers)= @_; - - # Remember if we restarted for this test case (count restarts) - $tinfo->{'restarted'}= 1; + my (@servers)= @_; if ( join('|', @servers) eq join('|', all_servers()) ) { @@ -4466,6 +4603,7 @@ sub start_servers($) { } if (-d $datadir ) { + preserve_error_log($mysqld); mtr_verbose(" - removing '$datadir'"); rmtree($datadir); } @@ -4491,6 +4629,7 @@ sub start_servers($) { unless -d $datadir; } + restore_error_log($mysqld); # Create the servers tmpdir my $tmpdir= $mysqld->value('tmpdir'); === modified file 'mysql-test/t/multi_update2.test' --- a/mysql-test/t/multi_update2.test 2008-11-19 18:17:26 +0000 +++ b/mysql-test/t/multi_update2.test 2009-03-20 14:18:22 +0000 @@ -2,6 +2,9 @@ # Test of update statement that uses many tables. # +# This is a big test. +--source include/big_test.inc + --disable_warnings DROP TABLE IF EXISTS t1,t2; --enable_warnings === modified file 'storage/maria/ma_loghandler.c' --- a/storage/maria/ma_loghandler.c 2009-02-19 09:01:25 +0000 +++ b/storage/maria/ma_loghandler.c 2009-03-17 19:05:01 +0000 @@ -333,6 +333,8 @@ struct st_translog_descriptor my_bool is_everything_flushed; /* True when flush pass is in progress */ my_bool flush_in_progress; + /* The flush number (used to distinguish two flushes goes one by one) */ + volatile int flush_no; /* Next flush pass variables */ TRANSLOG_ADDRESS next_pass_max_lsn; pthread_t max_lsn_requester; @@ -3484,6 +3486,8 @@ my_bool translog_init_with_table(const c id_to_share= NULL; log_descriptor.directory_fd= -1; log_descriptor.is_everything_flushed= 1; + log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no= 0; log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; (*init_table_func)(); @@ -7548,6 +7552,7 @@ void translog_flush_wait_for_end(LSN ls void translog_flush_set_new_goal_and_wait(TRANSLOG_ADDRESS lsn) { + int flush_no= log_descriptor.flush_no; DBUG_ENTER("translog_flush_set_new_goal_and_wait"); DBUG_PRINT("enter", ("LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); safe_mutex_assert_owner(&log_descriptor.log_flush_lock); @@ -7556,7 +7561,7 @@ void translog_flush_set_new_goal_and_wai log_descriptor.next_pass_max_lsn= lsn; log_descriptor.max_lsn_requester= pthread_self(); } - while (log_descriptor.flush_in_progress) + while (flush_no == log_descriptor.flush_no) { pthread_cond_wait(&log_descriptor.log_flush_cond, &log_descriptor.log_flush_lock); @@ -7735,6 +7740,7 @@ out: if (sent_to_disk != LSN_IMPOSSIBLE) log_descriptor.flushed= sent_to_disk; log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no++; DBUG_PRINT("info", ("flush_in_progress is dropped")); pthread_mutex_unlock(&log_descriptor.log_flush_lock);\ pthread_cond_broadcast(&log_descriptor.log_flush_cond);

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2687)
by Michael Widenius 22 Mar '09

22 Mar '09

#At lp:maria based on revid:monty@askmonty.org-20090320114554-e6kokfmsmyr4as9g 2687 Michael Widenius 2009-03-22 Apply patch by Antony Dovgal: - Move SAFE_MUTEX to be stored in config.h by configure.in (not as a flag used with compiler command line) - Generate my_config.h in configure modified: BUILD/SETUP.sh BUILD/compile-ia64-debug-max configure.in dbug/dbug.c include/Makefile.am mysys/my_wincond.c mysys/my_winthread.c per-file messages: BUILD/SETUP.sh Remove -DSAFE_MUTEX as the following --with-debug flag will automaticly add it BUILD/compile-ia64-debug-max Remove -DSAFE_MUTEX as the following --with-debug flag will automaticly add it configure.in Move SAFE_MUTEX and SAFE_MALLOC to [my_] config.h Generate my_config.h as part of configure process dbug/dbug.c Include my_global.h before we undef SAFE_MUTEX include/Makefile.am Update comment. For now, lets generate my_config.h if someone deletes it after configure mysys/my_wincond.c Include my_global.h before we undef SAFE_MUTEX mysys/my_winthread.c Include my_global.h before we undef SAFE_MUTEX === modified file 'BUILD/SETUP.sh' --- a/BUILD/SETUP.sh 2009-03-12 22:27:35 +0000 +++ b/BUILD/SETUP.sh 2009-03-22 12:16:09 +0000 @@ -124,7 +124,7 @@ valgrind_flags="$valgrind_flags -DMYSQL_ # # Used in -debug builds debug_cflags="-DUNIV_MUST_NOT_INLINE -DEXTRA_DEBUG -DFORCE_INIT_OF_VARS " -debug_cflags="$debug_cflags -DSAFEMALLOC -DPEDANTIC_SAFEMALLOC -DSAFE_MUTEX" +debug_cflags="$debug_cflags -DSAFEMALLOC -DPEDANTIC_SAFEMALLOC" error_inject="--with-error-inject " # # Base C++ flags for all builds === modified file 'BUILD/compile-ia64-debug-max' --- a/BUILD/compile-ia64-debug-max 2007-01-28 20:09:54 +0000 +++ b/BUILD/compile-ia64-debug-max 2009-03-22 12:16:09 +0000 @@ -4,5 +4,5 @@ gmake -k maintainer-clean || true path=`dirname $0` . "$path/autorun.sh" -CC=ecc CFLAGS="-w1 -DEXTRA_DEBUG -DSAFEMALLOC -DSAFE_MUTEX -O2" CXX=ecc CXXFLAGS="-w1 -DEXTRA_DEBUG -DSAFEMALLOC -DSAFE_MUTEX -O2" ./configure --prefix=/usr/local/mysql --with-extra-charsets=complex --enable-thread-safe-client --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static --with-debug --with-innodb --with-embedded-server --with-archive-storage-engine +CC=ecc CFLAGS="-w1 -DEXTRA_DEBUG -DSAFEMALLOC -O2" CXX=ecc CXXFLAGS="-w1 -DEXTRA_DEBUG -DSAFEMALLOC -O2" ./configure --prefix=/usr/local/mysql --with-extra-charsets=complex --enable-thread-safe-client --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static --with-debug --with-innodb --with-embedded-server --with-archive-storage-engine gmake === modified file 'configure.in' --- a/configure.in 2009-03-12 22:27:35 +0000 +++ b/configure.in 2009-03-22 12:16:09 +0000 @@ -1677,14 +1677,17 @@ if test "$with_debug" = "yes" then # Medium debug. AC_DEFINE([DBUG_ON], [1], [Use libdbug]) - CFLAGS="$DEBUG_CFLAGS $DEBUG_OPTIMIZE_CC -DSAFE_MUTEX $CFLAGS" - CXXFLAGS="$DEBUG_CXXFLAGS $DEBUG_OPTIMIZE_CXX -DSAFE_MUTEX $CXXFLAGS" + AC_DEFINE([SAFE_MUTEX], [1], [Use safe mutexes]) + CFLAGS="$DEBUG_CFLAGS $DEBUG_OPTIMIZE_CC $CFLAGS" + CXXFLAGS="$DEBUG_CXXFLAGS $DEBUG_OPTIMIZE_CXX $CXXFLAGS" elif test "$with_debug" = "full" then # Full debug. Very slow in some cases AC_DEFINE([DBUG_ON], [1], [Use libdbug]) - CFLAGS="$DEBUG_CFLAGS -DSAFE_MUTEX -DSAFEMALLOC $CFLAGS" - CXXFLAGS="$DEBUG_CXXFLAGS -DSAFE_MUTEX -DSAFEMALLOC $CXXFLAGS" + AC_DEFINE([SAFE_MUTEX], [1], [Use safe mutexes]) + AC_DEFINE([SAFEMALLOC], [1], [Use safe malloc]) + CFLAGS="$DEBUG_CFLAGS $CFLAGS" + CXXFLAGS="$DEBUG_CXXFLAGS $CXXFLAGS" else # Optimized version. No debug AC_DEFINE([DBUG_OFF], [1], [Don't use libdbug]) @@ -2812,6 +2815,7 @@ AC_CONFIG_FILES(Makefile extra/Makefile include/mysql_version.h plugin/Makefile win/Makefile) AC_CONFIG_COMMANDS([default], , test -z "$CONFIG_HEADERS" || echo timestamp > stamp-h) +AC_CONFIG_COMMANDS([my_config.h], cp include/config.h include/my_config.h) # Ensure that table handlers gets all modifications to CFLAGS/CXXFLAGS AC_CONFIG_COMMANDS_POST(ac_configure_args="$ac_configure_args CFLAGS='$CFLAGS' CXXFLAGS='$CXXFLAGS'") === modified file 'dbug/dbug.c' --- a/dbug/dbug.c 2009-03-12 22:27:35 +0000 +++ b/dbug/dbug.c 2009-03-22 12:16:09 +0000 @@ -84,8 +84,8 @@ in pthread_mutex_lock */ -#undef SAFE_MUTEX #include <my_global.h> +#undef SAFE_MUTEX #include <m_string.h> #include <errno.h> === modified file 'include/Makefile.am' --- a/include/Makefile.am 2008-11-21 14:21:50 +0000 +++ b/include/Makefile.am 2009-03-22 12:16:09 +0000 @@ -59,6 +59,8 @@ link_sources: # We want both "my_config.h" and "config.h" that are identical, as # MySQL sources assumes the name "my_config.h", and 3rd party sources # assumes the name "config.h". +# Normally this is generated by configure; This rule is left here in case +# someone deletes my_config.h and expect it to be generated by make my_config.h: config.h $(CP) config.h my_config.h === modified file 'mysys/my_wincond.c' --- a/mysys/my_wincond.c 2008-10-21 14:10:04 +0000 +++ b/mysys/my_wincond.c 2009-03-22 12:16:09 +0000 @@ -17,6 +17,7 @@ ** The following is a simple implementation of posix conditions *****************************************************************************/ +#include <my_global.h> #undef SAFE_MUTEX /* Avoid safe_mutex redefinitions */ #include "mysys_priv.h" #if defined(THREAD) && defined(__WIN__) === modified file 'mysys/my_winthread.c' --- a/mysys/my_winthread.c 2008-01-10 12:21:53 +0000 +++ b/mysys/my_winthread.c 2009-03-22 12:16:09 +0000 @@ -18,6 +18,7 @@ *****************************************************************************/ /* SAFE_MUTEX will not work until the thread structure is up to date */ +#include <my_global.h> #undef SAFE_MUTEX #include "mysys_priv.h"

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (sanja:2689)
by sanja＠askmonty.org 20 Mar '09

20 Mar '09

#At lp:maria 2689 sanja(a)askmonty.org 2009-03-20 [merge] Merge modified: storage/maria/ma_loghandler.c === modified file 'storage/maria/ma_loghandler.c' --- a/storage/maria/ma_loghandler.c 2009-02-19 09:01:25 +0000 +++ b/storage/maria/ma_loghandler.c 2009-03-17 19:05:01 +0000 @@ -333,6 +333,8 @@ struct st_translog_descriptor my_bool is_everything_flushed; /* True when flush pass is in progress */ my_bool flush_in_progress; + /* The flush number (used to distinguish two flushes goes one by one) */ + volatile int flush_no; /* Next flush pass variables */ TRANSLOG_ADDRESS next_pass_max_lsn; pthread_t max_lsn_requester; @@ -3484,6 +3486,8 @@ my_bool translog_init_with_table(const c id_to_share= NULL; log_descriptor.directory_fd= -1; log_descriptor.is_everything_flushed= 1; + log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no= 0; log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; (*init_table_func)(); @@ -7548,6 +7552,7 @@ void translog_flush_wait_for_end(LSN ls void translog_flush_set_new_goal_and_wait(TRANSLOG_ADDRESS lsn) { + int flush_no= log_descriptor.flush_no; DBUG_ENTER("translog_flush_set_new_goal_and_wait"); DBUG_PRINT("enter", ("LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); safe_mutex_assert_owner(&log_descriptor.log_flush_lock); @@ -7556,7 +7561,7 @@ void translog_flush_set_new_goal_and_wai log_descriptor.next_pass_max_lsn= lsn; log_descriptor.max_lsn_requester= pthread_self(); } - while (log_descriptor.flush_in_progress) + while (flush_no == log_descriptor.flush_no) { pthread_cond_wait(&log_descriptor.log_flush_cond, &log_descriptor.log_flush_lock); @@ -7735,6 +7740,7 @@ out: if (sent_to_disk != LSN_IMPOSSIBLE) log_descriptor.flushed= sent_to_disk; log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no++; DBUG_PRINT("info", ("flush_in_progress is dropped")); pthread_mutex_unlock(&log_descriptor.log_flush_lock);\ pthread_cond_broadcast(&log_descriptor.log_flush_cond);

1 0

[Maria-developers] MariaDB and Windows
by Patrick Crews 20 Mar '09

20 Mar '09

Hi there. I was wondering what is the policy / direction for MariaDB and Windows. Basically, are we aiming to maintain MariaDB for that platform as well? I am asking because of the following output log from a couple of attempted Windows builds of a Maria branch I pulled today 3/18/2009. This was on a 32-bit version of XP that I use to build MySQL on a regular basis without incident. To make a long story short: ========== Rebuild All: 26 succeeded, 39 failed, 1 skipped ========== A lot of things are failing now. Please see attached output for details. I haven't really researched the failures too much myself, I was mainly curious if we were taking a similar approach as Drizzle, etc. Let me know if I can provide further information.

3 4

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2688)
by knielsen＠knielsen-hq.org 20 Mar '09

20 Mar '09

#At lp:maria 2688 knielsen(a)knielsen-hq.org 2009-03-20 Remove redundant code, probably bad automerge modified: mysql-test/lib/mtr_cases.pm per-file messages: mysql-test/lib/mtr_cases.pm Remove redundant code, probably bad automerge === modified file 'mysql-test/lib/mtr_cases.pm' --- a/mysql-test/lib/mtr_cases.pm 2009-02-19 09:01:25 +0000 +++ b/mysql-test/lib/mtr_cases.pm 2009-03-20 14:39:37 +0000 @@ -799,15 +799,6 @@ sub collect_one_test_case { push(@{$tinfo->{'master_opt'}}, @$suite_opts); push(@{$tinfo->{'slave_opt'}}, @$suite_opts); - #----------------------------------------------------------------------- - # Check for test specific config file - #----------------------------------------------------------------------- - my $test_cnf_file= "$testdir/$tname.cnf"; - if ( -f $test_cnf_file) { - # Specifies the configuration file to use for this test - $tinfo->{'template_path'}= $test_cnf_file; - } - # ---------------------------------------------------------------------- # Check for test specific config file # ----------------------------------------------------------------------

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2687) Bug#43418
by knielsen＠knielsen-hq.org 20 Mar '09

20 Mar '09

#At lp:maria 2687 knielsen(a)knielsen-hq.org 2009-03-20 BUG#43418: MTR2: does not notice a memory leak occuring at shutdown of mysqld w/ --valgrind - Fixed by implementing parsing of error log messages generated outside of test case runs (eg. during server shutdown). Also make mysql-test-run.pl not delete the error log after server restart, which looses information about which warnings were found. Finally, make multi_update2 a --big test. modified: mysql-test/lib/My/Test.pm mysql-test/mysql-test-run.pl mysql-test/t/multi_update2.test per-file messages: mysql-test/lib/My/Test.pm Fix home-brewed (and broken) serialization in My::Test to use the standard Storable serializer. mysql-test/mysql-test-run.pl - Stop mysqld servers gracefully rather than kill -9 when warnings are being checked. - After stopping mysqld servers, do an additional parse of the error log to check for any warnings generated during shutdown. - Fix error log parsing to be careful not to skip parsing part of the file, by keeping track of previous file position rather than relying on mark_log markers. - Workers report warnings during shutdown to the master process with a new packet 'WARNINGS' which includes a list of names of test that might have caused the problem (could be any test run since last server start). - Fail entire test suite if warnings are found. - When we remove the server data dir before server restart, preserve the error log (don't delete it between restarts), as it may contain valuable information even for test cases which don't show direct failures. mysql-test/t/multi_update2.test Make test --big, as it takes a _long_ time to run and only tests a single bug. === modified file 'mysql-test/lib/My/Test.pm' --- a/mysql-test/lib/My/Test.pm 2008-11-14 10:49:12 +0000 +++ b/mysql-test/lib/My/Test.pm 2009-03-20 14:18:22 +0000 @@ -9,6 +9,7 @@ package My::Test; use strict; use warnings; use Carp; +use Storable(); sub new { @@ -30,18 +31,6 @@ sub key { } -sub _encode { - my ($value)= @_; - $value =~ s/([|\\\x{0a}\x{0d}])/sprintf('\%02X', ord($1))/eg; - return $value; -} - -sub _decode { - my ($value)= @_; - $value =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/ge; - return $value; -} - sub is_failed { my ($self)= @_; my $result= $self->{result}; @@ -58,66 +47,22 @@ sub write_test { # Give the test a unique key before serializing it $test->{key}= "$test" unless defined $test->{key}; - print $sock $header, "\n"; - while ((my ($key, $value)) = each(%$test)) { - print $sock $key, "= "; - if (ref $value eq "ARRAY") { - print $sock "[", _encode(join(", ", @$value)), "]"; - } else { - print $sock _encode($value); - } - print $sock "\n"; - } - print $sock "\n"; + my $serialized= Storable::freeze($test); + $serialized =~ s/([\x0d\x0a\\])/sprintf("\\%02x", ord($1))/eg; + print $sock $header, "\n", $serialized, "\n"; } sub read_test { my ($sock)= @_; - my $test= My::Test->new(); - # Read the : separated key value pairs until a - # single newline on it's own line - my $line; - while (defined($line= <$sock>)) { - # List is terminated by newline on it's own - if ($line eq "\n") { - # Correctly terminated reply - # print "Got newline\n"; - last; - } - chomp($line); - - # Split key/value on the first "=" - my ($key, $value)= split("= ", $line, 2); - - if ($value =~ /^\[(.*)\]/){ - my @values= split(", ", _decode($1)); - push(@{$test->{$key}}, @values); - } - else - { - $test->{$key}= _decode($value); - } - } + my $serialized= <$sock>; + chomp($serialized); + $serialized =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/eg; + my $test= Storable::thaw($serialized); + die "wrong class (hack attempt?)" + unless ref($test) eq 'My::Test'; return $test; } -sub print_test { - my ($self)= @_; - - print "[", $self->{name}, "]", "\n"; - while ((my ($key, $value)) = each(%$self)) { - print " ", $key, "= "; - if (ref $value eq "ARRAY") { - print "[", join(", ", @$value), "]"; - } else { - print $value; - } - print "\n"; - } - print "\n"; -} - - 1; === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2009-03-12 22:27:35 +0000 +++ b/mysql-test/mysql-test-run.pl 2009-03-20 14:18:22 +0000 @@ -357,7 +357,7 @@ sub main { mtr_print_thick_line(); mtr_print_header(); - my $completed= run_test_server($server, $tests, $opt_parallel); + my ($completed, $fail)= run_test_server($server, $tests, $opt_parallel); # Send Ctrl-C to any children still running kill("INT", keys(%children)); @@ -393,6 +393,10 @@ sub main { mtr_error("Not all tests completed"); } + if ($fail) { + mtr_error("Test suite failure."); + } + mtr_print_line(); if ( $opt_gcov ) { @@ -412,6 +416,7 @@ sub run_test_server ($$$) { my $num_saved_cores= 0; # Number of core files saved in vardir/log/ so far. my $num_saved_datadir= 0; # Number of datadirs saved in vardir/log/ so far. my $num_failed_test= 0; # Number of tests failed so far + my $test_failure= 0; # Scheduler variables my $max_ndb= $childs / 2; @@ -445,7 +450,7 @@ sub run_test_server ($$$) { $s->remove($sock); if (--$childs == 0){ $suite_timeout_proc->kill(); - return $completed; + return ($completed, $test_failure); } next; } @@ -509,18 +514,19 @@ sub run_test_server ($$$) { } $num_saved_datadir++; + $test_failure= 1; if ( !$opt_force ) { # Test has failed, force is off $suite_timeout_proc->kill(); push(@$completed, $result); - return $completed; + return ($completed, 1); } elsif ($opt_max_test_fail > 0 and $num_failed_test >= $opt_max_test_fail) { $suite_timeout_proc->kill(); mtr_report("Too many tests($num_failed_test) failed!", "Terminating..."); - return undef; + return (undef, 1); } $num_failed_test++; } @@ -571,7 +577,18 @@ sub run_test_server ($$$) { elsif ($line eq 'START'){ ; # Send first test } - else { + elsif ($line eq 'WARNINGS'){ + my $fake_test= My::Test::read_test($sock); + my $test_list= join (" ", @{$fake_test->{testnames}}); + mtr_report("***Warnings generated in error logs during shutdown ". + "after running tests: $test_list"); + $test_failure= 1; + if ( !$opt_force ) { + # Test failure due to warnings, force is off + $suite_timeout_proc->kill(); + return ($completed, 1); + } + } else { mtr_error("Unknown response: '$line' from client"); } @@ -649,7 +666,7 @@ sub run_test_server ($$$) { if ( ! $suite_timeout_proc->wait_one(0) ) { mtr_report("Test suite timeout! Terminating..."); - return undef; + return (undef, 1); } } } @@ -717,7 +734,7 @@ sub run_worker ($) { delete($test->{'comment'}); delete($test->{'logfile'}); - run_testcase($test); + run_testcase($test, $server); #$test->{result}= 'MTR_RES_PASSED'; # Send it back, now with results set #$test->print_test(); @@ -725,6 +742,15 @@ sub run_worker ($) { } elsif ($line eq 'BYE'){ mtr_report("Server said BYE"); + # We need to gracefully shut down the servers to see any + # Valgrind memory leak errors etc. since last server restart. + if ($opt_warnings) { + stop_servers(all_servers()); + if(check_warnings_post_shutdown($server)) { + # Warnings appeared in log file(s) during final server shutdown. + exit(1); + } + } exit(0); } else { @@ -732,9 +758,7 @@ sub run_worker ($) { } } - stop_all_servers(); - - exit(1); + die "Internal error: should not reach this place."; } @@ -3109,6 +3133,7 @@ sub run_on_all($$) sub mark_log { my ($log, $tinfo)= @_; my $log_msg= "CURRENT_TEST: $tinfo->{name}\n"; + pre_write_errorlog($log, $tinfo->{name}); mtr_tofile($log, $log_msg); } @@ -3181,8 +3206,8 @@ my %old_env; # > 0 failure # -sub run_testcase ($) { - my $tinfo= shift; +sub run_testcase ($$) { + my ($tinfo, $server_socket)= @_; mtr_verbose("Running test:", $tinfo->{name}); @@ -3197,7 +3222,12 @@ sub run_testcase ($) { { my @restart= servers_need_restart($tinfo); if ( @restart != 0) { - stop_servers($tinfo, @restart ); + # Remember that we restarted for this test case (count restarts) + $tinfo->{'restarted'}= 1; + stop_servers(@restart ); + if ($opt_warnings) { + check_warnings_post_shutdown($server_socket); + } } if ( started(all_servers()) == 0 ) @@ -3336,7 +3366,18 @@ sub run_testcase ($) { { if ($check_res == 1) { # Test case had sideeffects, not fatal error, just continue - stop_all_servers(); + if ($opt_warnings) { + # Checking error logs for warnings, so need to stop server + # gracefully so that memory leaks etc. can be properly detected. + stop_servers(all_servers()); + check_warnings_post_shutdown($server_socket); + # Even if we got warnings here, we should not fail this + # particular test, as the warnings may be caused by an earlier + # test. + } else { + # Not checking warnings, so can do a hard shutdown. + stop_all_servers(); + } mtr_report("Resuming tests...\n"); } else { @@ -3468,6 +3509,64 @@ sub run_testcase ($) { } +# We want to preserve the error log between server restarts, as it may contain +# valuable debugging information even if there is no test failure recorded. +sub _preserve_error_log_names { + my ($mysqld)= @_; + my $error_log_file= $mysqld->value('log-error'); + my $error_log_dir= dirname($error_log_file); + my $save_name= $error_log_dir ."/../". $mysqld->name() .".error.log"; + return ($error_log_file, $save_name); +} + +sub preserve_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($error_log_file, $save_name); + # Ignore any errors, as it's just a best-effort to keep the log if possible. +} + +sub restore_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($save_name, $error_log_file); +} + +# Keep track of last position in mysqld error log where we scanned for +# warnings, so we can attribute any warnings found to the correct test +# suite or server restart. +my $last_warning_position= { }; + +# Called just before a mysqld server is started or a testcase is run, +# to keep track of which tests have been run since last restart, and +# of when the error log is reset. +# +# Second argument $test_name is test name, or undef for server restart. +sub pre_write_errorlog { + my ($error_log, $test_name)= @_; + + if (! -e $error_log) { + # If the error log is moved away, reset the warning parse position. + delete $last_warning_position->{$error_log}; + } + + if (defined($test_name)) { + $last_warning_position->{$error_log}{test_names}= [] + unless exists($last_warning_position->{$error_log}{test_names}); + push @{$last_warning_position->{$error_log}{test_names}}, $test_name; + } else { + # Server restart, so clear the list of tests run since last restart. + # (except the last one (if any), which is the test about to be run). + if (defined($last_warning_position->{$error_log}{test_names}) && + @{$last_warning_position->{$error_log}{test_names}}) { + $last_warning_position->{$error_log}{test_names}= + [$last_warning_position->{$error_log}{test_names}[-1]]; + } else { + $last_warning_position->{$error_log}{test_names}= []; + } + } +} + # # Perform a rough examination of the servers # error log and write all lines that look @@ -3479,18 +3578,14 @@ sub extract_warning_lines ($) { # Open the servers .err log file and read all lines # belonging to current tets into @lines my $Ferr = IO::File->new($error_log) - or mtr_error("Could not open file '$error_log' for reading: $!"); + or return []; + my $last_pos= $last_warning_position->{$error_log}{seek_pos}; + $Ferr->seek($last_pos, 0) if defined($last_pos); + # If the seek fails, we will parse the whole error log, at least we will not + # miss any warnings. - my @lines; - while ( my $line = <$Ferr> ) - { - if ( $line =~ /^CURRENT_TEST:/ ) - { - # Throw away lines from previous tests - @lines = (); - } - push(@lines, $line); - } + my @lines= <$Ferr>; + $last_warning_position->{$error_log}{seek_pos}= $Ferr->tell(); $Ferr = undef; # Close error log file # mysql_client_test.test sends a COM_DEBUG packet to the server @@ -3537,20 +3632,47 @@ sub extract_warning_lines ($) { qr/Attempting backtrace/, qr/Assertion .* failed/, ); + # These are taken from the include/mtr_warnings.sql global suppression + # list. They occur delayed, so they can be parsed during shutdown rather + # than during the per-test check. + # + # ToDo: having the warning suppressions inside the mysqld we are trying to + # check is in any case horrible. We should change it to all be done here + # within the Perl code, which is both simpler, easier, faster, and more + # robust. We could still have individual test cases put in suppressions by + # parsing statically or by writing dynamically to a CSV table read by the + # Perl code. + my @antipatterns = + ( + qr/InnoDB: Error: in ALTER TABLE `test`.`t[12]`/, + qr/InnoDB: Error: table `test`.`t[12]` does not exist in the InnoDB internal/, + ); - foreach my $line ( @lines ) + my $match_count= 0; + LINE: foreach my $line ( @lines ) { - foreach my $pat ( @patterns ) + PAT: foreach my $pat ( @patterns ) { if ( $line =~ /$pat/ ) { + foreach my $apat (@antipatterns) + { + next LINE if $line =~ $apat; + } print $Fwarn $line; - last; + ++$match_count; + last PAT; } } } $Fwarn = undef; # Close file + if ($match_count > 0 && + defined($last_warning_position->{$error_log}{test_names})) { + return $last_warning_position->{$error_log}{test_names}; + } else { + return []; + } } @@ -3717,6 +3839,22 @@ sub check_warnings ($) { mtr_error("INTERNAL_ERROR: check_warnings"); } +# Check for warnings generated during shutdown of a mysqld server. +# If any, report them to master server, and return true; else just return false. +sub check_warnings_post_shutdown { + my ($server_socket)= @_; + my $testname_hash= { }; + foreach my $mysqld ( mysqlds()) + { + my $testlist= extract_warning_lines($mysqld->value('log-error')); + $testname_hash->{$_}= 1 for @$testlist; + } + my @warning_tests= keys(%$testname_hash); + if (@warning_tests) { + my $fake_test= My::Test->new(testnames => \@warning_tests); + $fake_test->write_test($server_socket, 'WARNINGS'); + } +} # # Loop through our list of processes and look for and entry @@ -3814,6 +3952,7 @@ sub clean_datadir { foreach my $mysqld ( mysqlds() ) { my $mysqld_dir= dirname($mysqld->value('datadir')); + preserve_error_log($mysqld); if (-d $mysqld_dir ) { mtr_verbose(" - removing '$mysqld_dir'"); rmtree($mysqld_dir); @@ -4142,6 +4281,7 @@ sub mysqld_start ($$) { if ( defined $exe ) { + pre_write_errorlog($output); $mysqld->{'proc'}= My::SafeProcess->new ( name => $mysqld->name(), @@ -4371,10 +4511,7 @@ sub get_extra_opts { sub stop_servers($$) { - my ($tinfo, @servers)= @_; - - # Remember if we restarted for this test case (count restarts) - $tinfo->{'restarted'}= 1; + my (@servers)= @_; if ( join('|', @servers) eq join('|', all_servers()) ) { @@ -4466,6 +4603,7 @@ sub start_servers($) { } if (-d $datadir ) { + preserve_error_log($mysqld); mtr_verbose(" - removing '$datadir'"); rmtree($datadir); } @@ -4491,6 +4629,7 @@ sub start_servers($) { unless -d $datadir; } + restore_error_log($mysqld); # Create the servers tmpdir my $tmpdir= $mysqld->value('tmpdir'); === modified file 'mysql-test/t/multi_update2.test' --- a/mysql-test/t/multi_update2.test 2008-11-19 18:17:26 +0000 +++ b/mysql-test/t/multi_update2.test 2009-03-20 14:18:22 +0000 @@ -2,6 +2,9 @@ # Test of update statement that uses many tables. # +# This is a big test. +--source include/big_test.inc + --disable_warnings DROP TABLE IF EXISTS t1,t2; --enable_warnings

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (sanja:2683)
by sanja＠askmonty.org 20 Mar '09

20 Mar '09

#At lp:maria 2683 sanja(a)askmonty.org 2009-03-17 Fixed bug of waiting for flush pass end. modified: storage/maria/ma_loghandler.c per-file messages: storage/maria/ma_loghandler.c Fixed bug of waiting for flush pass end: in case of two sequential flushes waiting thread can not detect end of the pass it is waiting because till the time it will check the flush in progress flag will be set on again. Solution is to have pass number and compare the number before before waits start and check it as condition of the flush pass end (of course it does not matter if the counter of pass number overflows). === modified file 'storage/maria/ma_loghandler.c' --- a/storage/maria/ma_loghandler.c 2009-02-19 09:01:25 +0000 +++ b/storage/maria/ma_loghandler.c 2009-03-17 19:05:01 +0000 @@ -333,6 +333,8 @@ struct st_translog_descriptor my_bool is_everything_flushed; /* True when flush pass is in progress */ my_bool flush_in_progress; + /* The flush number (used to distinguish two flushes goes one by one) */ + volatile int flush_no; /* Next flush pass variables */ TRANSLOG_ADDRESS next_pass_max_lsn; pthread_t max_lsn_requester; @@ -3484,6 +3486,8 @@ my_bool translog_init_with_table(const c id_to_share= NULL; log_descriptor.directory_fd= -1; log_descriptor.is_everything_flushed= 1; + log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no= 0; log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; (*init_table_func)(); @@ -7548,6 +7552,7 @@ void translog_flush_wait_for_end(LSN ls void translog_flush_set_new_goal_and_wait(TRANSLOG_ADDRESS lsn) { + int flush_no= log_descriptor.flush_no; DBUG_ENTER("translog_flush_set_new_goal_and_wait"); DBUG_PRINT("enter", ("LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); safe_mutex_assert_owner(&log_descriptor.log_flush_lock); @@ -7556,7 +7561,7 @@ void translog_flush_set_new_goal_and_wai log_descriptor.next_pass_max_lsn= lsn; log_descriptor.max_lsn_requester= pthread_self(); } - while (log_descriptor.flush_in_progress) + while (flush_no == log_descriptor.flush_no) { pthread_cond_wait(&log_descriptor.log_flush_cond, &log_descriptor.log_flush_lock); @@ -7735,6 +7740,7 @@ out: if (sent_to_disk != LSN_IMPOSSIBLE) log_descriptor.flushed= sent_to_disk; log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no++; DBUG_PRINT("info", ("flush_in_progress is dropped")); pthread_mutex_unlock(&log_descriptor.log_flush_lock);\ pthread_cond_broadcast(&log_descriptor.log_flush_cond);

2 1

Re: [Maria-developers] safe patches to backport -- pool-of-threads
by Michael Widenius 20 Mar '09

20 Mar '09

Hi! Sorry for replying this 'off-thread', but I was not by some strange reason subscribed to the maria-developers list :( MARK CALLAGHAN writes: mark> One of the features listed here mark> http://askmonty.org/wiki/index.php/MariaDB_versus_MySQL as a potential mark> backport is support for pool-of-threads. I am not sure how safe this mark> feature is today. mark> mark> http://mysqlha.blogspot.com/2009/01/no-new-global-mutexes-and-how-to-make.h… mark> describes some of the problems that might cause a server to hang when mark> pool-of-threads is used. The main port of the pool-of-threads code has been in extensive use by a high performance MySQL customer for a long time and, within their limits, it has given them a lot of extra performance. To be fair, the above customer is using a scheduler that is using getport instead of libevent. I am considering to add also getport support to the scheduler.cc in MariaDB, even if this only gives higher performance on Solaris. The patch has it's problems and is not complete, but for certain workloads and configurations it can definitely save the day. One example is if your application needs 200,000 connected users. One of the biggest problem with pool-of-threads is that when you run out of threads you are stuck as you can't see what's going on and you can connect and kill things. The extra-port options fixes that. It also allows you to mix pool-of-threads and one-thread-per-connection in an easy way. It would have been better to make this a connection option, but because connections are also handled by the thread pool, when you run with pool-of-threads, this doesn't work as if all worker threads are occupied the connection will never go trough. Adding another pool for connection would require so much new work that it wouldn't be safe for MariaDB 5.1. For MariaDB 5.2 we should consider looking at adding another thread pool for connections and making the scheduler a connection option. Drizzle is by default using pool-of-threads, so the code gets more tested every day. mark> http://bugs.mysql.com/bug.php?id=42288 describes the performance mark> overhead of pool-of-threads. The overhead is significant for workloads mark> with fast queries such as sysbench. Brian/Drizzle is, as far as I know, trying to benchmark the original getevent port code and the libevent code to find out how things really stands. Brian suspect that there may a problem in libevent that needs to be fixed and he has someone looking into it. When they or we have fixed it, we will collaborate to ensure both projects works properly. mark> I think that pool-of-threads needs a lot of work before it can be mark> considered safe. We backported this to our 5.0 branch at work and have mark> begun to search for fixes. At least pool-of-threads is not default in MariaDB and you have to turn it explicitely one if you want to use it; This makes it reasonable safe to include into MariaDB 5.1 at this point as old user will not be affected at all by it and new users trying it can do that as long as they are aware of the limitations. Regards, Monty

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2686)
by Michael Widenius 20 Mar '09

20 Mar '09

#At lp:maria based on revid:monty@askmonty.org-20090320114130-0mthxmxifg10e8qj 2686 Michael Widenius 2009-03-20 [merge] Merge with trunk removed: mysql-test/t/pool_of_threads-master.opt added: mysql-test/t/connect.cnf mysql-test/t/pool_of_threads.cnf modified: client/mysqltest.cc mysql-test/lib/My/ConfigFactory.pm mysql-test/r/connect.result mysql-test/r/pool_of_threads.result mysql-test/t/connect.test mysql-test/t/pool_of_threads.test === modified file 'client/mysqltest.cc' --- a/client/mysqltest.cc 2009-03-20 11:41:30 +0000 +++ b/client/mysqltest.cc 2009-03-20 11:45:54 +0000 @@ -76,7 +76,7 @@ enum { OPT_SKIP_SAFEMALLOC=OPT_MAX_CLIENT_OPTION, OPT_PS_PROTOCOL, OPT_SP_PROTOCOL, OPT_CURSOR_PROTOCOL, OPT_VIEW_PROTOCOL, OPT_MAX_CONNECT_RETRIES, OPT_MARK_PROGRESS, OPT_LOG_DIR, OPT_TAIL_LINES, - OPT_GLOBAL_SUBST + OPT_GLOBAL_SUBST, OPT_MY_CONNECT_TIMEOUT }; static int record= 0, opt_sleep= -1; @@ -87,6 +87,7 @@ const char *opt_include= 0, *opt_charset static int opt_port= 0; static int opt_max_connect_retries; static my_bool opt_compress= 0, silent= 0, verbose= 0; +static int opt_connect_timeout= -1; static my_bool debug_info_flag= 0, debug_check_flag= 0; static my_bool tty_password= 0; static my_bool opt_mark_progress= 0; @@ -4952,6 +4953,9 @@ void do_connect(struct st_command *comma if (opt_charsets_dir) mysql_options(&con_slot->mysql, MYSQL_SET_CHARSET_DIR, opt_charsets_dir); + if (opt_connect_timeout >= 0) + mysql_options(&con_slot->mysql, MYSQL_OPT_CONNECT_TIMEOUT, + &opt_connect_timeout); #ifdef HAVE_OPENSSL if (opt_use_ssl || con_ssl) @@ -5692,6 +5696,9 @@ static struct my_option my_long_options[ #include "sslopt-longopts.h" {"test-file", 'x', "Read test from/in this file (default stdin).", 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, + {"connect-timeout", OPT_MY_CONNECT_TIMEOUT, "Client connection timeout", + (uchar**) &opt_connect_timeout, (uchar**) &opt_connect_timeout, 0, + GET_INT, REQUIRED_ARG, -1, -1, 0, 0, 0, 0}, {"timer-file", 'm', "File where the timing in micro seconds is stored.", 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, {"tmpdir", 't', "Temporary directory where sockets are put.", === modified file 'mysql-test/lib/My/ConfigFactory.pm' --- a/mysql-test/lib/My/ConfigFactory.pm 2009-02-15 10:58:34 +0000 +++ b/mysql-test/lib/My/ConfigFactory.pm 2009-03-18 15:46:32 +0000 @@ -202,6 +202,7 @@ my @mysqld_rules= { 'pid-file' => \&fix_pidfile }, { '#host' => \&fix_host }, { 'port' => \&fix_port }, + { '#extra-port' => \&fix_port }, { 'socket' => \&fix_socket }, { 'log-error' => \&fix_log_error }, { 'general-log' => sub { return 1; } }, @@ -353,7 +354,7 @@ sub post_check_client_group { sub post_check_client_groups { my ($self, $config)= @_; - my $first_mysqld= $config->first_like('mysqld.'); + my $first_mysqld= $config->first_like('mysqld\.'); return unless $first_mysqld; === modified file 'mysql-test/r/connect.result' --- a/mysql-test/r/connect.result 2008-03-17 11:26:00 +0000 +++ b/mysql-test/r/connect.result 2009-03-18 15:46:32 +0000 @@ -215,6 +215,13 @@ SET GLOBAL event_scheduler = OFF; # -- End of Bug#35074. +SELECT 'Connection on extra port ok'; +Connection on extra port ok +Connection on extra port ok +SELECT 'Connection on extra port 2 ok'; +Connection on extra port 2 ok +Connection on extra port 2 ok +# -- Success: more than --extra-max-connections + 1 normal connections not possible # ------------------------------------------------------------------ # -- End of 5.1 tests # ------------------------------------------------------------------ === modified file 'mysql-test/r/pool_of_threads.result' --- a/mysql-test/r/pool_of_threads.result 2009-03-12 22:27:35 +0000 +++ b/mysql-test/r/pool_of_threads.result 2009-03-18 15:46:32 +0000 @@ -2151,3 +2151,23 @@ Privat (Private Nutzung) Mobilfunk Warnings: Warning 1052 Column 'kundentyp' in group statement is ambiguous drop table t1; +SELECT sleep(5); +SELECT sleep(5); +# -- Success: more than --thread-pool-size normal connections not possible +sleep(5) +0 +sleep(5) +0 +SELECT sleep(5); +SELECT sleep(5); +SELECT 'Connection on extra port ok'; +Connection on extra port ok +Connection on extra port ok +SELECT 'Connection on extra port 2 ok'; +Connection on extra port 2 ok +Connection on extra port 2 ok +# -- Success: more than --extra-max-connections + 1 normal connections not possible +sleep(5) +0 +sleep(5) +0 === added file 'mysql-test/t/connect.cnf' --- a/mysql-test/t/connect.cnf 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/connect.cnf 2009-03-18 15:46:32 +0000 @@ -0,0 +1,8 @@ +!include include/default_my.cnf + +[mysqld.1] +extra-port= @mysqld.1.#extra-port +extra-max-connections=1 + +[ENV] +MASTER_EXTRA_PORT= @mysqld.1.extra-port === modified file 'mysql-test/t/connect.test' --- a/mysql-test/t/connect.test 2008-03-17 11:26:00 +0000 +++ b/mysql-test/t/connect.test 2009-03-18 15:46:32 +0000 @@ -288,6 +288,33 @@ let $wait_condition = --echo # -- End of Bug#35074. --echo +# Test connections to the extra port. + +connect(extracon,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon; +SELECT 'Connection on extra port ok'; + +connect(extracon2,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon2; +SELECT 'Connection on extra port 2 ok'; + +--disable_abort_on_error +--disable_result_log +--disable_query_log +connect(extracon3,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +--enable_query_log +--enable_result_log +--enable_abort_on_error +let $error = $mysql_errno; +if (!$error) +{ + --echo # -- Error: managed to establish more than --extra-max-connections + 1 connections +} +if ($error) +{ + --echo # -- Success: more than --extra-max-connections + 1 normal connections not possible +} + --echo # ------------------------------------------------------------------ --echo # -- End of 5.1 tests --echo # ------------------------------------------------------------------ === removed file 'mysql-test/t/pool_of_threads-master.opt' --- a/mysql-test/t/pool_of_threads-master.opt 2009-03-12 22:27:35 +0000 +++ b/mysql-test/t/pool_of_threads-master.opt 1970-01-01 00:00:00 +0000 @@ -1 +0,0 @@ ---test-ignore-wrong-options --thread-handling=pool-of-threads === added file 'mysql-test/t/pool_of_threads.cnf' --- a/mysql-test/t/pool_of_threads.cnf 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/pool_of_threads.cnf 2009-03-18 15:46:32 +0000 @@ -0,0 +1,14 @@ +!include include/default_my.cnf + +[mysqld.1] +test-ignore-wrong-options +thread-handling= pool-of-threads +thread_pool_size= 2 +extra-port= @mysqld.1.#extra-port +extra-max-connections=1 + +[client] +connect-timeout= 2 + +[ENV] +MASTER_EXTRA_PORT= @mysqld.1.extra-port === modified file 'mysql-test/t/pool_of_threads.test' --- a/mysql-test/t/pool_of_threads.test 2009-03-12 22:27:35 +0000 +++ b/mysql-test/t/pool_of_threads.test 2009-03-18 15:46:32 +0000 @@ -5,3 +5,78 @@ -- source include/common-tests.inc + +# Test that we cannot have more simultaneous connections than +# --thread-pool-size on the standard port, but _can_ have additional +# connections on the extra port. + +# First set two connections running, and check that extra connection +# on normal port fails due to--thread-pool-size=2 +connection default; +send SELECT sleep(5); + +connect(con2,localhost,root,,); +connection con2; +send SELECT sleep(5); +--sleep 1 + +--disable_abort_on_error +--disable_result_log +--disable_query_log +connect(con3,localhost,root,,); +--enable_query_log +--enable_result_log +--enable_abort_on_error +let $error = $mysql_errno; +if (!$error) +{ + --echo # -- Error: managed to establish more than --thread-pool-size connections +} +if ($error) +{ + --echo # -- Success: more than --thread-pool-size normal connections not possible +} + +connection default; +--reap +connection con2; +--reap + +# Now try again, but this time use the extra port to successfully connect. + +connection default; +send SELECT sleep(5); + +connection con2; +send SELECT sleep(5); +--sleep 1 + +connect(extracon,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon; +SELECT 'Connection on extra port ok'; + +connect(extracon2,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon2; +SELECT 'Connection on extra port 2 ok'; + +--disable_abort_on_error +--disable_result_log +--disable_query_log +connect(extracon3,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +--enable_query_log +--enable_result_log +--enable_abort_on_error +let $error = $mysql_errno; +if (!$error) +{ + --echo # -- Error: managed to establish more than --extra-max-connections + 1 connections +} +if ($error) +{ + --echo # -- Success: more than --extra-max-connections + 1 normal connections not possible +} + +connection default; +--reap +connection con2; +--reap

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2685)
by Michael Widenius 20 Mar '09

20 Mar '09

#At lp:maria based on revid:monty@askmonty.org-20090318021520-fn3aisa99yt4oe97 2685 Michael Widenius 2009-03-20 Avoid compiler warnings on windows modified: client/mysqltest.cc include/config-win.h unittest/mysys/waiting_threads-t.c per-file messages: client/mysqltest.cc Avoid compiler warnings on windows include/config-win.h Added missing typedef ssize_t Added some other useful defines from MySQL 6.0 unittest/mysys/waiting_threads-t.c Fixed link failure for 'ftruncate' on Windows === modified file 'client/mysqltest.cc' --- a/client/mysqltest.cc 2009-02-19 09:01:25 +0000 +++ b/client/mysqltest.cc 2009-03-20 11:41:30 +0000 @@ -1622,7 +1622,7 @@ int compare_files2(File fd1, const char* { int error= RESULT_OK; File fd2; - my_off_t fd1_length, fd2_length; + size_t fd1_length, fd2_length; DYNAMIC_STRING fd1_result, fd2_result; if ((fd2= my_open(filename2, O_RDONLY, MYF(0))) < 0) @@ -1631,8 +1631,8 @@ int compare_files2(File fd1, const char* die("Failed to open second file: '%s'", filename2); } - fd1_length= my_seek(fd1, 0, SEEK_END, MYF(0)); - fd2_length= my_seek(fd2, 0, SEEK_END, MYF(0)); + fd1_length= (size_t) my_seek(fd1, 0, SEEK_END, MYF(0)); + fd2_length= (size_t) my_seek(fd2, 0, SEEK_END, MYF(0)); if (init_dynamic_string(&fd1_result, 0, fd1_length, 0) || init_dynamic_string(&fd2_result, 0, fd2_length, 0)) === modified file 'include/config-win.h' --- a/include/config-win.h 2009-03-12 22:27:35 +0000 +++ b/include/config-win.h 2009-03-20 11:41:30 +0000 @@ -94,6 +94,17 @@ functions */ #define S_IROTH S_IREAD /* for my_lib */ +/* for MY_S_ISFIFO() macro from my_lib */ +#if defined (_S_IFIFO) && !defined (S_IFIFO) +#define S_IFIFO _S_IFIFO +#endif + +/* Winsock2 constant (Vista SDK and later)*/ +#define IPPROTO_IPV6 41 +#ifndef IPV6_V6ONLY +#define IPV6_V6ONLY 27 +#endif + /* Constants used by chmod. Note, that group/others is ignored - because unsupported by Windows due to different access control model. @@ -160,10 +171,21 @@ typedef __int64 os_off_t; #ifdef _WIN64 typedef UINT_PTR rf_SetTimer; #else +typedef uint rf_SetTimer; +#endif + #ifndef HAVE_SIZE_T -typedef unsigned int size_t; +#ifndef _SIZE_T_DEFINED +typedef SIZE_T size_t; +#define _SIZE_T_DEFINED +#endif +#endif + +#ifndef HAVE_SSIZE_T +#ifndef _SSIZE_T_DEFINED +typedef SSIZE_T ssize_t; +#define _SSIZE_T_DEFINED #endif -typedef uint rf_SetTimer; #endif #define Socket_defined @@ -195,7 +217,7 @@ typedef uint rf_SetTimer; #define SIZEOF_CHARP 4 #endif #define HAVE_BROKEN_NETINET_INCLUDES -#ifdef __NT__ +#ifdef _WIN32 #define HAVE_NAMED_PIPE /* We can only create pipes on NT */ #endif @@ -316,7 +338,7 @@ inline ulonglong double2ulonglong(double #define strcasecmp stricmp #define strncasecmp strnicmp -#ifndef __NT__ +#ifndef _WIN32 #undef FILE_SHARE_DELETE #define FILE_SHARE_DELETE 0 /* Not implemented on Win 98/ME */ #endif @@ -368,7 +390,7 @@ inline ulonglong double2ulonglong(double #define thread_safe_increment(V,L) InterlockedIncrement((long*) &(V)) #define thread_safe_decrement(V,L) InterlockedDecrement((long*) &(V)) /* The following is only used for statistics, so it should be good enough */ -#ifdef __NT__ /* This should also work on Win98 but .. */ +#ifdef _WIN32 /* This should also work on Win98 but .. */ #define thread_safe_add(V,C,L) InterlockedExchangeAdd((long*) &(V),(C)) #define thread_safe_sub(V,C,L) InterlockedExchangeAdd((long*) &(V),-(long) (C)) #endif === modified file 'unittest/mysys/waiting_threads-t.c' --- a/unittest/mysys/waiting_threads-t.c 2009-02-19 09:01:25 +0000 +++ b/unittest/mysys/waiting_threads-t.c 2009-03-20 11:41:30 +0000 @@ -254,6 +254,7 @@ void do_tests() diag("timeout_long=%lu us, deadlock_search_depth_long=%lu", wt_timeout_long, wt_deadlock_search_depth_long); +#ifndef _WIN32 #define test_kill_strategy(X) \ diag("kill strategy: " #X); \ DBUG_EXECUTE("reset_file", \ @@ -261,6 +262,12 @@ void do_tests() DBUG_PRINT("info", ("kill strategy: " #X)); \ kill_strategy=X; \ do_one_test(); +#else + diag("kill strategy: " #X); \ + DBUG_PRINT("info", ("kill strategy: " #X)); \ + kill_strategy=X; \ + do_one_test(); +#endif test_kill_strategy(LATEST); test_kill_strategy(RANDOM);

1 0

[Maria-developers] Launchpad repoistories upgraded, bzr version 1.9 now required
by Kristian Nielsen 20 Mar '09

20 Mar '09

This morning I finished upgrading our three maria repositories at launchpad to repository format 1.9: lp:maria lp:maria/5.2 lp:maria/6.0 There should be no need to re-branch from launchpad or make any other changes to local trees or branches. However, I believe it will now be necessary to use bzr of at least version 1.9 to access the repositories on launchpad. It is recommended, but not required, to upgrade local shared repositores and branches with bzr upgrade --1.9 to get the following benefits 1. Stacked branches, which allows pushing private clones to lp:~USER/maria/xxx with only having to upload changes to main trees, for me pushing such a private branch now takes only ~30 seconds. 2. General faster operation of bzr in my experience (eg bzr log 10x faster). - Kristian.

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2686)
by knielsen＠knielsen-hq.org 19 Mar '09

19 Mar '09

#At lp:maria 2686 knielsen(a)knielsen-hq.org 2009-03-19 [merge] test merge modified: dbug/dbug.c mysql-test/lib/My/Test.pm mysql-test/mysql-test-run.pl mysql-test/t/multi_update2.test mysql-test/valgrind.supp sql/mysqld.cc sql/scheduler.cc sql/sql_select.cc === modified file 'dbug/dbug.c' --- a/dbug/dbug.c 2009-03-12 22:27:35 +0000 +++ b/dbug/dbug.c 2009-03-18 14:08:05 +0000 @@ -506,6 +506,7 @@ int DbugParse(CODE_STATE *cs, const char rel= control[0] == '+' || control[0] == '-'; if ((!rel || (!stack->out_file && !stack->next))) { + FreeState(cs, stack, 0); stack->flags= 0; stack->delay= 0; stack->maxdepth= 0; @@ -1648,10 +1649,12 @@ static void FreeState(CODE_STATE *cs, st FreeList(state->processes); if (!is_shared(state, p_functions)) FreeList(state->p_functions); - if (!is_shared(state, out_file)) + if (!is_shared(state, out_file) && + state->out_file != stderr && state->out_file != stdout) DBUGCloseFile(cs, state->out_file); (void) fflush(cs->stack->out_file); - if (state->prof_file) + if (state->prof_file && + state->out_file != stderr && state->out_file != stdout) DBUGCloseFile(cs, state->prof_file); if (free_state) free((void*) state); === modified file 'mysql-test/lib/My/Test.pm' --- a/mysql-test/lib/My/Test.pm 2008-11-14 10:49:12 +0000 +++ b/mysql-test/lib/My/Test.pm 2009-03-17 08:23:05 +0000 @@ -9,6 +9,7 @@ package My::Test; use strict; use warnings; use Carp; +use Storable(); sub new { @@ -30,18 +31,6 @@ sub key { } -sub _encode { - my ($value)= @_; - $value =~ s/([|\\\x{0a}\x{0d}])/sprintf('\%02X', ord($1))/eg; - return $value; -} - -sub _decode { - my ($value)= @_; - $value =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/ge; - return $value; -} - sub is_failed { my ($self)= @_; my $result= $self->{result}; @@ -58,66 +47,22 @@ sub write_test { # Give the test a unique key before serializing it $test->{key}= "$test" unless defined $test->{key}; - print $sock $header, "\n"; - while ((my ($key, $value)) = each(%$test)) { - print $sock $key, "= "; - if (ref $value eq "ARRAY") { - print $sock "[", _encode(join(", ", @$value)), "]"; - } else { - print $sock _encode($value); - } - print $sock "\n"; - } - print $sock "\n"; + my $serialized= Storable::freeze($test); + $serialized =~ s/([\x0d\x0a\\])/sprintf("\\%02x", ord($1))/eg; + print $sock $header, "\n", $serialized, "\n"; } sub read_test { my ($sock)= @_; - my $test= My::Test->new(); - # Read the : separated key value pairs until a - # single newline on it's own line - my $line; - while (defined($line= <$sock>)) { - # List is terminated by newline on it's own - if ($line eq "\n") { - # Correctly terminated reply - # print "Got newline\n"; - last; - } - chomp($line); - - # Split key/value on the first "=" - my ($key, $value)= split("= ", $line, 2); - - if ($value =~ /^\[(.*)\]/){ - my @values= split(", ", _decode($1)); - push(@{$test->{$key}}, @values); - } - else - { - $test->{$key}= _decode($value); - } - } + my $serialized= <$sock>; + chomp($serialized); + $serialized =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/eg; + my $test= Storable::thaw($serialized); + die "wrong class (hack attempt?)" + unless ref($test) eq 'My::Test'; return $test; } -sub print_test { - my ($self)= @_; - - print "[", $self->{name}, "]", "\n"; - while ((my ($key, $value)) = each(%$self)) { - print " ", $key, "= "; - if (ref $value eq "ARRAY") { - print "[", join(", ", @$value), "]"; - } else { - print $value; - } - print "\n"; - } - print "\n"; -} - - 1; === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2009-03-12 22:27:35 +0000 +++ b/mysql-test/mysql-test-run.pl 2009-03-17 11:58:05 +0000 @@ -357,7 +357,7 @@ sub main { mtr_print_thick_line(); mtr_print_header(); - my $completed= run_test_server($server, $tests, $opt_parallel); + my ($completed, $fail)= run_test_server($server, $tests, $opt_parallel); # Send Ctrl-C to any children still running kill("INT", keys(%children)); @@ -393,6 +393,10 @@ sub main { mtr_error("Not all tests completed"); } + if ($fail) { + mtr_error("Test suite failure."); + } + mtr_print_line(); if ( $opt_gcov ) { @@ -412,6 +416,7 @@ sub run_test_server ($$$) { my $num_saved_cores= 0; # Number of core files saved in vardir/log/ so far. my $num_saved_datadir= 0; # Number of datadirs saved in vardir/log/ so far. my $num_failed_test= 0; # Number of tests failed so far + my $test_failure= 0; # Scheduler variables my $max_ndb= $childs / 2; @@ -445,7 +450,7 @@ sub run_test_server ($$$) { $s->remove($sock); if (--$childs == 0){ $suite_timeout_proc->kill(); - return $completed; + return ($completed, $test_failure); } next; } @@ -509,18 +514,19 @@ sub run_test_server ($$$) { } $num_saved_datadir++; + $test_failure= 1; if ( !$opt_force ) { # Test has failed, force is off $suite_timeout_proc->kill(); push(@$completed, $result); - return $completed; + return ($completed, 1); } elsif ($opt_max_test_fail > 0 and $num_failed_test >= $opt_max_test_fail) { $suite_timeout_proc->kill(); mtr_report("Too many tests($num_failed_test) failed!", "Terminating..."); - return undef; + return (undef, 1); } $num_failed_test++; } @@ -571,7 +577,18 @@ sub run_test_server ($$$) { elsif ($line eq 'START'){ ; # Send first test } - else { + elsif ($line eq 'WARNINGS'){ + my $fake_test= My::Test::read_test($sock); + my $test_list= join (" ", @{$fake_test->{testnames}}); + mtr_report("***Warnings generated in error logs during shutdown ". + "after running tests: $test_list"); + $test_failure= 1; + if ( !$opt_force ) { + # Test failure due to warnings, force is off + $suite_timeout_proc->kill(); + return ($completed, 1); + } + } else { mtr_error("Unknown response: '$line' from client"); } @@ -649,7 +666,7 @@ sub run_test_server ($$$) { if ( ! $suite_timeout_proc->wait_one(0) ) { mtr_report("Test suite timeout! Terminating..."); - return undef; + return (undef, 1); } } } @@ -717,7 +734,7 @@ sub run_worker ($) { delete($test->{'comment'}); delete($test->{'logfile'}); - run_testcase($test); + run_testcase($test, $server); #$test->{result}= 'MTR_RES_PASSED'; # Send it back, now with results set #$test->print_test(); @@ -725,6 +742,15 @@ sub run_worker ($) { } elsif ($line eq 'BYE'){ mtr_report("Server said BYE"); + # We need to gracefully shut down the servers to see any + # Valgrind memory leak errors etc. since last server restart. + if ($opt_warnings) { + stop_servers(all_servers()); + if(check_warnings_post_shutdown($server)) { + # Warnings appeared in log file(s) during final server shutdown. + exit(1); + } + } exit(0); } else { @@ -732,8 +758,8 @@ sub run_worker ($) { } } + # NOTREACHED stop_all_servers(); - exit(1); } @@ -3109,6 +3135,7 @@ sub run_on_all($$) sub mark_log { my ($log, $tinfo)= @_; my $log_msg= "CURRENT_TEST: $tinfo->{name}\n"; + pre_write_errorlog($log, $tinfo->{name}); mtr_tofile($log, $log_msg); } @@ -3181,8 +3208,8 @@ my %old_env; # > 0 failure # -sub run_testcase ($) { - my $tinfo= shift; +sub run_testcase ($$) { + my ($tinfo, $server_socket)= @_; mtr_verbose("Running test:", $tinfo->{name}); @@ -3197,7 +3224,12 @@ sub run_testcase ($) { { my @restart= servers_need_restart($tinfo); if ( @restart != 0) { - stop_servers($tinfo, @restart ); + # Remember that we restarted for this test case (count restarts) + $tinfo->{'restarted'}= 1; + stop_servers(@restart ); + if ($opt_warnings) { + check_warnings_post_shutdown($server_socket); + } } if ( started(all_servers()) == 0 ) @@ -3336,7 +3368,18 @@ sub run_testcase ($) { { if ($check_res == 1) { # Test case had sideeffects, not fatal error, just continue - stop_all_servers(); + if ($opt_warnings) { + # Checking error logs for warnings, so need to stop server + # gracefully so that memory leaks etc. can be properly detected. + stop_servers(all_servers()); + check_warnings_post_shutdown($server_socket); + # Even if we got warnings here, we should not fail this + # particular test, as the warnings may be caused by an earlier + # test. + } else { + # Not checking warnings, so can do a hard shutdown. + stop_all_servers(); + } mtr_report("Resuming tests...\n"); } else { @@ -3468,6 +3511,64 @@ sub run_testcase ($) { } +# We want to preserve the error log between server restarts, as it may contain +# valuable debugging information even if there is no test failure recorded. +sub _preserve_error_log_names { + my ($mysqld)= @_; + my $error_log_file= $mysqld->value('log-error'); + my $error_log_dir= dirname($error_log_file); + my $save_name= $error_log_dir ."/../". $mysqld->name() .".error.log"; + return ($error_log_file, $save_name); +} + +sub preserve_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($error_log_file, $save_name); + # Ignore any errors, as it's just a best-effort to keep the log if possible. +} + +sub restore_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($save_name, $error_log_file); +} + +# Keep track of last position in mysqld error log where we scanned for +# warnings, so we can attribute any warnings found to the correct test +# suite or server restart. +my $last_warning_position= { }; + +# Called just before a mysqld server is started or a testcase is run, +# to keep track of which tests have been run since last restart, and +# of when the error log is reset. +# +# Second argument $test_name is test name, or undef for server restart. +sub pre_write_errorlog { + my ($error_log, $test_name)= @_; + + if (! -e $error_log) { + # If the error log is moved away, reset the warning parse position. + delete $last_warning_position->{$error_log}; + } + + if (defined($test_name)) { + $last_warning_position->{$error_log}{test_names}= [] + unless exists($last_warning_position->{$error_log}{test_names}); + push @{$last_warning_position->{$error_log}{test_names}}, $test_name; + } else { + # Server restart, so clear the list of tests run since last restart. + # (except the last one (if any), which is the test about to be run). + if (defined($last_warning_position->{$error_log}{test_names}) && + @{$last_warning_position->{$error_log}{test_names}}) { + $last_warning_position->{$error_log}{test_names}= + [$last_warning_position->{$error_log}{test_names}[-1]]; + } else { + $last_warning_position->{$error_log}{test_names}= []; + } + } +} + # # Perform a rough examination of the servers # error log and write all lines that look @@ -3479,18 +3580,12 @@ sub extract_warning_lines ($) { # Open the servers .err log file and read all lines # belonging to current tets into @lines my $Ferr = IO::File->new($error_log) - or mtr_error("Could not open file '$error_log' for reading: $!"); + or return []; + my $last_pos= $last_warning_position->{$error_log}{seek_pos}; + $Ferr->seek($last_pos, 0) if defined($last_pos); - my @lines; - while ( my $line = <$Ferr> ) - { - if ( $line =~ /^CURRENT_TEST:/ ) - { - # Throw away lines from previous tests - @lines = (); - } - push(@lines, $line); - } + my @lines= <$Ferr>; + $last_warning_position->{$error_log}{seek_pos}= $Ferr->tell(); $Ferr = undef; # Close error log file # mysql_client_test.test sends a COM_DEBUG packet to the server @@ -3537,20 +3632,47 @@ sub extract_warning_lines ($) { qr/Attempting backtrace/, qr/Assertion .* failed/, ); + # These are taken from the include/mtr_warnings.sql global suppression + # list. They occur delayed, so they can be parsed during shutdown rather + # than during the per-test check. + # + # ToDo: having the warning suppressions inside the mysqld we are trying to + # check is in any case horrible. We should change it to all be done here + # within the Perl code, which is both simpler, easier, faster, and more + # robust. We could still have individual test cases put in suppressions by + # parsing statically or by writing dynamically to a CSV table read by the + # Perl code. + my @antipatterns = + ( + qr/InnoDB: Error: in ALTER TABLE `test`.`t[12]`/, + qr/InnoDB: Error: table `test`.`t[12]` does not exist in the InnoDB internal/, + ); - foreach my $line ( @lines ) + my $match_count= 0; + LINE: foreach my $line ( @lines ) { - foreach my $pat ( @patterns ) + PAT: foreach my $pat ( @patterns ) { if ( $line =~ /$pat/ ) { + foreach my $apat (@antipatterns) + { + next LINE if $line =~ $apat; + } print $Fwarn $line; - last; + ++$match_count; + last PAT; } } } $Fwarn = undef; # Close file + if ($match_count > 0 && + defined($last_warning_position->{$error_log}{test_names})) { + return $last_warning_position->{$error_log}{test_names}; + } else { + return []; + } } @@ -3717,6 +3839,22 @@ sub check_warnings ($) { mtr_error("INTERNAL_ERROR: check_warnings"); } +# Check for warnings generated during shutdown of a mysqld server. +# If any, report them to master server, and return true; else just return false. +sub check_warnings_post_shutdown { + my ($server_socket)= @_; + my $testname_hash= { }; + foreach my $mysqld ( mysqlds()) + { + my $testlist= extract_warning_lines($mysqld->value('log-error')); + $testname_hash->{$_}= 1 for @$testlist; + } + my @warning_tests= keys(%$testname_hash); + if (@warning_tests) { + my $fake_test= My::Test->new(testnames => \@warning_tests); + $fake_test->write_test($server_socket, 'WARNINGS'); + } +} # # Loop through our list of processes and look for and entry @@ -3814,6 +3952,7 @@ sub clean_datadir { foreach my $mysqld ( mysqlds() ) { my $mysqld_dir= dirname($mysqld->value('datadir')); + preserve_error_log($mysqld); if (-d $mysqld_dir ) { mtr_verbose(" - removing '$mysqld_dir'"); rmtree($mysqld_dir); @@ -4142,6 +4281,7 @@ sub mysqld_start ($$) { if ( defined $exe ) { + pre_write_errorlog($output); $mysqld->{'proc'}= My::SafeProcess->new ( name => $mysqld->name(), @@ -4371,10 +4511,7 @@ sub get_extra_opts { sub stop_servers($$) { - my ($tinfo, @servers)= @_; - - # Remember if we restarted for this test case (count restarts) - $tinfo->{'restarted'}= 1; + my (@servers)= @_; if ( join('|', @servers) eq join('|', all_servers()) ) { @@ -4466,6 +4603,7 @@ sub start_servers($) { } if (-d $datadir ) { + preserve_error_log($mysqld); mtr_verbose(" - removing '$datadir'"); rmtree($datadir); } @@ -4491,6 +4629,7 @@ sub start_servers($) { unless -d $datadir; } + restore_error_log($mysqld); # Create the servers tmpdir my $tmpdir= $mysqld->value('tmpdir'); === modified file 'mysql-test/t/multi_update2.test' --- a/mysql-test/t/multi_update2.test 2008-11-19 18:17:26 +0000 +++ b/mysql-test/t/multi_update2.test 2009-03-16 09:25:20 +0000 @@ -2,6 +2,9 @@ # Test of update statement that uses many tables. # +# This is a big test. +--source include/big_test.inc + --disable_warnings DROP TABLE IF EXISTS t1,t2; --enable_warnings === modified file 'mysql-test/valgrind.supp' --- a/mysql-test/valgrind.supp 2008-11-21 14:21:50 +0000 +++ b/mysql-test/valgrind.supp 2009-03-18 14:08:05 +0000 @@ -229,6 +229,44 @@ } { + libz deflate_slow 1 + Memcheck:Cond + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azflush +} + +{ + libz deflate_slow 2 + Memcheck:Value8 + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azflush +} + +{ + libz deflate_slow 3 + Memcheck:Cond + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azclose +} + +{ + libz _tr_flush_block + Memcheck:Value8 + fun:compress_block + fun:_tr_flush_block + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azflush +} + +{ libz deflate Memcheck:Cond obj:*/libz.so.* @@ -256,6 +294,14 @@ fun:do_flush } +{ + libz deflate4 + Memcheck:Cond + fun:deflate + fun:do_flush + fun:azclose +} + # # Warning from my_thread_init becasue mysqld dies before kill thread exists # @@ -379,7 +425,7 @@ } { - dlclose memory loss from plugin + dlclose memory loss from plugin variant 1 Memcheck:Leak fun:calloc fun:_dlerror_run @@ -388,6 +434,19 @@ } { + dlclose memory loss from plugin variant 2 + Memcheck:Leak + fun:malloc + fun:_dl_close_worker + fun:_dl_close + fun:_dl_catch_error + fun:_dlerror_run + fun:dlclose + fun:_Z15free_plugin_memP12st_plugin_dl + fun:_Z13plugin_dl_delPK19st_mysql_lex_string +} + +{ dlopen / ptread_cancel_init memory loss on Suse Linux 10.3 32/64 bit Memcheck:Leak fun:*alloc @@ -588,3 +647,19 @@ fun:dlopen* } +# +# In glibc (checked version 2.7), inet_ntoa allocates an 18-byte +# per-thread static buffer for the return value. That memory is freed +# at thread exit, however if called from the main thread, Valgrind +# does not see the free (test main.no-threads). +# +# Since inet_ntoa() does not allocate memory dynamically per-call, this +# suppression is safe. +# + +{ + inet_ntoa thread local storage + Memcheck:Leak + fun:malloc + fun:inet_ntoa +} === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2009-03-18 02:14:22 +0000 +++ b/sql/mysqld.cc 2009-03-19 14:19:04 +0000 @@ -4815,10 +4815,10 @@ static bool read_init_file(char *file_na DBUG_ENTER("read_init_file"); DBUG_PRINT("enter",("name: %s",file_name)); if (!(file=my_fopen(file_name,O_RDONLY,MYF(MY_WME)))) - return(1); + DBUG_RETURN(1); bootstrap(file); (void) my_fclose(file,MYF(MY_WME)); - return 0; + DBUG_RETURN(0); } @@ -4839,6 +4839,7 @@ void handle_connection_in_main_thread(TH safe_mutex_assert_owner(&LOCK_thread_count); thread_cache_size=0; // Safety threads.append(thd); + thd->connect_utime= thd->start_utime= my_micro_time(); (void) pthread_mutex_unlock(&LOCK_thread_count); handle_one_connection((void*) thd); } === modified file 'sql/scheduler.cc' --- a/sql/scheduler.cc 2009-03-12 22:27:35 +0000 +++ b/sql/scheduler.cc 2009-03-18 14:08:05 +0000 @@ -470,6 +470,7 @@ static void libevent_add_connection(THD DBUG_VOID_RETURN; } threads.append(thd); + thd->connect_utime= thd->start_utime= my_micro_time(); libevent_thd_add(thd); pthread_mutex_unlock(&LOCK_thread_count); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2009-02-19 09:01:25 +0000 +++ b/sql/sql_select.cc 2009-03-18 14:08:05 +0000 @@ -1994,8 +1994,17 @@ JOIN::exec() tmp_fields_list2, tmp_all_fields2, fields_list.elements, tmp_all_fields1)) DBUG_VOID_RETURN; - curr_join->tmp_fields_list2= tmp_fields_list2; - curr_join->tmp_all_fields2= tmp_all_fields2; +#ifdef HAVE_purify + /* + Some GCCs use memcpy() for struct assignment, even for x=x. + GCC bug 19410: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19410 + */ + if (curr_join != this) +#endif + { + curr_join->tmp_fields_list2= tmp_fields_list2; + curr_join->tmp_all_fields2= tmp_all_fields2; + } } curr_fields_list= &curr_join->tmp_fields_list2; curr_all_fields= &curr_join->tmp_all_fields2;

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2685)
by knielsen＠knielsen-hq.org 18 Mar '09

18 Mar '09

#At lp:maria 2685 knielsen(a)knielsen-hq.org 2009-03-17 When we remove the server data dir before server restart, preserve the error log (don't delete it between restarts), as it may contain valuable information even for test cases which don't show direct failures. modified: mysql-test/mysql-test-run.pl === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2009-03-17 08:23:05 +0000 +++ b/mysql-test/mysql-test-run.pl 2009-03-17 11:58:05 +0000 @@ -3511,6 +3511,29 @@ sub run_testcase ($$) { } +# We want to preserve the error log between server restarts, as it may contain +# valuable debugging information even if there is no test failure recorded. +sub _preserve_error_log_names { + my ($mysqld)= @_; + my $error_log_file= $mysqld->value('log-error'); + my $error_log_dir= dirname($error_log_file); + my $save_name= $error_log_dir ."/../". $mysqld->name() .".error.log"; + return ($error_log_file, $save_name); +} + +sub preserve_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($error_log_file, $save_name); + # Ignore any errors, as it's just a best-effort to keep the log if possible. +} + +sub restore_error_log { + my ($mysqld)= @_; + my ($error_log_file, $save_name)= _preserve_error_log_names($mysqld); + my $res= rename($save_name, $error_log_file); +} + # Keep track of last position in mysqld error log where we scanned for # warnings, so we can attribute any warnings found to the correct test # suite or server restart. @@ -3929,6 +3952,7 @@ sub clean_datadir { foreach my $mysqld ( mysqlds() ) { my $mysqld_dir= dirname($mysqld->value('datadir')); + preserve_error_log($mysqld); if (-d $mysqld_dir ) { mtr_verbose(" - removing '$mysqld_dir'"); rmtree($mysqld_dir); @@ -4579,6 +4603,7 @@ sub start_servers($) { } if (-d $datadir ) { + preserve_error_log($mysqld); mtr_verbose(" - removing '$datadir'"); rmtree($datadir); } @@ -4604,6 +4629,7 @@ sub start_servers($) { unless -d $datadir; } + restore_error_log($mysqld); # Create the servers tmpdir my $tmpdir= $mysqld->value('tmpdir');

2 1

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2686)
by knielsen＠knielsen-hq.org 18 Mar '09

18 Mar '09

#At lp:maria 2686 knielsen(a)knielsen-hq.org 2009-03-18 Fix some Valgrind warnings. modified: dbug/dbug.c mysql-test/valgrind.supp sql/mysqld.cc sql/scheduler.cc sql/sql_select.cc === modified file 'dbug/dbug.c' --- a/dbug/dbug.c 2009-03-12 22:27:35 +0000 +++ b/dbug/dbug.c 2009-03-18 14:08:05 +0000 @@ -506,6 +506,7 @@ int DbugParse(CODE_STATE *cs, const char rel= control[0] == '+' || control[0] == '-'; if ((!rel || (!stack->out_file && !stack->next))) { + FreeState(cs, stack, 0); stack->flags= 0; stack->delay= 0; stack->maxdepth= 0; @@ -1648,10 +1649,12 @@ static void FreeState(CODE_STATE *cs, st FreeList(state->processes); if (!is_shared(state, p_functions)) FreeList(state->p_functions); - if (!is_shared(state, out_file)) + if (!is_shared(state, out_file) && + state->out_file != stderr && state->out_file != stdout) DBUGCloseFile(cs, state->out_file); (void) fflush(cs->stack->out_file); - if (state->prof_file) + if (state->prof_file && + state->out_file != stderr && state->out_file != stdout) DBUGCloseFile(cs, state->prof_file); if (free_state) free((void*) state); === modified file 'mysql-test/valgrind.supp' --- a/mysql-test/valgrind.supp 2008-11-21 14:21:50 +0000 +++ b/mysql-test/valgrind.supp 2009-03-18 14:08:05 +0000 @@ -229,6 +229,44 @@ } { + libz deflate_slow 1 + Memcheck:Cond + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azflush +} + +{ + libz deflate_slow 2 + Memcheck:Value8 + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azflush +} + +{ + libz deflate_slow 3 + Memcheck:Cond + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azclose +} + +{ + libz _tr_flush_block + Memcheck:Value8 + fun:compress_block + fun:_tr_flush_block + fun:deflate_slow + fun:deflate + fun:do_flush + fun:azflush +} + +{ libz deflate Memcheck:Cond obj:*/libz.so.* @@ -256,6 +294,14 @@ fun:do_flush } +{ + libz deflate4 + Memcheck:Cond + fun:deflate + fun:do_flush + fun:azclose +} + # # Warning from my_thread_init becasue mysqld dies before kill thread exists # @@ -379,7 +425,7 @@ } { - dlclose memory loss from plugin + dlclose memory loss from plugin variant 1 Memcheck:Leak fun:calloc fun:_dlerror_run @@ -388,6 +434,19 @@ } { + dlclose memory loss from plugin variant 2 + Memcheck:Leak + fun:malloc + fun:_dl_close_worker + fun:_dl_close + fun:_dl_catch_error + fun:_dlerror_run + fun:dlclose + fun:_Z15free_plugin_memP12st_plugin_dl + fun:_Z13plugin_dl_delPK19st_mysql_lex_string +} + +{ dlopen / ptread_cancel_init memory loss on Suse Linux 10.3 32/64 bit Memcheck:Leak fun:*alloc @@ -588,3 +647,19 @@ fun:dlopen* } +# +# In glibc (checked version 2.7), inet_ntoa allocates an 18-byte +# per-thread static buffer for the return value. That memory is freed +# at thread exit, however if called from the main thread, Valgrind +# does not see the free (test main.no-threads). +# +# Since inet_ntoa() does not allocate memory dynamically per-call, this +# suppression is safe. +# + +{ + inet_ntoa thread local storage + Memcheck:Leak + fun:malloc + fun:inet_ntoa +} === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2009-03-13 13:31:54 +0000 +++ b/sql/mysqld.cc 2009-03-18 14:08:05 +0000 @@ -4813,10 +4813,10 @@ static bool read_init_file(char *file_na DBUG_ENTER("read_init_file"); DBUG_PRINT("enter",("name: %s",file_name)); if (!(file=my_fopen(file_name,O_RDONLY,MYF(MY_WME)))) - return(1); + DBUG_RETURN(1); bootstrap(file); (void) my_fclose(file,MYF(MY_WME)); - return 0; + DBUG_RETURN(0); } @@ -4837,6 +4837,7 @@ void handle_connection_in_main_thread(TH safe_mutex_assert_owner(&LOCK_thread_count); thread_cache_size=0; // Safety threads.append(thd); + thd->connect_utime= thd->start_utime= my_micro_time(); (void) pthread_mutex_unlock(&LOCK_thread_count); handle_one_connection((void*) thd); } === modified file 'sql/scheduler.cc' --- a/sql/scheduler.cc 2009-03-12 22:27:35 +0000 +++ b/sql/scheduler.cc 2009-03-18 14:08:05 +0000 @@ -470,6 +470,7 @@ static void libevent_add_connection(THD DBUG_VOID_RETURN; } threads.append(thd); + thd->connect_utime= thd->start_utime= my_micro_time(); libevent_thd_add(thd); pthread_mutex_unlock(&LOCK_thread_count); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2009-02-19 09:01:25 +0000 +++ b/sql/sql_select.cc 2009-03-18 14:08:05 +0000 @@ -1994,8 +1994,17 @@ JOIN::exec() tmp_fields_list2, tmp_all_fields2, fields_list.elements, tmp_all_fields1)) DBUG_VOID_RETURN; - curr_join->tmp_fields_list2= tmp_fields_list2; - curr_join->tmp_all_fields2= tmp_all_fields2; +#ifdef HAVE_purify + /* + Some GCCs use memcpy() for struct assignment, even for x=x. + GCC bug 19410: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19410 + */ + if (curr_join != this) +#endif + { + curr_join->tmp_fields_list2= tmp_fields_list2; + curr_join->tmp_all_fields2= tmp_all_fields2; + } } curr_fields_list= &curr_join->tmp_fields_list2; curr_all_fields= &curr_join->tmp_all_fields2;

3 2

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2684) Bug#43418
by knielsen＠knielsen-hq.org 18 Mar '09

18 Mar '09

#At lp:maria 2684 knielsen(a)knielsen-hq.org 2009-03-17 BUG#43418: MTR2: does not notice a memory leak occuring at shutdown of mysqld w/ --valgrind Problem was that mysql-test-run.pl sometimes skipped parsing part of the error log, and sometimes shutdown mysqld with kill -9, preventing tools like Valgrind and safemalloc from outputting memory leak check in the error logs. Fixed by - Stopping mysqld servers gracefully rather than kill -9 when warnings are being checked. - After stopping mysqld servers, do an additional parse of the error log to check for any warnings generated during shutdown. - Fix error log parsing to be careful not to skip parsing part of the file, by keeping track of previous file position rather than relying on mark_log markers. - Workers report warnings during shutdown to the master process with a new packet 'WARNINGS' which includes a list of names of test that might have caused the problem (could be any test run since last server start). - Fail entire test suite if warnings are found. Also fixed home-brewed and broken serialization in My::Test to use the standard Storable serializer. modified: mysql-test/lib/My/Test.pm mysql-test/mysql-test-run.pl === modified file 'mysql-test/lib/My/Test.pm' --- a/mysql-test/lib/My/Test.pm 2008-11-14 10:49:12 +0000 +++ b/mysql-test/lib/My/Test.pm 2009-03-17 08:23:05 +0000 @@ -9,6 +9,7 @@ package My::Test; use strict; use warnings; use Carp; +use Storable(); sub new { @@ -30,18 +31,6 @@ sub key { } -sub _encode { - my ($value)= @_; - $value =~ s/([|\\\x{0a}\x{0d}])/sprintf('\%02X', ord($1))/eg; - return $value; -} - -sub _decode { - my ($value)= @_; - $value =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/ge; - return $value; -} - sub is_failed { my ($self)= @_; my $result= $self->{result}; @@ -58,66 +47,22 @@ sub write_test { # Give the test a unique key before serializing it $test->{key}= "$test" unless defined $test->{key}; - print $sock $header, "\n"; - while ((my ($key, $value)) = each(%$test)) { - print $sock $key, "= "; - if (ref $value eq "ARRAY") { - print $sock "[", _encode(join(", ", @$value)), "]"; - } else { - print $sock _encode($value); - } - print $sock "\n"; - } - print $sock "\n"; + my $serialized= Storable::freeze($test); + $serialized =~ s/([\x0d\x0a\\])/sprintf("\\%02x", ord($1))/eg; + print $sock $header, "\n", $serialized, "\n"; } sub read_test { my ($sock)= @_; - my $test= My::Test->new(); - # Read the : separated key value pairs until a - # single newline on it's own line - my $line; - while (defined($line= <$sock>)) { - # List is terminated by newline on it's own - if ($line eq "\n") { - # Correctly terminated reply - # print "Got newline\n"; - last; - } - chomp($line); - - # Split key/value on the first "=" - my ($key, $value)= split("= ", $line, 2); - - if ($value =~ /^\[(.*)\]/){ - my @values= split(", ", _decode($1)); - push(@{$test->{$key}}, @values); - } - else - { - $test->{$key}= _decode($value); - } - } + my $serialized= <$sock>; + chomp($serialized); + $serialized =~ s/\\([0-9a-fA-F]{2})/chr(hex($1))/eg; + my $test= Storable::thaw($serialized); + die "wrong class (hack attempt?)" + unless ref($test) eq 'My::Test'; return $test; } -sub print_test { - my ($self)= @_; - - print "[", $self->{name}, "]", "\n"; - while ((my ($key, $value)) = each(%$self)) { - print " ", $key, "= "; - if (ref $value eq "ARRAY") { - print "[", join(", ", @$value), "]"; - } else { - print $value; - } - print "\n"; - } - print "\n"; -} - - 1; === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2009-03-12 22:27:35 +0000 +++ b/mysql-test/mysql-test-run.pl 2009-03-17 08:23:05 +0000 @@ -357,7 +357,7 @@ sub main { mtr_print_thick_line(); mtr_print_header(); - my $completed= run_test_server($server, $tests, $opt_parallel); + my ($completed, $fail)= run_test_server($server, $tests, $opt_parallel); # Send Ctrl-C to any children still running kill("INT", keys(%children)); @@ -393,6 +393,10 @@ sub main { mtr_error("Not all tests completed"); } + if ($fail) { + mtr_error("Test suite failure."); + } + mtr_print_line(); if ( $opt_gcov ) { @@ -412,6 +416,7 @@ sub run_test_server ($$$) { my $num_saved_cores= 0; # Number of core files saved in vardir/log/ so far. my $num_saved_datadir= 0; # Number of datadirs saved in vardir/log/ so far. my $num_failed_test= 0; # Number of tests failed so far + my $test_failure= 0; # Scheduler variables my $max_ndb= $childs / 2; @@ -445,7 +450,7 @@ sub run_test_server ($$$) { $s->remove($sock); if (--$childs == 0){ $suite_timeout_proc->kill(); - return $completed; + return ($completed, $test_failure); } next; } @@ -509,18 +514,19 @@ sub run_test_server ($$$) { } $num_saved_datadir++; + $test_failure= 1; if ( !$opt_force ) { # Test has failed, force is off $suite_timeout_proc->kill(); push(@$completed, $result); - return $completed; + return ($completed, 1); } elsif ($opt_max_test_fail > 0 and $num_failed_test >= $opt_max_test_fail) { $suite_timeout_proc->kill(); mtr_report("Too many tests($num_failed_test) failed!", "Terminating..."); - return undef; + return (undef, 1); } $num_failed_test++; } @@ -571,7 +577,18 @@ sub run_test_server ($$$) { elsif ($line eq 'START'){ ; # Send first test } - else { + elsif ($line eq 'WARNINGS'){ + my $fake_test= My::Test::read_test($sock); + my $test_list= join (" ", @{$fake_test->{testnames}}); + mtr_report("***Warnings generated in error logs during shutdown ". + "after running tests: $test_list"); + $test_failure= 1; + if ( !$opt_force ) { + # Test failure due to warnings, force is off + $suite_timeout_proc->kill(); + return ($completed, 1); + } + } else { mtr_error("Unknown response: '$line' from client"); } @@ -649,7 +666,7 @@ sub run_test_server ($$$) { if ( ! $suite_timeout_proc->wait_one(0) ) { mtr_report("Test suite timeout! Terminating..."); - return undef; + return (undef, 1); } } } @@ -717,7 +734,7 @@ sub run_worker ($) { delete($test->{'comment'}); delete($test->{'logfile'}); - run_testcase($test); + run_testcase($test, $server); #$test->{result}= 'MTR_RES_PASSED'; # Send it back, now with results set #$test->print_test(); @@ -725,6 +742,15 @@ sub run_worker ($) { } elsif ($line eq 'BYE'){ mtr_report("Server said BYE"); + # We need to gracefully shut down the servers to see any + # Valgrind memory leak errors etc. since last server restart. + if ($opt_warnings) { + stop_servers(all_servers()); + if(check_warnings_post_shutdown($server)) { + # Warnings appeared in log file(s) during final server shutdown. + exit(1); + } + } exit(0); } else { @@ -732,8 +758,8 @@ sub run_worker ($) { } } + # NOTREACHED stop_all_servers(); - exit(1); } @@ -3109,6 +3135,7 @@ sub run_on_all($$) sub mark_log { my ($log, $tinfo)= @_; my $log_msg= "CURRENT_TEST: $tinfo->{name}\n"; + pre_write_errorlog($log, $tinfo->{name}); mtr_tofile($log, $log_msg); } @@ -3181,8 +3208,8 @@ my %old_env; # > 0 failure # -sub run_testcase ($) { - my $tinfo= shift; +sub run_testcase ($$) { + my ($tinfo, $server_socket)= @_; mtr_verbose("Running test:", $tinfo->{name}); @@ -3197,7 +3224,12 @@ sub run_testcase ($) { { my @restart= servers_need_restart($tinfo); if ( @restart != 0) { - stop_servers($tinfo, @restart ); + # Remember that we restarted for this test case (count restarts) + $tinfo->{'restarted'}= 1; + stop_servers(@restart ); + if ($opt_warnings) { + check_warnings_post_shutdown($server_socket); + } } if ( started(all_servers()) == 0 ) @@ -3336,7 +3368,18 @@ sub run_testcase ($) { { if ($check_res == 1) { # Test case had sideeffects, not fatal error, just continue - stop_all_servers(); + if ($opt_warnings) { + # Checking error logs for warnings, so need to stop server + # gracefully so that memory leaks etc. can be properly detected. + stop_servers(all_servers()); + check_warnings_post_shutdown($server_socket); + # Even if we got warnings here, we should not fail this + # particular test, as the warnings may be caused by an earlier + # test. + } else { + # Not checking warnings, so can do a hard shutdown. + stop_all_servers(); + } mtr_report("Resuming tests...\n"); } else { @@ -3468,6 +3511,41 @@ sub run_testcase ($) { } +# Keep track of last position in mysqld error log where we scanned for +# warnings, so we can attribute any warnings found to the correct test +# suite or server restart. +my $last_warning_position= { }; + +# Called just before a mysqld server is started or a testcase is run, +# to keep track of which tests have been run since last restart, and +# of when the error log is reset. +# +# Second argument $test_name is test name, or undef for server restart. +sub pre_write_errorlog { + my ($error_log, $test_name)= @_; + + if (! -e $error_log) { + # If the error log is moved away, reset the warning parse position. + delete $last_warning_position->{$error_log}; + } + + if (defined($test_name)) { + $last_warning_position->{$error_log}{test_names}= [] + unless exists($last_warning_position->{$error_log}{test_names}); + push @{$last_warning_position->{$error_log}{test_names}}, $test_name; + } else { + # Server restart, so clear the list of tests run since last restart. + # (except the last one (if any), which is the test about to be run). + if (defined($last_warning_position->{$error_log}{test_names}) && + @{$last_warning_position->{$error_log}{test_names}}) { + $last_warning_position->{$error_log}{test_names}= + [$last_warning_position->{$error_log}{test_names}[-1]]; + } else { + $last_warning_position->{$error_log}{test_names}= []; + } + } +} + # # Perform a rough examination of the servers # error log and write all lines that look @@ -3479,18 +3557,12 @@ sub extract_warning_lines ($) { # Open the servers .err log file and read all lines # belonging to current tets into @lines my $Ferr = IO::File->new($error_log) - or mtr_error("Could not open file '$error_log' for reading: $!"); + or return []; + my $last_pos= $last_warning_position->{$error_log}{seek_pos}; + $Ferr->seek($last_pos, 0) if defined($last_pos); - my @lines; - while ( my $line = <$Ferr> ) - { - if ( $line =~ /^CURRENT_TEST:/ ) - { - # Throw away lines from previous tests - @lines = (); - } - push(@lines, $line); - } + my @lines= <$Ferr>; + $last_warning_position->{$error_log}{seek_pos}= $Ferr->tell(); $Ferr = undef; # Close error log file # mysql_client_test.test sends a COM_DEBUG packet to the server @@ -3537,20 +3609,47 @@ sub extract_warning_lines ($) { qr/Attempting backtrace/, qr/Assertion .* failed/, ); + # These are taken from the include/mtr_warnings.sql global suppression + # list. They occur delayed, so they can be parsed during shutdown rather + # than during the per-test check. + # + # ToDo: having the warning suppressions inside the mysqld we are trying to + # check is in any case horrible. We should change it to all be done here + # within the Perl code, which is both simpler, easier, faster, and more + # robust. We could still have individual test cases put in suppressions by + # parsing statically or by writing dynamically to a CSV table read by the + # Perl code. + my @antipatterns = + ( + qr/InnoDB: Error: in ALTER TABLE `test`.`t[12]`/, + qr/InnoDB: Error: table `test`.`t[12]` does not exist in the InnoDB internal/, + ); - foreach my $line ( @lines ) + my $match_count= 0; + LINE: foreach my $line ( @lines ) { - foreach my $pat ( @patterns ) + PAT: foreach my $pat ( @patterns ) { if ( $line =~ /$pat/ ) { + foreach my $apat (@antipatterns) + { + next LINE if $line =~ $apat; + } print $Fwarn $line; - last; + ++$match_count; + last PAT; } } } $Fwarn = undef; # Close file + if ($match_count > 0 && + defined($last_warning_position->{$error_log}{test_names})) { + return $last_warning_position->{$error_log}{test_names}; + } else { + return []; + } } @@ -3717,6 +3816,22 @@ sub check_warnings ($) { mtr_error("INTERNAL_ERROR: check_warnings"); } +# Check for warnings generated during shutdown of a mysqld server. +# If any, report them to master server, and return true; else just return false. +sub check_warnings_post_shutdown { + my ($server_socket)= @_; + my $testname_hash= { }; + foreach my $mysqld ( mysqlds()) + { + my $testlist= extract_warning_lines($mysqld->value('log-error')); + $testname_hash->{$_}= 1 for @$testlist; + } + my @warning_tests= keys(%$testname_hash); + if (@warning_tests) { + my $fake_test= My::Test->new(testnames => \@warning_tests); + $fake_test->write_test($server_socket, 'WARNINGS'); + } +} # # Loop through our list of processes and look for and entry @@ -4142,6 +4257,7 @@ sub mysqld_start ($$) { if ( defined $exe ) { + pre_write_errorlog($output); $mysqld->{'proc'}= My::SafeProcess->new ( name => $mysqld->name(), @@ -4371,10 +4487,7 @@ sub get_extra_opts { sub stop_servers($$) { - my ($tinfo, @servers)= @_; - - # Remember if we restarted for this test case (count restarts) - $tinfo->{'restarted'}= 1; + my (@servers)= @_; if ( join('|', @servers) eq join('|', all_servers()) ) {

2 1

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2685)
by knielsen＠knielsen-hq.org 18 Mar '09

18 Mar '09

#At lp:maria 2685 knielsen(a)knielsen-hq.org 2009-03-18 Add testing of extra port for pool-of-threads. The additional test uses up all threads in the pool with SELECT SLEEP(), and tests that this makes normal connections block, but connections on the extra port still work. Also test connection limit on extra port with and without pool-of-threads enabled. Add --connect-timeout option to mysqltest program. Add facility for --extra-port option to ConfigFactory. Fix regexp typo in ConfigFactory.pm removed: mysql-test/t/pool_of_threads-master.opt added: mysql-test/t/connect.cnf mysql-test/t/pool_of_threads.cnf modified: client/mysqltest.cc mysql-test/lib/My/ConfigFactory.pm mysql-test/r/connect.result mysql-test/r/pool_of_threads.result mysql-test/t/connect.test mysql-test/t/pool_of_threads.test === modified file 'client/mysqltest.cc' --- a/client/mysqltest.cc 2009-02-19 09:01:25 +0000 +++ b/client/mysqltest.cc 2009-03-18 15:46:32 +0000 @@ -76,7 +76,7 @@ enum { OPT_SKIP_SAFEMALLOC=OPT_MAX_CLIENT_OPTION, OPT_PS_PROTOCOL, OPT_SP_PROTOCOL, OPT_CURSOR_PROTOCOL, OPT_VIEW_PROTOCOL, OPT_MAX_CONNECT_RETRIES, OPT_MARK_PROGRESS, OPT_LOG_DIR, OPT_TAIL_LINES, - OPT_GLOBAL_SUBST + OPT_GLOBAL_SUBST, OPT_MY_CONNECT_TIMEOUT }; static int record= 0, opt_sleep= -1; @@ -87,6 +87,7 @@ const char *opt_include= 0, *opt_charset static int opt_port= 0; static int opt_max_connect_retries; static my_bool opt_compress= 0, silent= 0, verbose= 0; +static int opt_connect_timeout= -1; static my_bool debug_info_flag= 0, debug_check_flag= 0; static my_bool tty_password= 0; static my_bool opt_mark_progress= 0; @@ -4952,6 +4953,9 @@ void do_connect(struct st_command *comma if (opt_charsets_dir) mysql_options(&con_slot->mysql, MYSQL_SET_CHARSET_DIR, opt_charsets_dir); + if (opt_connect_timeout >= 0) + mysql_options(&con_slot->mysql, MYSQL_OPT_CONNECT_TIMEOUT, + &opt_connect_timeout); #ifdef HAVE_OPENSSL if (opt_use_ssl || con_ssl) @@ -5692,6 +5696,9 @@ static struct my_option my_long_options[ #include "sslopt-longopts.h" {"test-file", 'x', "Read test from/in this file (default stdin).", 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, + {"connect-timeout", OPT_MY_CONNECT_TIMEOUT, "Client connection timeout", + (uchar**) &opt_connect_timeout, (uchar**) &opt_connect_timeout, 0, + GET_INT, REQUIRED_ARG, -1, -1, 0, 0, 0, 0}, {"timer-file", 'm', "File where the timing in micro seconds is stored.", 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, {"tmpdir", 't', "Temporary directory where sockets are put.", === modified file 'mysql-test/lib/My/ConfigFactory.pm' --- a/mysql-test/lib/My/ConfigFactory.pm 2009-02-15 10:58:34 +0000 +++ b/mysql-test/lib/My/ConfigFactory.pm 2009-03-18 15:46:32 +0000 @@ -202,6 +202,7 @@ my @mysqld_rules= { 'pid-file' => \&fix_pidfile }, { '#host' => \&fix_host }, { 'port' => \&fix_port }, + { '#extra-port' => \&fix_port }, { 'socket' => \&fix_socket }, { 'log-error' => \&fix_log_error }, { 'general-log' => sub { return 1; } }, @@ -353,7 +354,7 @@ sub post_check_client_group { sub post_check_client_groups { my ($self, $config)= @_; - my $first_mysqld= $config->first_like('mysqld.'); + my $first_mysqld= $config->first_like('mysqld\.'); return unless $first_mysqld; === modified file 'mysql-test/r/connect.result' --- a/mysql-test/r/connect.result 2008-03-17 11:26:00 +0000 +++ b/mysql-test/r/connect.result 2009-03-18 15:46:32 +0000 @@ -215,6 +215,13 @@ SET GLOBAL event_scheduler = OFF; # -- End of Bug#35074. +SELECT 'Connection on extra port ok'; +Connection on extra port ok +Connection on extra port ok +SELECT 'Connection on extra port 2 ok'; +Connection on extra port 2 ok +Connection on extra port 2 ok +# -- Success: more than --extra-max-connections + 1 normal connections not possible # ------------------------------------------------------------------ # -- End of 5.1 tests # ------------------------------------------------------------------ === modified file 'mysql-test/r/pool_of_threads.result' --- a/mysql-test/r/pool_of_threads.result 2009-03-12 22:27:35 +0000 +++ b/mysql-test/r/pool_of_threads.result 2009-03-18 15:46:32 +0000 @@ -2151,3 +2151,23 @@ Privat (Private Nutzung) Mobilfunk Warnings: Warning 1052 Column 'kundentyp' in group statement is ambiguous drop table t1; +SELECT sleep(5); +SELECT sleep(5); +# -- Success: more than --thread-pool-size normal connections not possible +sleep(5) +0 +sleep(5) +0 +SELECT sleep(5); +SELECT sleep(5); +SELECT 'Connection on extra port ok'; +Connection on extra port ok +Connection on extra port ok +SELECT 'Connection on extra port 2 ok'; +Connection on extra port 2 ok +Connection on extra port 2 ok +# -- Success: more than --extra-max-connections + 1 normal connections not possible +sleep(5) +0 +sleep(5) +0 === added file 'mysql-test/t/connect.cnf' --- a/mysql-test/t/connect.cnf 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/connect.cnf 2009-03-18 15:46:32 +0000 @@ -0,0 +1,8 @@ +!include include/default_my.cnf + +[mysqld.1] +extra-port= @mysqld.1.#extra-port +extra-max-connections=1 + +[ENV] +MASTER_EXTRA_PORT= @mysqld.1.extra-port === modified file 'mysql-test/t/connect.test' --- a/mysql-test/t/connect.test 2008-03-17 11:26:00 +0000 +++ b/mysql-test/t/connect.test 2009-03-18 15:46:32 +0000 @@ -288,6 +288,33 @@ let $wait_condition = --echo # -- End of Bug#35074. --echo +# Test connections to the extra port. + +connect(extracon,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon; +SELECT 'Connection on extra port ok'; + +connect(extracon2,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon2; +SELECT 'Connection on extra port 2 ok'; + +--disable_abort_on_error +--disable_result_log +--disable_query_log +connect(extracon3,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +--enable_query_log +--enable_result_log +--enable_abort_on_error +let $error = $mysql_errno; +if (!$error) +{ + --echo # -- Error: managed to establish more than --extra-max-connections + 1 connections +} +if ($error) +{ + --echo # -- Success: more than --extra-max-connections + 1 normal connections not possible +} + --echo # ------------------------------------------------------------------ --echo # -- End of 5.1 tests --echo # ------------------------------------------------------------------ === removed file 'mysql-test/t/pool_of_threads-master.opt' --- a/mysql-test/t/pool_of_threads-master.opt 2009-03-12 22:27:35 +0000 +++ b/mysql-test/t/pool_of_threads-master.opt 1970-01-01 00:00:00 +0000 @@ -1 +0,0 @@ ---test-ignore-wrong-options --thread-handling=pool-of-threads === added file 'mysql-test/t/pool_of_threads.cnf' --- a/mysql-test/t/pool_of_threads.cnf 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/pool_of_threads.cnf 2009-03-18 15:46:32 +0000 @@ -0,0 +1,14 @@ +!include include/default_my.cnf + +[mysqld.1] +test-ignore-wrong-options +thread-handling= pool-of-threads +thread_pool_size= 2 +extra-port= @mysqld.1.#extra-port +extra-max-connections=1 + +[client] +connect-timeout= 2 + +[ENV] +MASTER_EXTRA_PORT= @mysqld.1.extra-port === modified file 'mysql-test/t/pool_of_threads.test' --- a/mysql-test/t/pool_of_threads.test 2009-03-12 22:27:35 +0000 +++ b/mysql-test/t/pool_of_threads.test 2009-03-18 15:46:32 +0000 @@ -5,3 +5,78 @@ -- source include/common-tests.inc + +# Test that we cannot have more simultaneous connections than +# --thread-pool-size on the standard port, but _can_ have additional +# connections on the extra port. + +# First set two connections running, and check that extra connection +# on normal port fails due to--thread-pool-size=2 +connection default; +send SELECT sleep(5); + +connect(con2,localhost,root,,); +connection con2; +send SELECT sleep(5); +--sleep 1 + +--disable_abort_on_error +--disable_result_log +--disable_query_log +connect(con3,localhost,root,,); +--enable_query_log +--enable_result_log +--enable_abort_on_error +let $error = $mysql_errno; +if (!$error) +{ + --echo # -- Error: managed to establish more than --thread-pool-size connections +} +if ($error) +{ + --echo # -- Success: more than --thread-pool-size normal connections not possible +} + +connection default; +--reap +connection con2; +--reap + +# Now try again, but this time use the extra port to successfully connect. + +connection default; +send SELECT sleep(5); + +connection con2; +send SELECT sleep(5); +--sleep 1 + +connect(extracon,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon; +SELECT 'Connection on extra port ok'; + +connect(extracon2,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon2; +SELECT 'Connection on extra port 2 ok'; + +--disable_abort_on_error +--disable_result_log +--disable_query_log +connect(extracon3,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +--enable_query_log +--enable_result_log +--enable_abort_on_error +let $error = $mysql_errno; +if (!$error) +{ + --echo # -- Error: managed to establish more than --extra-max-connections + 1 connections +} +if ($error) +{ + --echo # -- Success: more than --extra-max-connections + 1 normal connections not possible +} + +connection default; +--reap +connection con2; +--reap

1 0

Re: [Maria-developers] PBXT and MariaDB
by Paul McCullagh 18 Mar '09

18 Mar '09

OK, thanks. Will do. On Mar 18, 2009, at 1:05 AM, Michael Widenius wrote: > > Hi! > >>>>>> "Paul" == Paul McCullagh <paul.mccullagh(a)primebase.org> writes: > >>> From: Paul McCullagh <paul.mccullagh(a)primebase.org> >>> Date: March 9, 2009 9:48:44 AM GMT+01:00 >>> To: maria-developers(a)lists.launchpad.net >>> Subject: [Maria-developers] PBXT and MariaDB >>> >>> Hi Monty, >>> >>> In our call last week you suggested that I push PBXT into the >>> MariaDB tree. >>> >>> Which branch should I push to? > > lp:maria > > In other words, the 'trunk' series at: > > https://launchpad.net/maria/trunk > > We are now looking at setting up a Buildbot network to start doing > 'builds-after-each-push' and monthly binaries. > > Regards, > Monty -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2684)
by Michael Widenius 18 Mar '09

18 Mar '09

#At lp:maria based on revid:monty@askmonty.org-20090318021422-jmb1ap5kougrzgzz 2684 Michael Widenius 2009-03-18 Ignore generated file event-config.h modified: .bzrignore === modified file '.bzrignore' --- a/.bzrignore 2009-02-19 09:01:25 +0000 +++ b/.bzrignore 2009-03-18 02:15:20 +0000 @@ -1902,3 +1902,4 @@ sql/share/spanish sql/share/swedish sql/share/ukrainian libmysqld/examples/mysqltest.cc +extra/libevent/event-config.h

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2683)
by Michael Widenius 18 Mar '09

18 Mar '09

#At lp:maria based on revid:knielsen@knielsen-hq.org-20090313133154-0qas21in74uxdtk8 2683 Michael Widenius 2009-03-18 Added mariadb and mariadb-version as my.conf option tags Fixed compiler error when configuring without --lib-event modified: include/mysql_version.h.in sql/mysqld.cc sql/scheduler.h per-file messages: include/mysql_version.h.in Added mariadb-'version' as a my.cnf option tag for mysqld sql/mysqld.cc Added mariadb and mariadb-version as my.conf option tags === modified file 'include/mysql_version.h.in' --- a/include/mysql_version.h.in 2007-09-13 14:19:46 +0000 +++ b/include/mysql_version.h.in 2009-03-18 02:14:22 +0000 @@ -11,6 +11,7 @@ #define PROTOCOL_VERSION @PROTOCOL_VERSION@ #define MYSQL_SERVER_VERSION "@VERSION@" #define MYSQL_BASE_VERSION "mysqld-@MYSQL_BASE_VERSION@" +#define MARIADB_BASE_VERSION "mariadb-@MYSQL_BASE_VERSION@" #define MYSQL_SERVER_SUFFIX_DEF "@MYSQL_SERVER_SUFFIX@" #define FRM_VER @DOT_FRM_VERSION@ #define MYSQL_VERSION_ID @MYSQL_VERSION_ID@ === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2009-03-13 13:31:54 +0000 +++ b/sql/mysqld.cc 2009-03-18 02:14:22 +0000 @@ -3014,7 +3014,9 @@ static const char *load_default_groups[] #ifdef WITH_NDBCLUSTER_STORAGE_ENGINE "mysql_cluster", #endif -"mysqld","server", MYSQL_BASE_VERSION, 0, 0}; +"mysqld", "server", MYSQL_BASE_VERSION, +"mariadb", MARIADB_BASE_VERSION, +0, 0}; #if defined(__WIN__) static const int load_default_groups_sz= === modified file 'sql/scheduler.h' --- a/sql/scheduler.h 2009-03-12 22:27:35 +0000 +++ b/sql/scheduler.h 2009-03-18 02:14:22 +0000 @@ -82,7 +82,9 @@ void pool_of_threads_scheduler(scheduler #else #define HAVE_POOL_OF_THREADS 0 /* For easyer tests */ -#define pool_of_threads_scheduler(A) one_thread_per_connection_scheduler(A) +#define pool_of_threads_scheduler(A) \ + one_thread_per_connection_scheduler(A, &max_connections, \ + &connection_count) class thd_scheduler {};

1 0

[Maria-developers] bzr commit into mariadb 5.1, with Maria 2.0:maria/2.0 branch (sanja:2722)
by sanja＠askmonty.org 17 Mar '09

17 Mar '09

#At lp:maria/2.0 2722 sanja(a)askmonty.org 2009-03-17 Group commit patch. Transaction log syncs statistic. Buffer finish optimisation in case of absence of CRC page protection or sector protetction. Fixed bug of waiting for flush pass end. (put together only Monty review, will be pushed separately) modified: mysql-test/suite/maria/r/maria3.result mysys/my_init.c storage/maria/ha_maria.cc storage/maria/ma_init.c storage/maria/ma_loghandler.c storage/maria/ma_loghandler.h tests/fork_big2.pl per-file messages: mysql-test/suite/maria/r/maria3.result Transaction log syncs statistic. mysys/my_init.c The initialization should be here but it do niot want to link (works without it because initialization do not do something except checks). storage/maria/ha_maria.cc Transaction log syncs statistic. Group commit control variables and routines. storage/maria/ma_init.c Finishing of soft group commit thread. storage/maria/ma_loghandler.c Group commit (soft and hard) added. Transaction log syncs statistic. Buffer finish optimisation in case of absence of CRC page protection or sector protetction. Fixed bug of waiting for flush pass end. storage/maria/ma_loghandler.h Group commit control variables and routines. Transaction log syncs statistic. tests/fork_big2.pl Fixed test written by windows to get it working under linux (line endings) === modified file 'mysql-test/suite/maria/r/maria3.result' --- a/mysql-test/suite/maria/r/maria3.result 2008-10-09 20:03:54 +0000 +++ b/mysql-test/suite/maria/r/maria3.result 2009-03-17 20:32:17 +0000 @@ -264,6 +264,8 @@ Variable_name Value maria_block_size 8192 maria_checkpoint_interval 30 maria_force_start_after_recovery_failures 0 +maria_group_commit none +maria_group_commit_rate 800 maria_log_file_size 4294959104 maria_log_purge_type immediate maria_max_sort_file_size 9223372036854775807 === modified file 'mysys/my_init.c' --- a/mysys/my_init.c 2008-12-10 09:02:25 +0000 +++ b/mysys/my_init.c 2009-03-17 20:32:17 +0000 @@ -40,6 +40,7 @@ static void netware_init(); #else #define netware_init() #endif +#include <my_atomic.h> my_bool my_init_done= 0; uint mysys_usage_id= 0; /* Incremented for each my_init() */ @@ -82,6 +83,10 @@ my_bool my_init(void) if (my_progname) my_progname_short= my_progname + dirname_length(my_progname); +/* + if (my_atomic_initialize()) + return 1; +*/ #if defined(THREAD) && defined(SAFE_MUTEX) safe_mutex_global_init(); /* Must be called early */ #endif === modified file 'storage/maria/ha_maria.cc' --- a/storage/maria/ha_maria.cc 2009-01-16 16:18:17 +0000 +++ b/storage/maria/ha_maria.cc 2009-03-17 20:32:17 +0000 @@ -101,22 +101,40 @@ TYPELIB maria_translog_purge_type_typeli array_elements(maria_translog_purge_type_names) - 1, "", maria_translog_purge_type_names, NULL }; + +/* transactional log directory sync */ const char *maria_sync_log_dir_names[]= { "NEVER", "NEWFILE", "ALWAYS", NullS }; - TYPELIB maria_sync_log_dir_typelib= { array_elements(maria_sync_log_dir_names) - 1, "", maria_sync_log_dir_names, NULL }; +/* transactional log group commit */ +const char *maria_group_commit_names[]= +{ + "none", "hard", "soft", NullS +}; +TYPELIB maria_group_commit_typelib= +{ + array_elements(maria_group_commit_names) - 1, "", + maria_group_commit_names, NULL +}; + /** Interval between background checkpoints in seconds */ static ulong checkpoint_interval; static void update_checkpoint_interval(MYSQL_THD thd, struct st_mysql_sys_var *var, void *var_ptr, const void *save); +static void update_maria_group_commit(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save); +static void update_maria_group_commit_rate(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save); /** After that many consecutive recovery failures, remove logs */ static ulong force_start_after_recovery_failures; static void update_log_file_size(MYSQL_THD thd, @@ -163,6 +181,22 @@ static MYSQL_SYSVAR_ULONG(log_file_size, NULL, update_log_file_size, TRANSLOG_FILE_SIZE, TRANSLOG_MIN_FILE_SIZE, 0xffffffffL, TRANSLOG_PAGE_SIZE); +static MYSQL_SYSVAR_ENUM(group_commit, group_commit, + PLUGIN_VAR_RQCMDARG, + "Specifies maria group commit mode. " + "Possible values are \"none\" (no group commit), " + "\"hard\" (with waiting to actual commit), " + "\"soft\" (no wait for commit (DANGEROUS!!!))", + NULL, update_maria_group_commit, + TRANSLOG_GCOMMIT_NONE, &maria_group_commit_typelib); + +static MYSQL_SYSVAR_ULONG(group_commit_rate, group_commit_rate, + PLUGIN_VAR_RQCMDARG, + "If group commits switched on commit will happens with about every " + "100/maria_group_commit_rate second. 0 is special value which switch " + "rate off", + NULL, update_maria_group_commit_rate, 800, 0, UINT_MAX, 1); + static MYSQL_SYSVAR_ENUM(log_purge_type, log_purge_type, PLUGIN_VAR_RQCMDARG, "Specifies how maria transactional log will be purged. " @@ -3247,6 +3281,8 @@ static struct st_mysql_sys_var* system_v MYSQL_SYSVAR(block_size), MYSQL_SYSVAR(checkpoint_interval), MYSQL_SYSVAR(force_start_after_recovery_failures), + MYSQL_SYSVAR(group_commit), + MYSQL_SYSVAR(group_commit_rate), MYSQL_SYSVAR(page_checksum), MYSQL_SYSVAR(log_dir_path), MYSQL_SYSVAR(log_file_size), @@ -3277,6 +3313,97 @@ static void update_checkpoint_interval(M } /** + @brief Updates group commit mode +*/ + +static void update_maria_group_commit(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save) +{ + ulong value= (ulong)*((long *)var_ptr); + DBUG_ENTER("update_maria_group_commit"); + DBUG_PRINT("enter", ("old value: %lu new value %lu rate %lu", + value, (ulong)(*(long *)save), group_commit_rate)); + /* old value */ + switch (value) + { + case TRANSLOG_GCOMMIT_NONE: + break; + case TRANSLOG_GCOMMIT_HARD: + translog_hard_group_commit(FALSE); + break; + case TRANSLOG_GCOMMIT_SOFT: + translog_soft_sync(FALSE); + if (group_commit_rate) + translog_soft_sync_end(); + break; + default: + DBUG_ASSERT(0); /* impossible */ + } + value= *(ulong *)var_ptr= (ulong)(*(long *)save); + translog_sync(); + /* new value */ + switch (value) + { + case TRANSLOG_GCOMMIT_NONE: + break; + case TRANSLOG_GCOMMIT_HARD: + translog_hard_group_commit(TRUE); + break; + case TRANSLOG_GCOMMIT_SOFT: + translog_soft_sync(TRUE); + /* variable change made under global lock so we can just read it */ + if (group_commit_rate) + translog_soft_sync_start(); + break; + default: + DBUG_ASSERT(0); /* impossible */ + } + DBUG_VOID_RETURN; +} + +/** + @brief Updates group commit rate +*/ + +static void update_maria_group_commit_rate(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save) +{ + ulong new_value= (ulong)*((long *)save); + DBUG_ENTER("update_maria_group_commit_rate"); + DBUG_PRINT("enter", ("old value: %lu new value %lu group commit %lu", + *(ulong *)var_ptr, new_value, group_commit)); + if (new_value && + ((TRANSLOG_RATE_BASE * 1000000000ULL / new_value + + TRANSLOG_RATE_BASE / 2) / + TRANSLOG_RATE_BASE) == 0) + new_value= 0; /* protection against too small value */ + /* variable change made under global lock so we can just read it */ + switch (group_commit) + { + case TRANSLOG_GCOMMIT_NONE: + *(ulong *)var_ptr= new_value; + translog_set_group_commit_rate(new_value); + break; + case TRANSLOG_GCOMMIT_HARD: + *(ulong *)var_ptr= new_value; + translog_set_group_commit_rate(new_value); + break; + case TRANSLOG_GCOMMIT_SOFT: + if (*(ulong *)var_ptr) + translog_soft_sync_end(); + translog_set_group_commit_rate(new_value); + if ((*(ulong *)var_ptr= new_value)) + translog_soft_sync_start(); + break; + default: + DBUG_ASSERT(0); /* impossible */ + } + DBUG_VOID_RETURN; +} + +/** @brief Updates the transaction log file limit. */ @@ -3298,6 +3425,7 @@ static SHOW_VAR status_variables[]= { {"Maria_pagecache_reads", (char*) &maria_pagecache_var.global_cache_read, SHOW_LONGLONG}, {"Maria_pagecache_write_requests", (char*) &maria_pagecache_var.global_cache_w_requests, SHOW_LONGLONG}, {"Maria_pagecache_writes", (char*) &maria_pagecache_var.global_cache_write, SHOW_LONGLONG}, + {"Maria_transaction_log_syncs", (char*) &translog_syncs, SHOW_LONGLONG}, {NullS, NullS, SHOW_LONG} }; === modified file 'storage/maria/ma_init.c' --- a/storage/maria/ma_init.c 2008-10-09 20:03:54 +0000 +++ b/storage/maria/ma_init.c 2009-03-17 20:32:17 +0000 @@ -82,6 +82,11 @@ void maria_end(void) maria_inited= maria_multi_threaded= FALSE; ft_free_stopwords(); ma_checkpoint_end(); + if (translog_status == TRANSLOG_OK) + { + translog_soft_sync_end(); + translog_sync(); + } if ((trid= trnman_get_max_trid()) > max_trid_in_control_file) { /* === modified file 'storage/maria/ma_loghandler.c' --- a/storage/maria/ma_loghandler.c 2009-01-16 09:38:02 +0000 +++ b/storage/maria/ma_loghandler.c 2009-03-17 20:32:17 +0000 @@ -18,6 +18,7 @@ #include "ma_blockrec.h" /* for some constants and in-write hooks */ #include "ma_key_recover.h" /* For some in-write hooks */ #include "ma_checkpoint.h" +#include "ma_servicethread.h" /* On Windows, neither my_open() nor my_sync() work for directories. @@ -47,6 +48,15 @@ #include <m_ctype.h> #endif +/** @brief protects checkpoint_in_progress */ +static pthread_mutex_t LOCK_soft_sync; +/** @brief for killing the background checkpoint thread */ +static pthread_cond_t COND_soft_sync; +/** @brief control structure for checkpoint background thread */ +static MA_SERVICE_THREAD_CONTROL soft_sync_control= + {THREAD_DEAD, FALSE, &LOCK_soft_sync, &COND_soft_sync}; + + /* transaction log file descriptor */ typedef struct st_translog_file { @@ -124,10 +134,20 @@ struct st_translog_buffer /* Previous buffer offset to detect it flush finish */ TRANSLOG_ADDRESS prev_buffer_offset; /* + If the buffer was forced to close it save value of its horizon + otherwise LSN_IMPOSSIBLE + */ + TRANSLOG_ADDRESS pre_force_close_horizon; + /* How much is written (or will be written when copy_to_buffer_in_progress become 0) to this buffer */ translog_size_t size; + /* + How much data was skipped during moving page from previous buffer + to this one (it is optimisation of forcing buffer to finish + */ + uint skipped_data; /* File handler for this buffer */ TRANSLOG_FILE *file; /* Threads which are waiting for buffer filling/freeing */ @@ -304,6 +324,7 @@ struct st_translog_descriptor */ pthread_mutex_t log_flush_lock; pthread_cond_t log_flush_cond; + pthread_cond_t new_goal_cond; /* Protects changing of headers of finished files (max_lsn) */ pthread_mutex_t file_header_lock; @@ -333,6 +354,8 @@ struct st_translog_descriptor my_bool is_everything_flushed; /* True when flush pass is in progress */ my_bool flush_in_progress; + /* The flush number (used to distinguish two flushes goes one by one) */ + volatile int flush_no; /* Next flush pass variables */ TRANSLOG_ADDRESS next_pass_max_lsn; pthread_t max_lsn_requester; @@ -343,12 +366,40 @@ static struct st_translog_descriptor log ulong log_purge_type= TRANSLOG_PURGE_IMMIDIATE; ulong log_file_size= TRANSLOG_FILE_SIZE; ulong sync_log_dir= TRANSLOG_SYNC_DIR_NEWFILE; +ulong group_commit= TRANSLOG_GCOMMIT_NONE; +ulong group_commit_rate= 0; /* Marker for end of log */ static uchar end_of_log= 0; #define END_OF_LOG &end_of_log +static my_atomic_rwlock_t soft_sync_rwl; +static my_atomic_rwlock_t hgroup_commit_rwl; +/** + Switch for "soft" sync (no real sync() but periodical sync by service + thread) +*/ +static volatile uint32 soft_sync= FALSE; +/** + Switch for "hard" group commit mode +*/ +static uint32 hgroup_commit= FALSE; +/** + File numbers interval which have to be sync() +*/ +static uint32 soft_sync_min= 0; +static uint32 soft_sync_max= 0; +static my_atomic_rwlock_t group_commit_wait_rwl; +/** + stores interval in nanoseconds/TRANSLOG_RATE_BASE (to + fit into uint32) +*/ +static uint32 group_commit_wait= 0; enum enum_translog_status translog_status= TRANSLOG_UNINITED; +ulonglong translog_syncs= 0; /* Number of sync()s */ + +/* time of last flush */ +static ulonglong flush_start= 0; /* chunk types */ #define TRANSLOG_CHUNK_LSN 0x00 /* 0 chunk refer as LSN (head or tail */ @@ -978,12 +1029,17 @@ static TRANSLOG_FILE *get_logfile_by_num static TRANSLOG_FILE *get_current_logfile() { TRANSLOG_FILE *file; + DBUG_ENTER("get_current_logfile"); rw_rdlock(&log_descriptor.open_files_lock); + DBUG_PRINT("info", ("max_file: %lu min_file: %lu open_files: %lu", + (ulong) log_descriptor.max_file, + (ulong) log_descriptor.min_file, + (ulong) log_descriptor.open_files.elements)); DBUG_ASSERT(log_descriptor.max_file - log_descriptor.min_file + 1 == log_descriptor.open_files.elements); file= *dynamic_element(&log_descriptor.open_files, 0, TRANSLOG_FILE **); rw_unlock(&log_descriptor.open_files_lock); - return (file); + DBUG_RETURN(file); } uchar NEAR maria_trans_file_magic[]= @@ -1067,6 +1123,7 @@ static my_bool translog_write_file_heade static my_bool translog_max_lsn_to_header(File file, LSN lsn) { uchar lsn_buff[LSN_STORE_SIZE]; + my_bool rc; DBUG_ENTER("translog_max_lsn_to_header"); DBUG_PRINT("enter", ("File descriptor: %ld " "lsn: (%lu,0x%lx)", @@ -1075,11 +1132,13 @@ static my_bool translog_max_lsn_to_heade lsn_store(lsn_buff, lsn); - DBUG_RETURN(my_pwrite(file, lsn_buff, - LSN_STORE_SIZE, - (LOG_HEADER_DATA_SIZE - LSN_STORE_SIZE), - log_write_flags) != 0 || - my_sync(file, MYF(MY_WME)) != 0); + if (!(rc= (my_pwrite(file, lsn_buff, + LSN_STORE_SIZE, + (LOG_HEADER_DATA_SIZE - LSN_STORE_SIZE), + log_write_flags) != 0 || + my_sync(file, MYF(MY_WME)) != 0))) + translog_syncs++; + DBUG_RETURN(rc); } @@ -1421,7 +1480,9 @@ LSN translog_get_file_max_lsn_stored(uin static my_bool translog_buffer_init(struct st_translog_buffer *buffer, int num) { DBUG_ENTER("translog_buffer_init"); - buffer->prev_last_lsn= buffer->last_lsn= LSN_IMPOSSIBLE; + buffer->pre_force_close_horizon= + buffer->prev_last_lsn= buffer->last_lsn= + LSN_IMPOSSIBLE; DBUG_PRINT("info", ("last_lsn and prev_last_lsn set to 0 buffer: 0x%lx", (ulong) buffer)); @@ -1433,6 +1494,7 @@ static my_bool translog_buffer_init(stru memset(buffer->buffer, TRANSLOG_FILLER, TRANSLOG_WRITE_BUFFER); /* Buffer size */ buffer->size= 0; + buffer->skipped_data= 0; /* cond of thread which is waiting for buffer filling */ if (pthread_cond_init(&buffer->waiting_filling_buffer, 0)) DBUG_RETURN(1); @@ -1487,7 +1549,10 @@ static my_bool translog_close_log_file(T TODO: sync only we have changed the log */ if (!file->is_sync) + { rc= my_sync(file->handler.file, MYF(MY_WME)); + translog_syncs++; + } rc|= my_close(file->handler.file, MYF(MY_WME)); my_free(file, MYF(0)); return test(rc); @@ -2042,7 +2107,8 @@ static void translog_start_buffer(struct (ulong) LSN_OFFSET(log_descriptor.horizon), (ulong) LSN_OFFSET(log_descriptor.horizon))); DBUG_ASSERT(buffer_no == buffer->buffer_no); - buffer->prev_last_lsn= buffer->last_lsn= LSN_IMPOSSIBLE; + buffer->pre_force_close_horizon= + buffer->prev_last_lsn= buffer->last_lsn= LSN_IMPOSSIBLE; DBUG_PRINT("info", ("last_lsn and prev_last_lsn set to 0 buffer: 0x%lx", (ulong) buffer)); buffer->offset= log_descriptor.horizon; @@ -2050,6 +2116,7 @@ static void translog_start_buffer(struct buffer->file= get_current_logfile(); buffer->overlay= 0; buffer->size= 0; + buffer->skipped_data= 0; translog_cursor_init(cursor, buffer, buffer_no); DBUG_PRINT("info", ("file: #%ld (%d) init cursor #%u: 0x%lx " "chaser: %d Size: %lu (%lu)", @@ -2521,6 +2588,7 @@ static my_bool translog_buffer_flush(str TRANSLOG_ADDRESS offset= buffer->offset; TRANSLOG_FILE *file= buffer->file; uint8 ver= buffer->ver; + uint skipped_data; DBUG_ENTER("translog_buffer_flush"); DBUG_PRINT("enter", ("Buffer: #%u 0x%lx file: %d offset: (%lu,0x%lx) size: %lu", @@ -2555,6 +2623,8 @@ static my_bool translog_buffer_flush(str disk */ file= buffer->file; + skipped_data= buffer->skipped_data; + DBUG_ASSERT(skipped_data < TRANSLOG_PAGE_SIZE); for (i= 0, pg= LSN_OFFSET(buffer->offset) / TRANSLOG_PAGE_SIZE; i < buffer->size; i+= TRANSLOG_PAGE_SIZE, pg++) @@ -2571,13 +2641,16 @@ static my_bool translog_buffer_flush(str DBUG_ASSERT(i + TRANSLOG_PAGE_SIZE <= buffer->size); if (translog_status != TRANSLOG_OK && translog_status != TRANSLOG_SHUTDOWN) DBUG_RETURN(1); - if (pagecache_inject(log_descriptor.pagecache, + if (pagecache_write_part(log_descriptor.pagecache, &file->handler, pg, 3, buffer->buffer + i, PAGECACHE_PLAIN_PAGE, PAGECACHE_LOCK_LEFT_UNLOCKED, - PAGECACHE_PIN_LEFT_UNPINNED, 0, - LSN_IMPOSSIBLE)) + PAGECACHE_PIN_LEFT_UNPINNED, + PAGECACHE_WRITE_DONE, 0, + LSN_IMPOSSIBLE, + skipped_data, + TRANSLOG_PAGE_SIZE - skipped_data)) { DBUG_PRINT("error", ("Can't write page (%lu,0x%lx) to pagecache, error: %d", @@ -2587,10 +2660,12 @@ static my_bool translog_buffer_flush(str translog_stop_writing(); DBUG_RETURN(1); } + skipped_data= 0; } file->is_sync= 0; - if (my_pwrite(file->handler.file, buffer->buffer, - buffer->size, LSN_OFFSET(buffer->offset), + if (my_pwrite(file->handler.file, buffer->buffer + buffer->skipped_data, + buffer->size - buffer->skipped_data, + LSN_OFFSET(buffer->offset) + buffer->skipped_data, log_write_flags)) { DBUG_PRINT("error", ("Can't write buffer (%lu,0x%lx) size %lu " @@ -3319,11 +3394,17 @@ static my_bool translog_truncate_log(TRA ((my_chsize(fd, next_page_offset, TRANSLOG_FILLER, MYF(MY_WME)) || (page_rest && my_pwrite(fd, page_buff, page_rest, LSN_OFFSET(addr), log_write_flags)) || - my_sync(fd, MYF(MY_WME))) | + my_sync(fd, MYF(MY_WME))) || my_close(fd, MYF(MY_WME))) || (sync_log_dir >= TRANSLOG_SYNC_DIR_ALWAYS && sync_dir(log_descriptor.directory_fd, MYF(MY_WME | MY_IGNORE_BADFD)))) DBUG_RETURN(1); + /* + if above 'if' failed all translog initialization failed so it does not + matter if sync will be done but translog_syncs will not be increased in + this case + */ + translog_syncs+= 2; /*sunc() log and sync() dir */ /* fix the horizon */ log_descriptor.horizon= addr; @@ -3485,6 +3566,8 @@ my_bool translog_init_with_table(const c id_to_share= NULL; log_descriptor.directory_fd= -1; log_descriptor.is_everything_flushed= 1; + log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no= 0; log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; (*init_table_func)(); @@ -3495,6 +3578,8 @@ my_bool translog_init_with_table(const c log_descriptor.open_flags= O_BINARY | O_RDONLY; else log_descriptor.open_flags= O_BINARY | O_RDWR; + my_atomic_rwlock_init(&soft_sync_rwl); + my_atomic_rwlock_init(&group_commit_wait_rwl); if (pthread_mutex_init(&log_descriptor.sent_to_disk_lock, MY_MUTEX_INIT_FAST) || pthread_mutex_init(&log_descriptor.file_header_lock, @@ -3508,6 +3593,7 @@ my_bool translog_init_with_table(const c pthread_mutex_init(&log_descriptor.dirty_buffer_mask_lock, MY_MUTEX_INIT_FAST) || pthread_cond_init(&log_descriptor.log_flush_cond, 0) || + pthread_cond_init(&log_descriptor.new_goal_cond, 0) || my_rwlock_init(&log_descriptor.open_files_lock, NULL) || my_init_dynamic_array(&log_descriptor.open_files, @@ -3909,7 +3995,6 @@ my_bool translog_init_with_table(const c log_descriptor.flushed= log_descriptor.horizon; log_descriptor.in_buffers_only= log_descriptor.bc.buffer->offset; log_descriptor.max_lsn= LSN_IMPOSSIBLE; /* set to 0 */ - log_descriptor.previous_flush_horizon= log_descriptor.horizon; /* Now 'flushed' is set to 'horizon' value, but 'horizon' is (potentially) address of the next LSN and we want indicate that all LSNs that are @@ -3992,6 +4077,10 @@ my_bool translog_init_with_table(const c It is beginning of the log => there is no LSNs in the log => There is no harm in leaving it "as-is". */ + log_descriptor.previous_flush_horizon= log_descriptor.horizon; + DBUG_PRINT("info", ("previous_flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(log_descriptor. + previous_flush_horizon))); DBUG_RETURN(0); } file_no--; @@ -4067,6 +4156,9 @@ my_bool translog_init_with_table(const c translog_free_record_header(&rec); } } + log_descriptor.previous_flush_horizon= log_descriptor.horizon; + DBUG_PRINT("info", ("previous_flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(log_descriptor.previous_flush_horizon))); DBUG_RETURN(0); err: ma_message_no_user(0, "log initialization failed"); @@ -4154,6 +4246,9 @@ void translog_destroy() pthread_mutex_destroy(&log_descriptor.log_flush_lock); pthread_mutex_destroy(&log_descriptor.dirty_buffer_mask_lock); pthread_cond_destroy(&log_descriptor.log_flush_cond); + pthread_cond_destroy(&log_descriptor.new_goal_cond); + my_atomic_rwlock_destroy(&soft_sync_rwl); + my_atomic_rwlock_destroy(&group_commit_wait_rwl); rwlock_destroy(&log_descriptor.open_files_lock); delete_dynamic(&log_descriptor.open_files); delete_dynamic(&log_descriptor.unfinished_files); @@ -7383,18 +7478,20 @@ static void translog_force_current_buffe */ DBUG_PRINT("info", ("left: %u", (uint) left)); + old_buffer->pre_force_close_horizon= + old_buffer->offset + old_buffer->size; /* decrease offset */ new_buff_beginning-= log_descriptor.bc.current_page_fill; current_page_fill= log_descriptor.bc.current_page_fill; memset(log_descriptor.bc.ptr, TRANSLOG_FILLER, left); - log_descriptor.bc.buffer->size+= left; + old_buffer->size+= left; DBUG_PRINT("info", ("Finish Page buffer #%u: 0x%lx " "Size: %lu", - (uint) log_descriptor.bc.buffer->buffer_no, - (ulong) log_descriptor.bc.buffer, - (ulong) log_descriptor.bc.buffer->size)); - DBUG_ASSERT(log_descriptor.bc.buffer->buffer_no == + (uint) old_buffer->buffer_no, + (ulong) old_buffer, + (ulong) old_buffer->size)); + DBUG_ASSERT(old_buffer->buffer_no == log_descriptor.bc.buffer_no); } else @@ -7425,6 +7522,10 @@ static void translog_force_current_buffe log_descriptor.bc.buffer->offset= new_buff_beginning; log_descriptor.bc.write_counter= write_counter; log_descriptor.bc.previous_offset= previous_offset; + new_buffer->prev_last_lsn= BUFFER_MAX_LSN(old_buffer); + DBUG_PRINT("info", ("prev_last_lsn set to (%lu,0x%lx) buffer: 0x%lx", + LSN_IN_PARTS(new_buffer->prev_last_lsn), + (ulong) new_buffer)); /* Advances this log pointer, increases writers and let other threads to @@ -7501,11 +7602,14 @@ static void translog_force_current_buffe if (left) { - /* - TODO: do not copy beginning of the page if we have no CRC or sector - checks on - */ - memcpy(new_buffer->buffer, data, current_page_fill); + if (log_descriptor.flags & + (TRANSLOG_PAGE_CRC | TRANSLOG_SECTOR_PROTECTION)) + memcpy(new_buffer->buffer, data, current_page_fill); + else + { + new_buffer->skipped_data= current_page_fill; + DBUG_ASSERT(new_buffer->skipped_data < TRANSLOG_PAGE_SIZE); + } } old_buffer->next_buffer_offset= new_buffer->offset; translog_buffer_lock(new_buffer); @@ -7545,6 +7649,7 @@ void translog_flush_wait_for_end(LSN ls void translog_flush_set_new_goal_and_wait(TRANSLOG_ADDRESS lsn) { + int flush_no= log_descriptor.flush_no; DBUG_ENTER("translog_flush_set_new_goal_and_wait"); DBUG_PRINT("enter", ("LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); safe_mutex_assert_owner(&log_descriptor.log_flush_lock); @@ -7552,8 +7657,9 @@ void translog_flush_set_new_goal_and_wai { log_descriptor.next_pass_max_lsn= lsn; log_descriptor.max_lsn_requester= pthread_self(); + pthread_cond_broadcast(&log_descriptor.new_goal_cond); } - while (log_descriptor.flush_in_progress) + while (flush_no == log_descriptor.flush_no) { pthread_cond_wait(&log_descriptor.log_flush_cond, &log_descriptor.log_flush_lock); @@ -7563,67 +7669,71 @@ void translog_flush_set_new_goal_and_wai /** - @brief Flush the log up to given LSN (included) - - @param lsn log record serial number up to which (inclusive) - the log has to be flushed - - @return Operation status - @retval 0 OK - @retval 1 Error + @brief sync() range of files (inclusive) and directory (by request) + @param min from file + @param max to file + @param sync_dir need sync directory */ -my_bool translog_flush(TRANSLOG_ADDRESS lsn) +static my_bool translog_sync_files(uint32 min, uint32 max, + my_bool sync_dir) { - LSN sent_to_disk= LSN_IMPOSSIBLE; - TRANSLOG_ADDRESS flush_horizon; - uint fn, i; - dirty_buffer_mask_t dirty_buffer_mask; - uint8 last_buffer_no, start_buffer_no; + uint fn; my_bool rc= 0; - DBUG_ENTER("translog_flush"); - DBUG_PRINT("enter", ("Flush up to LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); - DBUG_ASSERT(translog_status == TRANSLOG_OK || - translog_status == TRANSLOG_READONLY); - LINT_INIT(sent_to_disk); + DBUG_ENTER("translog_sync_files"); + DBUG_PRINT("info", ("min: %lu max: %lu sync dir: %d", + (ulong) min, (ulong) max, (int) sync_dir)); + DBUG_ASSERT(min <= max); - pthread_mutex_lock(&log_descriptor.log_flush_lock); - DBUG_PRINT("info", ("Everything is flushed up to (%lu,0x%lx)", - LSN_IN_PARTS(log_descriptor.flushed))); - if (cmp_translog_addr(log_descriptor.flushed, lsn) >= 0) - { - pthread_mutex_unlock(&log_descriptor.log_flush_lock); - DBUG_RETURN(0); - } - if (log_descriptor.flush_in_progress) + flush_start= my_getsystime(); + for (fn= min; fn <= max; fn++) { - translog_flush_set_new_goal_and_wait(lsn); - if (!pthread_equal(log_descriptor.max_lsn_requester, pthread_self())) + TRANSLOG_FILE *file= get_logfile_by_number(fn); + DBUG_ASSERT(file != NULL); + if (!file->is_sync) { - /* fix lsn if it was horizon */ - if (cmp_translog_addr(lsn, log_descriptor.bc.buffer->last_lsn) > 0) - lsn= BUFFER_MAX_LSN(log_descriptor.bc.buffer); - translog_flush_wait_for_end(lsn); - pthread_mutex_unlock(&log_descriptor.log_flush_lock); - DBUG_RETURN(0); + if (my_sync(file->handler.file, MYF(MY_WME))) + { + rc= 1; + translog_stop_writing(); + DBUG_RETURN(rc); + } + translog_syncs++; + file->is_sync= 1; } - log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; } - log_descriptor.flush_in_progress= 1; - flush_horizon= log_descriptor.previous_flush_horizon; - DBUG_PRINT("info", ("flush_in_progress is set")); - pthread_mutex_unlock(&log_descriptor.log_flush_lock); - translog_lock(); - if (log_descriptor.is_everything_flushed) + if (sync_dir) { - DBUG_PRINT("info", ("everything is flushed")); - rc= (translog_status == TRANSLOG_READONLY); - translog_unlock(); - goto out; + if (!(rc|= sync_dir(log_descriptor.directory_fd, + MYF(MY_WME | MY_IGNORE_BADFD)))) + translog_syncs++; } + DBUG_RETURN(rc); +} + + +/* + @brief Flushes buffers up to lsn + + @param lsn LSN to which we should flush + @parem sent_to_disk returns 'sent to disk' position + @param flush_horizon returns horizon of the flush + + @note See comment to translog_flush(). +*/ + +void translog_flush_buffers(TRANSLOG_ADDRESS *lsn, + TRANSLOG_ADDRESS *sent_to_disk, + TRANSLOG_ADDRESS *flush_horizon) +{ + dirty_buffer_mask_t dirty_buffer_mask; + uint i; + uint8 last_buffer_no, start_buffer_no; + DBUG_ENTER("translog_flush_buffers"); + /* We will recheck information when will lock buffers one by one so we can use unprotected read here (this is just for @@ -7647,15 +7757,15 @@ my_bool translog_flush(TRANSLOG_ADDRESS /* if LSN up to which we have to flush bigger then maximum LSN of previous buffer and at least one LSN was saved in the current buffer (last_lsn != - LSN_IMPOSSIBLE) then we better finish the current buffer. + LSN_IMPOSSIBLE) then we have to close the current buffer. */ - if (cmp_translog_addr(lsn, log_descriptor.bc.buffer->prev_last_lsn) > 0 && + if (cmp_translog_addr(*lsn, log_descriptor.bc.buffer->prev_last_lsn) > 0 && log_descriptor.bc.buffer->last_lsn != LSN_IMPOSSIBLE) { struct st_translog_buffer *buffer= log_descriptor.bc.buffer; - lsn= log_descriptor.bc.buffer->last_lsn; /* fix lsn if it was horizon */ + *lsn= log_descriptor.bc.buffer->last_lsn; /* fix lsn if it was horizon */ DBUG_PRINT("info", ("LSN to flush fixed to last lsn: (%lu,0x%lx)", - LSN_IN_PARTS(log_descriptor.bc.buffer->last_lsn))); + LSN_IN_PARTS(log_descriptor.bc.buffer->last_lsn))); last_buffer_no= log_descriptor.bc.buffer_no; log_descriptor.is_everything_flushed= 1; translog_force_current_buffer_to_finish(); @@ -7667,8 +7777,10 @@ my_bool translog_flush(TRANSLOG_ADDRESS TRANSLOG_BUFFERS_NO); translog_unlock(); } - sent_to_disk= translog_get_sent_to_disk(); - if (cmp_translog_addr(lsn, sent_to_disk) > 0) + + /* flush buffers */ + *sent_to_disk= translog_get_sent_to_disk(); + if (cmp_translog_addr(*lsn, *sent_to_disk) > 0) { DBUG_PRINT("info", ("Start buffer #: %u last buffer #: %u", @@ -7688,52 +7800,224 @@ my_bool translog_flush(TRANSLOG_ADDRESS LSN_IN_PARTS(buffer->last_lsn), (buffer->file ? "dirty" : "closed"))); - if (buffer->prev_last_lsn <= lsn && + if (buffer->prev_last_lsn <= *lsn && buffer->file != NULL) { - DBUG_ASSERT(flush_horizon <= buffer->offset + buffer->size); - flush_horizon= buffer->offset + buffer->size; + DBUG_ASSERT(*flush_horizon <= buffer->offset + buffer->size); + *flush_horizon= (buffer->pre_force_close_horizon ? + buffer->pre_force_close_horizon : + buffer->offset + buffer->size); + DBUG_PRINT("info", ("flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(*flush_horizon))); + DBUG_ASSERT(*flush_horizon <= log_descriptor.horizon); translog_buffer_flush(buffer); } translog_buffer_unlock(buffer); i= (i + 1) % TRANSLOG_BUFFERS_NO; } while (i != last_buffer_no); - sent_to_disk= translog_get_sent_to_disk(); + *sent_to_disk= translog_get_sent_to_disk(); } - /* sync files from previous flush till current one */ - for (fn= LSN_FILE_NO(log_descriptor.flushed); fn <= LSN_FILE_NO(lsn); fn++) + DBUG_VOID_RETURN; +} + +/** + @brief Flush the log up to given LSN (included) + + @param lsn log record serial number up to which (inclusive) + the log has to be flushed + + @return Operation status + @retval 0 OK + @retval 1 Error + + @note + - Non group commit logic: Commits made in passes. Thread which started + flush first is performing actual flush, other threads sets new goal (LSN) + of the next pass (if it is maximum) and waits for the pass end or just + wait for the pass end. + - If hard group commit switched on and rate set to zero the first thread + repeat pass if finds at the end of pass new goals (just take them as its + new goal). The process can not loop forever because we have limited number + of threads. + - If hard group commit switched ON and rate is not zero then at the end of + the pass first tread waits for new goals but not more then need to have + actual rate close to set value. If time exceed it stop pass and let other + thread start new pass if time is not over take new goal and repeats the + pass. + - If soft group commit switched on everything work as without group commit + but this procedure do not perform real sync(). If rate is not zero real + sync() will be performed by service thread with the rate and if there was + new LSN appeared. + + @note Terminology: + 'sent to disk' means written to disk but not sync()ed, + 'flushed' mean sent to disk and synced(). +*/ + +my_bool translog_flush(TRANSLOG_ADDRESS lsn) +{ + struct timespec abstime; + ulonglong flush_interval; + ulonglong time_spent; + LSN sent_to_disk= LSN_IMPOSSIBLE; + TRANSLOG_ADDRESS flush_horizon; + my_bool rc= 0; + my_bool hgroup_commit_at_start; + + DBUG_ENTER("translog_flush"); + DBUG_PRINT("enter", ("Flush up to LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); + DBUG_ASSERT(translog_status == TRANSLOG_OK || + translog_status == TRANSLOG_READONLY); + LINT_INIT(sent_to_disk); + + pthread_mutex_lock(&log_descriptor.log_flush_lock); + DBUG_PRINT("info", ("Everything is flushed up to (%lu,0x%lx)", + LSN_IN_PARTS(log_descriptor.flushed))); + if (cmp_translog_addr(log_descriptor.flushed, lsn) >= 0) { - TRANSLOG_FILE *file= get_logfile_by_number(fn); - DBUG_ASSERT(file != NULL); - if (!file->is_sync) + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + DBUG_RETURN(0); + } + if (log_descriptor.flush_in_progress) + { + translog_lock(); + /* fix lsn if it was horizon */ + if (cmp_translog_addr(lsn, log_descriptor.bc.buffer->last_lsn) > 0) + lsn= BUFFER_MAX_LSN(log_descriptor.bc.buffer); + translog_unlock(); + translog_flush_set_new_goal_and_wait(lsn); + if (!pthread_equal(log_descriptor.max_lsn_requester, pthread_self())) { - if (my_sync(file->handler.file, MYF(MY_WME))) + translog_flush_wait_for_end(lsn); + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + DBUG_RETURN(0); + } + log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; + } + log_descriptor.flush_in_progress= 1; + flush_horizon= log_descriptor.previous_flush_horizon; + DBUG_PRINT("info", ("flush_in_progress is set, flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(flush_horizon))); + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + + my_atomic_rwlock_wrlock(&hgroup_commit_rwl); + hgroup_commit_at_start= my_atomic_load32(&hgroup_commit); + my_atomic_rwlock_wrunlock(&hgroup_commit_rwl); + if (hgroup_commit_at_start) + { + my_atomic_rwlock_rdlock(&group_commit_wait_rwl); + flush_interval= my_atomic_load32(&group_commit_wait) * TRANSLOG_RATE_BASE; + my_atomic_rwlock_rdunlock(&group_commit_wait_rwl); + } + + translog_lock(); + if (log_descriptor.is_everything_flushed) + { + DBUG_PRINT("info", ("everything is flushed")); + rc= (translog_status == TRANSLOG_READONLY); + translog_unlock(); + goto out; + } + + for (;;) + { + translog_flush_buffers(&lsn, &sent_to_disk, &flush_horizon); + + if (!hgroup_commit_at_start) + break; /* flush pass is ended */ + +retest: + if (flush_interval != 0 && + (my_getsystime() - flush_start) >= flush_interval) + break; /* flush pass is ended */ + + pthread_mutex_lock(&log_descriptor.log_flush_lock); + if (log_descriptor.next_pass_max_lsn != LSN_IMPOSSIBLE) + { + /* take next goal */ + lsn= log_descriptor.next_pass_max_lsn; + log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; + /* prevent other thread from continue */ + log_descriptor.max_lsn_requester= pthread_self(); + DBUG_PRINT("info", ("flush took next goal: (%lu,0x%lx)", + LSN_IN_PARTS(lsn))); + } + else + { + if (flush_interval == 0 || + (time_spent= (my_getsystime() - flush_start)) >= flush_interval) { - rc= 1; - translog_stop_writing(); - sent_to_disk= LSN_IMPOSSIBLE; - goto out; + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + break; } - file->is_sync= 1; + DBUG_PRINT("info", ("flush waits: %llu interval: %llu spent: %llu", + flush_interval - time_spent, + flush_interval, time_spent)); + /* wait time or next goal */ + set_timespec_nsec(abstime, flush_interval - time_spent); + pthread_cond_timedwait(&log_descriptor.new_goal_cond, + &log_descriptor.log_flush_lock, + &abstime); + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + DBUG_PRINT("info", ("retest conditions")); + goto retest; } + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + + /* next flush pass */ + DBUG_PRINT("info", ("next flush pass")); + translog_lock(); } - if (sync_log_dir >= TRANSLOG_SYNC_DIR_ALWAYS && - (LSN_FILE_NO(log_descriptor.previous_flush_horizon) != - LSN_FILE_NO(flush_horizon) || - ((LSN_OFFSET(log_descriptor.previous_flush_horizon) - 1) / - TRANSLOG_PAGE_SIZE) != - ((LSN_OFFSET(flush_horizon) - 1) / TRANSLOG_PAGE_SIZE))) - rc|= sync_dir(log_descriptor.directory_fd, MYF(MY_WME | MY_IGNORE_BADFD)); + /* + sync() files from previous flush till current one + + We read soft_sync unprotected because it is actually 1 bit + value and we do not care much about "a bit old value" speed + of test is much more valuable here. + */ + if (!soft_sync || hgroup_commit_at_start) + { + if ((rc= + translog_sync_files(LSN_FILE_NO(log_descriptor.flushed), + LSN_FILE_NO(lsn), + sync_log_dir >= TRANSLOG_SYNC_DIR_ALWAYS && + (LSN_FILE_NO(log_descriptor. + previous_flush_horizon) != + LSN_FILE_NO(flush_horizon) || + ((LSN_OFFSET(log_descriptor. + previous_flush_horizon) - 1) / + TRANSLOG_PAGE_SIZE) != + ((LSN_OFFSET(flush_horizon) - 1) / + TRANSLOG_PAGE_SIZE))))) + { + sent_to_disk= LSN_IMPOSSIBLE; + goto out; + } + /* keep values for soft sync() and forced sync() actual */ + my_atomic_rwlock_wrlock(&soft_sync_rwl); + my_atomic_store32(&soft_sync_min, LSN_FILE_NO(lsn)); + my_atomic_store32(&soft_sync_max, LSN_FILE_NO(lsn)); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); + } + else + { + my_atomic_rwlock_wrlock(&soft_sync_rwl); + my_atomic_store32(&soft_sync_max, LSN_FILE_NO(lsn)); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); + } + + DBUG_ASSERT(flush_horizon <= log_descriptor.horizon); log_descriptor.previous_flush_horizon= flush_horizon; out: pthread_mutex_lock(&log_descriptor.log_flush_lock); if (sent_to_disk != LSN_IMPOSSIBLE) log_descriptor.flushed= sent_to_disk; log_descriptor.flush_in_progress= 0; + log_descriptor.flush_no++; DBUG_PRINT("info", ("flush_in_progress is dropped")); - pthread_mutex_unlock(&log_descriptor.log_flush_lock);\ + pthread_mutex_unlock(&log_descriptor.log_flush_lock); pthread_cond_broadcast(&log_descriptor.log_flush_cond); DBUG_RETURN(rc); } @@ -8103,6 +8387,7 @@ LSN translog_first_theoretical_lsn() my_bool translog_purge(TRANSLOG_ADDRESS low) { uint32 last_need_file= LSN_FILE_NO(low); + uint32 min_unsync; TRANSLOG_ADDRESS horizon= translog_get_horizon(); int rc= 0; DBUG_ENTER("translog_purge"); @@ -8110,6 +8395,12 @@ my_bool translog_purge(TRANSLOG_ADDRESS DBUG_ASSERT(translog_status == TRANSLOG_OK || translog_status == TRANSLOG_READONLY); + my_atomic_rwlock_wrlock(&soft_sync_rwl); + min_unsync= my_atomic_load32(&soft_sync_min); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); + if (min_unsync < last_need_file) + last_need_file= min_unsync; + pthread_mutex_lock(&log_descriptor.purger_lock); if (LSN_FILE_NO(log_descriptor.last_lsn_checked) < last_need_file) { @@ -8346,6 +8637,160 @@ my_bool translog_log_debug_info(TRN *trn } + +/** + Sets soft sync mode + + @param mode TRUE if we need switch soft sync on else off +*/ + +void translog_soft_sync(my_bool mode) +{ + my_atomic_rwlock_wrlock(&soft_sync_rwl); + my_atomic_store32(&soft_sync, (uint32) mode); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); +} + + +/** + Sets hard group commit + + @param mode TRUE if we need switch hard group commit on else off +*/ + +void translog_hard_group_commit(my_bool mode) +{ + my_atomic_rwlock_wrlock(&hgroup_commit_rwl); + my_atomic_store32(&hgroup_commit, (uint32) mode); + my_atomic_rwlock_wrunlock(&hgroup_commit_rwl); +} + + +/** + @brief forced log sync (used when we are switching modes) +*/ + +void translog_sync() +{ + uint32 max= get_current_logfile()->number; + uint32 min; + DBUG_ENTER("ma_translog_sync"); + + my_atomic_rwlock_rdlock(&soft_sync_rwl); + min= my_atomic_load32(&soft_sync_min); + my_atomic_rwlock_rdunlock(&soft_sync_rwl); + if (!min) + min= max; + + translog_sync_files(min, max, sync_log_dir >= TRANSLOG_SYNC_DIR_ALWAYS); + + DBUG_VOID_RETURN; +} + + +/** + @brief set rate for group commit + + @param rate rate to set. + + @note internally it stores interval in nanoseconds/TRANSLOG_RATE_BASE (to + fit into uint32) +*/ + +void translog_set_group_commit_rate(uint32 rate) +{ + DBUG_ENTER("translog_set_group_commit_rate"); + ulonglong wait_time; + if (rate) + { + wait_time= ((TRANSLOG_RATE_BASE * 1000000000ULL / rate + + TRANSLOG_RATE_BASE / 2) / + TRANSLOG_RATE_BASE); + if (wait_time == 0) + wait_time= 1; /* protection from getting special value */ + } + else + wait_time= 0; + my_atomic_rwlock_wrlock(&group_commit_wait_rwl); + my_atomic_store32(&group_commit_wait, (uint32)wait_time); + my_atomic_rwlock_wrunlock(&group_commit_wait_rwl); + DBUG_PRINT("info", ("rate: %lu wait: %llu", + (ulong)rate, (ulonglong)wait_time)); + DBUG_VOID_RETURN; +} + + +/** + @brief syncing service thread +*/ + +static pthread_handler_t +ma_soft_sync_background( void *arg __attribute__((unused))) +{ + + my_thread_init(); + DBUG_ENTER("ma_soft_sync_background"); + for(;;) + { + ulonglong prev_loop= my_getsystime(); + ulonglong time, sleep; + uint32 min, max; + my_atomic_rwlock_rdlock(&soft_sync_rwl); + min= my_atomic_load32(&soft_sync_min); + max= my_atomic_load32(&soft_sync_max); + my_atomic_store32(&soft_sync_min, max); + my_atomic_rwlock_rdunlock(&soft_sync_rwl); + + my_atomic_rwlock_rdlock(&group_commit_wait_rwl); + sleep= my_atomic_load32(&group_commit_wait) * TRANSLOG_RATE_BASE; + my_atomic_rwlock_rdunlock(&group_commit_wait_rwl); + translog_sync_files(min, max, FALSE); + time= my_getsystime() - prev_loop; + if (time > sleep) + sleep= 0; + else + sleep-= time; + if (my_service_thread_sleep(&soft_sync_control, sleep)) + break; + } + my_service_thread_signal_end(&soft_sync_control); + my_thread_end(); + DBUG_RETURN(0); +} + + +/** + @brief Starts syncing thread +*/ + +int translog_soft_sync_start(void) +{ + pthread_t th; + int res= 0; + DBUG_ENTER("translog_soft_sync_start"); + if (ma_service_thread_control_init(&soft_sync_control)) + res= 1; + else if (!(res= pthread_create(&th, NULL, ma_soft_sync_background, NULL))) + soft_sync_control.status= THREAD_RUNNING; + DBUG_RETURN(res); +} + + +/** + @brief Stops syncing thread +*/ + +void translog_soft_sync_end(void) +{ + DBUG_ENTER("translog_soft_sync_end"); + if (soft_sync_control.inited) + { + ma_service_thread_control_end(&soft_sync_control); + } + DBUG_VOID_RETURN; +} + + #ifdef MARIA_DUMP_LOG #include <my_getopt.h> extern void translog_example_table_init(); === modified file 'storage/maria/ma_loghandler.h' --- a/storage/maria/ma_loghandler.h 2009-01-15 22:25:53 +0000 +++ b/storage/maria/ma_loghandler.h 2009-03-17 20:32:17 +0000 @@ -342,6 +342,14 @@ enum enum_translog_status TRANSLOG_SHUTDOWN /* going to shutdown the loghandler */ }; extern enum enum_translog_status translog_status; +extern ulonglong translog_syncs; /* Number of sync()s */ + +void translog_soft_sync(my_bool mode); +void translog_hard_group_commit(my_bool mode); +int translog_soft_sync_start(void); +void translog_soft_sync_end(void); +void translog_sync(); +void translog_set_group_commit_rate(uint32 rate); /* all the rest added because of recovery; should we make @@ -441,6 +449,18 @@ extern LOG_DESC log_record_type_descript typedef enum { + TRANSLOG_GCOMMIT_NONE, + TRANSLOG_GCOMMIT_HARD, + TRANSLOG_GCOMMIT_SOFT +} enum_maria_group_commit; +extern ulong group_commit; +extern ulong group_commit_rate; +/** + group commit interval is TRANSLOG_RATE_BASE/<rate> seconds +*/ +#define TRANSLOG_RATE_BASE 100 +typedef enum +{ TRANSLOG_PURGE_IMMIDIATE, TRANSLOG_PURGE_EXTERNAL, TRANSLOG_PURGE_ONDEMAND === modified file 'tests/fork_big2.pl' --- a/tests/fork_big2.pl 2006-02-12 21:26:30 +0000 +++ b/tests/fork_big2.pl 2009-03-17 20:32:17 +0000 @@ -16,21 +16,21 @@ package main; $opt_skip_create=$opt_skip_in=$opt_verbose=$opt_fast_insert= $opt_lock_tables=$opt_debug=$opt_skip_drop=$opt_fast=$opt_force=0; -$opt_thread_factor=1; -$opt_insert=1; -$opt_select=6;$opt_join=4; -$opt_select_count=$opt_join_count=0; -$opt_update=1;$opt_delete=0; -$opt_flush=$opt_check=$opt_repair=$opt_alter=0; -$opt_join_range=100; +$opt_thread_factor=1; +$opt_insert=1; +$opt_select=6;$opt_join=4; +$opt_select_count=$opt_join_count=0; +$opt_update=1;$opt_delete=0; +$opt_flush=$opt_check=$opt_repair=$opt_alter=0; +$opt_join_range=100; $opt_resize_interval=0; $opt_time=0; $opt_host=$opt_user=$opt_password=""; $opt_db="test"; $opt_verbose=$opt_debug=$opt_lock_tables=$opt_fast_insert=$opt_fast=$opt_skip_in=$opt_force=undef; # Ignore warnings from these -GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", - "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", - "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", +GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", + "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", + "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", "flush=i", "check=i", "repair=i", "alter=i", "resize-interval=i", "max-join_range=i", "time=i") || die "Aborted"; print "Test of multiple connections that test the following things:\n"; @@ -48,20 +48,20 @@ srand 100; # Make random numbers repea #### #### Start timeing and start test -#### - +#### + $opt_insert*=$opt_thread_factor; -$opt_select*=$opt_thread_factor; -$opt_join*=$opt_thread_factor; -$opt_select_count*=$opt_thread_factor; -$opt_join_count*=$opt_thread_factor; -$opt_update*=$opt_thread_factor; -$opt_delete*=$opt_thread_factor; - -if ($opt_time == 0 && $opt_insert == 0) -{ - $opt_insert=1; -} +$opt_select*=$opt_thread_factor; +$opt_join*=$opt_thread_factor; +$opt_select_count*=$opt_thread_factor; +$opt_join_count*=$opt_thread_factor; +$opt_update*=$opt_thread_factor; +$opt_delete*=$opt_thread_factor; + +if ($opt_time == 0 && $opt_insert == 0) +{ + $opt_insert=1; +} $start_time=new Benchmark; $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", @@ -100,71 +100,71 @@ $|= 1; # Autoflush #### #### Start the tests #### -if ($opt_time != 0) -{ - test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; +if ($opt_time != 0) +{ + test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; } for ($i=0 ; $i < $opt_insert ; $i ++) { test_insert() if (($pid=fork()) == 0); $work{$pid}="insert"; -} +} $threads=$i; -for ($i=0 ; $i < $opt_select ; $i ++) -{ - test_select() if (($pid=fork()) == 0); $work{$pid}="select"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_join ; $i ++) -{ - test_join() if (($pid=fork()) == 0); $work{$pid}="join"; -} -$threads+=$i; +for ($i=0 ; $i < $opt_select ; $i ++) +{ + test_select() if (($pid=fork()) == 0); $work{$pid}="select"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_join ; $i ++) +{ + test_join() if (($pid=fork()) == 0); $work{$pid}="join"; +} +$threads+=$i; for ($i=0 ; $i < $opt_select_count ; $i ++) { test_select_count() if (($pid=fork()) == 0); $work{$pid}="select_count"; } -$threads+=$i; -for ($i=0 ; $i < $opt_join_count ; $i ++) -{ - test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_update ; $i ++) -{ - test_update() if (($pid=fork()) == 0); $work{$pid}="update"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_delete ; $i ++) -{ - test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_flush ; $i ++) -{ - test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_check ; $i ++) -{ - test_check() if (($pid=fork()) == 0); $work{$pid}="check"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_repair ; $i ++) -{ - test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_alter ; $i ++) -{ - test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; -} -$threads+=$i; +$threads+=$i; +for ($i=0 ; $i < $opt_join_count ; $i ++) +{ + test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_update ; $i ++) +{ + test_update() if (($pid=fork()) == 0); $work{$pid}="update"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_delete ; $i ++) +{ + test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_flush ; $i ++) +{ + test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_check ; $i ++) +{ + test_check() if (($pid=fork()) == 0); $work{$pid}="check"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_repair ; $i ++) +{ + test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_alter ; $i ++) +{ + test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; +} +$threads+=$i; if ($opt_resize_interval != 0) { test_resize() if (($pid=fork()) == 0); $work{$pid}="resize"; $threads+=1; } - + print "Started $threads threads\n"; $errors=0; @@ -172,17 +172,17 @@ $running_insert_threads=$opt_insert; while (($pid=wait()) != -1) { $ret=$?/256; - print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; - if ($opt_time == 0) + print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; + if ($opt_time == 0) { if ($work{$pid} =~ /^insert/) { if (!--$running_insert_threads) - { - + { + # Time to stop other threads signal_abort(); - } + } } } $errors++ if ($ret != 0); @@ -214,17 +214,17 @@ print "Total time: " . exit(0); -# -# Sleep and then abort other threads -# - -sub test_abort -{ - sleep($opt_time); - signal_abort(); - exit(0); -} - +# +# Sleep and then abort other threads +# + +sub test_abort +{ + sleep($opt_time); + signal_abort(); + exit(0); +} + # # Insert records in the table @@ -363,58 +363,58 @@ sub test_join $dbh->disconnect; $dbh=0; print "Test_join: Executed $count joins\n"; exit(0); -} - -# -# select records -# Do continously joins between the first and second for range and count selected rows -# - -sub test_join_count -{ - my ($dbh, $i, $j, $count, $loop); - - $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", - $opt_user, $opt_password, - { PrintError => 0}) || die $DBI::errstr; - - $count_query=make_count_query($numtables); - $count=0; - $loop=9999; - $sum=0; - - srand(); - - $i=0; - while (($i++ % 10) || !test_if_abort($dbh)) - { - if ($loop++ >= 10) - { - $loop=0; - $row_counts=simple_query($dbh, $count_query); - } - for ($j=0 ; $j < $numtables-1 ; $j++) - { - my ($id1)= int rand $row_counts->[$j]; - my ($id2)= int rand $row_counts->[$j]; - if ($id1 > $id2) - { - my $id0=$id1; $id1=$id2; $id2=$id0; - if ($id2-$id1 > $opt_join_range) - { - $id2=$id1+$opt_join_range; - } - } - my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); - $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); - $sum+=$row->[0]; - $count++; - } - } - $dbh->disconnect; $dbh=0; - print "Test_join_count: Executed $count joins: total $sum rows\n"; - exit(0); -} +} + +# +# select records +# Do continously joins between the first and second for range and count selected rows +# + +sub test_join_count +{ + my ($dbh, $i, $j, $count, $loop); + + $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", + $opt_user, $opt_password, + { PrintError => 0}) || die $DBI::errstr; + + $count_query=make_count_query($numtables); + $count=0; + $loop=9999; + $sum=0; + + srand(); + + $i=0; + while (($i++ % 10) || !test_if_abort($dbh)) + { + if ($loop++ >= 10) + { + $loop=0; + $row_counts=simple_query($dbh, $count_query); + } + for ($j=0 ; $j < $numtables-1 ; $j++) + { + my ($id1)= int rand $row_counts->[$j]; + my ($id2)= int rand $row_counts->[$j]; + if ($id1 > $id2) + { + my $id0=$id1; $id1=$id2; $id2=$id0; + if ($id2-$id1 > $opt_join_range) + { + $id2=$id1+$opt_join_range; + } + } + my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); + $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); + $sum+=$row->[0]; + $count++; + } + } + $dbh->disconnect; $dbh=0; + print "Test_join_count: Executed $count joins: total $sum rows\n"; + exit(0); +} #

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2683)
by knielsen＠knielsen-hq.org 16 Mar '09

16 Mar '09

#At lp:maria 2683 knielsen(a)knielsen-hq.org 2009-03-16 Remove redundant code, probably bad automerge modified: mysql-test/lib/mtr_cases.pm === modified file 'mysql-test/lib/mtr_cases.pm' --- a/mysql-test/lib/mtr_cases.pm 2009-02-19 09:01:25 +0000 +++ b/mysql-test/lib/mtr_cases.pm 2009-03-16 09:27:05 +0000 @@ -799,15 +799,6 @@ sub collect_one_test_case { push(@{$tinfo->{'master_opt'}}, @$suite_opts); push(@{$tinfo->{'slave_opt'}}, @$suite_opts); - #----------------------------------------------------------------------- - # Check for test specific config file - #----------------------------------------------------------------------- - my $test_cnf_file= "$testdir/$tname.cnf"; - if ( -f $test_cnf_file) { - # Specifies the configuration file to use for this test - $tinfo->{'template_path'}= $test_cnf_file; - } - # ---------------------------------------------------------------------- # Check for test specific config file # ----------------------------------------------------------------------

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2683)
by knielsen＠knielsen-hq.org 16 Mar '09

16 Mar '09

#At lp:maria 2683 knielsen(a)knielsen-hq.org 2009-03-16 Make test --big, as it takes a _long_ time to run and only tests a single bug. modified: mysql-test/t/multi_update2.test === modified file 'mysql-test/t/multi_update2.test' --- a/mysql-test/t/multi_update2.test 2008-11-19 18:17:26 +0000 +++ b/mysql-test/t/multi_update2.test 2009-03-16 09:25:20 +0000 @@ -2,6 +2,9 @@ # Test of update statement that uses many tables. # +# This is a big test. +--source include/big_test.inc + --disable_warnings DROP TABLE IF EXISTS t1,t2; --enable_warnings

1 0

Re: [Maria-developers] MySQL / Maria Windows Port
by Michael Widenius 15 Mar '09

15 Mar '09

Hi! <cut> >> The my_pwrite() code now looks like: >> >> .... >> #if defined(_WIN32) >> readbytes= my_win_pread(Filedes, Buffer, Count, offset); >> #else >> ... >> >> And there is ew my_winfile.c file. >> >> Is that your code? <cut> Jeremiah> Yeah, this is definitely 99% my code. Someone spent the time to clean Jeremiah> it, normalize it, and add a few fixes and polish, but it's essentially Jeremiah> the same as what I submitted. Unfortunately, there's no one line Jeremiah> attribution I was promised for contributing the code. :) But at least Jeremiah> it's in there, that's a great start. Sorry about the attribution; I will ensure that when we pull/implement this to MariaDB we will add it! <cut> >> Could you take a look a the MySQL-6.0 mysys/windows code and see if >> you think if it's now good enough. >> >> If not, do you think we should take the 6.0 code as a base for our >> tree or should we start with your code? >> Jeremiah> The MySQL 6.0 code for Windows file handling is a good starting point, Jeremiah> but you can see the primary "hack" of my code is the my_open_osfhandle Jeremiah> function, which is a table specifically for emulating the C style Jeremiah> integer file handles. If I recall correctly (it has been years) the Jeremiah> only reasons I had to implement this were: Jeremiah> As you said, the my_file_info structure, which uses the file descriptor Jeremiah> itself as an index. Jeremiah> my_file.c Jeremiah> if ((uint) fd < my_file_limit && my_file_info[fd].type != UNOPEN) Jeremiah> There you can see the opacity of the file descriptor is violated by Jeremiah> using it as a bounds check for the array, although that's obviously Jeremiah> follows using it as an index into the array. When I made the patch my Jeremiah> understanding was that I needed to keep as many of the changes as Jeremiah> possible within Windows ifdefs, and that changing the structure that Jeremiah> contains all of the file handles significantly would be too much change Jeremiah> to non-Windows code. Yes, we should to strive to keep most of the code 'similar' to ensure easy pulls between MySQL versions. Jeremiah> There are obviously a lot of ways to resolve this. If I were writing it Jeremiah> from scratch, I wouldn't pass the file descriptor around to different Jeremiah> functions, I would pass a pointer to the my_file_info structure around Jeremiah> instead, and all of the mysys file handling code would take a Jeremiah> my_file_info structure pointer as the file desc parameter. There's no Jeremiah> need to index into an array if you already have the pointer to the Jeremiah> struct in question. The code would become portable, and the Jeremiah> serialization on file open/close (fairly minor) would disappear. Agree that a much better way would be to move away from int's and have a structure instead. It should however not be that hard to do this properly as most of the MySQL code is using the type 'File' for the file handle. My old idea was to create a new library, my_file, and copy all the file handling and file name functions from this to my_file and then change this to use a pointer to a structure instead of a File object. The File objects should also contains the file name in parts and one should be able to manipulate the file name without having the file open. The changes in the code would mainly be: - Create the new library. - Add a separate call to populate the File object with the name parts - Change all calls to my_open / my_read to my_file_open, my_file_read etc... (Can be done trivially with one replace command) - Change error handling to instead of checking if my_file_open returns -1 see if it returns MY_INVALID_FILE - Over time, change all file and file name handling functions to use the new file handling functions. You can find the original description of this task here: http://forge.mysql.com/worklog/task.php?id=314 I have included a halfdone version if this code in this email. basharteg> If things are basharteg> being designed in a way that isn't entirely UNIX-centric, it's easy to make basharteg> MySQL for Windows fly. If we get to the point where we're willing to look basharteg> at async socket I/O and thread queues / I/O Completion Ports for work basharteg> dispatch, we can do even more. >> I don't know much about that, so I would be really thankful for any >> help I can get in this area. Jeremiah> It's going to take a fair amount of design work to figure out how to Jeremiah> adapt MySQL to this kind of model. It could be implemented on all Jeremiah> platforms with epoll on Linux, kqueue on FreeBSD, and I/O Completion Jeremiah> Ports on Windows. Event notification I/O is the key to serious Jeremiah> scalability, but it requires significant design changes, especially when Jeremiah> you're coming from a 1:1 threading model. I have always been worried about the fact that if you have other threads doing IO, you will always get a thread switch + some atomic operations / mutex when doing any read or write operations. The 1:1 model makes handling io operations much easier and if things are buffered then this can be superior compared to having a pool for data. Also, if all the threads are mixing cpu bound and io operations, this gives you somewhat balanced load that works reasonable good in many cases. I understand that pools are superior in some context; I am just not yet sure when it's best to use a pool and when to use 1:1 threads. <cut> >> In MySQL 6.0 we you can choose at startup if you want to have one >> thread per connection or a pool of threads to handle all connections. >> I wrote the patch in such a way that it's < 10 min of works to pull >> this code back to 5.1 (or MariaDB); Will do this next week. >> >> http://dev.mysql.com/doc/refman/6.0/en/connection-threads.html >> Jeremiah> This change makes significant progress towards moving to event Jeremiah> notification I/O. That means you broke the state information up between Jeremiah> thread and connection and eliminated the bad thread == connection Jeremiah> assumptions. Yes. The patch is now in lp:maria basharteg> So in terms of MySQL, it definitely runs with a 1:1 threading model using basharteg> blocking I/O. While this isn't uncommon for UNIX applications, you'll basharteg> notice the "hot" UNIX apps that are more interested in the C10K ( basharteg> http://www.kegel.com/c10k.html) target for their scalability are starting to basharteg> move to async I/O implementations. This has produced some good output like basharteg> lighttpd (http://www.lighttpd.net/) >> I haven't yet made up my mind what's the best route for MariaDB in >> this case; I am waiting for a benchmark or nice code that will make up >> my mind for me... The thread goes both ways. When MySQL was created, it was more common for people to use using async instead of an 1:1 thread model. As far as people have told me, after MySQL and as the thread libraries got better a lot of programs changed to us a 1:1 thread model. Don't think it has yet been proved what is the best model/mix long term. A lot depends on the worklog and what the threads are doing. For example in the MySQL case: - If all your queries are short and there are very little conflicts a pool-of-threads version is much better than a 1:1 model. - If you have a big mix long running queries and short running queries, a 1:1 thread application is superior to simple pole based one. Jeremiah> Yeah, benchmarks are tough too, because serial benchmarks will tend to Jeremiah> show the overhead involved in asynchronous I/O operations. You have to Jeremiah> decide that you're developing for the multi-core world and accept the Jeremiah> minor overhead in exchange for the significant scalability enhancements, Jeremiah> and design your benchmarks to implement concurrent loads. See above. What is best is mostly depending one what the threads are supposed to do. Multi-core is not the most critical part here. <cut> >> Note that the thread pool is still 'beta' quality, in the sense it >> still have some notable week spots: >> >> - If all threads are working or hang (in lock tables) then no new >> connections can be established, one can't examine what is going on >> and one can't kill offending threads (as one can't send in new queries) >> >> The easyest fix would be to have another port where one could connect >> with the 1:1 model and check what's going on. I added this to the MariaDB thread-pool implementation that is now pushed. Jeremiah> Having an administrative connection is something MySQL has needed for a Jeremiah> very long time anyway. However, as someone who uses thread pools Jeremiah> extensively on a daily basis, you can handle the cases of thread pool Jeremiah> exhaustion in several ways, including safety timer events, timeouts on Jeremiah> lock waits, event based notification on lock acquisition (may be Jeremiah> difficult). This is not easy when you don't want to abort any of the running queries. As the lock between threads can bappen very low, for example inside a storage engine for which you don't have any control, none of the above solutions are easy or even possible to implement. Jeremiah> But as someone who runs a fairly large MySQL system for a Jeremiah> telephony company, I can tell you that the 1:1 threading model doesn't Jeremiah> save you from thread exhaustion any more than having a thread pool Jeremiah> would. When we get threads stuck waiting for locks our servers can Jeremiah> easily become thread exhausted with no way to kill the threads. As long as you limit max_connections under the number of threads your system can handle, you should always be able to connect as a SUPER user and fix things. Jeremiah> Besides, in a good design, you shouldn't have just one thread pool. The Jeremiah> thread pool that handles I/O events (like accepting new connections) Jeremiah> should be different from the worker thread pool that executes SQL Jeremiah> statements. In the current thread pool implementation, the threads are waiting in the pool for any request on connection port; In this case the connection/login handling and reading commands from the user are both io events. Jeremiah> In the Windows async I/O world, we consider connections to be relatively Jeremiah> inexpensive. I don't set connection limits in my applications. Each Jeremiah> connection is a little bit of non-paged pool kernel memory, plus the Jeremiah> little struct I use to keep the connection state information, plus one Jeremiah> async I/O read pending (among thousands). You also need to keep the connection state, which in MySQL means the THD object. This is a semi-expensive object (10 K) Jeremiah> But yes, one of the basics of our design is having at least two thread Jeremiah> pools, one for socket I/O (at least so far as new connections, posting Jeremiah> reads, and receiving queries) and one for SQL statements. The socket Jeremiah> I/O thread pool should not be something that deadlocks, since whatever Jeremiah> locking is used, it would be far more deterministic than the locking Jeremiah> involved in executing SQL statements. In the current implementation, we have only one pool and in MySQL:s case it may be enough. The reason for this is that when the thread gets a signal that there is data to be read, it will have all data it needs in the buffer (as the client commands are mostly very short and if they are not short, the client is working full time to send it as they are always send whole). This means that there is in practice no deadlocks when it comes to reading data and you win the context switch from reading data and then giving it to a worker thread. Jeremiah> Another thing to throw out there since I'm already being terribly Jeremiah> verbose is, each query should have a timeout value, and when that Jeremiah> timeout value is exceeded, the query should terminate. One of the worst Jeremiah> parts about working in MySQL is the idea of a stuck query, especially Jeremiah> when you need a separate connection to kill it. In systems like Jeremiah> Microsoft SQL, the default query timeout is 2 minutes, and if you need Jeremiah> to exceed that or you don't want a timeout, you can change it as part of Jeremiah> the connection settings. Runaway queries running indefinitely doesn't Jeremiah> make any practical sense. This agains depends on the application. Most applications need to do long statistical queries from time to time and you don't want to kill these! Same thing with any queries that does critical updates or moves things around (like ALTER TABLE, REPAIR etc). In other words; I agree we need a timeout mechanism. I am however not sure what should be the best default value for this. That said, Guilhem did once implement a timeout for queries but this was unfortunately never pushed becasue it changed the code too much at the time when the patch was made and we where close to GA. We should dig out this patch and fix it for the current code and add it. Jeremiah> Also, when we get async I/O in place, you Jeremiah> could eventually add an extension to the protocol that allows the client Jeremiah> to cancel the query in progress. Since the incoming I/Os are handled by Jeremiah> I/O threads rather than the SQL threads, you can still receive commands Jeremiah> while the query is in progress. So even as you're executing a query for Jeremiah> someone, you have an async read on the socket waiting for the next Jeremiah> command. If the next command is received while the query is still in Jeremiah> progress, you check to see if it's a cancel command, or simply a Jeremiah> pipelined SQL query. If it's a cancel command, you flag your SQL thread Jeremiah> to stop when it can, and if it's a pipelined SQL query they're probably Jeremiah> violating the MySQL protocol anyway since they need to wait for a Jeremiah> response to the previous query, but if we allowed pipelined queries some Jeremiah> day in the future, they've lost their right to cancel this query and you Jeremiah> simply queue the SQL query until the current query is finished. In the Jeremiah> far future, multiple queries on the same connection are fine, and are Jeremiah> given query identifiers and all handled through async I/O, and all Jeremiah> queries are cancellable. Now you can achieve the same thing by just sending the kill one another connection (could for example be used through the extra-port). Drizzle has done some work in the direction of sending many queries on the same connection. We should look at this when their implementation is ready. I am still a bit sceptical of having two thread pools; My major worry is that it will for normal users slow down things instead of giving a speed increase (because of the extra context switch and mutex/atomic operations to shuffle things around) We need to find someone that can write a prototype and show the benefits trough some agreed to benchmark... basharteg> We need to find the points in there where I/Os are taking place that can be basharteg> adapted to an async model. Basically, the disk reads that are happening to basharteg> satisfy the query, the network reads to get the query, the network writes to basharteg> send the results, and the network waits for the next query. All of those basharteg> things need to happen without a thread being associated with this basharteg> connection/query. We need a proof of concept and benchmark of this... >> For mysqld this isn't trivial as we can't easily change the code for >> the engines. >> >> What I don't like with the async model is that for reads on disk it >> doesn't speed up things as much and it may even slow down thing as you >> now have two threads communication and the original is just waiting >> for the other to take the message, execute it, and tell the other that >> it's now ready. Jeremiah> Well, you can avoid async I/O for disk I/O if you think it's going to be Jeremiah> a performance negative. On single or few drive SATA systems, you're Jeremiah> right. On high end I/O subsystems, async I/O can definitely have a Jeremiah> throughput benefit, especially when you have an advanced storage Jeremiah> subsystem driver that combines operations. As we get to Solid State Jeremiah> Disks in the future, it's going to be the same thing as with the Jeremiah> multi-core CPUs we're adapting to. If you're not doing multiple things Jeremiah> at once, your performance is going to be weak. For now though, you can Jeremiah> leave out async disk I/O, but async socket I/O is the only way to get up Jeremiah> to tens of thousands of connections and beyond. You shouldn't be afraid Jeremiah> of the overhead of using multiple threads and multiple thread pools. If Jeremiah> done properly, the penalty to serial operations is minimal, and the Jeremiah> performance gains are significant (in addition to the robustness Jeremiah> enhancements previously discussed). The pool-of-threads code we have now already gives us 100,000 of connections. But I am still afread of splitting the current thread pool to 2 parts... <cut> >> What would be the advantages of using Apache portable runtime? >> Should this be instead of mysys ? >> >> The benefits of using mysys are: >> - Well working and integrated with all MySQL parts >> - Well integrated with dbug statements >> - Provides a lot of extra features, like hiding windows thread >> implementation. >> Jeremiah> I can understand moving to apr from mysys might not be feasible, I just Jeremiah> bring it up because they've abstracted async and event driven I/O and a Jeremiah> lot of other platform differences very well. It works great on Jeremiah> Windows, FreeBSD, and Linux. Probably not something we can go for Jeremiah> right now, but they may have some useful designs and hints. I am happy to take any good ideas from their libraires and add them to ours and even move some of our code in their direction (if there is a clear gain for this). Regards, Monty

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2683)
by knielsen＠knielsen-hq.org 13 Mar '09

13 Mar '09

#At lp:maria 2683 knielsen(a)knielsen-hq.org 2009-03-13 [merge] Merged. removed: extra/libevent/event-config.h modified: sql/mysqld.cc === removed file 'extra/libevent/event-config.h' --- a/extra/libevent/event-config.h 2009-03-12 22:27:35 +0000 +++ b/extra/libevent/event-config.h 1970-01-01 00:00:00 +0000 @@ -1,1409 +0,0 @@ -/* event-config.h - * Generated by autoconf; post-processed by libevent. - * Do not edit this file. - * Do not rely on macros in this file existing in later versions. - */ -#ifndef _EVENT_CONFIG_H_ -#define _EVENT_CONFIG_H_ -/* include/config.h. Generated from config.h.in by configure. */ -/* config.h.in. Generated from configure.in by autoheader. */ - -/* Support big tables */ -#define _EVENT_BIG_TABLES 1 - -/* Whether features provided by the user community should be included */ -#define _EVENT_COMMUNITY_SERVER 1 - -/* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP - systems. This function is required for `alloca.c' support on those systems. - */ -/* #undef _EVENT_CRAY_STACKSEG_END */ - -/* Define to 1 if using `alloca.c'. */ -/* #undef _EVENT_C_ALLOCA */ - -/* Don't use libdbug */ -/* #undef _EVENT_DBUG_OFF */ - -/* Use libdbug */ -#define _EVENT_DBUG_ON 1 - -/* all charsets are available */ -#define _EVENT_DEFINE_ALL_CHARACTER_SETS 1 - -/* Disables the use of --init-file, --skip-grant-tables and --bootstrap - options */ -/* #undef _EVENT_DISABLE_GRANT_OPTIONS */ - -/* Define if clock_gettime is available in libc */ -/* #undef _EVENT_DNS_USE_CPU_CLOCK_FOR_ID */ - -/* Define is no secure id variant is available */ -#define _EVENT_DNS_USE_GETTIMEOFDAY_FOR_ID 1 - -/* Version of .frm files */ -#define _EVENT_DOT_FRM_VERSION 6 - -/* If LOAD DATA LOCAL INFILE should be enabled by default */ -#define _EVENT_ENABLED_LOCAL_INFILE 1 - -/* If SHOW PROFILE should be enabled */ -#define _EVENT_ENABLED_PROFILING 1 - -/* Enable error injection in MySQL Server */ -/* #undef _EVENT_ERROR_INJECT_SUPPORT */ - -/* Do we have FIONREAD */ -#define _EVENT_FIONREAD_IN_SYS_IOCTL 1 - -/* READLINE: your system defines TIOCGWINSZ in sys/ioctl.h. */ -#define _EVENT_GWINSZ_IN_SYS_IOCTL 1 - -/* Define to 1 if you have the `abi::__cxa_demangle' function. */ -/* #undef _EVENT_HAVE_ABI_CXA_DEMANGLE */ - -/* Define to 1 if you have the <aio.h> header file. */ -#define _EVENT_HAVE_AIO_H 1 - -/* Define to 1 if you have the `alarm' function. */ -#define _EVENT_HAVE_ALARM 1 - -/* Define to 1 if you have `alloca', as a function or macro. */ -#define _EVENT_HAVE_ALLOCA 1 - -/* Define to 1 if you have <alloca.h> and it should be used (not on Ultrix). - */ -#define _EVENT_HAVE_ALLOCA_H 1 - -/* Define to 1 if you have the <arpa/inet.h> header file. */ -#define _EVENT_HAVE_ARPA_INET_H 1 - -/* Define to 1 if you have the <asm/termbits.h> header file. */ -#define _EVENT_HAVE_ASM_TERMBITS_H 1 - -/* Define to 1 if you have the `backtrace' function. */ -#define _EVENT_HAVE_BACKTRACE 1 - -/* Define to 1 if you have the `backtrace_symbols' function. */ -#define _EVENT_HAVE_BACKTRACE_SYMBOLS 1 - -/* Define to 1 if you have the `backtrace_symbols_fd' function. */ -#define _EVENT_HAVE_BACKTRACE_SYMBOLS_FD 1 - -/* Define to 1 if you have the `bcmp' function. */ -#define _EVENT_HAVE_BCMP 1 - -/* Define to 1 if you have the `bfill' function. */ -/* #undef _EVENT_HAVE_BFILL */ - -/* Define to 1 if you have the `bmove' function. */ -/* #undef _EVENT_HAVE_BMOVE */ - -/* bool is not defined by all C++ compilators */ -#define _EVENT_HAVE_BOOL 1 - -/* Can netinet be included */ -/* #undef _EVENT_HAVE_BROKEN_NETINET_INCLUDES */ - -/* BSD style signals */ -/* #undef _EVENT_HAVE_BSD_SIGNALS */ - -/* Define to 1 if you have the `bsearch' function. */ -#define _EVENT_HAVE_BSEARCH 1 - -/* Define to 1 if compiler defines __bss_start. */ -#define _EVENT_HAVE_BSS_START 1 - -/* Define to 1 if you have the `bzero' function. */ -#define _EVENT_HAVE_BZERO 1 - -/* Define to enable charset armscii8 */ -/* #undef _EVENT_HAVE_CHARSET_armscii8 */ - -/* Define to enable ascii character set */ -/* #undef _EVENT_HAVE_CHARSET_ascii */ - -/* Define to enable charset big5 */ -#define _EVENT_HAVE_CHARSET_big5 1 - -/* Define to enable cp1250 */ -#define _EVENT_HAVE_CHARSET_cp1250 1 - -/* Define to enable charset cp1251 */ -/* #undef _EVENT_HAVE_CHARSET_cp1251 */ - -/* Define to enable charset cp1256 */ -/* #undef _EVENT_HAVE_CHARSET_cp1256 */ - -/* Define to enable charset cp1257 */ -/* #undef _EVENT_HAVE_CHARSET_cp1257 */ - -/* Define to enable charset cp850 */ -/* #undef _EVENT_HAVE_CHARSET_cp850 */ - -/* Define to enable charset cp852 */ -/* #undef _EVENT_HAVE_CHARSET_cp852 */ - -/* Define to enable charset cp866 */ -/* #undef _EVENT_HAVE_CHARSET_cp866 */ - -/* Define to enable charset cp932 */ -#define _EVENT_HAVE_CHARSET_cp932 1 - -/* Define to enable charset dec8 */ -/* #undef _EVENT_HAVE_CHARSET_dec8 */ - -/* Define to enable charset eucjpms */ -#define _EVENT_HAVE_CHARSET_eucjpms 1 - -/* Define to enable charset euckr */ -#define _EVENT_HAVE_CHARSET_euckr 1 - -/* Define to enable charset gb2312 */ -#define _EVENT_HAVE_CHARSET_gb2312 1 - -/* Define to enable charset gbk */ -#define _EVENT_HAVE_CHARSET_gbk 1 - -/* Define to enable charset geostd8 */ -/* #undef _EVENT_HAVE_CHARSET_geostd8 */ - -/* Define to enable charset greek */ -/* #undef _EVENT_HAVE_CHARSET_greek */ - -/* Define to enable charset hebrew */ -/* #undef _EVENT_HAVE_CHARSET_hebrew */ - -/* Define to enable charset hp8 */ -/* #undef _EVENT_HAVE_CHARSET_hp8 */ - -/* Define to enable charset keybcs2 */ -/* #undef _EVENT_HAVE_CHARSET_keybcs2 */ - -/* Define to enable charset koi8r */ -/* #undef _EVENT_HAVE_CHARSET_koi8r */ - -/* Define to enable charset koi8u */ -/* #undef _EVENT_HAVE_CHARSET_koi8u */ - -/* Define to enable charset latin1 */ -#define _EVENT_HAVE_CHARSET_latin1 1 - -/* Define to enable charset latin2 */ -#define _EVENT_HAVE_CHARSET_latin2 1 - -/* Define to enable charset latin5 */ -/* #undef _EVENT_HAVE_CHARSET_latin5 */ - -/* Define to enable charset latin7 */ -/* #undef _EVENT_HAVE_CHARSET_latin7 */ - -/* Define to enable charset macce */ -/* #undef _EVENT_HAVE_CHARSET_macce */ - -/* Define to enable charset macroman */ -/* #undef _EVENT_HAVE_CHARSET_macroman */ - -/* Define to enable charset sjis */ -#define _EVENT_HAVE_CHARSET_sjis 1 - -/* Define to enable charset swe7 */ -/* #undef _EVENT_HAVE_CHARSET_swe7 */ - -/* Define to enable charset tis620 */ -#define _EVENT_HAVE_CHARSET_tis620 1 - -/* Define to enable charset ucs2 */ -#define _EVENT_HAVE_CHARSET_ucs2 1 - -/* Define to enable charset ujis */ -#define _EVENT_HAVE_CHARSET_ujis 1 - -/* Define to enable ut8 */ -#define _EVENT_HAVE_CHARSET_utf8 1 - -/* Define to 1 if you have the `chsize' function. */ -/* #undef _EVENT_HAVE_CHSIZE */ - -/* Define to 1 if you have the `clock_gettime' function. */ -/* #undef _EVENT_HAVE_CLOCK_GETTIME */ - -/* Define to enable compression support */ -#define _EVENT_HAVE_COMPRESS 1 - -/* crypt */ -#define _EVENT_HAVE_CRYPT 1 - -/* Define to 1 if you have the <crypt.h> header file. */ -#define _EVENT_HAVE_CRYPT_H 1 - -/* Define to 1 if you have the <curses.h> header file. */ -#define _EVENT_HAVE_CURSES_H 1 - -/* Define to 1 if you have the `cuserid' function. */ -#define _EVENT_HAVE_CUSERID 1 - -/* Define to 1 if you have the <cxxabi.h> header file. */ -#define _EVENT_HAVE_CXXABI_H 1 - -/* Define to 1 if you have the declaration of `madvise', and to 0 if you - don't. */ -#define _EVENT_HAVE_DECL_MADVISE 1 - -/* Define to 1 if you have the declaration of `SHM_HUGETLB', and to 0 if you - don't. */ -#define _EVENT_HAVE_DECL_SHM_HUGETLB 1 - -/* Define to 1 if you have the declaration of `tgoto', and to 0 if you don't. - */ -#define _EVENT_HAVE_DECL_TGOTO 1 - -/* Whether we are using DEC threads */ -/* #undef _EVENT_HAVE_DEC_THREADS */ - -/* Define if /dev/poll is available */ -/* #undef _EVENT_HAVE_DEVPOLL */ - -/* Define to 1 if you have the <dirent.h> header file. */ -#define _EVENT_HAVE_DIRENT_H 1 - -/* Define to 1 if you have the `dlerror' function. */ -#define _EVENT_HAVE_DLERROR 1 - -/* Define to 1 if you have the <dlfcn.h> header file. */ -#define _EVENT_HAVE_DLFCN_H 1 - -/* Define to 1 if you have the `dlopen' function. */ -#define _EVENT_HAVE_DLOPEN 1 - -/* Define to 1 if you don't have `vprintf' but do have `_doprnt.' */ -/* #undef _EVENT_HAVE_DOPRNT */ - -/* Access checks in embedded library */ -/* #undef _EVENT_HAVE_EMBEDDED_PRIVILEGE_CONTROL */ - -/* Define if your system supports the epoll system calls */ -#define _EVENT_HAVE_EPOLL 1 - -/* Define to 1 if you have the `epoll_ctl' function. */ -#define _EVENT_HAVE_EPOLL_CTL 1 - -/* Define if your system supports event ports */ -/* #undef _EVENT_HAVE_EVENT_PORTS */ - -/* Define to 1 if you have the <execinfo.h> header file. */ -#define _EVENT_HAVE_EXECINFO_H 1 - -/* Defined by configure. Use explicit template instantiation. */ -#define _EVENT_HAVE_EXPLICIT_TEMPLATE_INSTANTIATION 1 - -/* Define to 1 if you have the `fchmod' function. */ -#define _EVENT_HAVE_FCHMOD 1 - -/* Define to 1 if you have the `fcntl' function. */ -#define _EVENT_HAVE_FCNTL 1 - -/* Define to 1 if you have the <fcntl.h> header file. */ -#define _EVENT_HAVE_FCNTL_H 1 - -/* Define to 1 if you have the `fconvert' function. */ -/* #undef _EVENT_HAVE_FCONVERT */ - -/* Define to 1 if you have the `fdatasync' function. */ -#define _EVENT_HAVE_FDATASYNC 1 - -/* Define to 1 if you have the `fgetln' function. */ -/* #undef _EVENT_HAVE_FGETLN */ - -/* Define to 1 if you have the `finite' function. */ -#define _EVENT_HAVE_FINITE 1 - -/* Define to 1 if you have the <floatingpoint.h> header file. */ -/* #undef _EVENT_HAVE_FLOATINGPOINT_H */ - -/* Define to 1 if you have the <float.h> header file. */ -#define _EVENT_HAVE_FLOAT_H 1 - -/* Define to 1 if you have the `flockfile' function. */ -#define _EVENT_HAVE_FLOCKFILE 1 - -/* Define to 1 if you have the <fnmatch.h> header file. */ -#define _EVENT_HAVE_FNMATCH_H 1 - -/* Define to 1 if you have the `fpresetsticky' function. */ -/* #undef _EVENT_HAVE_FPRESETSTICKY */ - -/* Define to 1 if you have the `fpsetmask' function. */ -/* #undef _EVENT_HAVE_FPSETMASK */ - -/* Define to 1 if the system has the type `fp_except'. */ -/* #undef _EVENT_HAVE_FP_EXCEPT */ - -/* Define to 1 if you have the `fsync' function. */ -#define _EVENT_HAVE_FSYNC 1 - -/* Define to 1 if you have the `ftruncate' function. */ -#define _EVENT_HAVE_FTRUNCATE 1 - -/* Define to 1 if compiler provides atomic builtins. */ -/* #undef _EVENT_HAVE_GCC_ATOMIC_BUILTINS */ - -/* Define to 1 if you have the `getaddrinfo' function. */ -#define _EVENT_HAVE_GETADDRINFO 1 - -/* Define to 1 if you have the `getcwd' function. */ -#define _EVENT_HAVE_GETCWD 1 - -/* Define to 1 if you have the `gethostbyaddr_r' function. */ -#define _EVENT_HAVE_GETHOSTBYADDR_R 1 - -/* Define to 1 if you have the `gethostbyname_r' function. */ -#define _EVENT_HAVE_GETHOSTBYNAME_R 1 - -/* Solaris define gethostbyname_r with 5 arguments. glibc2 defines this with 6 - arguments */ -#define _EVENT_HAVE_GETHOSTBYNAME_R_GLIBC2_STYLE 1 - -/* In OSF 4.0f the 3'd argument to gethostbyname_r is hostent_data * */ -/* #undef _EVENT_HAVE_GETHOSTBYNAME_R_RETURN_INT */ - -/* Define to 1 if you have the `gethrtime' function. */ -/* #undef _EVENT_HAVE_GETHRTIME */ - -/* Define to 1 if you have the `getline' function. */ -#define _EVENT_HAVE_GETLINE 1 - -/* Define to 1 if you have the `getnameinfo' function. */ -#define _EVENT_HAVE_GETNAMEINFO 1 - -/* Define to 1 if you have the `getpagesize' function. */ -#define _EVENT_HAVE_GETPAGESIZE 1 - -/* Define to 1 if you have the `getpass' function. */ -#define _EVENT_HAVE_GETPASS 1 - -/* Define to 1 if you have the `getpassphrase' function. */ -/* #undef _EVENT_HAVE_GETPASSPHRASE */ - -/* Define to 1 if you have the `getpwnam' function. */ -#define _EVENT_HAVE_GETPWNAM 1 - -/* Define to 1 if you have the `getpwuid' function. */ -#define _EVENT_HAVE_GETPWUID 1 - -/* getpwent() declaration present */ -/* #undef _EVENT_HAVE_GETPW_DECLS */ - -/* Define to 1 if you have the `getrlimit' function. */ -#define _EVENT_HAVE_GETRLIMIT 1 - -/* Define to 1 if you have the `getrusage' function. */ -#define _EVENT_HAVE_GETRUSAGE 1 - -/* Define to 1 if you have the `gettimeofday' function. */ -#define _EVENT_HAVE_GETTIMEOFDAY 1 - -/* Define to 1 if you have the `getwd' function. */ -#define _EVENT_HAVE_GETWD 1 - -/* Define to 1 if you have the `gmtime_r' function. */ -#define _EVENT_HAVE_GMTIME_R 1 - -/* Define to 1 if you have the <grp.h> header file. */ -#define _EVENT_HAVE_GRP_H 1 - -/* HIST_ENTRY is defined in the outer libeditreadline */ -/* #undef _EVENT_HAVE_HIST_ENTRY */ - -/* Define to 1 if you have the <ieeefp.h> header file. */ -/* #undef _EVENT_HAVE_IEEEFP_H */ - -/* Define to 1 if you have the `index' function. */ -#define _EVENT_HAVE_INDEX 1 - -/* Define to 1 if you have the `inet_ntop' function. */ -#define _EVENT_HAVE_INET_NTOP 1 - -/* Define to 1 if you have the `initgroups' function. */ -#define _EVENT_HAVE_INITGROUPS 1 - -/* Define to 1 if the system has the type `int16'. */ -/* #undef _EVENT_HAVE_INT16 */ - -/* Define to 1 if the system has the type `int32'. */ -/* #undef _EVENT_HAVE_INT32 */ - -/* Define to 1 if the system has the type `int64'. */ -/* #undef _EVENT_HAVE_INT64 */ - -/* Define to 1 if the system has the type `int8'. */ -/* #undef _EVENT_HAVE_INT8 */ - -/* Define to 1 if you have the <inttypes.h> header file. */ -#define _EVENT_HAVE_INTTYPES_H 1 - -/* Define to 1 if the system has the type `in_addr_t'. */ -#define _EVENT_HAVE_IN_ADDR_T 1 - -/* isinf() macro or function */ -#define _EVENT_HAVE_ISINF 1 - -/* Define to 1 if you have the `isnan' function. */ -#define _EVENT_HAVE_ISNAN 1 - -/* Define to 1 if you have the `issetugid' function. */ -/* #undef _EVENT_HAVE_ISSETUGID */ - -/* Define to 1 if you have the `kqueue' function. */ -/* #undef _EVENT_HAVE_KQUEUE */ - -/* Define if mysql_cv_langinfo_codeset=yes */ -#define _EVENT_HAVE_LANGINFO_CODESET - -/* Define to 1 if you have the <langinfo.h> header file. */ -#define _EVENT_HAVE_LANGINFO_H 1 - -/* Define if you have large pages support */ -#define _EVENT_HAVE_LARGE_PAGES 1 - -/* Define to 1 if you have the `bind' library (-lbind). */ -/* #undef _EVENT_HAVE_LIBBIND */ - -/* Define to 1 if you have the `compat' library (-lcompat). */ -/* #undef _EVENT_HAVE_LIBCOMPAT */ - -/* Define to 1 if you have the `c_r' library (-lc_r). */ -/* #undef _EVENT_HAVE_LIBC_R */ - -/* Define to 1 if you have the `dl' library (-ldl). */ -#define _EVENT_HAVE_LIBDL 1 - -/* If we want to use libevent and have connection pooling */ -#define _EVENT_HAVE_LIBEVENT 1 - -/* Define to 1 if you have the `gen' library (-lgen). */ -/* #undef _EVENT_HAVE_LIBGEN */ - -/* Define to 1 if you have the `m' library (-lm). */ -#define _EVENT_HAVE_LIBM 1 - -/* Define to 1 if you have the `nsl' library (-lnsl). */ -#define _EVENT_HAVE_LIBNSL 1 - -/* Define to 1 if you have the `nsl_r' library (-lnsl_r). */ -/* #undef _EVENT_HAVE_LIBNSL_R */ - -/* Define to 1 if you have the `posix4' library (-lposix4). */ -/* #undef _EVENT_HAVE_LIBPOSIX4 */ - -/* Define to 1 if you have the `pthread' library (-lpthread). */ -#define _EVENT_HAVE_LIBPTHREAD 1 - -/* Define to 1 if you have the `resolv' library (-lresolv). */ -#define _EVENT_HAVE_LIBRESOLV 1 - -/* Define to 1 if you have the `rt' library (-lrt). */ -#define _EVENT_HAVE_LIBRT 1 - -/* Define to 1 if you have the `socket' library (-lsocket). */ -/* #undef _EVENT_HAVE_LIBSOCKET */ - -/* Define if have -lwrap */ -/* #undef _EVENT_HAVE_LIBWRAP */ - -/* Define to 1 if you have the <limits.h> header file. */ -#define _EVENT_HAVE_LIMITS_H 1 - -/* Whether we are using Xavier Leroy's LinuxThreads */ -/* #undef _EVENT_HAVE_LINUXTHREADS */ - -/* Define to 1 if you have the <linux/config.h> header file. */ -/* #undef _EVENT_HAVE_LINUX_CONFIG_H */ - -/* Define to 1 if you have the <locale.h> header file. */ -#define _EVENT_HAVE_LOCALE_H 1 - -/* Define to 1 if you have the `localtime_r' function. */ -#define _EVENT_HAVE_LOCALTIME_R 1 - -/* Define to 1 if you have the `locking' function. */ -/* #undef _EVENT_HAVE_LOCKING */ - -/* Define to 1 if you have the `longjmp' function. */ -#define _EVENT_HAVE_LONGJMP 1 - -/* Define to 1 if you have the `lrand48' function. */ -#define _EVENT_HAVE_LRAND48 1 - -/* Define to 1 if you have the `lstat' function. */ -#define _EVENT_HAVE_LSTAT 1 - -/* Define to 1 if you have the `madvise' function. */ -#define _EVENT_HAVE_MADVISE 1 - -/* Define to 1 if you have the `mallinfo' function. */ -#define _EVENT_HAVE_MALLINFO 1 - -/* Define to 1 if you have the <malloc.h> header file. */ -#define _EVENT_HAVE_MALLOC_H 1 - -/* Define if you have mbrlen */ -#define _EVENT_HAVE_MBRLEN - -/* Define if you have mbrtowc */ -#define _EVENT_HAVE_MBRTOWC - -/* Define if you have mbsrtowcs */ -#define _EVENT_HAVE_MBSRTOWCS - -/* Define if mysql_cv_have_mbstate_t=yes */ -#define _EVENT_HAVE_MBSTATE_T - -/* Define to 1 if you have the `memcpy' function. */ -#define _EVENT_HAVE_MEMCPY 1 - -/* Define to 1 if you have the `memmove' function. */ -#define _EVENT_HAVE_MEMMOVE 1 - -/* Define to 1 if you have the <memory.h> header file. */ -#define _EVENT_HAVE_MEMORY_H 1 - -/* Define to 1 if you have the `mkstemp' function. */ -#define _EVENT_HAVE_MKSTEMP 1 - -/* Define to 1 if you have the `mlockall' function. */ -#define _EVENT_HAVE_MLOCKALL 1 - -/* Define to 1 if you have the `mmap' function. */ -#define _EVENT_HAVE_MMAP 1 - -/* Define to 1 if you have the `mmap64' function. */ -#define _EVENT_HAVE_MMAP64 1 - -/* Define to 1 if you have the <ndir.h> header file. */ -/* #undef _EVENT_HAVE_NDIR_H */ - -/* Define to 1 if you have the <netinet/in6.h> header file. */ -/* #undef _EVENT_HAVE_NETINET_IN6_H */ - -/* Define to 1 if you have the <netinet/in.h> header file. */ -#define _EVENT_HAVE_NETINET_IN_H 1 - -/* For some non posix threads */ -/* #undef _EVENT_HAVE_NONPOSIX_PTHREAD_GETSPECIFIC */ - -/* For some non posix threads */ -/* #undef _EVENT_HAVE_NONPOSIX_PTHREAD_MUTEX_INIT */ - -/* sigwait with one argument */ -/* #undef _EVENT_HAVE_NONPOSIX_SIGWAIT */ - -/* NPTL threads implementation */ -#define _EVENT_HAVE_NPTL 1 - -/* Define to 1 if the system has the type `off_t'. */ -#define _EVENT_HAVE_OFF_T 1 - -/* OpenSSL */ -#define _EVENT_HAVE_OPENSSL 1 - -/* Define to 1 if you have the <paths.h> header file. */ -#define _EVENT_HAVE_PATHS_H 1 - -/* Define to 1 if you have the `perror' function. */ -#define _EVENT_HAVE_PERROR 1 - -/* Define to 1 if you have the `poll' function. */ -#define _EVENT_HAVE_POLL 1 - -/* Define to 1 if you have the <poll.h> header file. */ -#define _EVENT_HAVE_POLL_H 1 - -/* Define to 1 if you have the `port_create' function. */ -/* #undef _EVENT_HAVE_PORT_CREATE */ - -/* Define to 1 if you have the <port.h> header file. */ -/* #undef _EVENT_HAVE_PORT_H */ - -/* Define to 1 if you have the `posix_fallocate' function. */ -#define _EVENT_HAVE_POSIX_FALLOCATE 1 - -/* Signal handling is POSIX (sigset/sighold, etc) */ -#define _EVENT_HAVE_POSIX_SIGNALS 1 - -/* Define to 1 if you have the `pread' function. */ -#define _EVENT_HAVE_PREAD 1 - -/* Define to 1 if you have the `pthread_attr_create' function. */ -/* #undef _EVENT_HAVE_PTHREAD_ATTR_CREATE */ - -/* Define to 1 if you have the `pthread_attr_getstacksize' function. */ -#define _EVENT_HAVE_PTHREAD_ATTR_GETSTACKSIZE 1 - -/* Define to 1 if you have the `pthread_attr_setprio' function. */ -/* #undef _EVENT_HAVE_PTHREAD_ATTR_SETPRIO */ - -/* Define to 1 if you have the `pthread_attr_setschedparam' function. */ -#define _EVENT_HAVE_PTHREAD_ATTR_SETSCHEDPARAM 1 - -/* pthread_attr_setscope */ -#define _EVENT_HAVE_PTHREAD_ATTR_SETSCOPE 1 - -/* Define to 1 if you have the `pthread_attr_setstacksize' function. */ -#define _EVENT_HAVE_PTHREAD_ATTR_SETSTACKSIZE 1 - -/* Define to 1 if you have the `pthread_condattr_create' function. */ -/* #undef _EVENT_HAVE_PTHREAD_CONDATTR_CREATE */ - -/* Define to 1 if you have the `pthread_getsequence_np' function. */ -/* #undef _EVENT_HAVE_PTHREAD_GETSEQUENCE_NP */ - -/* Define to 1 if you have the `pthread_init' function. */ -/* #undef _EVENT_HAVE_PTHREAD_INIT */ - -/* Define to 1 if you have the `pthread_key_delete' function. */ -#define _EVENT_HAVE_PTHREAD_KEY_DELETE 1 - -/* Define to 1 if you have the `pthread_rwlock_rdlock' function. */ -#define _EVENT_HAVE_PTHREAD_RWLOCK_RDLOCK 1 - -/* Define to 1 if you have the `pthread_setprio' function. */ -/* #undef _EVENT_HAVE_PTHREAD_SETPRIO */ - -/* Define to 1 if you have the `pthread_setprio_np' function. */ -/* #undef _EVENT_HAVE_PTHREAD_SETPRIO_NP */ - -/* Define to 1 if you have the `pthread_setschedparam' function. */ -#define _EVENT_HAVE_PTHREAD_SETSCHEDPARAM 1 - -/* Define to 1 if you have the `pthread_setschedprio' function. */ -#define _EVENT_HAVE_PTHREAD_SETSCHEDPRIO 1 - -/* Define to 1 if you have the `pthread_sigmask' function. */ -#define _EVENT_HAVE_PTHREAD_SIGMASK 1 - -/* pthread_yield function with one argument */ -/* #undef _EVENT_HAVE_PTHREAD_YIELD_ONE_ARG */ - -/* pthread_yield that doesn't take any arguments */ -#define _EVENT_HAVE_PTHREAD_YIELD_ZERO_ARG 1 - -/* Define to 1 if you have the `putenv' function. */ -#define _EVENT_HAVE_PUTENV 1 - -/* Define to 1 if you have the <pwd.h> header file. */ -#define _EVENT_HAVE_PWD_H 1 - -/* If we want to have query cache */ -#define _EVENT_HAVE_QUERY_CACHE 1 - -/* POSIX readdir_r */ -#define _EVENT_HAVE_READDIR_R 1 - -/* Define to 1 if you have the `readlink' function. */ -#define _EVENT_HAVE_READLINK 1 - -/* Define to 1 if you have the `realpath' function. */ -#define _EVENT_HAVE_REALPATH 1 - -/* Define to 1 if you have the `regcomp' function. */ -#define _EVENT_HAVE_REGCOMP 1 - -/* Define to 1 if you have the `rename' function. */ -#define _EVENT_HAVE_RENAME 1 - -/* Define to 1 if system calls automatically restart after interruption by a - signal. */ -#define _EVENT_HAVE_RESTARTABLE_SYSCALLS 1 - -/* Define to 1 if you have the `re_comp' function. */ -#define _EVENT_HAVE_RE_COMP 1 - -/* Define to 1 if you have the `rint' function. */ -#define _EVENT_HAVE_RINT 1 - -/* RTree keys */ -#define _EVENT_HAVE_RTREE_KEYS 1 - -/* Define to 1 if you have the `rwlock_init' function. */ -/* #undef _EVENT_HAVE_RWLOCK_INIT */ - -/* Define to 1 if you have the <sched.h> header file. */ -#define _EVENT_HAVE_SCHED_H 1 - -/* Define to 1 if you have the `sched_yield' function. */ -#define _EVENT_HAVE_SCHED_YIELD 1 - -/* Define to 1 if you have the `select' function. */ -#define _EVENT_HAVE_SELECT 1 - -/* Define to 1 if you have the <select.h> header file. */ -/* #undef _EVENT_HAVE_SELECT_H */ - -/* Define to 1 if you have the <semaphore.h> header file. */ -#define _EVENT_HAVE_SEMAPHORE_H 1 - -/* Define to 1 if you have the `setenv' function. */ -#define _EVENT_HAVE_SETENV 1 - -/* Define if F_SETFD is defined in <fcntl.h> */ -#define _EVENT_HAVE_SETFD 1 - -/* Define to 1 if you have the `setlocale' function. */ -#define _EVENT_HAVE_SETLOCALE 1 - -/* Define to 1 if you have the `setupterm' function. */ -/* #undef _EVENT_HAVE_SETUPTERM */ - -/* Define to 1 if you have the `shmat' function. */ -#define _EVENT_HAVE_SHMAT 1 - -/* Define to 1 if you have the `shmctl' function. */ -#define _EVENT_HAVE_SHMCTL 1 - -/* Define to 1 if you have the `shmdt' function. */ -#define _EVENT_HAVE_SHMDT 1 - -/* Define to 1 if you have the `shmget' function. */ -#define _EVENT_HAVE_SHMGET 1 - -/* Define to 1 if you have the `sigaction' function. */ -#define _EVENT_HAVE_SIGACTION 1 - -/* Define to 1 if you have the `sigaddset' function. */ -#define _EVENT_HAVE_SIGADDSET 1 - -/* Define to 1 if you have the `sigemptyset' function. */ -#define _EVENT_HAVE_SIGEMPTYSET 1 - -/* Define to 1 if you have the `sighold' function. */ -#define _EVENT_HAVE_SIGHOLD 1 - -/* Define to 1 if you have the `signal' function. */ -#define _EVENT_HAVE_SIGNAL 1 - -/* Define to 1 if you have the <signal.h> header file. */ -#define _EVENT_HAVE_SIGNAL_H 1 - -/* Define to 1 if you have the `sigset' function. */ -#define _EVENT_HAVE_SIGSET 1 - -/* Define to 1 if the system has the type `sigset_t'. */ -#define _EVENT_HAVE_SIGSET_T 1 - -/* Define to 1 if you have the `sigthreadmask' function. */ -/* #undef _EVENT_HAVE_SIGTHREADMASK */ - -/* POSIX sigwait */ -#define _EVENT_HAVE_SIGWAIT 1 - -/* Define to 1 if the system has the type `size_t'. */ -#define _EVENT_HAVE_SIZE_T 1 - -/* Define to 1 if you have the `sleep' function. */ -#define _EVENT_HAVE_SLEEP 1 - -/* Define to 1 if you have the `snprintf' function. */ -#define _EVENT_HAVE_SNPRINTF 1 - -/* Define to 1 if you have the `socket' function. */ -#define _EVENT_HAVE_SOCKET 1 - -/* Solaris define gethostbyaddr_r with 7 arguments. glibc2 defines this with 8 - arguments */ -/* #undef _EVENT_HAVE_SOLARIS_STYLE_GETHOST */ - -/* Spatial extentions */ -#define _EVENT_HAVE_SPATIAL 1 - -/* Define to 1 if you have the <stdarg.h> header file. */ -#define _EVENT_HAVE_STDARG_H 1 - -/* Define to 1 if you have the <stddef.h> header file. */ -#define _EVENT_HAVE_STDDEF_H 1 - -/* Define to 1 if you have the <stdint.h> header file. */ -#define _EVENT_HAVE_STDINT_H 1 - -/* Define to 1 if you have the <stdlib.h> header file. */ -#define _EVENT_HAVE_STDLIB_H 1 - -/* Define to 1 if you have the `stpcpy' function. */ -#define _EVENT_HAVE_STPCPY 1 - -/* Define to 1 if you have the `strcasecmp' function. */ -#define _EVENT_HAVE_STRCASECMP 1 - -/* Define to 1 if you have the `strcoll' function. */ -#define _EVENT_HAVE_STRCOLL 1 - -/* Define to 1 if you have the `strdup' function. */ -#define _EVENT_HAVE_STRDUP 1 - -/* Define to 1 if you have the `strerror' function. */ -#define _EVENT_HAVE_STRERROR 1 - -/* Define to 1 if you have the <strings.h> header file. */ -#define _EVENT_HAVE_STRINGS_H 1 - -/* Define to 1 if you have the <string.h> header file. */ -#define _EVENT_HAVE_STRING_H 1 - -/* Define to 1 if you have the `strlcat' function. */ -/* #undef _EVENT_HAVE_STRLCAT */ - -/* Define to 1 if you have the `strlcpy' function. */ -/* #undef _EVENT_HAVE_STRLCPY */ - -/* Define to 1 if you have the `strnlen' function. */ -#define _EVENT_HAVE_STRNLEN 1 - -/* Define to 1 if you have the `strpbrk' function. */ -#define _EVENT_HAVE_STRPBRK 1 - -/* Define to 1 if you have the `strsep' function. */ -#define _EVENT_HAVE_STRSEP 1 - -/* Define to 1 if you have the `strsignal' function. */ -#define _EVENT_HAVE_STRSIGNAL 1 - -/* Define to 1 if you have the `strstr' function. */ -#define _EVENT_HAVE_STRSTR 1 - -/* Define to 1 if you have the `strtok_r' function. */ -#define _EVENT_HAVE_STRTOK_R 1 - -/* Define to 1 if you have the `strtol' function. */ -#define _EVENT_HAVE_STRTOL 1 - -/* Define to 1 if you have the `strtoll' function. */ -#define _EVENT_HAVE_STRTOLL 1 - -/* Define to 1 if you have the `strtoul' function. */ -#define _EVENT_HAVE_STRTOUL 1 - -/* Define to 1 if you have the `strtoull' function. */ -#define _EVENT_HAVE_STRTOULL 1 - -/* Define to 1 if the system has the type `struct in6_addr'. */ -#define _EVENT_HAVE_STRUCT_IN6_ADDR 1 - -/* Define to 1 if `st_rdev' is member of `struct stat'. */ -#define _EVENT_HAVE_STRUCT_STAT_ST_RDEV 1 - -/* Define to 1 if your `struct stat' has `st_rdev'. Deprecated, use - `HAVE_STRUCT_STAT_ST_RDEV' instead. */ -#define _EVENT_HAVE_ST_RDEV 1 - -/* Define to 1 if you have the <synch.h> header file. */ -/* #undef _EVENT_HAVE_SYNCH_H */ - -/* Define to 1 if you have the <sys/cdefs.h> header file. */ -#define _EVENT_HAVE_SYS_CDEFS_H 1 - -/* Define to 1 if you have the <sys/devpoll.h> header file. */ -/* #undef _EVENT_HAVE_SYS_DEVPOLL_H */ - -/* Define to 1 if you have the <sys/dir.h> header file. */ -#define _EVENT_HAVE_SYS_DIR_H 1 - -/* Define to 1 if you have the <sys/epoll.h> header file. */ -#define _EVENT_HAVE_SYS_EPOLL_H 1 - -/* Define to 1 if you have the <sys/event.h> header file. */ -/* #undef _EVENT_HAVE_SYS_EVENT_H */ - -/* Define to 1 if you have the <sys/file.h> header file. */ -#define _EVENT_HAVE_SYS_FILE_H 1 - -/* Define to 1 if you have the <sys/ioctl.h> header file. */ -#define _EVENT_HAVE_SYS_IOCTL_H 1 - -/* Define to 1 if you have the <sys/ipc.h> header file. */ -#define _EVENT_HAVE_SYS_IPC_H 1 - -/* Define to 1 if you have the <sys/malloc.h> header file. */ -/* #undef _EVENT_HAVE_SYS_MALLOC_H */ - -/* Define to 1 if you have the <sys/mman.h> header file. */ -#define _EVENT_HAVE_SYS_MMAN_H 1 - -/* Define to 1 if you have the <sys/ndir.h> header file. */ -/* #undef _EVENT_HAVE_SYS_NDIR_H */ - -/* Define to 1 if you have the <sys/param.h> header file. */ -#define _EVENT_HAVE_SYS_PARAM_H 1 - -/* Define to 1 if you have the <sys/prctl.h> header file. */ -#define _EVENT_HAVE_SYS_PRCTL_H 1 - -/* Define to 1 if you have the <sys/ptem.h> header file. */ -/* #undef _EVENT_HAVE_SYS_PTEM_H */ - -/* Define to 1 if you have the <sys/pte.h> header file. */ -/* #undef _EVENT_HAVE_SYS_PTE_H */ - -/* Define to 1 if you have the <sys/queue.h> header file. */ -#define _EVENT_HAVE_SYS_QUEUE_H 1 - -/* Define to 1 if you have the <sys/resource.h> header file. */ -#define _EVENT_HAVE_SYS_RESOURCE_H 1 - -/* Define to 1 if you have the <sys/select.h> header file. */ -#define _EVENT_HAVE_SYS_SELECT_H 1 - -/* Define to 1 if you have the <sys/shm.h> header file. */ -#define _EVENT_HAVE_SYS_SHM_H 1 - -/* Define to 1 if you have the <sys/socket.h> header file. */ -#define _EVENT_HAVE_SYS_SOCKET_H 1 - -/* Define to 1 if you have the <sys/stat.h> header file. */ -#define _EVENT_HAVE_SYS_STAT_H 1 - -/* Define to 1 if you have the <sys/stream.h> header file. */ -/* #undef _EVENT_HAVE_SYS_STREAM_H */ - -/* Define to 1 if you have the <sys/timeb.h> header file. */ -#define _EVENT_HAVE_SYS_TIMEB_H 1 - -/* Define to 1 if you have the <sys/time.h> header file. */ -#define _EVENT_HAVE_SYS_TIME_H 1 - -/* Define to 1 if you have the <sys/types.h> header file. */ -#define _EVENT_HAVE_SYS_TYPES_H 1 - -/* Define to 1 if you have the <sys/un.h> header file. */ -#define _EVENT_HAVE_SYS_UN_H 1 - -/* Define to 1 if you have the <sys/utime.h> header file. */ -/* #undef _EVENT_HAVE_SYS_UTIME_H */ - -/* Define to 1 if you have the <sys/vadvise.h> header file. */ -/* #undef _EVENT_HAVE_SYS_VADVISE_H */ - -/* Define to 1 if you have the <sys/wait.h> header file. */ -#define _EVENT_HAVE_SYS_WAIT_H 1 - -/* Define if TAILQ_FOREACH is defined in <sys/queue.h> */ -#define _EVENT_HAVE_TAILQFOREACH 1 - -/* Define to 1 if you have the `tcgetattr' function. */ -#define _EVENT_HAVE_TCGETATTR 1 - -/* Define to 1 if you have the `tell' function. */ -/* #undef _EVENT_HAVE_TELL */ - -/* Define to 1 if you have the `tempnam' function. */ -#define _EVENT_HAVE_TEMPNAM 1 - -/* Define to 1 if you have the <termbits.h> header file. */ -/* #undef _EVENT_HAVE_TERMBITS_H */ - -/* Define to 1 if you have the <termcap.h> header file. */ -#define _EVENT_HAVE_TERMCAP_H 1 - -/* Define to 1 if you have the <termios.h> header file. */ -#define _EVENT_HAVE_TERMIOS_H 1 - -/* Define to 1 if you have the <termio.h> header file. */ -#define _EVENT_HAVE_TERMIO_H 1 - -/* Define to 1 if you have the <term.h> header file. */ -#define _EVENT_HAVE_TERM_H 1 - -/* Define to 1 if you have the `thr_setconcurrency' function. */ -/* #undef _EVENT_HAVE_THR_SETCONCURRENCY */ - -/* Define to 1 if you have the `thr_yield' function. */ -/* #undef _EVENT_HAVE_THR_YIELD */ - -/* Define if timeradd is defined in <sys/time.h> */ -#define _EVENT_HAVE_TIMERADD 1 - -/* Define if timerclear is defined in <sys/time.h> */ -#define _EVENT_HAVE_TIMERCLEAR 1 - -/* Define if timercmp is defined in <sys/time.h> */ -#define _EVENT_HAVE_TIMERCMP 1 - -/* Define if timerisset is defined in <sys/time.h> */ -#define _EVENT_HAVE_TIMERISSET 1 - -/* Timespec has a ts_sec instead of tv_sev */ -/* #undef _EVENT_HAVE_TIMESPEC_TS_SEC */ - -/* Have the tzname variable */ -#define _EVENT_HAVE_TZNAME 1 - -/* national Unicode collations */ -#define _EVENT_HAVE_UCA_COLLATIONS 1 - -/* Define to 1 if the system has the type `uchar'. */ -/* #undef _EVENT_HAVE_UCHAR */ - -/* Define to 1 if the system has the type `uint'. */ -#define _EVENT_HAVE_UINT 1 - -/* Define to 1 if the system has the type `uint16'. */ -/* #undef _EVENT_HAVE_UINT16 */ - -/* Define to 1 if the system has the type `uint16_t'. */ -#define _EVENT_HAVE_UINT16_T 1 - -/* Define to 1 if the system has the type `uint32'. */ -/* #undef _EVENT_HAVE_UINT32 */ - -/* Define to 1 if the system has the type `uint32_t'. */ -#define _EVENT_HAVE_UINT32_T 1 - -/* Define to 1 if the system has the type `uint64'. */ -/* #undef _EVENT_HAVE_UINT64 */ - -/* Define to 1 if the system has the type `uint64_t'. */ -#define _EVENT_HAVE_UINT64_T 1 - -/* Define to 1 if the system has the type `uint8'. */ -/* #undef _EVENT_HAVE_UINT8 */ - -/* Define to 1 if the system has the type `uint8_t'. */ -#define _EVENT_HAVE_UINT8_T 1 - -/* Define to 1 if the system has the type `ulong'. */ -#define _EVENT_HAVE_ULONG 1 - -/* Define to 1 if you have the <unistd.h> header file. */ -#define _EVENT_HAVE_UNISTD_H 1 - -/* Have UnixWare 7 (or similar) almost-POSIX threading library */ -/* #undef _EVENT_HAVE_UNIXWARE7_THREADS */ - -/* sighold() is present and usable */ -/* #undef _EVENT_HAVE_USG_SIGHOLD */ - -/* certain Japanese customer */ -/* #undef _EVENT_HAVE_UTF8_GENERAL_CS */ - -/* Define to 1 if you have the <utime.h> header file. */ -#define _EVENT_HAVE_UTIME_H 1 - -/* Define to 1 if `utime(file, NULL)' sets file's timestamp to the present. */ -#define _EVENT_HAVE_UTIME_NULL 1 - -/* Define to 1 if the system has the type `u_int32_t'. */ -#define _EVENT_HAVE_U_INT32_T 1 - -/* Define to 1 if you have the <varargs.h> header file. */ -/* #undef _EVENT_HAVE_VARARGS_H */ - -/* Define to 1 if you have the `vasprintf' function. */ -#define _EVENT_HAVE_VASPRINTF 1 - -/* Define to 1 if you have the `vidattr' function. */ -/* #undef _EVENT_HAVE_VIDATTR */ - -/* Define to enable buffered read. This works only if syscalls read/recv - return as soon as there is some data in the kernel buffer, no matter how - big the given buffer is. */ -#define _EVENT_HAVE_VIO_READ_BUFF 1 - -/* Found vis.h and the strvis() function */ -/* #undef _EVENT_HAVE_VIS_H */ - -/* Define to 1 if you have the `vprintf' function. */ -#define _EVENT_HAVE_VPRINTF 1 - -/* Define to 1 if you have the <wchar.h> header file. */ -#define _EVENT_HAVE_WCHAR_H 1 - -/* Define if you check wcsdup */ -#define _EVENT_HAVE_WCSDUP - -/* Define if you have wctomb */ -#define _EVENT_HAVE_WCTOMB - -/* Define to 1 if you have the <wctype.h> header file. */ -#define _EVENT_HAVE_WCTYPE_H 1 - -/* Define if you have wcwidth */ -#define _EVENT_HAVE_WCWIDTH - -/* Define to 1 if compiler supports weak symbol attribute. */ -#define _EVENT_HAVE_WEAK_SYMBOL 1 - -/* Define if kqueue works correctly with pipes */ -/* #undef _EVENT_HAVE_WORKING_KQUEUE */ - -/* Define to 1 if you have the <xfs/xfs.h> header file. */ -/* #undef _EVENT_HAVE_XFS_XFS_H */ - -/* Defined by configure. Using yaSSL for SSL. */ -#define _EVENT_HAVE_YASSL 1 - -/* Define if /proc/meminfo shows the huge page size (Linux only) */ -#define _EVENT_HUGETLB_USE_PROC_MEMINFO 1 - -/* Define if you have -lwrap */ -/* #undef _EVENT_LIBWRAP */ - -/* Machine type name, eg sparc */ -#define _EVENT_MACHINE_TYPE "x86_64" - -/* Maximum number of indexes per table */ -#define _EVENT_MAX_INDEXES 64 - -/* Define the default charset name */ -#define _EVENT_MYSQL_DEFAULT_CHARSET_NAME "latin1" - -/* Define the default charset name */ -#define _EVENT_MYSQL_DEFAULT_COLLATION_NAME "latin1_swedish_ci" - -/* Assume single-CPU mode, no concurrency */ -/* #undef _EVENT_MY_ATOMIC_MODE_DUMMY */ - -/* Use pthread rwlocks for atomic ops */ -/* #undef _EVENT_MY_ATOMIC_MODE_RWLOCKS */ - -/* Define to 1 if you want to use fast mutexes */ -/* #undef _EVENT_MY_PTHREAD_FASTMUTEX */ - -/* Including Ndb Cluster DB sci transporter */ -/* #undef _EVENT_NDB_SCI_TRANSPORTER */ - -/* Including Ndb Cluster DB shared memory transporter */ -/* #undef _EVENT_NDB_SHM_TRANSPORTER */ - -/* NDB build version */ -/* #undef _EVENT_NDB_VERSION_BUILD */ - -/* NDB major version */ -/* #undef _EVENT_NDB_VERSION_MAJOR */ - -/* NDB minor version */ -/* #undef _EVENT_NDB_VERSION_MINOR */ - -/* NDB status version */ -/* #undef _EVENT_NDB_VERSION_STATUS */ - -/* Name of package */ -#define _EVENT_PACKAGE "mysql" - -/* Define to the address where bug reports for this package should be sent. */ -#define _EVENT_PACKAGE_BUGREPORT "" - -/* Define to the full name of this package. */ -#define _EVENT_PACKAGE_NAME "" - -/* Define to the full name and version of this package. */ -#define _EVENT_PACKAGE_STRING "" - -/* Define to the one symbol short name of this package. */ -#define _EVENT_PACKAGE_TARNAME "" - -/* Define to the version of this package. */ -#define _EVENT_PACKAGE_VERSION "" - -/* mysql client protocol version */ -#define _EVENT_PROTOCOL_VERSION 10 - -/* qsort returns void */ -#define _EVENT_QSORT_TYPE_IS_VOID 1 - -/* The return type of qsort (int or void). */ -#define _EVENT_RETQSORTTYPE void - -/* Define as the return type of signal handlers (`int' or `void'). */ -#define _EVENT_RETSIGTYPE void - -/* The size of `char', as computed by sizeof. */ -#define _EVENT_SIZEOF_CHAR 1 - -/* The size of `char*', as computed by sizeof. */ -#define _EVENT_SIZEOF_CHARP 8 - -/* The size of `int', as computed by sizeof. */ -#define _EVENT_SIZEOF_INT 4 - -/* The size of `long', as computed by sizeof. */ -#define _EVENT_SIZEOF_LONG 8 - -/* The size of `long long', as computed by sizeof. */ -#define _EVENT_SIZEOF_LONG_LONG 8 - -/* The size of `off_t', as computed by sizeof. */ -#define _EVENT_SIZEOF_OFF_T 8 - -/* The size of `short', as computed by sizeof. */ -#define _EVENT_SIZEOF_SHORT 2 - -/* The size of `void*', as computed by sizeof. */ -#define _EVENT_SIZEOF_VOIDP 8 - -/* The base type of the last arg to accept */ -#define _EVENT_SOCKET_SIZE_TYPE socklen_t - -/* Last argument to get/setsockopt */ -/* #undef _EVENT_SOCKOPT_OPTLEN_TYPE */ - -/* Broken sprintf */ -/* #undef _EVENT_SPRINTF_RETURNS_GARBAGE */ - -/* POSIX sprintf */ -#define _EVENT_SPRINTF_RETURNS_INT 1 - -/* Broken sprintf */ -/* #undef _EVENT_SPRINTF_RETURNS_PTR */ - -/* If using the C implementation of alloca, define if you know the - direction of stack growth for your system; otherwise it will be - automatically deduced at runtime. - STACK_DIRECTION > 0 => grows toward higher addresses - STACK_DIRECTION < 0 => grows toward lower addresses - STACK_DIRECTION = 0 => direction of growth unknown */ -#define _EVENT_STACK_DIRECTION -1 - -/* Define to 1 if the `S_IS*' macros in <sys/stat.h> do not work properly. */ -/* #undef _EVENT_STAT_MACROS_BROKEN */ - -/* Define to 1 if you have the ANSI C header files. */ -#define _EVENT_STDC_HEADERS 1 - -/* d_ino member present in struct dirent */ -#define _EVENT_STRUCT_DIRENT_HAS_D_INO 1 - -/* d_namlen member present in struct dirent */ -/* #undef _EVENT_STRUCT_DIRENT_HAS_D_NAMLEN */ - -/* The struct rlimit type to use with setrlimit */ -#define _EVENT_STRUCT_RLIMIT struct rlimit - -/* Name of system, eg sun-solaris */ -#define _EVENT_SYSTEM_TYPE "suse-linux-gnu" - -/* Whether we build for Linux */ -#define _EVENT_TARGET_OS_LINUX 1 - -/* Define if you want to have threaded code. This may be undef on client code - */ -#define _EVENT_THREAD 1 - -/* Should the client be thread safe */ -#define _EVENT_THREAD_SAFE_CLIENT 1 - -/* Define to 1 if time_t is unsigned */ -/* #undef _EVENT_TIME_T_UNSIGNED */ - -/* Define to 1 if you can safely include both <sys/time.h> and <time.h>. */ -#define _EVENT_TIME_WITH_SYS_TIME 1 - -/* declaration of TIOCSTAT in sys/ioctl.h */ -/* #undef _EVENT_TIOCSTAT_IN_SYS_IOCTL */ - -/* Define to 1 if your <sys/time.h> declares `struct tm'. */ -/* #undef _EVENT_TM_IN_SYS_TIME */ - -/* used libedit interface (can we dereference result of - rl_completion_entry_function) */ -/* #undef _EVENT_USE_LIBEDIT_INTERFACE */ - -/* Maria is used for internal temporary tables */ -#define _EVENT_USE_MARIA_FOR_TMP_TABLES 1 - -/* Use multi-byte character routines */ -#define _EVENT_USE_MB 1 - -/* */ -#define _EVENT_USE_MB_IDENT 1 - -/* Needs to use mysys_new helpers */ -#define _EVENT_USE_MYSYS_NEW 1 - -/* used new readline interface (are rl_completion_func_t and - rl_compentry_func_t defined) */ -#define _EVENT_USE_NEW_READLINE_INTERFACE 1 - -/* the pstack backtrace library */ -/* #undef _EVENT_USE_PSTACK */ - -/* Version number of package */ -#define _EVENT_VERSION "5.1.32-maria-beta" - -/* sighandler type is void (*signal ()) (); */ -#define _EVENT_VOID_SIGHANDLER 1 - -/* Include Archive Storage Engine into mysqld */ -#define _EVENT_WITH_ARCHIVE_STORAGE_ENGINE 1 - -/* Include Basic Write-only Read-never tables into mysqld */ -#define _EVENT_WITH_BLACKHOLE_STORAGE_ENGINE 1 - -/* Include Stores tables in text CSV format into mysqld */ -#define _EVENT_WITH_CSV_STORAGE_ENGINE 1 - -/* Include Example for Storage Engines for developers into mysqld */ -/* #undef _EVENT_WITH_EXAMPLE_STORAGE_ENGINE */ - -/* Include Connects to tables on remote MySQL servers into mysqld */ -#define _EVENT_WITH_FEDERATED_STORAGE_ENGINE 1 - -/* Include Volatile memory based tables into mysqld */ -#define _EVENT_WITH_HEAP_STORAGE_ENGINE 1 - -/* Include Transactional Tables using InnoDB into mysqld */ -#define _EVENT_WITH_INNOBASE_STORAGE_ENGINE 1 - -/* Include Crash-safe tables with MyISAM heritage into mysqld */ -#define _EVENT_WITH_MARIA_STORAGE_ENGINE 1 - -/* Include Merge multiple MySQL tables into one into mysqld */ -#define _EVENT_WITH_MYISAMMRG_STORAGE_ENGINE 1 - -/* Include Traditional non-transactional MySQL tables into mysqld */ -#define _EVENT_WITH_MYISAM_STORAGE_ENGINE 1 - -/* Include High Availability Clustered tables into mysqld */ -/* #undef _EVENT_WITH_NDBCLUSTER_STORAGE_ENGINE */ - -/* Including Ndb Cluster Binlog */ -/* #undef _EVENT_WITH_NDB_BINLOG */ - -/* Include MySQL Partitioning Support into mysqld */ -#define _EVENT_WITH_PARTITION_STORAGE_ENGINE 1 - -/* Define to 1 if your processor stores words with the most significant byte - first (like Motorola and SPARC, unlike Intel and VAX). */ -/* #undef _EVENT_WORDS_BIGENDIAN */ - -/* Number of bits in a file offset, on hosts where this is settable. */ -/* #undef _EVENT__FILE_OFFSET_BITS */ - -/* makes fseeko etc. visible, on some hosts. */ -/* #undef _EVENT__LARGEFILE_SOURCE */ - -/* Large files support on AIX-style hosts. */ -/* #undef _EVENT__LARGE_FILES */ - -/* Define to appropriate substitue if compiler doesnt have __func__ */ -/* #undef _EVENT___func__ */ - -/* Define to empty if `const' does not conform to ANSI C. */ -/* #undef _EVENT_const */ - -/* Define to `__inline__' or `__inline' if that's what the C compiler - calls it, or to nothing if 'inline' is not supported under any name. */ -#ifndef _EVENT___cplusplus -/* #undef _EVENT_inline */ -#endif - -/* Define to `long int' if <sys/types.h> does not define. */ -/* #undef _EVENT_off_t */ - -/* Define to `int' if <sys/types.h> does not define. */ -/* #undef _EVENT_pid_t */ - -/* Define to `unsigned int' if <sys/types.h> does not define. */ -/* #undef _EVENT_size_t */ - -/* Define to unsigned int if you dont have it */ -/* #undef _EVENT_socklen_t */ -#endif === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2009-03-12 22:27:35 +0000 +++ b/sql/mysqld.cc 2009-03-13 13:31:54 +0000 @@ -1613,6 +1613,7 @@ static my_socket activate_tcp_port(uint uint retry; DBUG_ENTER("activate_tcp_port"); DBUG_PRINT("enter",("port: %u", port)); + LINT_INIT(ret); ip_sock = socket(AF_INET, SOCK_STREAM, 0); if (ip_sock == INVALID_SOCKET) @@ -1679,7 +1680,6 @@ static void network_init(void) #endif int arg=1; DBUG_ENTER("network_init"); - LINT_INIT(ret); if (thread_scheduler.init()) unireg_abort(1); /* purecov: inspected */

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2682)
by knielsen＠knielsen-hq.org 13 Mar '09

13 Mar '09

#At lp:maria 2682 knielsen(a)knielsen-hq.org 2009-03-13 Add testing of extra port for pool-of-threads. The additional test uses up all threads in the pool with SELECT SLEEP(), and tests that this makes normal connections block, but connections on the extra port still work. Add --connect-timeout option to mysqltest program. Add facility for --extra-port option to ConfigFactory. Fix regexp typo in ConfigFactory.pm removed: mysql-test/t/pool_of_threads-master.opt added: mysql-test/t/pool_of_threads.cnf modified: client/mysqltest.cc mysql-test/lib/My/ConfigFactory.pm mysql-test/r/pool_of_threads.result mysql-test/t/pool_of_threads.test === modified file 'client/mysqltest.cc' --- a/client/mysqltest.cc 2009-02-19 09:01:25 +0000 +++ b/client/mysqltest.cc 2009-03-13 15:13:29 +0000 @@ -76,7 +76,7 @@ enum { OPT_SKIP_SAFEMALLOC=OPT_MAX_CLIENT_OPTION, OPT_PS_PROTOCOL, OPT_SP_PROTOCOL, OPT_CURSOR_PROTOCOL, OPT_VIEW_PROTOCOL, OPT_MAX_CONNECT_RETRIES, OPT_MARK_PROGRESS, OPT_LOG_DIR, OPT_TAIL_LINES, - OPT_GLOBAL_SUBST + OPT_GLOBAL_SUBST, OPT_MY_CONNECT_TIMEOUT }; static int record= 0, opt_sleep= -1; @@ -87,6 +87,7 @@ const char *opt_include= 0, *opt_charset static int opt_port= 0; static int opt_max_connect_retries; static my_bool opt_compress= 0, silent= 0, verbose= 0; +static int opt_connect_timeout= -1; static my_bool debug_info_flag= 0, debug_check_flag= 0; static my_bool tty_password= 0; static my_bool opt_mark_progress= 0; @@ -4952,6 +4953,9 @@ void do_connect(struct st_command *comma if (opt_charsets_dir) mysql_options(&con_slot->mysql, MYSQL_SET_CHARSET_DIR, opt_charsets_dir); + if (opt_connect_timeout >= 0) + mysql_options(&con_slot->mysql, MYSQL_OPT_CONNECT_TIMEOUT, + &opt_connect_timeout); #ifdef HAVE_OPENSSL if (opt_use_ssl || con_ssl) @@ -5692,6 +5696,9 @@ static struct my_option my_long_options[ #include "sslopt-longopts.h" {"test-file", 'x', "Read test from/in this file (default stdin).", 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, + {"connect-timeout", OPT_MY_CONNECT_TIMEOUT, "Client connection timeout", + (uchar**) &opt_connect_timeout, (uchar**) &opt_connect_timeout, 0, + GET_INT, REQUIRED_ARG, -1, -1, 0, 0, 0, 0}, {"timer-file", 'm', "File where the timing in micro seconds is stored.", 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, {"tmpdir", 't', "Temporary directory where sockets are put.", === modified file 'mysql-test/lib/My/ConfigFactory.pm' --- a/mysql-test/lib/My/ConfigFactory.pm 2009-02-15 10:58:34 +0000 +++ b/mysql-test/lib/My/ConfigFactory.pm 2009-03-13 15:13:29 +0000 @@ -202,6 +202,7 @@ my @mysqld_rules= { 'pid-file' => \&fix_pidfile }, { '#host' => \&fix_host }, { 'port' => \&fix_port }, + { '#extra-port' => \&fix_port }, { 'socket' => \&fix_socket }, { 'log-error' => \&fix_log_error }, { 'general-log' => sub { return 1; } }, @@ -353,7 +354,7 @@ sub post_check_client_group { sub post_check_client_groups { my ($self, $config)= @_; - my $first_mysqld= $config->first_like('mysqld.'); + my $first_mysqld= $config->first_like('mysqld\.'); return unless $first_mysqld; === modified file 'mysql-test/r/pool_of_threads.result' --- a/mysql-test/r/pool_of_threads.result 2009-03-12 22:27:35 +0000 +++ b/mysql-test/r/pool_of_threads.result 2009-03-13 15:13:29 +0000 @@ -2151,3 +2151,20 @@ Privat (Private Nutzung) Mobilfunk Warnings: Warning 1052 Column 'kundentyp' in group statement is ambiguous drop table t1; +SELECT sleep(5); +SELECT sleep(5); +connect(localhost,root,,test,12500,/home/knielsen/devel/maria/my/work-1.5/mysql-test/var/tmp/mysqld.1.sock); +# -- Success: more than --thread-pool-size normal connections not possible +sleep(5) +0 +sleep(5) +0 +SELECT sleep(5); +SELECT sleep(5); +SELECT 'Connection on extra port ok'; +Connection on extra port ok +Connection on extra port ok +sleep(5) +0 +sleep(5) +0 === removed file 'mysql-test/t/pool_of_threads-master.opt' --- a/mysql-test/t/pool_of_threads-master.opt 2009-03-12 22:27:35 +0000 +++ b/mysql-test/t/pool_of_threads-master.opt 1970-01-01 00:00:00 +0000 @@ -1 +0,0 @@ ---test-ignore-wrong-options --thread-handling=pool-of-threads === added file 'mysql-test/t/pool_of_threads.cnf' --- a/mysql-test/t/pool_of_threads.cnf 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/pool_of_threads.cnf 2009-03-13 15:13:29 +0000 @@ -0,0 +1,13 @@ +!include include/default_my.cnf + +[mysqld.1] +test-ignore-wrong-options +thread-handling= pool-of-threads +thread_pool_size= 2 +extra-port= @mysqld.1.#extra-port + +[client] +connect-timeout= 2 + +[ENV] +MASTER_EXTRA_PORT= @mysqld.1.extra-port === modified file 'mysql-test/t/pool_of_threads.test' --- a/mysql-test/t/pool_of_threads.test 2009-03-12 22:27:35 +0000 +++ b/mysql-test/t/pool_of_threads.test 2009-03-13 15:13:29 +0000 @@ -5,3 +5,55 @@ -- source include/common-tests.inc + +# Test that we cannot have more simultaneous connections than +# --thread-pool-size on the standard port, but _can_ have additional +# connections on the extra port. + +# First set two connections running, and check that extra connection +# on normal port fails due to--thread-pool-size=2 +connection default; +send SELECT sleep(5); + +connect(con2,localhost,root,,); +connection con2; +send SELECT sleep(5); +--sleep 1 + +--disable_abort_on_error +--disable_result_log +connect(con3,localhost,root,,); +--enable_result_log +--enable_abort_on_error +let $error = $mysql_errno; +if (!$error) +{ + --echo # -- Error: managed to establish more than --thread-pool-size connections +} +if ($error) +{ + --echo # -- Success: more than --thread-pool-size normal connections not possible +} + +connection default; +--reap +connection con2; +--reap + +# Now try again, but this time use the extra port to successfully connect. + +connection default; +send SELECT sleep(5); + +connection con2; +send SELECT sleep(5); +--sleep 1 + +connect(extracon,127.0.0.1,root,,test,$MASTER_EXTRA_PORT,); +connection extracon; +SELECT 'Connection on extra port ok'; + +connection default; +--reap +connection con2; +--reap

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2681)
by Michael Widenius 12 Mar '09

12 Mar '09

#At lp:maria based on revid:monty@askmonty.org-20090312222735-zopxqrhf3xjnfnyd 2681 Michael Widenius 2009-03-13 Added missing fix from last commit modified: mysql-test/r/merge-big.result === modified file 'mysql-test/r/merge-big.result' --- a/mysql-test/r/merge-big.result 2008-02-07 11:04:19 +0000 +++ b/mysql-test/r/merge-big.result 2009-03-12 22:29:39 +0000 @@ -7,7 +7,7 @@ drop table if exists t1,t2,t3,t4,t5,t6; CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; LOCK TABLE t1 WRITE; # connection con1 -SET SESSION debug="+d,sleep_open_and_lock_after_open"; +SET GLOBAL debug="+d,sleep_open_and_lock_after_open"; INSERT INTO t1 VALUES (1); # connection default # Let INSERT go into thr_multi_lock(). @@ -21,6 +21,6 @@ SELECT * FROM t1; c1 UNLOCK TABLES; # connection con1 -SET SESSION debug="-d,sleep_open_and_lock_after_open"; +SET GLOBAL debug="-d,sleep_open_and_lock_after_open"; # connection default DROP TABLE t1;

1 0

[Maria-developers] bzr commit into mariadb 5.1, with Maria 1.5:maria branch (knielsen:2681)
by knielsen＠knielsen-hq.org 12 Mar '09

12 Mar '09

#At lp:maria 2681 knielsen(a)knielsen-hq.org 2009-03-12 Remove redundant code, probably bad automerge modified: mysql-test/lib/mtr_cases.pm === modified file 'mysql-test/lib/mtr_cases.pm' --- a/mysql-test/lib/mtr_cases.pm 2009-02-19 09:01:25 +0000 +++ b/mysql-test/lib/mtr_cases.pm 2009-03-12 14:38:43 +0000 @@ -799,15 +799,6 @@ sub collect_one_test_case { push(@{$tinfo->{'master_opt'}}, @$suite_opts); push(@{$tinfo->{'slave_opt'}}, @$suite_opts); - #----------------------------------------------------------------------- - # Check for test specific config file - #----------------------------------------------------------------------- - my $test_cnf_file= "$testdir/$tname.cnf"; - if ( -f $test_cnf_file) { - # Specifies the configuration file to use for this test - $tinfo->{'template_path'}= $test_cnf_file; - } - # ---------------------------------------------------------------------- # Check for test specific config file # ----------------------------------------------------------------------

1 0

[Maria-developers] Merge Analysis Daemon
by Kurt von Finck 12 Mar '09

12 Mar '09

lp:mad blog post by Tim Penhey. Seems interesting, but as I'm not writing code, I'm not sure how useful anyone would find this. http://how-bazaar.blogspot.com/2009/03/lpmad.html -- ./k kurt von finck http://www.mneptok.com public key: keyserver.ubuntu.com key id: 5229D26A fingerprint: 127A A484 ADBF A5AD E7FB 8CD2 8913 18F4 5229 D26A Music is the space between the notes. - Debussy

1 0

[Maria-developers] XtraDB merge
by Vadim Tkachenko 12 Mar '09

12 Mar '09

Hello, We had talk with Monty and I am going to propose merge for MariaDB-MySQL tree with InnoDB->XtraDB replacement. Merge only related to InnoDB, no other changes in main MySQL tree. Is that OK ? What is the best way to do that - just merge proposal ? Thanks, Vadim -- Vadim Tkachenko, CTO Percona Inc. ICQ: 369-510-335, Skype: vadimtk153, Phone +1-888-401-3403 MySQL Performance Blog - http://www.mysqlperformanceblog.com MySQL Consulting http://www.percona.com/ Attend the 2009 Percona Performance Conference April 22-23 - http://conferences.percona.com/

2 2

[Maria-developers] bzr commit into mariadb 5.1, with Maria 1.5:maria branch (knielsen:2680)
by knielsen＠knielsen-hq.org 12 Mar '09

12 Mar '09

#At lp:maria 2680 knielsen(a)knielsen-hq.org 2009-03-12 Make test --big, as it takes a _long_ time to run and only tests a single bug. modified: mysql-test/t/multi_update2.test === modified file 'mysql-test/t/multi_update2.test' --- a/mysql-test/t/multi_update2.test 2008-11-19 18:17:26 +0000 +++ b/mysql-test/t/multi_update2.test 2009-03-12 12:29:17 +0000 @@ -2,6 +2,9 @@ # Test of update statement that uses many tables. # +# This is a big test. +--source include/big_test.inc + --disable_warnings DROP TABLE IF EXISTS t1,t2; --enable_warnings

2 1

[Maria-developers] WL#3859: Plug-in Service API discussion
by Kristian Nielsen 12 Mar '09

12 Mar '09

While we were discussing a generic I/O layer in MariaDB, Sergei brought up the issue of WL#3859. This worklog carries a principal issue that I wanted to mention and discuss. Sergei, if I understood you correctly, you said that Sun MySQL would never take a new internal API that does not follow the WL#3859 framework. Correct? My understanding (which may or may not be correct) is that at least part of MySQL/Sun wants to develop the storage engine plugin layer (and all other plugin layers) to be a complete ABI, that is complete binary compatibility. So that the user can obtain a pre-compiled plugin independently and load it into the server without risk of crashing if plugin is compiled against a different source version of the server (but may get an error message if versions are incompatible). This basically means that all data and code needs to be exported via the WL#3859 framework, not using the system linker as is currently done. Advantages include - Avoiding crash on incompatible version is good. - Supports selling proprietary (non Open Source) plugins. - Works on Windows where the system linker cannot export symbols from executables (though there is a standard workaround of just compiling the server into a library that is called from a stub main executable). Disadvantages include - Supports selling proprietary (non Open Source) plugins (yes, can be an advantage or disadvantage depending on point-of-view ;-). - Will tend to discourage making desirable changes, as the procedure becomes more complicated (more work to track versioned API, less flexibitily in what C++ language constructs can be used); and there will be the desire to minimize binary incompatibility. It would be great if some of you still inside Sun could confirm or reject that this is the way things are. This raises at least two important questions for MariaDB: 1. Do we want to adopt this direction, moving all existing exposed server internals into this framework, and implementing any new ones within it as well? Or reserve the right to use old-style direct linking where it makes more sense? (We can of course in any case use the WL#3859 framework where it makes sense, the issue is the principle of using it for everything eventually). 2. It seems impossible that we can keep the Sun/MySQL API versions and the MariaDB API versions 100% synchronised. So how will we handle version compatibility (ie. Sun/MySQL version 0x0109 may be different from Maria version 0x0109 due to different paces of development)? Will all plugins need to track two different versions, one for Sun/MySQL and one for MariaDB? Or will we somehow coordinate between the two teams to avoid version conflicts? Any opinions? My personal opinion is that I am sceptical if this worklog is a good idea; most of the big projects of this kind that have been successful have used the old MySQL approach (Apache, Perl, the Linux kernel). Though I'm not 100% decided one way or the other yet. But more importantly I think we need to have a conscious decision on how to deal with this in MariaDB, as it is a fundamental design issue that will impact many other desicions along the way. - Kristian.

2 3

[Maria-developers] bzr commit into mariadb 5.1, with Maria 1.5:maria branch (knielsen:2679) Bug#43418
by knielsen＠knielsen-hq.org 12 Mar '09

12 Mar '09

#At lp:maria 2679 knielsen(a)knielsen-hq.org 2009-03-12 BUG#43418: MTR2: does not notice a memory leak occuring at shutdown of mysqld w/ --valgrind Partial fix. Do a graceful shutdown at end of testsuite instead of kill -9. Do a simplified parsing for log errors after final server shutdown. modified: mysql-test/mysql-test-run.pl === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2009-02-19 09:01:25 +0000 +++ b/mysql-test/mysql-test-run.pl 2009-03-12 11:39:00 +0000 @@ -357,7 +357,7 @@ sub main { mtr_print_thick_line(); mtr_print_header(); - my $completed= run_test_server($server, $tests, $opt_parallel); + my ($completed, $fail)= run_test_server($server, $tests, $opt_parallel); # Send Ctrl-C to any children still running kill("INT", keys(%children)); @@ -393,6 +393,10 @@ sub main { mtr_error("Not all tests completed"); } + if ($fail) { + mtr_error("Test suite failure."); + } + mtr_print_line(); if ( $opt_gcov ) { @@ -412,6 +416,7 @@ sub run_test_server ($$$) { my $num_saved_cores= 0; # Number of core files saved in vardir/log/ so far. my $num_saved_datadir= 0; # Number of datadirs saved in vardir/log/ so far. my $num_failed_test= 0; # Number of tests failed so far + my $test_failure= 0; # Scheduler variables my $max_ndb= $childs / 2; @@ -445,7 +450,7 @@ sub run_test_server ($$$) { $s->remove($sock); if (--$childs == 0){ $suite_timeout_proc->kill(); - return $completed; + return ($completed, $test_failure); } next; } @@ -509,18 +514,19 @@ sub run_test_server ($$$) { } $num_saved_datadir++; + $test_failure= 1; if ( !$opt_force ) { # Test has failed, force is off $suite_timeout_proc->kill(); push(@$completed, $result); - return $completed; + return ($completed, 1); } elsif ($opt_max_test_fail > 0 and $num_failed_test >= $opt_max_test_fail) { $suite_timeout_proc->kill(); mtr_report("Too many tests($num_failed_test) failed!", "Terminating..."); - return undef; + return (undef, 1); } $num_failed_test++; } @@ -571,6 +577,10 @@ sub run_test_server ($$$) { elsif ($line eq 'START'){ ; # Send first test } + elsif ($line eq 'WARNINGS'){ + mtr_report("***Warnings found in log after final shutdown."); + $test_failure= 1; + } else { mtr_error("Unknown response: '$line' from client"); } @@ -649,7 +659,7 @@ sub run_test_server ($$$) { if ( ! $suite_timeout_proc->wait_one(0) ) { mtr_report("Test suite timeout! Terminating..."); - return undef; + return (undef, 1); } } } @@ -706,10 +716,11 @@ sub run_worker ($) { # Ask server for first test print $server "START\n"; + my $test= undef; while(my $line= <$server>){ chomp($line); if ($line eq 'TESTCASE'){ - my $test= My::Test::read_test($server); + $test= My::Test::read_test($server); #$test->print_test(); # Clear comment and logfile, to avoid @@ -725,6 +736,22 @@ sub run_worker ($) { } elsif ($line eq 'BYE'){ mtr_report("Server said BYE"); + # We need to gracefully shut down the servers to see any + # Valgrind memory leak errors etc. since last server restart. + if ($opt_warnings && defined($test)) { + # Set marker so we only consider log messages generated during shutdown + foreach my $mysqld ( mysqlds() ) { + if (defined $mysqld->value('log-error')) { + mark_log($mysqld->value('log-error'), "<final shutdown>"); + } + } + stop_all_servers(1); + if (check_warnings($test)) { + # Warnings appeared in log file(s) during final server shutdown. + print $server "WARNINGS\n"; + exit(1); + } + } exit(0); } else { @@ -3095,8 +3122,8 @@ sub run_on_all($$) sub mark_log { - my ($log, $tinfo)= @_; - my $log_msg= "CURRENT_TEST: $tinfo->{name}\n"; + my ($log, $name)= @_; + my $log_msg= "CURRENT_TEST: $name\n"; mtr_tofile($log, $log_msg); } @@ -3247,7 +3274,7 @@ sub run_testcase ($) { } # Write start of testcase to log - mark_log($path_current_testlog, $tinfo); + mark_log($path_current_testlog, $tinfo->{name}); if (start_servers($tinfo)) { @@ -3526,6 +3553,7 @@ sub extract_warning_lines ($) { qr/Assertion .* failed/, ); + my $match_count= 0; foreach my $line ( @lines ) { foreach my $pat ( @patterns ) @@ -3533,12 +3561,14 @@ sub extract_warning_lines ($) { if ( $line =~ /$pat/ ) { print $Fwarn $line; + ++$match_count; last; } } } $Fwarn = undef; # Close file + return $match_count; } @@ -3554,8 +3584,6 @@ sub start_check_warnings ($$) { my $name= "warnings-".$mysqld->name(); - extract_warning_lines($mysqld->value('log-error')); - my $args; mtr_init_args(\$args); @@ -3614,17 +3642,37 @@ sub check_warnings ($) { # Start the mysqltest processes in parallel to save time # also makes it possible to wait for any process to exit during the check my %started; + my $total_found= 0; foreach my $mysqld ( mysqlds() ) { - if ( defined $mysqld->{'proc'} ) + my $found= 0; + if (defined $mysqld->value('log-error')) { + $found= extract_warning_lines($mysqld->value('log-error')); + } + + if ( defined $mysqld->{'proc'}) { my $proc= start_check_warnings($tinfo, $mysqld); $started{$proc->pid()}= $proc; } + else + { + $total_found+= $found; + } } # Return immediately if no check proceess was started - return 0 unless ( keys %started ); + unless ( keys %started ) { + # When we check warnings after shutdown of mysqld, we cannot use the + # SQL implementation of warning checking, so we just use the Perl + # checking instead. + # (eventually, it would be better to do all the parsing here in Perl). + if ($total_found > 0) { + return 1; + } else { + return 0; + } + } my $timeout_proc= My::SafeProcess->timer(check_timeout()); @@ -4161,12 +4209,14 @@ sub mysqld_start ($$) { sub stop_all_servers () { + my ($graceful)= @_; mtr_verbose("Stopping all servers..."); # Kill all started servers - My::SafeProcess::shutdown(0, # shutdown timeout 0 => kill - started(all_servers())); + my $timeout = $graceful ? + $opt_shutdown_timeout : 0; # shutdown timeout 0 => kill + My::SafeProcess::shutdown($timeout, started(all_servers())); # Remove pidfiles foreach my $server ( all_servers() ) @@ -4426,7 +4476,7 @@ sub start_servers($) { # Already started # Write start of testcase to log file - mark_log($mysqld->value('log-error'), $tinfo); + mark_log($mysqld->value('log-error'), $tinfo->{name}); next; } @@ -4485,7 +4535,7 @@ sub start_servers($) { mkpath($tmpdir) unless -d $tmpdir; # Write start of testcase to log file - mark_log($mysqld->value('log-error'), $tinfo); + mark_log($mysqld->value('log-error'), $tinfo->{name}); # Run <tname>-master.sh if ($mysqld->option('#!run-master-sh') and

1 0

[Maria-developers] what triggers row-header compaction for Maria
by MARK CALLAGHAN 09 Mar '09

09 Mar '09

What triggers row-header compaction for Maria? There are fields in the row-header that can be removed under certain conditions (transaction has committed, no other transactions can see that row). Is this done when OPTIMIZE TABLE is run? Is this done in other cases? -- Mark Callaghan mdcallag(a)gmail.com

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2679)
by Michael Widenius 09 Mar '09

09 Mar '09

#At lp:maria based on revid:cjac@dev0-20090305222554-cxws3ke1ktvhhwku 2679 Michael Widenius 2009-03-09 Merge with lp:maria modified: .bzr-mysql/default.conf === modified file '.bzr-mysql/default.conf' --- a/.bzr-mysql/default.conf 2009-03-04 11:31:27 +0000 +++ b/.bzr-mysql/default.conf 2009-03-09 14:13:44 +0000 @@ -3,4 +3,4 @@ tree_location = lp:maria post_commit_to = maria-developers(a)lists.launchpad.net post_commit_url = lp:maria tree_name = maria -project_name = "mariadb 5.1, with Maria 1.5" +project_name = "MariaDB 5.1, with Maria 1.5"

1 0

[Maria-developers] safe patches to backport -- pool-of-threads
by MARK CALLAGHAN 09 Mar '09

09 Mar '09

One of the features listed here http://askmonty.org/wiki/index.php/MariaDB_versus_MySQL as a potential backport is support for pool-of-threads. I am not sure how safe this feature is today. http://mysqlha.blogspot.com/2009/01/no-new-global-mutexes-and-how-to-make.h… describes some of the problems that might cause a server to hang when pool-of-threads is used. http://bugs.mysql.com/bug.php?id=42288 describes the performance overhead of pool-of-threads. The overhead is significant for workloads with fast queries such as sysbench. I think that pool-of-threads needs a lot of work before it can be considered safe. We backported this to our 5.0 branch at work and have begun to search for fixes. -- Mark Callaghan mdcallag(a)gmail.com

3 2

[Maria-developers] the _BOOK_ in http://forge.mysql.com/worklog/task.php?id=3134
by MARK CALLAGHAN 09 Mar '09

09 Mar '09

Is the _BOOK_ the Gray and Reuter book? http://www.informatik.uni-trier.de/~ley/db/books/dbtext/Gray93.html -- Mark Callaghan mdcallag(a)gmail.com

2 1

[Maria-developers] GCOV / LCOV data for storage/maria
by Patrick Crews 05 Mar '09

05 Mar '09

Hello all, Here are some lcov files for code coverage of storage/maria. I did 3 runs: unit only (make test in storage/maria), mtr-only, and both (mtr then make test). Included in the tarball are 3 directories, labelled for each run. You can open the index.html file to take a look at code coverage. I have some reservations about these numbers as OSX 10.5 was a little iffy on gcov a while back. I'll try to put together some numbers on OpenSolaris or Ubuntu as time permits (or someone else can post their numbers for comparison if they are feeling particularly industrious) Thanks, Patrick

1 0

[Maria-developers] bzr commit into mariadb 6.0, with Maria 1.5:maria/6.0 branch (sanja:3039)
by sanja＠askmonty.org 05 Mar '09

05 Mar '09

#At file:///home/bell/mysql/bzr/maria-6.0/ 3039 sanja(a)askmonty.org 2009-03-05 We need only commit e-mail (not push one). modified: .bzr-mysql/default.conf per-file messages: .bzr-mysql/default.conf We need only commit e-mail (not push one). === modified file '.bzr-mysql/default.conf' --- a/.bzr-mysql/default.conf 2009-03-04 22:15:38 +0000 +++ b/.bzr-mysql/default.conf 2009-03-05 09:19:00 +0000 @@ -1,6 +1,5 @@ [MYSQL] tree_location = lp:maria/6.0 post_commit_to = "maria-developers(a)lists.launchpad.net" -post_push_to = "maria-developers(a)lists.launchpad.net" tree_name = maria/6.0 project_name = "mariadb 6.0, with Maria 1.5"

1 0

[Maria-developers] max #rows per page -- DIR_COUNT_SIZE
by MARK CALLAGHAN 04 Mar '09

04 Mar '09

DIR_COUNT_SIZE is 1. Doesn't that mean that the max #rows on a page is 256? Do you want that limit when Maria pages can be large? -- Mark Callaghan mdcallag(a)gmail.com

1 0

[Maria-developers] bzr push into mariadb 6.0, with Maria 1.5:maria/6.0 branch (sanja:3037 to 3038)
by sanja＠askmonty.org 04 Mar '09

04 Mar '09

3038 sanja(a)askmonty.org 2009-03-05 The tree config modified: .bzr-mysql/default.conf 3037 Michael Widenius 2009-02-15 (empty) merge with main branch === modified file '.bzr-mysql/default.conf' --- a/.bzr-mysql/default.conf 2009-02-12 17:56:03 +0000 +++ b/.bzr-mysql/default.conf 2009-03-04 22:15:38 +0000 @@ -1,5 +1,6 @@ [MYSQL] -tree_location = bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-6.0-bugteam -post_commit_to = "commits(a)lists.mysql.com" -post_push_to = "commits(a)lists.mysql.com" -tree_name = "mysql-6.0-bugteam" +tree_location = lp:maria/6.0 +post_commit_to = "maria-developers(a)lists.launchpad.net" +post_push_to = "maria-developers(a)lists.launchpad.net" +tree_name = maria/6.0 +project_name = "mariadb 6.0, with Maria 1.5"

1 0

[Maria-developers] bzr commit into mariadb 6.0, with Maria 1.5:maria/6.0 branch (sanja:3038)
by sanja＠askmonty.org 04 Mar '09

04 Mar '09

#At file:///home/bell/mysql/bzr/maria-6.0/ 3038 sanja(a)askmonty.org 2009-03-05 The tree config modified: .bzr-mysql/default.conf per-file messages: .bzr-mysql/default.conf The tree config === modified file '.bzr-mysql/default.conf' --- a/.bzr-mysql/default.conf 2009-02-12 17:56:03 +0000 +++ b/.bzr-mysql/default.conf 2009-03-04 22:15:38 +0000 @@ -1,5 +1,6 @@ [MYSQL] -tree_location = bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-6.0-bugteam -post_commit_to = "commits(a)lists.mysql.com" -post_push_to = "commits(a)lists.mysql.com" -tree_name = "mysql-6.0-bugteam" +tree_location = lp:maria/6.0 +post_commit_to = "maria-developers(a)lists.launchpad.net" +post_push_to = "maria-developers(a)lists.launchpad.net" +tree_name = maria/6.0 +project_name = "mariadb 6.0, with Maria 1.5"

1 0

[Maria-developers] bzr commit into mariadb 5.1, with Maria 2.0:maria/2.0 branch (sanja:2722)
by sanja＠askmonty.org 04 Mar '09

04 Mar '09

#At lp:maria/2.0 2722 sanja(a)askmonty.org 2009-03-04 Group commit (only for review). modified: mysql-test/suite/maria/r/maria3.result mysys/my_init.c storage/maria/ha_maria.cc storage/maria/ma_init.c storage/maria/ma_loghandler.c storage/maria/ma_loghandler.h tests/fork_big2.pl per-file messages: mysql-test/suite/maria/r/maria3.result new maria variables. mysys/my_init.c commented my_atomic_initialize() call create problem with linker but it should be here. storage/maria/ha_maria.cc Group commit controls added. storage/maria/ma_init.c Ending soft sync thread. storage/maria/ma_loghandler.c Group commit added. storage/maria/ma_loghandler.h Functions for controlling group commit. tests/fork_big2.pl Testing script fixed (now can run under Linux). === modified file 'mysql-test/suite/maria/r/maria3.result' --- a/mysql-test/suite/maria/r/maria3.result 2008-10-09 20:03:54 +0000 +++ b/mysql-test/suite/maria/r/maria3.result 2009-03-04 16:13:55 +0000 @@ -264,6 +264,8 @@ Variable_name Value maria_block_size 8192 maria_checkpoint_interval 30 maria_force_start_after_recovery_failures 0 +maria_group_commit none +maria_group_commit_rate 800 maria_log_file_size 4294959104 maria_log_purge_type immediate maria_max_sort_file_size 9223372036854775807 === modified file 'mysys/my_init.c' --- a/mysys/my_init.c 2008-12-10 09:02:25 +0000 +++ b/mysys/my_init.c 2009-03-04 16:13:55 +0000 @@ -40,6 +40,7 @@ static void netware_init(); #else #define netware_init() #endif +#include <my_atomic.h> my_bool my_init_done= 0; uint mysys_usage_id= 0; /* Incremented for each my_init() */ @@ -82,6 +83,10 @@ my_bool my_init(void) if (my_progname) my_progname_short= my_progname + dirname_length(my_progname); +/* + if (my_atomic_initialize()) + return 1; +*/ #if defined(THREAD) && defined(SAFE_MUTEX) safe_mutex_global_init(); /* Must be called early */ #endif === modified file 'storage/maria/ha_maria.cc' --- a/storage/maria/ha_maria.cc 2009-01-16 16:18:17 +0000 +++ b/storage/maria/ha_maria.cc 2009-03-04 16:13:55 +0000 @@ -101,22 +101,40 @@ TYPELIB maria_translog_purge_type_typeli array_elements(maria_translog_purge_type_names) - 1, "", maria_translog_purge_type_names, NULL }; + +/* transactional log directory sync */ const char *maria_sync_log_dir_names[]= { "NEVER", "NEWFILE", "ALWAYS", NullS }; - TYPELIB maria_sync_log_dir_typelib= { array_elements(maria_sync_log_dir_names) - 1, "", maria_sync_log_dir_names, NULL }; +/* transactional log group commit */ +const char *maria_group_commit_names[]= +{ + "none", "hard", "soft", NullS +}; +TYPELIB maria_group_commit_typelib= +{ + array_elements(maria_group_commit_names) - 1, "", + maria_group_commit_names, NULL +}; + /** Interval between background checkpoints in seconds */ static ulong checkpoint_interval; static void update_checkpoint_interval(MYSQL_THD thd, struct st_mysql_sys_var *var, void *var_ptr, const void *save); +static void update_maria_group_commit(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save); +static void update_maria_group_commit_rate(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save); /** After that many consecutive recovery failures, remove logs */ static ulong force_start_after_recovery_failures; static void update_log_file_size(MYSQL_THD thd, @@ -163,6 +181,22 @@ static MYSQL_SYSVAR_ULONG(log_file_size, NULL, update_log_file_size, TRANSLOG_FILE_SIZE, TRANSLOG_MIN_FILE_SIZE, 0xffffffffL, TRANSLOG_PAGE_SIZE); +static MYSQL_SYSVAR_ENUM(group_commit, group_commit, + PLUGIN_VAR_RQCMDARG, + "Specifies maria group commit mode. " + "Possible values are \"none\" (no group commit), " + "\"hard\" (with waiting to actual commit), " + "\"soft\" (no wait for commit (DANGEROUS!!!))", + NULL, update_maria_group_commit, + TRANSLOG_GCOMMIT_NONE, &maria_group_commit_typelib); + +static MYSQL_SYSVAR_ULONG(group_commit_rate, group_commit_rate, + PLUGIN_VAR_RQCMDARG, + "If group commits switched on commit will happens with about every " + "100/maria_group_commit_rate second. 0 is special value which switch " + "rate off", + NULL, update_maria_group_commit_rate, 800, 0, UINT_MAX, 1); + static MYSQL_SYSVAR_ENUM(log_purge_type, log_purge_type, PLUGIN_VAR_RQCMDARG, "Specifies how maria transactional log will be purged. " @@ -3247,6 +3281,8 @@ static struct st_mysql_sys_var* system_v MYSQL_SYSVAR(block_size), MYSQL_SYSVAR(checkpoint_interval), MYSQL_SYSVAR(force_start_after_recovery_failures), + MYSQL_SYSVAR(group_commit), + MYSQL_SYSVAR(group_commit_rate), MYSQL_SYSVAR(page_checksum), MYSQL_SYSVAR(log_dir_path), MYSQL_SYSVAR(log_file_size), @@ -3277,6 +3313,97 @@ static void update_checkpoint_interval(M } /** + @brief Updates group commit mode +*/ + +static void update_maria_group_commit(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save) +{ + ulong value= (ulong)*((long *)var_ptr); + DBUG_ENTER("update_maria_group_commit"); + DBUG_PRINT("enter", ("old value: %lu new value %lu rate %lu", + value, (ulong)(*(long *)save), group_commit_rate)); + /* old value */ + switch (value) + { + case TRANSLOG_GCOMMIT_NONE: + break; + case TRANSLOG_GCOMMIT_HARD: + translog_hard_group_commit(FALSE); + break; + case TRANSLOG_GCOMMIT_SOFT: + translog_soft_sync(FALSE); + if (group_commit_rate) + translog_soft_sync_end(); + break; + default: + DBUG_ASSERT(0); /* impossible */ + } + value= *(ulong *)var_ptr= (ulong)(*(long *)save); + translog_sync(); + /* new value */ + switch (value) + { + case TRANSLOG_GCOMMIT_NONE: + break; + case TRANSLOG_GCOMMIT_HARD: + translog_hard_group_commit(TRUE); + break; + case TRANSLOG_GCOMMIT_SOFT: + translog_soft_sync(TRUE); + /* variable change made under global lock so we can just read it */ + if (group_commit_rate) + translog_soft_sync_start(); + break; + default: + DBUG_ASSERT(0); /* impossible */ + } + DBUG_VOID_RETURN; +} + +/** + @brief Updates group commit rate +*/ + +static void update_maria_group_commit_rate(MYSQL_THD thd, + struct st_mysql_sys_var *var, + void *var_ptr, const void *save) +{ + ulong new_value= (ulong)*((long *)save); + DBUG_ENTER("update_maria_group_commit_rate"); + DBUG_PRINT("enter", ("old value: %lu new value %lu group commit %lu", + *(ulong *)var_ptr, new_value, group_commit)); + if (new_value && + ((TRANSLOG_RATE_BASE * 1000000000ULL / new_value + + TRANSLOG_RATE_BASE / 2) / + TRANSLOG_RATE_BASE) == 0) + new_value= 0; /* protection against too small value */ + /* variable change made under global lock so we can just read it */ + switch (group_commit) + { + case TRANSLOG_GCOMMIT_NONE: + *(ulong *)var_ptr= new_value; + translog_set_group_commit_rate(new_value); + break; + case TRANSLOG_GCOMMIT_HARD: + *(ulong *)var_ptr= new_value; + translog_set_group_commit_rate(new_value); + break; + case TRANSLOG_GCOMMIT_SOFT: + if (*(ulong *)var_ptr) + translog_soft_sync_end(); + translog_set_group_commit_rate(new_value); + if ((*(ulong *)var_ptr= new_value)) + translog_soft_sync_start(); + break; + default: + DBUG_ASSERT(0); /* impossible */ + } + DBUG_VOID_RETURN; +} + +/** @brief Updates the transaction log file limit. */ === modified file 'storage/maria/ma_init.c' --- a/storage/maria/ma_init.c 2008-10-09 20:03:54 +0000 +++ b/storage/maria/ma_init.c 2009-03-04 16:13:55 +0000 @@ -82,6 +82,11 @@ void maria_end(void) maria_inited= maria_multi_threaded= FALSE; ft_free_stopwords(); ma_checkpoint_end(); + if (translog_status == TRANSLOG_OK) + { + translog_soft_sync_end(); + translog_sync(); + } if ((trid= trnman_get_max_trid()) > max_trid_in_control_file) { /* === modified file 'storage/maria/ma_loghandler.c' --- a/storage/maria/ma_loghandler.c 2009-01-16 09:38:02 +0000 +++ b/storage/maria/ma_loghandler.c 2009-03-04 16:13:55 +0000 @@ -18,6 +18,7 @@ #include "ma_blockrec.h" /* for some constants and in-write hooks */ #include "ma_key_recover.h" /* For some in-write hooks */ #include "ma_checkpoint.h" +#include "ma_servicethread.h" /* On Windows, neither my_open() nor my_sync() work for directories. @@ -47,6 +48,15 @@ #include <m_ctype.h> #endif +/** @brief protects checkpoint_in_progress */ +static pthread_mutex_t LOCK_soft_sync; +/** @brief for killing the background checkpoint thread */ +static pthread_cond_t COND_soft_sync; +/** @brief control structure for checkpoint background thread */ +static MA_SERVICE_THREAD_CONTROL soft_sync_control= + {THREAD_DEAD, FALSE, &LOCK_soft_sync, &COND_soft_sync}; + + /* transaction log file descriptor */ typedef struct st_translog_file { @@ -124,6 +134,11 @@ struct st_translog_buffer /* Previous buffer offset to detect it flush finish */ TRANSLOG_ADDRESS prev_buffer_offset; /* + If the buffer was forced to close it save value of its horizon + otherwise LSN_IMPOSSIBLE + */ + TRANSLOG_ADDRESS pre_force_close_horizon; + /* How much is written (or will be written when copy_to_buffer_in_progress become 0) to this buffer */ @@ -304,6 +319,7 @@ struct st_translog_descriptor */ pthread_mutex_t log_flush_lock; pthread_cond_t log_flush_cond; + pthread_cond_t new_goal_cond; /* Protects changing of headers of finished files (max_lsn) */ pthread_mutex_t file_header_lock; @@ -343,13 +359,40 @@ static struct st_translog_descriptor log ulong log_purge_type= TRANSLOG_PURGE_IMMIDIATE; ulong log_file_size= TRANSLOG_FILE_SIZE; ulong sync_log_dir= TRANSLOG_SYNC_DIR_NEWFILE; +ulong group_commit= TRANSLOG_GCOMMIT_NONE; +ulong group_commit_rate= 0; /* Marker for end of log */ static uchar end_of_log= 0; #define END_OF_LOG &end_of_log +static my_atomic_rwlock_t soft_sync_rwl; +static my_atomic_rwlock_t hgroup_commit_rwl; +/** + Switch for "soft" sync (no real sync() but periodical sync by service + thread) +*/ +static volatile uint32 soft_sync= FALSE; +/** + Switch for "hard" group commit mode +*/ +static uint32 hgroup_commit= FALSE; +/** + File numbers interval which have to be sync() +*/ +static uint32 soft_sync_min= 0; +static uint32 soft_sync_max= 0; +static my_atomic_rwlock_t group_commit_wait_rwl; +/** + stores interval in nanoseconds/TRANSLOG_RATE_BASE (to + fit into uint32) +*/ +static uint32 group_commit_wait= 0; enum enum_translog_status translog_status= TRANSLOG_UNINITED; +/* time of last flush */ +static ulonglong flush_start= 0; + /* chunk types */ #define TRANSLOG_CHUNK_LSN 0x00 /* 0 chunk refer as LSN (head or tail */ #define TRANSLOG_CHUNK_FIXED (1 << 6) /* 1 (pseudo)fixed record (also LSN) */ @@ -978,12 +1021,17 @@ static TRANSLOG_FILE *get_logfile_by_num static TRANSLOG_FILE *get_current_logfile() { TRANSLOG_FILE *file; + DBUG_ENTER("get_current_logfile"); rw_rdlock(&log_descriptor.open_files_lock); + DBUG_PRINT("info", ("max_file: %lu min_file: %lu open_files: %lu", + (ulong) log_descriptor.max_file, + (ulong) log_descriptor.min_file, + (ulong) log_descriptor.open_files.elements)); DBUG_ASSERT(log_descriptor.max_file - log_descriptor.min_file + 1 == log_descriptor.open_files.elements); file= *dynamic_element(&log_descriptor.open_files, 0, TRANSLOG_FILE **); rw_unlock(&log_descriptor.open_files_lock); - return (file); + DBUG_RETURN(file); } uchar NEAR maria_trans_file_magic[]= @@ -1421,7 +1469,9 @@ LSN translog_get_file_max_lsn_stored(uin static my_bool translog_buffer_init(struct st_translog_buffer *buffer, int num) { DBUG_ENTER("translog_buffer_init"); - buffer->prev_last_lsn= buffer->last_lsn= LSN_IMPOSSIBLE; + buffer->pre_force_close_horizon= + buffer->prev_last_lsn= buffer->last_lsn= + LSN_IMPOSSIBLE; DBUG_PRINT("info", ("last_lsn and prev_last_lsn set to 0 buffer: 0x%lx", (ulong) buffer)); @@ -2042,7 +2092,8 @@ static void translog_start_buffer(struct (ulong) LSN_OFFSET(log_descriptor.horizon), (ulong) LSN_OFFSET(log_descriptor.horizon))); DBUG_ASSERT(buffer_no == buffer->buffer_no); - buffer->prev_last_lsn= buffer->last_lsn= LSN_IMPOSSIBLE; + buffer->pre_force_close_horizon= + buffer->prev_last_lsn= buffer->last_lsn= LSN_IMPOSSIBLE; DBUG_PRINT("info", ("last_lsn and prev_last_lsn set to 0 buffer: 0x%lx", (ulong) buffer)); buffer->offset= log_descriptor.horizon; @@ -3495,6 +3546,8 @@ my_bool translog_init_with_table(const c log_descriptor.open_flags= O_BINARY | O_RDONLY; else log_descriptor.open_flags= O_BINARY | O_RDWR; + my_atomic_rwlock_init(&soft_sync_rwl); + my_atomic_rwlock_init(&group_commit_wait_rwl); if (pthread_mutex_init(&log_descriptor.sent_to_disk_lock, MY_MUTEX_INIT_FAST) || pthread_mutex_init(&log_descriptor.file_header_lock, @@ -3508,6 +3561,7 @@ my_bool translog_init_with_table(const c pthread_mutex_init(&log_descriptor.dirty_buffer_mask_lock, MY_MUTEX_INIT_FAST) || pthread_cond_init(&log_descriptor.log_flush_cond, 0) || + pthread_cond_init(&log_descriptor.new_goal_cond, 0) || my_rwlock_init(&log_descriptor.open_files_lock, NULL) || my_init_dynamic_array(&log_descriptor.open_files, @@ -3909,7 +3963,6 @@ my_bool translog_init_with_table(const c log_descriptor.flushed= log_descriptor.horizon; log_descriptor.in_buffers_only= log_descriptor.bc.buffer->offset; log_descriptor.max_lsn= LSN_IMPOSSIBLE; /* set to 0 */ - log_descriptor.previous_flush_horizon= log_descriptor.horizon; /* Now 'flushed' is set to 'horizon' value, but 'horizon' is (potentially) address of the next LSN and we want indicate that all LSNs that are @@ -3992,6 +4045,10 @@ my_bool translog_init_with_table(const c It is beginning of the log => there is no LSNs in the log => There is no harm in leaving it "as-is". */ + log_descriptor.previous_flush_horizon= log_descriptor.horizon; + DBUG_PRINT("info", ("previous_flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(log_descriptor. + previous_flush_horizon))); DBUG_RETURN(0); } file_no--; @@ -4067,6 +4124,9 @@ my_bool translog_init_with_table(const c translog_free_record_header(&rec); } } + log_descriptor.previous_flush_horizon= log_descriptor.horizon; + DBUG_PRINT("info", ("previous_flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(log_descriptor.previous_flush_horizon))); DBUG_RETURN(0); err: ma_message_no_user(0, "log initialization failed"); @@ -4154,6 +4214,9 @@ void translog_destroy() pthread_mutex_destroy(&log_descriptor.log_flush_lock); pthread_mutex_destroy(&log_descriptor.dirty_buffer_mask_lock); pthread_cond_destroy(&log_descriptor.log_flush_cond); + pthread_cond_destroy(&log_descriptor.new_goal_cond); + my_atomic_rwlock_destroy(&soft_sync_rwl); + my_atomic_rwlock_destroy(&group_commit_wait_rwl); rwlock_destroy(&log_descriptor.open_files_lock); delete_dynamic(&log_descriptor.open_files); delete_dynamic(&log_descriptor.unfinished_files); @@ -7383,18 +7446,20 @@ static void translog_force_current_buffe */ DBUG_PRINT("info", ("left: %u", (uint) left)); + old_buffer->pre_force_close_horizon= + old_buffer->offset + old_buffer->size; /* decrease offset */ new_buff_beginning-= log_descriptor.bc.current_page_fill; current_page_fill= log_descriptor.bc.current_page_fill; memset(log_descriptor.bc.ptr, TRANSLOG_FILLER, left); - log_descriptor.bc.buffer->size+= left; + old_buffer->size+= left; DBUG_PRINT("info", ("Finish Page buffer #%u: 0x%lx " "Size: %lu", - (uint) log_descriptor.bc.buffer->buffer_no, - (ulong) log_descriptor.bc.buffer, - (ulong) log_descriptor.bc.buffer->size)); - DBUG_ASSERT(log_descriptor.bc.buffer->buffer_no == + (uint) old_buffer->buffer_no, + (ulong) old_buffer, + (ulong) old_buffer->size)); + DBUG_ASSERT(old_buffer->buffer_no == log_descriptor.bc.buffer_no); } else @@ -7425,6 +7490,10 @@ static void translog_force_current_buffe log_descriptor.bc.buffer->offset= new_buff_beginning; log_descriptor.bc.write_counter= write_counter; log_descriptor.bc.previous_offset= previous_offset; + new_buffer->prev_last_lsn= BUFFER_MAX_LSN(old_buffer); + DBUG_PRINT("info", ("prev_last_lsn set to (%lu,0x%lx) buffer: 0x%lx", + LSN_IN_PARTS(new_buffer->prev_last_lsn), + (ulong) new_buffer)); /* Advances this log pointer, increases writers and let other threads to @@ -7552,6 +7621,7 @@ void translog_flush_set_new_goal_and_wai { log_descriptor.next_pass_max_lsn= lsn; log_descriptor.max_lsn_requester= pthread_self(); + pthread_cond_broadcast(&log_descriptor.new_goal_cond); } while (log_descriptor.flush_in_progress) { @@ -7563,66 +7633,65 @@ void translog_flush_set_new_goal_and_wai /** - @brief Flush the log up to given LSN (included) - - @param lsn log record serial number up to which (inclusive) - the log has to be flushed - - @return Operation status - @retval 0 OK - @retval 1 Error + @brief sync() range of files (inclusive) and directory (by request) + @param min from file + @param max to file + @param sync_dir need sync directory */ -my_bool translog_flush(TRANSLOG_ADDRESS lsn) +static my_bool translog_sync_files(uint32 min, uint32 max, + my_bool sync_dir) { - LSN sent_to_disk= LSN_IMPOSSIBLE; - TRANSLOG_ADDRESS flush_horizon; - uint fn, i; - dirty_buffer_mask_t dirty_buffer_mask; - uint8 last_buffer_no, start_buffer_no; + uint fn; my_bool rc= 0; - DBUG_ENTER("translog_flush"); - DBUG_PRINT("enter", ("Flush up to LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); - DBUG_ASSERT(translog_status == TRANSLOG_OK || - translog_status == TRANSLOG_READONLY); - LINT_INIT(sent_to_disk); + DBUG_ENTER("translog_sync_files"); + DBUG_PRINT("info", ("min: %lu max: %lu sync dir: %d", + (ulong) min, (ulong) max, (int) sync_dir)); + DBUG_ASSERT(min <= max); - pthread_mutex_lock(&log_descriptor.log_flush_lock); - DBUG_PRINT("info", ("Everything is flushed up to (%lu,0x%lx)", - LSN_IN_PARTS(log_descriptor.flushed))); - if (cmp_translog_addr(log_descriptor.flushed, lsn) >= 0) - { - pthread_mutex_unlock(&log_descriptor.log_flush_lock); - DBUG_RETURN(0); - } - if (log_descriptor.flush_in_progress) + flush_start= my_getsystime(); + for (fn= min; fn <= max; fn++) { - translog_flush_set_new_goal_and_wait(lsn); - if (!pthread_equal(log_descriptor.max_lsn_requester, pthread_self())) + TRANSLOG_FILE *file= get_logfile_by_number(fn); + DBUG_ASSERT(file != NULL); + if (!file->is_sync) { - /* fix lsn if it was horizon */ - if (cmp_translog_addr(lsn, log_descriptor.bc.buffer->last_lsn) > 0) - lsn= BUFFER_MAX_LSN(log_descriptor.bc.buffer); - translog_flush_wait_for_end(lsn); - pthread_mutex_unlock(&log_descriptor.log_flush_lock); - DBUG_RETURN(0); + if (my_sync(file->handler.file, MYF(MY_WME))) + { + rc= 1; + translog_stop_writing(); + DBUG_RETURN(rc); + } + file->is_sync= 1; } - log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; } - log_descriptor.flush_in_progress= 1; - flush_horizon= log_descriptor.previous_flush_horizon; - DBUG_PRINT("info", ("flush_in_progress is set")); - pthread_mutex_unlock(&log_descriptor.log_flush_lock); - translog_lock(); - if (log_descriptor.is_everything_flushed) - { - DBUG_PRINT("info", ("everything is flushed")); - rc= (translog_status == TRANSLOG_READONLY); - translog_unlock(); - goto out; - } + if (sync_dir) + rc|= sync_dir(log_descriptor.directory_fd, MYF(MY_WME | MY_IGNORE_BADFD)); + + DBUG_RETURN(rc); +} + + +/* + @brief Flushes buffers up to lsn + + @param lsn LSN to which we should flush + @parem sent_to_disk returns 'sent to disk' position + @param flush_horizon returns horizon of the flush + + @note See comment to translog_flush(). +*/ + +void translog_flush_buffers(TRANSLOG_ADDRESS *lsn, + TRANSLOG_ADDRESS *sent_to_disk, + TRANSLOG_ADDRESS *flush_horizon) +{ + dirty_buffer_mask_t dirty_buffer_mask; + uint i; + uint8 last_buffer_no, start_buffer_no; + DBUG_ENTER("translog_flush_buffers"); /* We will recheck information when will lock buffers one by @@ -7647,15 +7716,15 @@ my_bool translog_flush(TRANSLOG_ADDRESS /* if LSN up to which we have to flush bigger then maximum LSN of previous buffer and at least one LSN was saved in the current buffer (last_lsn != - LSN_IMPOSSIBLE) then we better finish the current buffer. + LSN_IMPOSSIBLE) then we have to close the current buffer. */ - if (cmp_translog_addr(lsn, log_descriptor.bc.buffer->prev_last_lsn) > 0 && + if (cmp_translog_addr(*lsn, log_descriptor.bc.buffer->prev_last_lsn) > 0 && log_descriptor.bc.buffer->last_lsn != LSN_IMPOSSIBLE) { struct st_translog_buffer *buffer= log_descriptor.bc.buffer; - lsn= log_descriptor.bc.buffer->last_lsn; /* fix lsn if it was horizon */ + *lsn= log_descriptor.bc.buffer->last_lsn; /* fix lsn if it was horizon */ DBUG_PRINT("info", ("LSN to flush fixed to last lsn: (%lu,0x%lx)", - LSN_IN_PARTS(log_descriptor.bc.buffer->last_lsn))); + LSN_IN_PARTS(log_descriptor.bc.buffer->last_lsn))); last_buffer_no= log_descriptor.bc.buffer_no; log_descriptor.is_everything_flushed= 1; translog_force_current_buffer_to_finish(); @@ -7667,8 +7736,10 @@ my_bool translog_flush(TRANSLOG_ADDRESS TRANSLOG_BUFFERS_NO); translog_unlock(); } - sent_to_disk= translog_get_sent_to_disk(); - if (cmp_translog_addr(lsn, sent_to_disk) > 0) + + /* flush buffers */ + *sent_to_disk= translog_get_sent_to_disk(); + if (cmp_translog_addr(*lsn, *sent_to_disk) > 0) { DBUG_PRINT("info", ("Start buffer #: %u last buffer #: %u", @@ -7688,44 +7759,215 @@ my_bool translog_flush(TRANSLOG_ADDRESS LSN_IN_PARTS(buffer->last_lsn), (buffer->file ? "dirty" : "closed"))); - if (buffer->prev_last_lsn <= lsn && + if (buffer->prev_last_lsn <= *lsn && buffer->file != NULL) { - DBUG_ASSERT(flush_horizon <= buffer->offset + buffer->size); - flush_horizon= buffer->offset + buffer->size; + DBUG_ASSERT(*flush_horizon <= buffer->offset + buffer->size); + *flush_horizon= (buffer->pre_force_close_horizon ? + buffer->pre_force_close_horizon : + buffer->offset + buffer->size); + DBUG_PRINT("info", ("flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(*flush_horizon))); + DBUG_ASSERT(*flush_horizon <= log_descriptor.horizon); translog_buffer_flush(buffer); } translog_buffer_unlock(buffer); i= (i + 1) % TRANSLOG_BUFFERS_NO; } while (i != last_buffer_no); - sent_to_disk= translog_get_sent_to_disk(); + *sent_to_disk= translog_get_sent_to_disk(); } - /* sync files from previous flush till current one */ - for (fn= LSN_FILE_NO(log_descriptor.flushed); fn <= LSN_FILE_NO(lsn); fn++) + DBUG_VOID_RETURN; +} + +/** + @brief Flush the log up to given LSN (included) + + @param lsn log record serial number up to which (inclusive) + the log has to be flushed + + @return Operation status + @retval 0 OK + @retval 1 Error + + @note + - Non group commit logic: Commits made in passes. Thread which started + flush first is performing actual flush, other threads sets new goal (LSN) + of the next pass (if it is maximum) and waits for the pass end or just + wait for the pass end. + - If hard group commit switched on and rate set to zero the first thread + repeat pass if finds at the end of pass new goals (just take them as its + new goal). The process can not loop forever because we have limited number + of threads. + - If hard group commit switched ON and rate is not zero then at the end of + the pass first tread waits for new goals but not more then need to have + actual rate close to set value. If time exceed it stop pass and let other + thread start new pass if time is not over take new goal and repeats the + pass. + - If soft group commit switched on everything work as without group commit + but this procedure do not perform real sync(). If rate is not zero real + sync() will be performed by service thread with the rate and if there was + new LSN appeared. + + @note Terminology: + 'sent to disk' means written to disk but not sync()ed, + 'flushed' mean sent to disk and synced(). +*/ + +my_bool translog_flush(TRANSLOG_ADDRESS lsn) +{ + struct timespec abstime; + ulonglong flush_interval; + ulonglong time_spent; + LSN sent_to_disk= LSN_IMPOSSIBLE; + TRANSLOG_ADDRESS flush_horizon; + my_bool rc= 0; + my_bool hgroup_commit_at_start; + + DBUG_ENTER("translog_flush"); + DBUG_PRINT("enter", ("Flush up to LSN: (%lu,0x%lx)", LSN_IN_PARTS(lsn))); + DBUG_ASSERT(translog_status == TRANSLOG_OK || + translog_status == TRANSLOG_READONLY); + LINT_INIT(sent_to_disk); + + pthread_mutex_lock(&log_descriptor.log_flush_lock); + DBUG_PRINT("info", ("Everything is flushed up to (%lu,0x%lx)", + LSN_IN_PARTS(log_descriptor.flushed))); + if (cmp_translog_addr(log_descriptor.flushed, lsn) >= 0) { - TRANSLOG_FILE *file= get_logfile_by_number(fn); - DBUG_ASSERT(file != NULL); - if (!file->is_sync) + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + DBUG_RETURN(0); + } + if (log_descriptor.flush_in_progress) + { + translog_lock(); + /* fix lsn if it was horizon */ + if (cmp_translog_addr(lsn, log_descriptor.bc.buffer->last_lsn) > 0) + lsn= BUFFER_MAX_LSN(log_descriptor.bc.buffer); + translog_unlock(); + translog_flush_set_new_goal_and_wait(lsn); + if (!pthread_equal(log_descriptor.max_lsn_requester, pthread_self())) { - if (my_sync(file->handler.file, MYF(MY_WME))) + translog_flush_wait_for_end(lsn); + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + DBUG_RETURN(0); + } + log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; + } + log_descriptor.flush_in_progress= 1; + flush_horizon= log_descriptor.previous_flush_horizon; + DBUG_PRINT("info", ("flush_in_progress is set, flush_horizon: (%lu,0x%lx)", + LSN_IN_PARTS(flush_horizon))); + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + + my_atomic_rwlock_wrlock(&hgroup_commit_rwl); + hgroup_commit_at_start= my_atomic_load32(&hgroup_commit); + my_atomic_rwlock_wrunlock(&hgroup_commit_rwl); + if (hgroup_commit_at_start) + { + my_atomic_rwlock_rdlock(&group_commit_wait_rwl); + flush_interval= my_atomic_load32(&group_commit_wait) * TRANSLOG_RATE_BASE; + my_atomic_rwlock_rdunlock(&group_commit_wait_rwl); + } + + translog_lock(); + if (log_descriptor.is_everything_flushed) + { + DBUG_PRINT("info", ("everything is flushed")); + rc= (translog_status == TRANSLOG_READONLY); + translog_unlock(); + goto out; + } + + for (;;) + { + translog_flush_buffers(&lsn, &sent_to_disk, &flush_horizon); + + if (!hgroup_commit_at_start) + break; /* flush pass is ended */ + +retest: + if (flush_interval != 0 && + (my_getsystime() - flush_start) >= flush_interval) + break; /* flush pass is ended */ + + pthread_mutex_lock(&log_descriptor.log_flush_lock); + if (log_descriptor.next_pass_max_lsn != LSN_IMPOSSIBLE) + { + /* take next goal */ + lsn= log_descriptor.next_pass_max_lsn; + log_descriptor.next_pass_max_lsn= LSN_IMPOSSIBLE; + /* prevent other thread from continue */ + log_descriptor.max_lsn_requester= pthread_self(); + DBUG_PRINT("info", ("flush took next goal: (%lu,0x%lx)", + LSN_IN_PARTS(lsn))); + } + else + { + if (flush_interval == 0 || + (time_spent= (my_getsystime() - flush_start)) >= flush_interval) { - rc= 1; - translog_stop_writing(); - sent_to_disk= LSN_IMPOSSIBLE; - goto out; + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + break; } - file->is_sync= 1; + DBUG_PRINT("info", ("flush waits: %llu interval: %llu spent: %llu", + flush_interval - time_spent, + flush_interval, time_spent)); + /* wait time or next goal */ + set_timespec_nsec(abstime, flush_interval - time_spent); + pthread_cond_timedwait(&log_descriptor.new_goal_cond, + &log_descriptor.log_flush_lock, + &abstime); + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + DBUG_PRINT("info", ("retest conditions")); + goto retest; } + pthread_mutex_unlock(&log_descriptor.log_flush_lock); + + /* next flush pass */ + DBUG_PRINT("info", ("next flush pass")); + translog_lock(); } - if (sync_log_dir >= TRANSLOG_SYNC_DIR_ALWAYS && - (LSN_FILE_NO(log_descriptor.previous_flush_horizon) != - LSN_FILE_NO(flush_horizon) || - ((LSN_OFFSET(log_descriptor.previous_flush_horizon) - 1) / - TRANSLOG_PAGE_SIZE) != - ((LSN_OFFSET(flush_horizon) - 1) / TRANSLOG_PAGE_SIZE))) - rc|= sync_dir(log_descriptor.directory_fd, MYF(MY_WME | MY_IGNORE_BADFD)); + /* + sync() files from previous flush till current one + + We read soft_sync unprotected because it is actually 1 bit + value and we do not care much about "a bit old value" speed + of test is much more valuable here. + */ + if (!soft_sync || hgroup_commit_at_start) + { + if ((rc= + translog_sync_files(LSN_FILE_NO(log_descriptor.flushed), + LSN_FILE_NO(lsn), + sync_log_dir >= TRANSLOG_SYNC_DIR_ALWAYS && + (LSN_FILE_NO(log_descriptor. + previous_flush_horizon) != + LSN_FILE_NO(flush_horizon) || + ((LSN_OFFSET(log_descriptor. + previous_flush_horizon) - 1) / + TRANSLOG_PAGE_SIZE) != + ((LSN_OFFSET(flush_horizon) - 1) / + TRANSLOG_PAGE_SIZE))))) + { + sent_to_disk= LSN_IMPOSSIBLE; + goto out; + } + /* keep values for soft sync() and forced sync() actual */ + my_atomic_rwlock_wrlock(&soft_sync_rwl); + my_atomic_store32(&soft_sync_min, LSN_FILE_NO(lsn)); + my_atomic_store32(&soft_sync_max, LSN_FILE_NO(lsn)); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); + } + else + { + my_atomic_rwlock_wrlock(&soft_sync_rwl); + my_atomic_store32(&soft_sync_max, LSN_FILE_NO(lsn)); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); + } + + DBUG_ASSERT(flush_horizon <= log_descriptor.horizon); log_descriptor.previous_flush_horizon= flush_horizon; out: pthread_mutex_lock(&log_descriptor.log_flush_lock); @@ -7733,7 +7975,7 @@ out: log_descriptor.flushed= sent_to_disk; log_descriptor.flush_in_progress= 0; DBUG_PRINT("info", ("flush_in_progress is dropped")); - pthread_mutex_unlock(&log_descriptor.log_flush_lock);\ + pthread_mutex_unlock(&log_descriptor.log_flush_lock); pthread_cond_broadcast(&log_descriptor.log_flush_cond); DBUG_RETURN(rc); } @@ -8103,6 +8345,7 @@ LSN translog_first_theoretical_lsn() my_bool translog_purge(TRANSLOG_ADDRESS low) { uint32 last_need_file= LSN_FILE_NO(low); + uint32 min_unsync; TRANSLOG_ADDRESS horizon= translog_get_horizon(); int rc= 0; DBUG_ENTER("translog_purge"); @@ -8110,6 +8353,12 @@ my_bool translog_purge(TRANSLOG_ADDRESS DBUG_ASSERT(translog_status == TRANSLOG_OK || translog_status == TRANSLOG_READONLY); + my_atomic_rwlock_wrlock(&soft_sync_rwl); + min_unsync= my_atomic_load32(&soft_sync_min); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); + if (min_unsync < last_need_file) + last_need_file= min_unsync; + pthread_mutex_lock(&log_descriptor.purger_lock); if (LSN_FILE_NO(log_descriptor.last_lsn_checked) < last_need_file) { @@ -8346,6 +8595,160 @@ my_bool translog_log_debug_info(TRN *trn } + +/** + Sets soft sync mode + + @param mode TRUE if we need switch soft sync on else off +*/ + +void translog_soft_sync(my_bool mode) +{ + my_atomic_rwlock_wrlock(&soft_sync_rwl); + my_atomic_store32(&soft_sync, (uint32) mode); + my_atomic_rwlock_wrunlock(&soft_sync_rwl); +} + + +/** + Sets hard group commit + + @param mode TRUE if we need switch hard group commit on else off +*/ + +void translog_hard_group_commit(my_bool mode) +{ + my_atomic_rwlock_wrlock(&hgroup_commit_rwl); + my_atomic_store32(&hgroup_commit, (uint32) mode); + my_atomic_rwlock_wrunlock(&hgroup_commit_rwl); +} + + +/** + @brief forced log sync (used when we are switching modes) +*/ + +void translog_sync() +{ + uint32 max= get_current_logfile()->number; + uint32 min; + DBUG_ENTER("ma_translog_sync"); + + my_atomic_rwlock_rdlock(&soft_sync_rwl); + min= my_atomic_load32(&soft_sync_min); + my_atomic_rwlock_rdunlock(&soft_sync_rwl); + if (!min) + min= max; + + translog_sync_files(min, max, sync_log_dir >= TRANSLOG_SYNC_DIR_ALWAYS); + + DBUG_VOID_RETURN; +} + + +/** + @brief set rate for group commit + + @param rate rate to set. + + @note internally it stores interval in nanoseconds/TRANSLOG_RATE_BASE (to + fit into uint32) +*/ + +void translog_set_group_commit_rate(uint32 rate) +{ + DBUG_ENTER("translog_set_group_commit_rate"); + ulonglong wait_time; + if (rate) + { + wait_time= ((TRANSLOG_RATE_BASE * 1000000000ULL / rate + + TRANSLOG_RATE_BASE / 2) / + TRANSLOG_RATE_BASE); + if (wait_time == 0) + wait_time= 1; /* protection from getting special value */ + } + else + wait_time= 0; + my_atomic_rwlock_wrlock(&group_commit_wait_rwl); + my_atomic_store32(&group_commit_wait, (uint32)wait_time); + my_atomic_rwlock_wrunlock(&group_commit_wait_rwl); + DBUG_PRINT("info", ("rate: %lu wait: %llu", + (ulong)rate, (ulonglong)wait_time)); + DBUG_VOID_RETURN; +} + + +/** + @brief syncing service thread +*/ + +static pthread_handler_t +ma_soft_sync_background( void *arg __attribute__((unused))) +{ + + my_thread_init(); + DBUG_ENTER("ma_soft_sync_background"); + for(;;) + { + ulonglong prev_loop= my_getsystime(); + ulonglong time, sleep; + uint32 min, max; + my_atomic_rwlock_rdlock(&soft_sync_rwl); + min= my_atomic_load32(&soft_sync_min); + max= my_atomic_load32(&soft_sync_max); + my_atomic_store32(&soft_sync_min, max); + my_atomic_rwlock_rdunlock(&soft_sync_rwl); + + my_atomic_rwlock_rdlock(&group_commit_wait_rwl); + sleep= my_atomic_load32(&group_commit_wait) * TRANSLOG_RATE_BASE; + my_atomic_rwlock_rdunlock(&group_commit_wait_rwl); + translog_sync_files(min, max, FALSE); + time= my_getsystime() - prev_loop; + if (time > sleep) + sleep= 0; + else + sleep-= time; + if (my_service_thread_sleep(&soft_sync_control, sleep)) + break; + } + my_service_thread_signal_end(&soft_sync_control); + my_thread_end(); + DBUG_RETURN(0); +} + + +/** + @brief Starts syncing thread +*/ + +int translog_soft_sync_start(void) +{ + pthread_t th; + int res= 0; + DBUG_ENTER("translog_soft_sync_start"); + if (ma_service_thread_control_init(&soft_sync_control)) + res= 1; + else if (!(res= pthread_create(&th, NULL, ma_soft_sync_background, NULL))) + soft_sync_control.status= THREAD_RUNNING; + DBUG_RETURN(res); +} + + +/** + @brief Stops syncing thread +*/ + +void translog_soft_sync_end(void) +{ + DBUG_ENTER("translog_soft_sync_end"); + if (soft_sync_control.inited) + { + ma_service_thread_control_end(&soft_sync_control); + } + DBUG_VOID_RETURN; +} + + #ifdef MARIA_DUMP_LOG #include <my_getopt.h> extern void translog_example_table_init(); === modified file 'storage/maria/ma_loghandler.h' --- a/storage/maria/ma_loghandler.h 2009-01-15 22:25:53 +0000 +++ b/storage/maria/ma_loghandler.h 2009-03-04 16:13:55 +0000 @@ -343,6 +343,13 @@ enum enum_translog_status }; extern enum enum_translog_status translog_status; +void translog_soft_sync(my_bool mode); +void translog_hard_group_commit(my_bool mode); +int translog_soft_sync_start(void); +void translog_soft_sync_end(void); +void translog_sync(); +void translog_set_group_commit_rate(uint32 rate); + /* all the rest added because of recovery; should we make ma_loghandler_for_recovery.h ? @@ -441,6 +448,18 @@ extern LOG_DESC log_record_type_descript typedef enum { + TRANSLOG_GCOMMIT_NONE, + TRANSLOG_GCOMMIT_HARD, + TRANSLOG_GCOMMIT_SOFT +} enum_maria_group_commit; +extern ulong group_commit; +extern ulong group_commit_rate; +/** + group commit interval is TRANSLOG_RATE_BASE/<rate> seconds +*/ +#define TRANSLOG_RATE_BASE 100 +typedef enum +{ TRANSLOG_PURGE_IMMIDIATE, TRANSLOG_PURGE_EXTERNAL, TRANSLOG_PURGE_ONDEMAND === modified file 'tests/fork_big2.pl' --- a/tests/fork_big2.pl 2006-02-12 21:26:30 +0000 +++ b/tests/fork_big2.pl 2009-03-04 16:13:55 +0000 @@ -16,21 +16,21 @@ package main; $opt_skip_create=$opt_skip_in=$opt_verbose=$opt_fast_insert= $opt_lock_tables=$opt_debug=$opt_skip_drop=$opt_fast=$opt_force=0; -$opt_thread_factor=1; -$opt_insert=1; -$opt_select=6;$opt_join=4; -$opt_select_count=$opt_join_count=0; -$opt_update=1;$opt_delete=0; -$opt_flush=$opt_check=$opt_repair=$opt_alter=0; -$opt_join_range=100; +$opt_thread_factor=1; +$opt_insert=1; +$opt_select=6;$opt_join=4; +$opt_select_count=$opt_join_count=0; +$opt_update=1;$opt_delete=0; +$opt_flush=$opt_check=$opt_repair=$opt_alter=0; +$opt_join_range=100; $opt_resize_interval=0; $opt_time=0; $opt_host=$opt_user=$opt_password=""; $opt_db="test"; $opt_verbose=$opt_debug=$opt_lock_tables=$opt_fast_insert=$opt_fast=$opt_skip_in=$opt_force=undef; # Ignore warnings from these -GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", - "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", - "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", +GetOptions("host=s","db=s","user=s","password=s","loop-count=i","skip-create","skip-in","skip-drop", + "verbose","fast-insert","lock-tables","debug","fast","force","thread-factor=i", + "insert=i", "select=i", "join=i", "select-count=i", "join-count=i", "update=i", "delete=i", "flush=i", "check=i", "repair=i", "alter=i", "resize-interval=i", "max-join_range=i", "time=i") || die "Aborted"; print "Test of multiple connections that test the following things:\n"; @@ -48,20 +48,20 @@ srand 100; # Make random numbers repea #### #### Start timeing and start test -#### - +#### + $opt_insert*=$opt_thread_factor; -$opt_select*=$opt_thread_factor; -$opt_join*=$opt_thread_factor; -$opt_select_count*=$opt_thread_factor; -$opt_join_count*=$opt_thread_factor; -$opt_update*=$opt_thread_factor; -$opt_delete*=$opt_thread_factor; - -if ($opt_time == 0 && $opt_insert == 0) -{ - $opt_insert=1; -} +$opt_select*=$opt_thread_factor; +$opt_join*=$opt_thread_factor; +$opt_select_count*=$opt_thread_factor; +$opt_join_count*=$opt_thread_factor; +$opt_update*=$opt_thread_factor; +$opt_delete*=$opt_thread_factor; + +if ($opt_time == 0 && $opt_insert == 0) +{ + $opt_insert=1; +} $start_time=new Benchmark; $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", @@ -100,71 +100,71 @@ $|= 1; # Autoflush #### #### Start the tests #### -if ($opt_time != 0) -{ - test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; +if ($opt_time != 0) +{ + test_abort() if (($pid=fork()) == 0); $work{$pid}="abort"; } for ($i=0 ; $i < $opt_insert ; $i ++) { test_insert() if (($pid=fork()) == 0); $work{$pid}="insert"; -} +} $threads=$i; -for ($i=0 ; $i < $opt_select ; $i ++) -{ - test_select() if (($pid=fork()) == 0); $work{$pid}="select"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_join ; $i ++) -{ - test_join() if (($pid=fork()) == 0); $work{$pid}="join"; -} -$threads+=$i; +for ($i=0 ; $i < $opt_select ; $i ++) +{ + test_select() if (($pid=fork()) == 0); $work{$pid}="select"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_join ; $i ++) +{ + test_join() if (($pid=fork()) == 0); $work{$pid}="join"; +} +$threads+=$i; for ($i=0 ; $i < $opt_select_count ; $i ++) { test_select_count() if (($pid=fork()) == 0); $work{$pid}="select_count"; } -$threads+=$i; -for ($i=0 ; $i < $opt_join_count ; $i ++) -{ - test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_update ; $i ++) -{ - test_update() if (($pid=fork()) == 0); $work{$pid}="update"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_delete ; $i ++) -{ - test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_flush ; $i ++) -{ - test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_check ; $i ++) -{ - test_check() if (($pid=fork()) == 0); $work{$pid}="check"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_repair ; $i ++) -{ - test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; -} -$threads+=$i; -for ($i=0 ; $i < $opt_alter ; $i ++) -{ - test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; -} -$threads+=$i; +$threads+=$i; +for ($i=0 ; $i < $opt_join_count ; $i ++) +{ + test_join_count() if (($pid=fork()) == 0); $work{$pid}="join_count"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_update ; $i ++) +{ + test_update() if (($pid=fork()) == 0); $work{$pid}="update"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_delete ; $i ++) +{ + test_delete() if (($pid=fork()) == 0); $work{$pid}="delete"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_flush ; $i ++) +{ + test_flush() if (($pid=fork()) == 0); $work{$pid}="flush"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_check ; $i ++) +{ + test_check() if (($pid=fork()) == 0); $work{$pid}="check"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_repair ; $i ++) +{ + test_repair() if (($pid=fork()) == 0); $work{$pid}="repair"; +} +$threads+=$i; +for ($i=0 ; $i < $opt_alter ; $i ++) +{ + test_alter() if (($pid=fork()) == 0); $work{$pid}="alter"; +} +$threads+=$i; if ($opt_resize_interval != 0) { test_resize() if (($pid=fork()) == 0); $work{$pid}="resize"; $threads+=1; } - + print "Started $threads threads\n"; $errors=0; @@ -172,17 +172,17 @@ $running_insert_threads=$opt_insert; while (($pid=wait()) != -1) { $ret=$?/256; - print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; - if ($opt_time == 0) + print "thread '" . $work{$pid} . "' finished with exit code $ret\n"; + if ($opt_time == 0) { if ($work{$pid} =~ /^insert/) { if (!--$running_insert_threads) - { - + { + # Time to stop other threads signal_abort(); - } + } } } $errors++ if ($ret != 0); @@ -214,17 +214,17 @@ print "Total time: " . exit(0); -# -# Sleep and then abort other threads -# - -sub test_abort -{ - sleep($opt_time); - signal_abort(); - exit(0); -} - +# +# Sleep and then abort other threads +# + +sub test_abort +{ + sleep($opt_time); + signal_abort(); + exit(0); +} + # # Insert records in the table @@ -363,58 +363,58 @@ sub test_join $dbh->disconnect; $dbh=0; print "Test_join: Executed $count joins\n"; exit(0); -} - -# -# select records -# Do continously joins between the first and second for range and count selected rows -# - -sub test_join_count -{ - my ($dbh, $i, $j, $count, $loop); - - $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", - $opt_user, $opt_password, - { PrintError => 0}) || die $DBI::errstr; - - $count_query=make_count_query($numtables); - $count=0; - $loop=9999; - $sum=0; - - srand(); - - $i=0; - while (($i++ % 10) || !test_if_abort($dbh)) - { - if ($loop++ >= 10) - { - $loop=0; - $row_counts=simple_query($dbh, $count_query); - } - for ($j=0 ; $j < $numtables-1 ; $j++) - { - my ($id1)= int rand $row_counts->[$j]; - my ($id2)= int rand $row_counts->[$j]; - if ($id1 > $id2) - { - my $id0=$id1; $id1=$id2; $id2=$id0; - if ($id2-$id1 > $opt_join_range) - { - $id2=$id1+$opt_join_range; - } - } - my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); - $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); - $sum+=$row->[0]; - $count++; - } - } - $dbh->disconnect; $dbh=0; - print "Test_join_count: Executed $count joins: total $sum rows\n"; - exit(0); -} +} + +# +# select records +# Do continously joins between the first and second for range and count selected rows +# + +sub test_join_count +{ + my ($dbh, $i, $j, $count, $loop); + + $dbh = DBI->connect("DBI:mysql:$opt_db:$opt_host", + $opt_user, $opt_password, + { PrintError => 0}) || die $DBI::errstr; + + $count_query=make_count_query($numtables); + $count=0; + $loop=9999; + $sum=0; + + srand(); + + $i=0; + while (($i++ % 10) || !test_if_abort($dbh)) + { + if ($loop++ >= 10) + { + $loop=0; + $row_counts=simple_query($dbh, $count_query); + } + for ($j=0 ; $j < $numtables-1 ; $j++) + { + my ($id1)= int rand $row_counts->[$j]; + my ($id2)= int rand $row_counts->[$j]; + if ($id1 > $id2) + { + my $id0=$id1; $id1=$id2; $id2=$id0; + if ($id2-$id1 > $opt_join_range) + { + $id2=$id1+$opt_join_range; + } + } + my ($t1,$t2)= ($testtables[$j]->[0],$testtables[$j+1]->[0]); + $row=simple_query($dbh, "select count(*) from $t1, $t2 where $t1.id=$t2.id and $t1.id between $id1 and $id2"); + $sum+=$row->[0]; + $count++; + } + } + $dbh->disconnect; $dbh=0; + print "Test_join_count: Executed $count joins: total $sum rows\n"; + exit(0); +} #

1 0

[Maria-developers] bzr commit into mariadb 5.1, with Maria 2.0:maria/2.0 branch (sanja:2721)
by sanja＠askmonty.org 04 Mar '09

04 Mar '09

#At lp:maria/2.0 2721 sanja(a)askmonty.org 2009-03-04 Tree config modified: .bzr-mysql/default.conf per-file messages: .bzr-mysql/default.conf Tree config === modified file '.bzr-mysql/default.conf' --- a/.bzr-mysql/default.conf 2009-01-19 08:25:12 +0000 +++ b/.bzr-mysql/default.conf 2009-03-04 13:44:41 +0000 @@ -1,6 +1,6 @@ [MYSQL] -tree_location = bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-5.1-maria-2.0/ -post_commit_to = maria(a)lists.mysql.com -post_commit_url = bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-5.1-maria-2.0/ -tree_name = mysql-5.1-maria-2.0 -project_name = MySQL/Maria +tree_location = lp:maria/2.0 +post_commit_to = maria-developers(a)lists.launchpad.net +post_commit_url = lp:maria/2.0 +tree_name = maria/2.0 +project_name = "mariadb 5.1, with Maria 2.0"

1 0

[Maria-developers] bzr commit into mariadb 5.1, with Maria 1.5:maria branch (sanja:2677)
by sanja＠askmonty.org 04 Mar '09

04 Mar '09

#At lp:maria 2677 sanja(a)askmonty.org 2009-03-04 Tree config per-file messages: .bzr-mysql/default.conf Tree config

1 0

[Maria-developers] questions about Maria
by MARK CALLAGHAN 02 Mar '09

02 Mar '09

Are checksums used for log pages? I did not see anything in ma_loghandler.c for this. Can I do a hot copy/backup of a table? That is can I copy the data and index files while transactions may be in progress for them and then replay log files to recover them? I prefer a FLUSH LOGS command that is limited to Maria not all things that might be considered a log in MySQL. Are there plans to provide more engine specific commands? What is the purpose of a 'Row checksum'? Comments in the code state that this uses 1 byte and another that it uses 4 bytes. How many bytes on disk are used? -- Mark Callaghan mdcallag(a)gmail.com

2 1

[Maria-developers] Fw: Launchpad offline 00.00 - 01.00 UTC 26th February 2009
by Kurt von Finck 25 Feb '09

25 Feb '09

Begin forwarded message: Date: Tue, 24 Feb 2009 18:09:57 +0000 From: Matthew Revell To: redacted internal Canonical list Subject: Launchpad offline 00.00 - 01.00 UTC 26th February 2009 Launchpad will be offline from 00.00 - 01.00 UTC on Thursday 26th February. This is to allow us to roll-out the code for our latest release, Launchpad 2.2.2. Please see the Launchpad blog -- http://blog.launchpad.net -- for details of the release! -- Matthew Revell - Launchpad.net - free software collaboration and project hosting -- Launchpad-announce mailing list Launchpad-announce(a)lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/launchpad-announce -- ./k kurt von finck http://www.mneptok.com public key: keyserver.ubuntu.com key id: 5229D26A fingerprint: 127A A484 ADBF A5AD E7FB 8CD2 8913 18F4 5229 D26A Music is the space between the notes. - Debussy

1 0

Re: [Maria-developers] bzr commit into MySQL/Maria:mysql-maria branch (monty:2725) Bug#40711
by Michael Widenius 24 Feb '09

24 Feb '09

Hi! >>>>> "Guilhem" == Guilhem Bichot <guilhem(a)sun.com> writes: <cut> >> +++ b/storage/maria/ma_close.c 2009-02-01 07:29:56 +0000 >> @@ -155,7 +155,7 @@ int maria_close(register MARIA_HA *info) >> MARIA_STATE_HISTORY_CLOSED *history; >> /* >> Here we ignore the unlikely case that we don't have memory to >> - store the case. In the worst case what happens is that any transaction >> + store the state. In the worst case what happens is that any transaction >> that tries to access this table will get a wrong status information. >> */ >> if ((history= (MARIA_STATE_HISTORY_CLOSED *) >> @@ -166,6 +166,8 @@ int maria_close(register MARIA_HA *info) >> if (my_hash_insert(&maria_stored_state, (uchar*) history)) >> my_free(history, MYF(0)); >> } >> + /* Marker for concurrent checkpoint */ >> + share->state_history= 0; >> } >> } >> pthread_mutex_unlock(&THR_LOCK_maria); Guilhem> So, what was the scenario? Guilhem> I imagined this one: Thread1 has just closed the table, but Thread2 had Guilhem> started a checkpoint, so Thread1 leaves the share in existence; Thread2 Guilhem> looks at share->state_history, but why would that point to freed memory? Guilhem> Is it because there is a Thread3 which did this freeing? No, the problem happens with only two threads. T1 one goes into the above code and creates a history entry that points to share->state_history. In checkpoint thread we will then execute the following code: _ma_remove_not_visible_states_with_lock(share, FALSE); Which uses share->state_history entry and may even free it. If free happens, we are likely to get a crash as history entry now points to freed memory. Regards, Monty

1 0