- developers - lists.mariadb.org

[Maria-developers] per-partition attributes in the CREATE TABLE
by Sergei Golubchik 09 Jun '10

09 Jun '10

Hi, we've talked about engine attributes in the CREATE TABLE, and that one should be able to specify them per partition as well. Now, thinking about it, I'm not quite sure what the semantics shuld be. What is your use case ? How do you want them to work ? I see different possibilities. Say, there is create table ... (.....) XXX=1 partition by list (a) ( partition p0 values in (1) YYY=2, partition p1 values in (2) ); 1. We can say that XXX should be listed in the engine's hton->table_options, and YYY - in the hton->partition_options. this works fine because the engine can use table_share->option_struct and it will contain correct values, independent from whether a table is partitioned or not. but it will break when partitioning will support different engines in different partitions. 2. We can say that XXX is partition engine options, and a pluggable engine can only see YYY. YYY should come from hton->partition_options, and the engine's table level attributes are not applicable. The drawback - every engine needs to have special code to take care of the partitioned case, and to duplicate all table-level options in the partition-level options. 3. Same as 2, but YYY can come from either table level or partition level arrays. Every engine still needs the special code for partitioned case, but does not need to duplicate the table_options array. Regards, Sergei

5 6

[Maria-developers] Ever wanted to save/restore your GDB breakpoints? Here's a solution.
by Timour Katchaounov 09 Jun '10

09 Jun '10

Hi, Today I got really annoyed while debugging a server crash with GDB. I never managed to make the same GDB reattach to a new process after a crash. This means that after each crash one has to restart GDB, and reattach to the new process. As a result all breakpoints are lost, and need to be set each time, which is pretty annoying. Today I thought that others might have been equally annoyed, and might have figured a solution. Indeed, Google is our friend, and I found the following short snippet that should be added to .gdbinit: define bsave shell rm -f brestore.txt set logging file brestore.txt set logging on info break set logging off # reformat on-the-fly to a valid gdb command file shell perl -n -e 'print "break $1\n" if /^\d+.+?(\S+)$/g' brestore.txt > brestore.gdb end document bsave store actual breakpoints end define brestore source brestore.gdb end document brestore restore breakpoints saved by bsave end The credits go to this page: http://stackoverflow.com/questions/501486/getting-gdb-to-save-a-list-of-bre… Happy debugging, Timour

1 1

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2850)
by knielsen＠knielsen-hq.org 09 Jun '10

09 Jun '10

#At lp:maria 2850 knielsen(a)knielsen-hq.org 2010-06-09 MWL#116: Fix a couple of races in group commit. modified: include/atomic/gcc_builtins.h include/atomic/x86-gcc.h sql/handler.cc === modified file 'include/atomic/gcc_builtins.h' --- a/include/atomic/gcc_builtins.h 2008-02-06 16:55:04 +0000 +++ b/include/atomic/gcc_builtins.h 2010-06-09 11:17:39 +0000 @@ -19,8 +19,9 @@ v= __sync_lock_test_and_set(a, v); #define make_atomic_cas_body(S) \ int ## S sav; \ - sav= __sync_val_compare_and_swap(a, *cmp, set); \ - if (!(ret= (sav == *cmp))) *cmp= sav; + int ## S cmp_val= *cmp; \ + sav= __sync_val_compare_and_swap(a, cmp_val, set);\ + if (!(ret= (sav == cmp_val))) *cmp= sav #ifdef MY_ATOMIC_MODE_DUMMY #define make_atomic_load_body(S) ret= *a === modified file 'include/atomic/x86-gcc.h' --- a/include/atomic/x86-gcc.h 2007-02-28 16:50:51 +0000 +++ b/include/atomic/x86-gcc.h 2010-06-09 11:17:39 +0000 @@ -38,15 +38,33 @@ #define asm __asm__ #endif +/* + The atomic operations imply a memory barrier for the CPU, to ensure that all + prior writes are flushed from cache, and all subsequent reads reloaded into + cache. + + We need to imply a similar memory barrier for the compiler, so that all + pending stores (to memory that may be aliased in other parts of the code) + will be flushed to memory before the operation, and all reads from such + memory be re-loaded. This is achieved by adding the "memory" pseudo-register + to the clobber list, see GCC documentation for more explanation. + + The compiler and CPU memory barriers are needed to make sure changes in one + thread are made visible in another by the atomic operation. +*/ #ifndef MY_ATOMIC_NO_XADD #define make_atomic_add_body(S) \ - asm volatile (LOCK_prefix "; xadd %0, %1;" : "+r" (v) , "+m" (*a)) + asm volatile (LOCK_prefix "; xadd %0, %1;" : "+r" (v) , "+m" (*a): : "memory") #endif #define make_atomic_fas_body(S) \ - asm volatile ("xchg %0, %1;" : "+r" (v) , "+m" (*a)) + asm volatile ("xchg %0, %1;" : "+r" (v) , "+m" (*a) : : "memory") #define make_atomic_cas_body(S) \ + int ## S sav; \ asm volatile (LOCK_prefix "; cmpxchg %3, %0; setz %2;" \ - : "+m" (*a), "+a" (*cmp), "=q" (ret): "r" (set)) + : "+m" (*a), "=a" (sav), "=q" (ret) \ + : "r" (set), "a" (*cmp) : "memory"); \ + if (!ret) \ + *cmp= sav #ifdef MY_ATOMIC_MODE_DUMMY #define make_atomic_load_body(S) ret=*a @@ -59,9 +77,9 @@ #define make_atomic_load_body(S) \ ret=0; \ asm volatile (LOCK_prefix "; cmpxchg %2, %0" \ - : "+m" (*a), "+a" (ret): "r" (ret)) + : "+m" (*a), "+a" (ret) : "r" (ret) : "memory") #define make_atomic_store_body(S) \ - asm volatile ("; xchg %0, %1;" : "+m" (*a), "+r" (v)) + asm volatile ("; xchg %0, %1;" : "+m" (*a), "+r" (v) : : "memory") #endif /* TODO test on intel whether the below helps. on AMD it makes no difference */ === modified file 'sql/handler.cc' --- a/sql/handler.cc 2010-05-26 08:16:18 +0000 +++ b/sql/handler.cc 2010-06-09 11:17:39 +0000 @@ -1103,14 +1103,30 @@ ha_check_and_coalesce_trx_read_only(THD static THD * enqueue_atomic(THD *thd) { - my_atomic_rwlock_wrlock(&LOCK_group_commit_queue); + THD *orig_queue; + thd->next_commit_ordered= group_commit_queue; + + my_atomic_rwlock_wrlock(&LOCK_group_commit_queue); + do + { + /* + Save the read value of group_commit_queue in each iteration of the loop. + When my_atomic_casptr() returns TRUE, we know that orig_queue is equal + to the value of group_commit_queue when we enqueued. + + However, as soon as we enqueue, thd->next_commit_ordered may be + invalidated by another thread (the group commit leader). So we need to + save the old queue value in a local variable orig_queue like this. + */ + orig_queue= thd->next_commit_ordered; + } while (!my_atomic_casptr((void **)(&group_commit_queue), (void **)(&thd->next_commit_ordered), - thd)) - ; + thd)); my_atomic_rwlock_wrunlock(&LOCK_group_commit_queue); - return thd->next_commit_ordered; + + return orig_queue; } static THD * @@ -1399,6 +1415,9 @@ int ha_commit_trans(THD *thd, bool all) int cookie; if (tc_log->use_group_log_xid) { + // ToDo: if xid==NULL here, we may use is_group_commit_leader uninitialised. + // ToDo: Same for cookie below when xid==NULL. + // Seems we generally need to check the case xid==NULL. if (is_group_commit_leader) { pthread_mutex_lock(&LOCK_group_commit); @@ -1434,9 +1453,18 @@ int ha_commit_trans(THD *thd, bool all) } pthread_mutex_unlock(&LOCK_group_commit); - /* Wake up everyone except ourself. */ - while ((queue= queue->next_commit_ordered) != NULL) - group_commit_wakeup_other(queue); + /* Wake up everyone except ourself. */ + THD *current= queue->next_commit_ordered; + while (current != NULL) + { + /* + Careful not to access current->next_commit_ordered after waking up + the other thread! As it may change immediately after wakeup. + */ + THD *next= current->next_commit_ordered; + group_commit_wakeup_other(current); + current= next; + } } else {

1 0

[Maria-developers] Fixes for an issue with my_atomic
by Kristian Nielsen 09 Jun '10

09 Jun '10

Hi, I had a problem with my group commit patch, and tracked it down to a problem in my_atomic. The issue is that my_atomic_cas*(val, cmp, new) accesses *cmp after successful CAS operation (in one place reading it, in another place writing it). Here is the fix: Index: work-5.1-groupcommit/include/atomic/gcc_builtins.h =================================================================== --- work-5.1-groupcommit.orig/include/atomic/gcc_builtins.h 2010-06-09 11:53:59.000000000 +0200 +++ work-5.1-groupcommit/include/atomic/gcc_builtins.h 2010-06-09 11:54:06.000000000 +0200 @@ -19,8 +19,9 @@ v= __sync_lock_test_and_set(a, v); #define make_atomic_cas_body(S) \ int ## S sav; \ - sav= __sync_val_compare_and_swap(a, *cmp, set); \ - if (!(ret= (sav == *cmp))) *cmp= sav; + int ## S cmp_val= *cmp; \ + sav= __sync_val_compare_and_swap(a, cmp_val, set);\ + if (!(ret= (sav == cmp_val))) *cmp= sav #ifdef MY_ATOMIC_MODE_DUMMY #define make_atomic_load_body(S) ret= *a Index: work-5.1-groupcommit/include/atomic/x86-gcc.h =================================================================== --- work-5.1-groupcommit.orig/include/atomic/x86-gcc.h 2010-06-09 11:53:59.000000000 +0200 +++ work-5.1-groupcommit/include/atomic/x86-gcc.h 2010-06-09 11:54:06.000000000 +0200 @@ -45,8 +45,12 @@ #define make_atomic_fas_body(S) \ asm volatile ("xchg %0, %1;" : "+r" (v) , "+m" (*a)) #define make_atomic_cas_body(S) \ + int ## S sav; \ asm volatile (LOCK_prefix "; cmpxchg %3, %0; setz %2;" \ - : "+m" (*a), "+a" (*cmp), "=q" (ret): "r" (set)) + : "+m" (*a), "=a" (sav), "=q" (ret) \ + : "r" (set), "a" (*cmp)); \ + if (!ret) \ + *cmp= sav #ifdef MY_ATOMIC_MODE_DUMMY #define make_atomic_load_body(S) ret=*a This makes the behaviour consistent with the other implementations, see for example generic-msvc.h. (It is also pretty important for correct operation. In my code, I use my_atomic_casptr() to atomically enqueue a struct from one thread, and as soon as it is enqueued another thread may grab the struct and change it, including the field pointed to by cmp. So it is essential that my_atomic_casptr() neither reads nor writes *cmp after successful CAS, as the above patch ensures.) It was btw. a bit funny how I tracked this down. After digging for a few hours in my own code without success I got the idea to check the CAS implementation, and spotted the problem in gcc_builtins.h. After fixing I was baffled as to why my code stil failed, until I realised I was not using gcc_builtins.h but x86-gcc.h, and found and fixed the similar problem there ;-) Now the question is, where should I push this (if at all)? Any opinions? ----------------------------------------------------------------------- While I was there, I also noticed another potential problem in gcc_builtins.h, suggesting this patch: Index: work-5.1-groupcommit/include/atomic/x86-gcc.h =================================================================== --- work-5.1-groupcommit.orig/include/atomic/x86-gcc.h 2010-06-09 11:37:12.000000000 +0200 +++ work-5.1-groupcommit/include/atomic/x86-gcc.h 2010-06-09 11:52:47.000000000 +0200 @@ -38,17 +38,31 @@ #define asm __asm__ #endif +/* + The atomic operations imply a memory barrier for the CPU, to ensure that all + prior writes are flushed from cache, and all subsequent reads reloaded into + cache. + + We need to imply a similar memory barrier for the compiler, so that all + pending stores (to memory that may be aliased in other parts of the code) + will be flushed to memory before the operation, and all reads from such + memory be re-loaded. This is achieved by adding the "memory" pseudo-register + to the clobber list, see GCC documentation for more explanation. + + The compiler and CPU memory barriers are needed to make sure changes in one + thread are made visible in another by the atomic operation. +*/ #ifndef MY_ATOMIC_NO_XADD #define make_atomic_add_body(S) \ - asm volatile (LOCK_prefix "; xadd %0, %1;" : "+r" (v) , "+m" (*a)) + asm volatile (LOCK_prefix "; xadd %0, %1;" : "+r" (v) , "+m" (*a): : "memory") #endif #define make_atomic_fas_body(S) \ - asm volatile ("xchg %0, %1;" : "+r" (v) , "+m" (*a)) + asm volatile ("xchg %0, %1;" : "+r" (v) , "+m" (*a) : : "memory") #define make_atomic_cas_body(S) \ int ## S sav; \ asm volatile (LOCK_prefix "; cmpxchg %3, %0; setz %2;" \ : "+m" (*a), "=a" (sav), "=q" (ret) \ - : "r" (set), "a" (*cmp)); \ + : "r" (set), "a" (*cmp) : "memory"); \ if (!ret) \ *cmp= sav @@ -63,9 +77,9 @@ #define make_atomic_load_body(S) \ ret=0; \ asm volatile (LOCK_prefix "; cmpxchg %2, %0" \ - : "+m" (*a), "+a" (ret): "r" (ret)) + : "+m" (*a), "+a" (ret) : "r" (ret) : "memory") #define make_atomic_store_body(S) \ - asm volatile ("; xchg %0, %1;" : "+m" (*a), "+r" (v)) + asm volatile ("; xchg %0, %1;" : "+m" (*a), "+r" (v) : : "memory") #endif /* TODO test on intel whether the below helps. on AMD it makes no difference */ The comment in the patch explains the idea I think. Basically, these memory barrier operations need to be a compiler barrier also. Otherwise there is nothing that prevents GCC from moving unrelated stores across the memory barrier operation. This is a problem for example when filling in a structure and then atomically linking it into a shared list. The CPU memory barrier ensures that the fields in the structure will be visible to other threads on the CPU level, but it is also necessary to tell GCC to keep the stores into the struct fields prior to the store linking into the shared list. (It also makes the volatile declaration on the updated memory location unnecessary, but maybe it is needed for other implementations (though it shouldn't be, volatile is almost always wrong)). - Kristian.

1 0

[Maria-developers] Patch to build xtradb and federatedx statically on Windows
by Bo Thorsen 09 Jun '10

09 Jun '10

Hi everyone, Recently, we found out that xtradb and federatedx were compiled as plugins on Windows. This is not correct because they are core storage engines and should be linked statically instead. On Unix, it's possible to build both types, but the CMake files doesn't really handle this. They just look in plug.in, and if there is a dynamic line, the storage engine is built dynamically. I'm hesitant to fix this properly, as it's probably already done in MySQL 5.5, or at least it will be. I see no reason to come up with a good solution for a temporary problem - there are many other issues on Windows that are more pressing. FederatedX couldn't compile as a statically linked plugin at all on Windows. I have fixed this now, but I *really* don't like the way we do these hacks on Windows. It's absurdly difficult to keep track of when a variable says xtradb and when it says innobase. And debugging CMakeLists.txt files is not too much fun. Anyway, I did the same thing on FederatedX as is done in XtraDB, and it works now. Finally, I removed a subdir that didn't have a CMakeLists.txt. Removed a warning during the cmake run. With this patch, the Windows zip file release build should work correctly and produce a MariaDB with both XtraDB and FederatedX in it. Should I check this into ~maria-captains/maria/5.1, or is there a branch more suitable at the moment? Bo Thorsen. Monty Program AB. -- MariaDB: MySQL replacement Community developed. Feature enhanced. Backward compatible.

2 2

[Maria-developers] Rev 2795: Uninitialized memory problem fixed. in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 08 Jun '10

08 Jun '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2795 revision-id: sanja(a)askmonty.org-20100608122454-dtlvue8n2s55yi67 parent: sanja(a)askmonty.org-20100608120847-v4loj2gdqjflbarv committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Tue 2010-06-08 15:24:54 +0300 message: Uninitialized memory problem fixed. === modified file 'sql/table.cc' --- a/sql/table.cc 2010-06-07 07:58:45 +0000 +++ b/sql/table.cc 2010-06-08 12:24:54 +0000 @@ -5176,6 +5176,7 @@ key_part_info->offset= (*reg_field)->offset(record[0]); key_part_info->length= (uint16) (*reg_field)->pack_length(); keyinfo->key_length+= key_part_info->length; + key_part_info->key_part_flag= 0; /* TODO: The below method of computing the key format length of the key part is a copy/paste from opt_range.cc, and table.cc.

1 0

[Maria-developers] Rev 2794: Fixed uninitialized memory in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 08 Jun '10

08 Jun '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2794 revision-id: sanja(a)askmonty.org-20100608120847-v4loj2gdqjflbarv parent: sanja(a)askmonty.org-20100608114156-36q3me04ecxh2by6 committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Tue 2010-06-08 15:08:47 +0300 message: Fixed uninitialized memory === modified file 'sql/sql_subquery_cache.cc' --- a/sql/sql_subquery_cache.cc 2010-06-08 11:41:56 +0000 +++ b/sql/sql_subquery_cache.cc 2010-06-08 12:08:47 +0000 @@ -50,6 +50,7 @@ tab_ref->null_rejecting= 1; tab_ref->disable_cache= FALSE; tab_ref->has_record= 0; + tab_ref->use_count= 0; KEY_PART_INFO *cur_key_part= tmp_key->key_part; store_key **ref_key= tab_ref->key_copy;

1 0

[Maria-developers] Rev 2793: Fixed uninitialized memory. in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 08 Jun '10

08 Jun '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2793 revision-id: sanja(a)askmonty.org-20100608114156-36q3me04ecxh2by6 parent: sanja(a)askmonty.org-20100608093610-efg156vu4mi4u9pp committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Tue 2010-06-08 14:41:56 +0300 message: Fixed uninitialized memory. === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-05-31 21:25:54 +0000 +++ b/sql/mysql_priv.h 2010-06-08 11:41:56 +0000 @@ -1274,7 +1274,7 @@ select_result *result, SELECT_LEX_UNIT *unit, SELECT_LEX *select_lex); -struct st_join_table *create_index_lookup_join_tab(TABLE *table); +struct st_join_table *create_index_lookup_join_tab(TABLE *table, int key_no); int join_read_key2(THD *thd, struct st_join_table *tab, TABLE *table, struct st_table_ref *table_ref); void free_underlaid_joins(THD *thd, SELECT_LEX *select); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-31 21:25:54 +0000 +++ b/sql/sql_select.cc 2010-06-08 11:41:56 +0000 @@ -7627,11 +7627,12 @@ Creates and fills JOIN_TAB for index look up in temporary table @param table The table where to look up + @param key_no Number of key @return JOIN_TAB object or NULL in case of error */ -JOIN_TAB *create_index_lookup_join_tab(TABLE *table) +JOIN_TAB *create_index_lookup_join_tab(TABLE *table, int key_no) { JOIN_TAB *tab; DBUG_ENTER("create_index_lookup_join_tab"); @@ -7640,13 +7641,12 @@ DBUG_RETURN(NULL); tab->read_record.table= table; tab->read_record.file=table->file; - /*tab->read_record.unlock_row= rr_unlock_row;*/ tab->next_select=0; tab->sorted= 1; + tab->ref.key= key_no; table->status= STATUS_NO_RECORD; tab->read_first_record= join_read_key; - /*tab->read_record.unlock_row= join_read_key_unlock_row;*/ tab->read_record.read_record= join_no_more_records; if (table->covering_keys.is_set(tab->ref.key) && !table->no_keyread) === modified file 'sql/sql_subquery_cache.cc' --- a/sql/sql_subquery_cache.cc 2010-05-31 21:25:54 +0000 +++ b/sql/sql_subquery_cache.cc 2010-06-08 11:41:56 +0000 @@ -225,7 +225,7 @@ (uchar*)&field_counter) < 0) || createtmp_table_search_structures(table_thd, cache_table, li_items, &tab_ref) || - !(tab= create_index_lookup_join_tab(cache_table))) + !(tab= create_index_lookup_join_tab(cache_table, 0))) { DBUG_PRINT("error", ("creating index failed")); goto error;

1 0

[Maria-developers] Rev 2792: Fixed memory management problem (Item can't contain other Item due to way of Items destruction). in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 08 Jun '10

08 Jun '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2792 revision-id: sanja(a)askmonty.org-20100608093610-efg156vu4mi4u9pp parent: sanja(a)askmonty.org-20100608074734-1m60ib2tac7y9m33 committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Tue 2010-06-08 12:36:10 +0300 message: Fixed memory management problem (Item can't contain other Item due to way of Items destruction). === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-05-31 21:25:54 +0000 +++ b/sql/item_cmpfunc.cc 2010-06-08 09:36:10 +0000 @@ -1737,13 +1737,16 @@ not_null_tables_cache|= args[1]->not_null_tables(); const_item_cache&= args[1]->const_item(); DBUG_ASSERT(scache == NULL); + DBUG_ASSERT(value_for_scache == NULL); if (args[0]->cols() ==1 && thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE && !(sub->engine->uncacheable() & (UNCACHEABLE_RAND | UNCACHEABLE_SIDEEFFECT))) { sub->depends_on.push_front((Item**)&cache); - scache= new Subquery_cache_tmptable(thd, sub->depends_on, &result); + value_for_scache= new Item_bool_cache; + scache= new Subquery_cache_tmptable(thd, sub->depends_on, + value_for_scache); } fixed= 1; return FALSE; @@ -1851,8 +1854,8 @@ /* put result in the cache */ if (scache) { - result.set(tmp, null_value); - scache->put_value(&result); + value_for_scache->set(tmp, null_value); + scache->put_value(value_for_scache); } DBUG_RETURN(tmp); } @@ -1876,6 +1879,7 @@ delete scache; scache= 0; } + value_for_scache= 0; DBUG_VOID_RETURN; } === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-05-31 21:25:54 +0000 +++ b/sql/item_cmpfunc.h 2010-06-08 09:36:10 +0000 @@ -241,7 +241,7 @@ /* Subquery cache */ Subquery_cache *scache; /* result representation for the subquery cache */ - Item_bool_cache result; + Item_bool_cache *value_for_scache; bool save_cache; /* Stores the value of "NULL IN (SELECT ...)" for uncorrelated subqueries: @@ -252,7 +252,8 @@ my_bool result_for_null_param; public: Item_in_optimizer(Item *a, Item_in_subselect *b): - Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), scache(NULL), + Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), + scache(NULL), value_for_scache(NULL), save_cache(0), result_for_null_param(UNKNOWN) {} bool fix_fields(THD *, Item **); === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-05-31 21:25:54 +0000 +++ b/sql/item_subselect.cc 2010-06-08 09:36:10 +0000 @@ -34,10 +34,10 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), - engine(0), old_engine(0), scache(0), used_tables_cache(0), - have_to_be_excluded(0), const_item_cache(1), inside_first_fix_fields(0), - done_first_fix_fields(FALSE), eliminated(FALSE), engine_changed(0), - changed(0), is_correlated(FALSE) + engine(0), old_engine(0), scache(0), value_for_scache(0), + used_tables_cache(0), have_to_be_excluded(0), const_item_cache(1), + inside_first_fix_fields(0), done_first_fix_fields(FALSE), + eliminated(FALSE), engine_changed(0), changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -121,6 +121,7 @@ delete scache; scache= 0; } + value_for_scache= 0; reset(); value_assigned= 0; DBUG_VOID_RETURN; @@ -129,7 +130,7 @@ void Item_singlerow_subselect::cleanup() { DBUG_ENTER("Item_singlerow_subselect::cleanup"); - value= 0; row= 0; + value= 0; row= 0; null_value_item= 0; Item_subselect::cleanup(); DBUG_VOID_RETURN; } @@ -572,7 +573,7 @@ Item_singlerow_subselect::Item_singlerow_subselect(st_select_lex *select_lex) - :Item_subselect(), value(0) + :Item_subselect(), value(0), null_value_item(0) { DBUG_ENTER("Item_singlerow_subselect::Item_singlerow_subselect"); init(select_lex, new select_singlerow_subselect(this)); @@ -760,6 +761,7 @@ (uint)depends_on.elements, (uint)test(thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE))); engine->fix_length_and_dec(row= &value); + DBUG_ASSERT(scache == NULL); if (depends_on.elements && optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && !(engine->uncacheable() & (UNCACHEABLE_RAND | @@ -839,6 +841,22 @@ DBUG_RETURN(NULL); } + +/** + Puts NULL value as result in the cache +*/ + +void Item_singlerow_subselect::put_null_value_in_scache() +{ + if (!value_for_scache) + { + value_for_scache= new Item_bool_cache; + value_for_scache->set(0, TRUE); // NULL + } + DBUG_ASSERT(value_for_scache->null_value); + scache->put_value(value_for_scache); +} + double Item_singlerow_subselect::val_real() { Item *cached_value; @@ -870,7 +888,7 @@ reset(); DBUG_PRINT("info", ("error: %u", (uint)err)); if (scache && !err) - scache->put_value(&const_null_value); + put_null_value_in_scache(); DBUG_RETURN(0); } } @@ -906,7 +924,7 @@ reset(); DBUG_PRINT("info", ("error: %u", (uint)err)); if (scache && !err) - scache->put_value(&const_null_value); + put_null_value_in_scache(); DBUG_RETURN(0); } } @@ -942,7 +960,7 @@ reset(); DBUG_PRINT("info", ("error: %u", (uint)err)); if (scache && !err) - scache->put_value(&const_null_value); + put_null_value_in_scache(); DBUG_RETURN(0); } } @@ -979,7 +997,7 @@ reset(); DBUG_PRINT("info", ("error: %u", (uint)err)); if (scache && !err) - scache->put_value(&const_null_value); + put_null_value_in_scache(); DBUG_RETURN(0); } } @@ -1016,7 +1034,7 @@ reset(); DBUG_PRINT("info", ("error: %u", (uint)err)); if (scache && !err) - scache->put_value(&const_null_value); + put_null_value_in_scache(); DBUG_RETURN(0); } } @@ -1108,13 +1126,16 @@ max_columns= engine->cols(); /* We need only 1 row to determine existence */ unit->global_parameters->select_limit= new Item_int((int32) 1); + + DBUG_ASSERT(scache == NULL); + DBUG_ASSERT(value_for_scache == NULL); if (substype() == EXISTS_SUBS && depends_on.elements && optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && !(engine->uncacheable() & (UNCACHEABLE_RAND | UNCACHEABLE_SIDEEFFECT))) { - DBUG_ASSERT(scache == NULL); - scache= new Subquery_cache_tmptable(thd, depends_on, &result); + value_for_scache= new Item_bool_cache; + scache= new Subquery_cache_tmptable(thd, depends_on, value_for_scache); DBUG_PRINT("info", ("cache: 0x%lx", (ulong) scache)); } DBUG_VOID_RETURN; @@ -1141,8 +1162,8 @@ if (scache) { - result.set(value, FALSE); - scache->put_value(&result); + value_for_scache->set(value, FALSE); + scache->put_value(value_for_scache); } DBUG_RETURN((double) value); @@ -1170,8 +1191,8 @@ if (scache) { - result.set(value, FALSE); - scache->put_value(&result); + value_for_scache->set(value, FALSE); + scache->put_value(value_for_scache); } DBUG_RETURN(value); @@ -1213,8 +1234,8 @@ if (scache) { - result.set(value, FALSE); - scache->put_value(&result); + value_for_scache->set(value, FALSE); + scache->put_value(value_for_scache); } str->set((ulonglong)value,&my_charset_bin); @@ -1257,8 +1278,8 @@ if (scache) { - result.set(value, FALSE); - scache->put_value(&result); + value_for_scache->set(value, FALSE); + scache->put_value(value_for_scache); } int2my_decimal(E_DEC_FATAL_ERROR, value, 0, decimal_value); @@ -1287,8 +1308,8 @@ if (scache) { - result.set(value, FALSE); - scache->put_value(&result); + value_for_scache->set(value, FALSE); + scache->put_value(value_for_scache); } DBUG_RETURN(value != 0); === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-05-31 21:25:54 +0000 +++ b/sql/item_subselect.h 2010-06-08 09:36:10 +0000 @@ -60,8 +60,8 @@ subselect_engine *old_engine; /* subquery cache */ Subquery_cache *scache; - /* null consrtant for caching */ - Item_null const_null_value; + /* subquery cache value for NULL and TRUE/FALSE subqueries */ + Item_bool_cache *value_for_scache; /* cache of used external tables */ table_map used_tables_cache; /* allowed number of columns (1 for single value subqueries) */ @@ -217,10 +217,15 @@ { protected: Item_cache *value, **row; + /* null value for subquery cache value */ + Item_null *null_value_item; + + void put_null_value_in_scache(); public: Item_singlerow_subselect(st_select_lex *select_lex); - Item_singlerow_subselect() :Item_subselect(), value(0), row (0) {} + Item_singlerow_subselect() :Item_subselect(), value(0), row (0), + null_value_item(0) {} void cleanup(); subs_type substype() { return SINGLEROW_SUBS; } @@ -284,8 +289,6 @@ { protected: bool value; /* value of this item (boolean: exists/not-exists) */ - /* result representation for the subquery cache */ - Item_bool_cache result; public: Item_exists_subselect(st_select_lex *select_lex);

1 0

[Maria-developers] Build Slaves
by Adam M. Dutko 08 Jun '10

08 Jun '10

Is the adutko-ultrasparc3 buildbot too slow? would it be missed?

2 1

[Maria-developers] Rev 2791: bugfixes lost in moving between trees in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 08 Jun '10

08 Jun '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2791 revision-id: sanja(a)askmonty.org-20100608074734-1m60ib2tac7y9m33 parent: sanja(a)askmonty.org-20100607075845-lo3tcaiuk54qqlw0 committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Tue 2010-06-08 10:47:34 +0300 message: bugfixes lost in moving between trees === modified file 'sql/item.cc' --- a/sql/item.cc 2010-05-31 21:25:54 +0000 +++ b/sql/item.cc 2010-06-08 07:47:34 +0000 @@ -5152,6 +5152,10 @@ int Item_field::save_in_field(Field *to, bool no_conversions) { + /* if it is external field */ + if (unlikely(depended_from)) + return save_field_in_field(field, &null_value, to, no_conversions); + return save_field_in_field(result_field, &null_value, to, no_conversions); } @@ -6359,7 +6363,7 @@ int Item_ref::save_in_field(Field *to, bool no_conversions) { int res; - if (result_field) + if (result_field && !depended_from) return save_field_in_field(result_field, &null_value, to, no_conversions); res= (*ref)->save_in_field(to, no_conversions); null_value= (*ref)->null_value;

1 0

[Maria-developers] Question about License...
by Mateusz Matan 08 Jun '10

08 Jun '10

Welcome, We designed a custom Storage Engine (CLDB) for MySQL/MariaDB which now pass all premature tests. I wonder, how can we solve license problem... We would not share our sources, but we will just use shared library (*.so)... Which licenses should our customer who bought MariaDB with CLDB use to stay legally... If it is possible, can we use in corporation use MariaDB on GPL and just sell CLDB (storage engine) on other license (which one) ?? Thank You for Your quick reply... -- ___________________________________________________________ Mateusz Matan IT Security R&D Department, C/C++ programmer ComArch S.A., Al. Jana Pawła II 41d, 31-864 Kraków tel: (+48 12) 684 8411 e-mail: Mateusz.Matan(a)comarch.pl

1 0

Re: [Maria-developers] [Commits] Rev 2797: MWL#90, code movearound to unify merged and non-merged semi-join materialization processing in file:///home/psergey/dev/maria-5.3-subqueries-r12/
by Sergey Petrunya 07 Jun '10

07 Jun '10

Hi Monty, On Fri, Jun 04, 2010 at 05:36:59PM +0300, Michael Widenius wrote: > Note that this is not a full review, but just a quick scan of some of > the things in the commit. (One suspicious thing found...) Thanks for the feedback! Reply summary: - This commit was primarily to get a buildbot run, hence the presence of loads of commented-out old code and lack of real comments. I'll address this a bit later. - The suspicious thing confirmed and fixed. - Style suggestions followed. > >>>>> "Sergey" == Sergey Petrunya <psergey(a)askmonty.org> writes: > > <cut> > > Sergey> === modified file 'sql/item_cmpfunc.cc' > Sergey> --- a/sql/item_cmpfunc.cc 2010-05-25 06:32:15 +0000 > Sergey> +++ b/sql/item_cmpfunc.cc 2010-06-04 13:40:57 +0000 > Sergey> @@ -5733,6 +5733,8 @@ > Sergey> It's a field from an materialized semi-join. We can substitute it only > Sergey> for a field from the same semi-join. > Sergey> */ > Sergey> +#if 0 > Sergey> + psergey3:remove: > > Please don't ever use #if 0; Instead use something like: > > #ifdef LEFT_FOR_TESTING_WILL_BE_REMOVED_BY_PSERGEY_SOON > > Even better to just remove the code (after all, we can always find it > in bzr) > > <cut> > Sergey> - if (item->field->table->reginfo.join_tab >= first) > Sergey> + //if (item->field->table->reginfo.join_tab >= first) > > Same here; Don't leave the old code around > > <cut> > > Sergey> +bool join_tab_execution_startup(JOIN_TAB *tab) > Sergey> { > Sergey> + DBUG_ENTER("join_tab_execution_startup"); > Sergey> Item_in_subselect *in_subs; > > Please put DBUG_ENTER after all declarations. > (So that we have same layout in C and C++) > > <cut> > > Sergey> +#if 0 > > Replace with proper #if or remove code > <cut> > > Sergey> +++ b/sql/sql_select.cc 2010-06-04 13:40:57 +0000 > Sergey> @@ -1008,15 +1006,26 @@ > Sergey> /* > Sergey> Permorm the the optimization on fields evaluation mentioned above > Sergey> for all on expressions. > Sergey> - */ > Sergey> - for (JOIN_TAB *tab= join_tab + const_tables; tab < join_tab + tables ; tab++) > Sergey> + */ > Sergey> + > Sergey> { > Sergey> - if (*tab->on_expr_ref) > Sergey> + List_iterator<JOIN_TAB_RANGE> it(join_tab_ranges); > Sergey> + JOIN_TAB_RANGE *jt_range; > Sergey> + bool first= TRUE; > > Wouldn't it be better to set first to const_tables and then to 0 ? > > Sergey> + while ((jt_range= it++)) > Sergey> { > Sergey> + for (JOIN_TAB *tab= jt_range->start + (first ? const_tables : 0); > > If you do the above, then you can just do 'jt_range->start + first' here > > Sergey> + tab < jt_range->end; tab++) > Sergey> + { > Sergey> + if (*tab->on_expr_ref) > Sergey> + { > Sergey> + *tab->on_expr_ref= substitute_for_best_equal_field(*tab->on_expr_ref, > Sergey> + tab->cond_equal, > Sergey> + map2table); > Sergey> + (*tab->on_expr_ref)->update_used_tables(); > Sergey> + } > Sergey> + } > Sergey> + first= FALSE; > Sergey> } > Sergey> } > > A comment for the above outer loop would be nice. > (It's not obvious why only the first element in join_tab_ranges has > const tables) > > Sergey> @@ -1026,6 +1035,7 @@ > Sergey> { > Sergey> conds=new Item_int((longlong) 0,1); // Always false > Sergey> } > Sergey> + > Sergey> if (make_join_select(this, select, conds)) > Sergey> { > Sergey> zero_result_cause= > Sergey> @@ -1289,7 +1299,8 @@ > Sergey> if (need_tmp || select_distinct || group_list || order) > Sergey> { > Sergey> for (uint i = const_tables; i < tables; i++) > Sergey> - join_tab[i].table->prepare_for_position(); > Sergey> + table[i]->prepare_for_position(); > Sergey> + > > Isn't table[] in other order than join_tab? > (I thought that only join_tab has the const tables first) Right. Will fix this. > <cut> > > Sergey> +JOIN_TAB *first_linear_tab(JOIN *join, bool after_const_tables) > Sergey> +{ > Sergey> + JOIN_TAB *first= join->join_tab; > Sergey> + if (after_const_tables) > Sergey> + first += join->const_tables; > > remove space before '+' > > Sergey> + if (first < join->join_tab + join->top_jtrange_tables) > Sergey> + return first; > Sergey> + else > Sergey> + return NULL; > Sergey> +} > > Better to do: > > return (first < join->join_tab + join->top_jtrange_tables) ? first : 0; > > Or: > > if (first < join->join_tab + join->top_jtrange_tables) > return first; > return NULL; > Changed. > <cut> > > Sergey> +JOIN_TAB *next_linear_tab(JOIN* join, JOIN_TAB* tab, bool include_bush_roots) //psergey2: added > > Remove comments; It's trival to see that the function was added :) > > Sergey> +{ > Sergey> + if (include_bush_roots && tab->bush_children) > Sergey> + return tab->bush_children->start; > Sergey> + > Sergey> + if (tab->last_leaf_in_bush) > Sergey> + tab= tab->bush_root_tab; > Sergey> + > Sergey> + if (tab->bush_root_tab) > Sergey> + return ++tab; > > Add an assert before if (tab->last_leaf_in_bush): > > DBUG_ASSERT(!tab->last_leaf_in_bush || tab->bush_root_tab); > Just to declare that the above code is safe! Done. > Sergey> + > Sergey> + if (++tab == join->join_tab + join->top_jtrange_tables /*join->join_tab_ranges.head()->end*/) > > Move the comment to previous row and make it more clear what you are testing > (The current comment doesn't tell me much) > > Sergey> + return NULL; > Sergey> + > Sergey> + if (!include_bush_roots && tab->bush_children) > Sergey> + { > Sergey> + tab= tab->bush_children->start; > Sergey> + } > Sergey> + return tab; > > Why not do: > > return ((!include_bush_roots && tab->bush_children) ? > tab->bush_children->start : tab); > > Sergey> + if ((start? tab: ++tab) == join->join_tab_ranges.head()->end) > Sergey> + return NULL; /* End */ > > I think the above code would be more clear if you would do: > > if (start) > tab++; > if (...) > > This makes it clear that the ++tab is not just for the test but also > for future usage of tab. Changed. > Sergey> + > Sergey> + if (tab->bush_children) > Sergey> + return tab->bush_children->start; > Sergey> + > Sergey> + return tab; > > Could be combined with ? > > Sergey> +} > Sergey> + > Sergey> + > Sergey> +static Item *null_ptr= NULL; > > Can we make this const, so that if anyone tried to change this memory > location we would get an exception? Done > <cut> > > Sergey> DBUG_RETURN(TRUE); /* purecov: inspected */ > > Sergey> join_tab= parent->join_tab_reexec; > Sergey> + //psergey2: hopefully this is ok: > Sergey> + // join_tab_ranges.head()->start= join_tab; > Sergey> + // join_tab_ranges.head()->end= join_tab + 1; > > Better to know than to hope :) It wasn't ok actually, already changed. > > (sorry, don't have time to look at the rest now) BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] New (by Knielsen test): Replication API for stacked event generators (120)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Replication API for stacked event generators CREATION DATE..: Mon, 07 Jun 2010, 13:13 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 120 (http://askmonty.org/worklog/?tid=120) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: A part of the replication project, MWL#107. Events are produced by event Generators. Examples are - Generation of statement-based replication events - Generation of row-based events - Generation of PBXT engine-level replication events and maybe reading of events from relay log on slave may also be an example of generating events. Event generators can be stacked, and a generator may defer event generation to the next one down the stack. For example, the row-level replication event generator may defer DLL to the statement-level replication event generator. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Knielsen test): Replication API for stacked event generators (120)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Replication API for stacked event generators CREATION DATE..: Mon, 07 Jun 2010, 13:13 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 120 (http://askmonty.org/worklog/?tid=120) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: A part of the replication project, MWL#107. Events are produced by event Generators. Examples are - Generation of statement-based replication events - Generation of row-based events - Generation of PBXT engine-level replication events and maybe reading of events from relay log on slave may also be an example of generating events. Event generators can be stacked, and a generator may defer event generation to the next one down the stack. For example, the row-level replication event generator may defer DLL to the statement-level replication event generator. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): New replication APIs (107)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: New replication APIs CREATION DATE..: Mon, 15 Mar 2010, 13:55 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Sergei CATEGORY.......: Server-Sprint TASK ID........: 107 (http://askmonty.org/worklog/?tid=107) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 50 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 12:11)=-=- High Level Description modified. --- /tmp/wklog.107.old.31097 2010-06-07 12:11:57.000000000 +0000 +++ /tmp/wklog.107.new.31097 2010-06-07 12:11:57.000000000 +0000 @@ -7,3 +7,6 @@ https://lists.launchpad.net/maria-developers/msg01998.html +Wiki page for the project: + + http://askmonty.org/wiki/ReplicationProject -=-=(Knielsen - Mon, 29 Mar 2010, 07:33)=-=- Research and design discussions: Galera, 2pc/XA, group commit, multi-engine transactions. Worked 14 hours and estimate 0 hours remain (original estimate increased by 14 hours). -=-=(Knielsen - Wed, 24 Mar 2010, 10:39)=-=- Design discussions Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Mon, 15 Mar 2010, 14:28)=-=- Research into the problem, and discussions on phone/mailing list Worked 25 hours and estimate 0 hours remain (original estimate increased by 25 hours). -=-=(Guest - Mon, 15 Mar 2010, 14:18)=-=- High-Level Specification modified. --- /tmp/wklog.107.old.9086 2010-03-15 14:18:18.000000000 +0000 +++ /tmp/wklog.107.new.9086 2010-03-15 14:18:18.000000000 +0000 @@ -1 +1,43 @@ +Current ideas/status after discussions on the mailing list: + + - Implement a set of plugin APIs and use them to move all of the existing + MySQL replication into a (set of) plugins. + + - Design the APIs so that they can support full MySQL replication, but also + so that they do not hardcode assumptions about how this replication + implementation is done, and so that they will be suitable for other types of + replication (Tungsten, Galera, parallel replication, ...). + + - APIs need to include the concept of a global transaction ID. Need to + determine the extent to which the semantics of such ID will be defined + by the API, and to which extend it will be defined by the plugin + implementations. + + - APIs should properly support reliable crash-recovery with decent + performance (eg. not require multiple mandatory fsync()s per commit, and + not make group commit impossible). + + - Would be nice if the API provided facilities for implementing good + consistency checking support (mainly checking master tables against slave + tables is hard here I think, but also applying wrong binlog data and + individual event checksums). + + +Steps to make this more concrete: + + - Investigate the current MySQL replication, and list all of the places where + a plugin implementation will need to connect/hook into the MySQL server. + * handler::{write,update,delete}_row() + * Statement execution + * Transaction start/commit + * Table open + * Query safe/not/safe for statement based replication + * Statement-based logging details (user variables, random seed, etc.) + * ... + + - Use this list to make an initial sketch of the set of APIs we need. + + - Use the list to determine the feasibility of this project and the level of + detail in the API needed to support a full replication implementation as a + plugin. -=-=(Serg - Mon, 15 Mar 2010, 14:13)=-=- Observers changed: Sergei DESCRIPTION: This is a top-level task for the project of designing a new set of replication APIs for MariaDB. This task is for the initial discussion of what to do and where to focus. The project is started in this email thread: https://lists.launchpad.net/maria-developers/msg01998.html Wiki page for the project: http://askmonty.org/wiki/ReplicationProject HIGH-LEVEL SPECIFICATION: Current ideas/status after discussions on the mailing list: - Implement a set of plugin APIs and use them to move all of the existing MySQL replication into a (set of) plugins. - Design the APIs so that they can support full MySQL replication, but also so that they do not hardcode assumptions about how this replication implementation is done, and so that they will be suitable for other types of replication (Tungsten, Galera, parallel replication, ...). - APIs need to include the concept of a global transaction ID. Need to determine the extent to which the semantics of such ID will be defined by the API, and to which extend it will be defined by the plugin implementations. - APIs should properly support reliable crash-recovery with decent performance (eg. not require multiple mandatory fsync()s per commit, and not make group commit impossible). - Would be nice if the API provided facilities for implementing good consistency checking support (mainly checking master tables against slave tables is hard here I think, but also applying wrong binlog data and individual event checksums). Steps to make this more concrete: - Investigate the current MySQL replication, and list all of the places where a plugin implementation will need to connect/hook into the MySQL server. * handler::{write,update,delete}_row() * Statement execution * Transaction start/commit * Table open * Query safe/not/safe for statement based replication * Statement-based logging details (user variables, random seed, etc.) * ... - Use this list to make an initial sketch of the set of APIs we need. - Use the list to determine the feasibility of this project and the level of detail in the API needed to support a full replication implementation as a plugin. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): New replication APIs (107)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: New replication APIs CREATION DATE..: Mon, 15 Mar 2010, 13:55 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Sergei CATEGORY.......: Server-Sprint TASK ID........: 107 (http://askmonty.org/worklog/?tid=107) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 50 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 12:11)=-=- High Level Description modified. --- /tmp/wklog.107.old.31097 2010-06-07 12:11:57.000000000 +0000 +++ /tmp/wklog.107.new.31097 2010-06-07 12:11:57.000000000 +0000 @@ -7,3 +7,6 @@ https://lists.launchpad.net/maria-developers/msg01998.html +Wiki page for the project: + + http://askmonty.org/wiki/ReplicationProject -=-=(Knielsen - Mon, 29 Mar 2010, 07:33)=-=- Research and design discussions: Galera, 2pc/XA, group commit, multi-engine transactions. Worked 14 hours and estimate 0 hours remain (original estimate increased by 14 hours). -=-=(Knielsen - Wed, 24 Mar 2010, 10:39)=-=- Design discussions Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Mon, 15 Mar 2010, 14:28)=-=- Research into the problem, and discussions on phone/mailing list Worked 25 hours and estimate 0 hours remain (original estimate increased by 25 hours). -=-=(Guest - Mon, 15 Mar 2010, 14:18)=-=- High-Level Specification modified. --- /tmp/wklog.107.old.9086 2010-03-15 14:18:18.000000000 +0000 +++ /tmp/wklog.107.new.9086 2010-03-15 14:18:18.000000000 +0000 @@ -1 +1,43 @@ +Current ideas/status after discussions on the mailing list: + + - Implement a set of plugin APIs and use them to move all of the existing + MySQL replication into a (set of) plugins. + + - Design the APIs so that they can support full MySQL replication, but also + so that they do not hardcode assumptions about how this replication + implementation is done, and so that they will be suitable for other types of + replication (Tungsten, Galera, parallel replication, ...). + + - APIs need to include the concept of a global transaction ID. Need to + determine the extent to which the semantics of such ID will be defined + by the API, and to which extend it will be defined by the plugin + implementations. + + - APIs should properly support reliable crash-recovery with decent + performance (eg. not require multiple mandatory fsync()s per commit, and + not make group commit impossible). + + - Would be nice if the API provided facilities for implementing good + consistency checking support (mainly checking master tables against slave + tables is hard here I think, but also applying wrong binlog data and + individual event checksums). + + +Steps to make this more concrete: + + - Investigate the current MySQL replication, and list all of the places where + a plugin implementation will need to connect/hook into the MySQL server. + * handler::{write,update,delete}_row() + * Statement execution + * Transaction start/commit + * Table open + * Query safe/not/safe for statement based replication + * Statement-based logging details (user variables, random seed, etc.) + * ... + + - Use this list to make an initial sketch of the set of APIs we need. + + - Use the list to determine the feasibility of this project and the level of + detail in the API needed to support a full replication implementation as a + plugin. -=-=(Serg - Mon, 15 Mar 2010, 14:13)=-=- Observers changed: Sergei DESCRIPTION: This is a top-level task for the project of designing a new set of replication APIs for MariaDB. This task is for the initial discussion of what to do and where to focus. The project is started in this email thread: https://lists.launchpad.net/maria-developers/msg01998.html Wiki page for the project: http://askmonty.org/wiki/ReplicationProject HIGH-LEVEL SPECIFICATION: Current ideas/status after discussions on the mailing list: - Implement a set of plugin APIs and use them to move all of the existing MySQL replication into a (set of) plugins. - Design the APIs so that they can support full MySQL replication, but also so that they do not hardcode assumptions about how this replication implementation is done, and so that they will be suitable for other types of replication (Tungsten, Galera, parallel replication, ...). - APIs need to include the concept of a global transaction ID. Need to determine the extent to which the semantics of such ID will be defined by the API, and to which extend it will be defined by the plugin implementations. - APIs should properly support reliable crash-recovery with decent performance (eg. not require multiple mandatory fsync()s per commit, and not make group commit impossible). - Would be nice if the API provided facilities for implementing good consistency checking support (mainly checking master tables against slave tables is hard here I think, but also applying wrong binlog data and individual event checksums). Steps to make this more concrete: - Investigate the current MySQL replication, and list all of the places where a plugin implementation will need to connect/hook into the MySQL server. * handler::{write,update,delete}_row() * Statement execution * Transaction start/commit * Table open * Query safe/not/safe for statement based replication * Statement-based logging details (user variables, random seed, etc.) * ... - Use this list to make an initial sketch of the set of APIs we need. - Use the list to determine the feasibility of this project and the level of detail in the API needed to support a full replication implementation as a plugin. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 2790: bugfixes in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 07 Jun '10

07 Jun '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2790 revision-id: sanja(a)askmonty.org-20100607075845-lo3tcaiuk54qqlw0 parent: sanja(a)askmonty.org-20100531212554-oal32d5v360l6cul committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Mon 2010-06-07 10:58:45 +0300 message: bugfixes === modified file 'mysql-test/r/subquery_cache.result' --- a/mysql-test/r/subquery_cache.result 2010-05-31 21:25:54 +0000 +++ b/mysql-test/r/subquery_cache.result 2010-06-07 07:58:45 +0000 @@ -588,4 +588,28 @@ Subquery_cache_hit 0 Subquery_cache_miss 4 drop table t1; +#test of sql_big_tables switch and outer table reference in subquery with grouping +set option sql_big_tables=1; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT); +INSERT INTO t1 VALUES (1,1),(2,1),(3,2),(4,2),(5,3),(6,3); +SELECT (SELECT t1_outer.a FROM t1 AS t1_inner GROUP BY b LIMIT 1) FROM t1 AS t1_outer; +(SELECT t1_outer.a FROM t1 AS t1_inner GROUP BY b LIMIT 1) +1 +2 +3 +4 +5 +6 +drop table t1; +set option sql_big_tables=0; +#test of function reference to outer query +set local group_concat_max_len=400; +create table t2 (a int, b int); +insert into t2 values (1,1), (2,2); +select b x, (select group_concat(x) from t2) from t2; +x (select group_concat(x) from t2) +1 1,1 +2 2,2 +drop table t2; +set local group_concat_max_len=default; set optimizer_switch='subquery_cache=default'; === modified file 'mysql-test/t/subquery_cache.test' --- a/mysql-test/t/subquery_cache.test 2010-05-31 21:25:54 +0000 +++ b/mysql-test/t/subquery_cache.test 2010-06-07 07:58:45 +0000 @@ -201,4 +201,20 @@ show status like "subquery_cache%"; drop table t1; +--echo #test of sql_big_tables switch and outer table reference in subquery with grouping +set option sql_big_tables=1; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT); +INSERT INTO t1 VALUES (1,1),(2,1),(3,2),(4,2),(5,3),(6,3); +SELECT (SELECT t1_outer.a FROM t1 AS t1_inner GROUP BY b LIMIT 1) FROM t1 AS t1_outer; +drop table t1; +set option sql_big_tables=0; + +--echo #test of function reference to outer query +set local group_concat_max_len=400; +create table t2 (a int, b int); +insert into t2 values (1,1), (2,2); +select b x, (select group_concat(x) from t2) from t2; +drop table t2; +set local group_concat_max_len=default; + set optimizer_switch='subquery_cache=default'; === modified file 'sql/table.cc' --- a/sql/table.cc 2010-05-31 21:25:54 +0000 +++ b/sql/table.cc 2010-06-07 07:58:45 +0000 @@ -5187,10 +5187,16 @@ key_part_info->store_length= key_part_info->length; if ((*reg_field)->real_maybe_null()) + { key_part_info->store_length+= HA_KEY_NULL_LENGTH; + keyinfo->key_length+= HA_KEY_NULL_LENGTH; + } if ((*reg_field)->type() == MYSQL_TYPE_BLOB || (*reg_field)->real_type() == MYSQL_TYPE_VARCHAR) + { key_part_info->store_length+= HA_KEY_BLOB_LENGTH; + keyinfo->key_length+= HA_KEY_BLOB_LENGTH; // ??? + } key_part_info->type= (uint8) (*reg_field)->key_type(); key_part_info->key_type =

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 74 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:16)=-=- Missing hours in previous progress report. Worked 3 hours and estimate 0 hours remain (original estimate increased by 3 hours). -=-=(Knielsen - Mon, 07 Jun 2010, 07:15)=-=- Some more benchmarking. Blog about results and architecture. Fix two bugs in the proof-of-concept patch, races that causes hangs in benchmarks. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Guest - Tue, 01 Jun 2010, 14:20)=-=- Status updated. --- /tmp/wklog.116.old.32652 2010-06-01 14:20:15.000000000 +0000 +++ /tmp/wklog.116.new.32652 2010-06-01 14:20:15.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 74 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:16)=-=- Missing hours in previous progress report. Worked 3 hours and estimate 0 hours remain (original estimate increased by 3 hours). -=-=(Knielsen - Mon, 07 Jun 2010, 07:15)=-=- Some more benchmarking. Blog about results and architecture. Fix two bugs in the proof-of-concept patch, races that causes hangs in benchmarks. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Guest - Tue, 01 Jun 2010, 14:20)=-=- Status updated. --- /tmp/wklog.116.old.32652 2010-06-01 14:20:15.000000000 +0000 +++ /tmp/wklog.116.new.32652 2010-06-01 14:20:15.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 71 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:15)=-=- Some more benchmarking. Blog about results and architecture. Fix two bugs in the proof-of-concept patch, races that causes hangs in benchmarks. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Guest - Tue, 01 Jun 2010, 14:20)=-=- Status updated. --- /tmp/wklog.116.old.32652 2010-06-01 14:20:15.000000000 +0000 +++ /tmp/wklog.116.new.32652 2010-06-01 14:20:15.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 71 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:15)=-=- Some more benchmarking. Blog about results and architecture. Fix two bugs in the proof-of-concept patch, races that causes hangs in benchmarks. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Guest - Tue, 01 Jun 2010, 14:20)=-=- Status updated. --- /tmp/wklog.116.old.32652 2010-06-01 14:20:15.000000000 +0000 +++ /tmp/wklog.116.new.32652 2010-06-01 14:20:15.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 41 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:13)=-=- Help debug some test failures seen in Buildbot. Worked 6 hours and estimate 0 hours remain (original estimate increased by 6 hours). -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). ------------------------------------------------------------ -=-=(View All Progress Notes, 33 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 41 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:13)=-=- Help debug some test failures seen in Buildbot. Worked 6 hours and estimate 0 hours remain (original estimate increased by 6 hours). -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). ------------------------------------------------------------ -=-=(View All Progress Notes, 33 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 41 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:13)=-=- Help debug some test failures seen in Buildbot. Worked 6 hours and estimate 0 hours remain (original estimate increased by 6 hours). -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). ------------------------------------------------------------ -=-=(View All Progress Notes, 33 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 41 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:13)=-=- Help debug some test failures seen in Buildbot. Worked 6 hours and estimate 0 hours remain (original estimate increased by 6 hours). -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). ------------------------------------------------------------ -=-=(View All Progress Notes, 33 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 9 ESTIMATE.......: 7 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:12)=-=- Help Andrew with the integration. Worked 2 hours and estimate 7 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Wrote patch that allows to test SphinxSE in mysql-test-run, using external Sphinx daemon. Worked 7 hours and estimate 9 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 28 May 2010, 07:49)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.4369 2010-05-28 07:49:13.000000000 +0000 +++ /tmp/wklog.42.new.4369 2010-05-28 07:49:13.000000000 +0000 @@ -49,6 +49,10 @@ might be possible to pre-generate the necessary data/index files and store them in the source tree. +I pushed a proof-of-concept patch for this here: + + lp:~knielsen/maria/5.2-sphinxse + Here is a sample test case using this: --source include/have_sphinx.inc -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. I pushed a proof-of-concept patch for this here: lp:~knielsen/maria/5.2-sphinxse Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 07 Jun '10

07 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 9 ESTIMATE.......: 7 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Mon, 07 Jun 2010, 07:12)=-=- Help Andrew with the integration. Worked 2 hours and estimate 7 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Wrote patch that allows to test SphinxSE in mysql-test-run, using external Sphinx daemon. Worked 7 hours and estimate 9 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 28 May 2010, 07:49)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.4369 2010-05-28 07:49:13.000000000 +0000 +++ /tmp/wklog.42.new.4369 2010-05-28 07:49:13.000000000 +0000 @@ -49,6 +49,10 @@ might be possible to pre-generate the necessary data/index files and store them in the source tree. +I pushed a proof-of-concept patch for this here: + + lp:~knielsen/maria/5.2-sphinxse + Here is a sample test case using this: --source include/have_sphinx.inc -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. I pushed a proof-of-concept patch for this here: lp:~knielsen/maria/5.2-sphinxse Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 2795: Bugfixes in file:///home/bell/maria/bzr/work-maria-5.3-scache/
by sanja＠askmonty.org 05 Jun '10

05 Jun '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache/ ------------------------------------------------------------ revno: 2795 revision-id: sanja(a)askmonty.org-20100605195727-7rrc5k75lr0a4o9z parent: sanja(a)askmonty.org-20100527182744-1tu96cgyiaodzs32 committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache timestamp: Sat 2010-06-05 22:57:27 +0300 message: Bugfixes === modified file 'mysql-test/r/myisam_mrr.result' --- a/mysql-test/r/myisam_mrr.result 2010-03-11 21:43:31 +0000 +++ b/mysql-test/r/myisam_mrr.result 2010-06-05 19:57:27 +0000 @@ -394,7 +394,7 @@ # - engine_condition_pushdown does not affect ICP select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (a int, b int, key(a)); === modified file 'mysql-test/r/subquery_cache.result' --- a/mysql-test/r/subquery_cache.result 2010-05-27 17:41:38 +0000 +++ b/mysql-test/r/subquery_cache.result 2010-06-05 19:57:27 +0000 @@ -588,4 +588,28 @@ Subquery_cache_hit 0 Subquery_cache_miss 4 drop table t1; +#test of sql_big_tables switch and outer table reference in subquery with grouping +set option sql_big_tables=1; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT); +INSERT INTO t1 VALUES (1,1),(2,1),(3,2),(4,2),(5,3),(6,3); +SELECT (SELECT t1_outer.a FROM t1 AS t1_inner GROUP BY b LIMIT 1) FROM t1 AS t1_outer; +(SELECT t1_outer.a FROM t1 AS t1_inner GROUP BY b LIMIT 1) +1 +2 +3 +4 +5 +6 +drop table t1; +set option sql_big_tables=0; +#test of function reference to outer query +set local group_concat_max_len=400; +create table t2 (a int, b int); +insert into t2 values (1,1), (2,2); +select b x, (select group_concat(x) from t2) from t2; +x (select group_concat(x) from t2) +1 1,1 +2 2,2 +drop table t2; +set local group_concat_max_len=default; set optimizer_switch='subquery_cache=default'; === modified file 'mysql-test/t/subquery_cache.test' --- a/mysql-test/t/subquery_cache.test 2010-05-27 17:41:38 +0000 +++ b/mysql-test/t/subquery_cache.test 2010-06-05 19:57:27 +0000 @@ -201,4 +201,20 @@ show status like "subquery_cache%"; drop table t1; +--echo #test of sql_big_tables switch and outer table reference in subquery with grouping +set option sql_big_tables=1; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT); +INSERT INTO t1 VALUES (1,1),(2,1),(3,2),(4,2),(5,3),(6,3); +SELECT (SELECT t1_outer.a FROM t1 AS t1_inner GROUP BY b LIMIT 1) FROM t1 AS t1_outer; +drop table t1; +set option sql_big_tables=0; + +--echo #test of function reference to outer query +set local group_concat_max_len=400; +create table t2 (a int, b int); +insert into t2 values (1,1), (2,2); +select b x, (select group_concat(x) from t2) from t2; +drop table t2; +set local group_concat_max_len=default; + set optimizer_switch='subquery_cache=default'; === modified file 'sql/item.cc' --- a/sql/item.cc 2010-05-27 17:41:38 +0000 +++ b/sql/item.cc 2010-06-05 19:57:27 +0000 @@ -5110,6 +5110,19 @@ } +/** + Saves one Fields of an Item of in other Field + + @param from Field to copy value from + @param null_value reference on item null_value to set it if it is needed + @param to Field to cope value to + @param no_conversions how to deal with NULL value (see + set_field_to_null_with_conversions()) + + @retval FALSE OK + @retval TRUE Error +*/ + static int save_field_in_field(Field *from, my_bool *null_value, Field *to, bool no_conversions) { @@ -5139,6 +5152,10 @@ int Item_field::save_in_field(Field *to, bool no_conversions) { + /* if it is external field */ + if (unlikely(depended_from)) + return save_field_in_field(field, &null_value, to, no_conversions); + return save_field_in_field(result_field, &null_value, to, no_conversions); } @@ -6346,7 +6363,7 @@ int Item_ref::save_in_field(Field *to, bool no_conversions) { int res; - if (result_field) + if (result_field && !depended_from) return save_field_in_field(result_field, &null_value, to, no_conversions); res= (*ref)->save_in_field(to, no_conversions); null_value= (*ref)->null_value; === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-05-25 18:29:14 +0000 +++ b/sql/item_subselect.cc 2010-06-05 19:57:27 +0000 @@ -1,4 +1,4 @@ -/* Copyrigh (C) 2000 MySQL AB +/* Copyright (C) 2000 MySQL AB This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -818,6 +818,12 @@ exec(); } +/** + Checks subquery cache for value + + @retval NULL nothing found + @retval reference on item representing value found in the cache +*/ Item *Item_subselect::check_cache() { === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-05-24 17:29:56 +0000 +++ b/sql/item_subselect.h 2010-06-05 19:57:27 +0000 @@ -95,7 +95,10 @@ st_select_lex *parent_select; /** - List of items subquery depends on (externally resolved); + List of references on items subquery depends on (externally resolved); + + @note We can't store direct links on Items because it could be + substituted with other item (for example for grouping). */ List<Item*> depends_on; === modified file 'sql/sql_subquery_cache.cc' --- a/sql/sql_subquery_cache.cc 2010-05-27 18:27:44 +0000 +++ b/sql/sql_subquery_cache.cc 2010-06-05 19:57:27 +0000 @@ -96,6 +96,10 @@ /** Creates equalities expression. + @note For some type of fields index lookup do not return failure but set + pointer on the next record. To check exact match we use expression like: + field1=value1 and field2=value2 ... + @retval FALSE OK @retval TRUE Error */ @@ -111,6 +115,7 @@ for (uint i= 1 /* skip result filed */; (ref= li++); i++) { Field *fld= cache_table->field[i]; + /* Only some field types should be checked after lookup */ if (fld->type() == MYSQL_TYPE_VARCHAR || fld->type() == MYSQL_TYPE_TINY_BLOB || fld->type() == MYSQL_TYPE_MEDIUM_BLOB || @@ -140,11 +145,22 @@ } +/** + Enumerates all fields in field number order. + + @param arg reference on current field number + + @return field number +*/ + static uint field_enumerator(uchar *arg) { return ((uint*)arg)[0]++; } +/** + Initializes temporary table and index for this cache +*/ void Subquery_cache_tmptable::init() { @@ -182,8 +198,10 @@ if (!(cache_table= create_tmp_table(table_thd, &cache_table_param, items, (ORDER*) NULL, FALSE, FALSE, - (table_thd->options | - TMP_TABLE_ALL_COLUMNS), + ((table_thd->options | + TMP_TABLE_ALL_COLUMNS) & + ~(OPTION_BIG_TABLES | + TMP_TABLE_FORCE_MYISAM)), HA_POS_ERROR, (char *)"subquery-cache-table"))) { @@ -191,14 +209,16 @@ DBUG_VOID_RETURN; } - if (cache_table->s->blob_fields) + if (cache_table->s->db_type() != heap_hton) { - DBUG_PRINT("error", ("we do not need blobs")); + DBUG_PRINT("error", ("we need only heap table")); goto error; } + /* first field in the table is result value, so we skip it */ li_items++; field_counter=1; + if (cache_table->alloc_keys(1) || (cache_table->add_tmp_key(0, items.elements - 1, &field_enumerator, @@ -224,6 +244,7 @@ DBUG_PRINT("error", ("Creating Item_field failed")); goto error; } + if (make_equalities()) { DBUG_PRINT("error", ("Creating equalities failed")); @@ -247,11 +268,26 @@ } +/** + Checks if current key present in the cache and returns value if it is true + + @param value assigned Item with value from the cache if key + is found + @return result of the key lookup +*/ + Subquery_cache::result Subquery_cache_tmptable::check_value(Item **value) { int res; DBUG_ENTER("Subquery_cache_tmptable::check_value"); + /* + We delay cache initialization to get item references which should be + used at the moment of query execution. I.e. we store reference on item + reference at the moment of class creation but for table creation and + index supply structures (join_tab) we need real Items which used at the + moment of execution so we can resolve reference only at this point. + */ if (!inited) init(); @@ -275,6 +311,15 @@ } +/** + Puts given value in the cache + + @param value Value to put in the cache + + @retval FALSE OK + @retval TRUE Error +*/ + my_bool Subquery_cache_tmptable::put_value(Item *value) { int error; @@ -313,9 +358,3 @@ cache_table= NULL; DBUG_RETURN(TRUE); } - - -void Subquery_cache_tmptable::cleanup() -{ - cache_table->file->ha_delete_all_rows(); -} === modified file 'sql/sql_subquery_cache.h' --- a/sql/sql_subquery_cache.h 2010-05-25 10:45:36 +0000 +++ b/sql/sql_subquery_cache.h 2010-06-05 19:57:27 +0000 @@ -23,10 +23,6 @@ Puts value into this cache (key should be taken from cache owner) */ virtual my_bool put_value(Item *value)= 0; - /** - Cleans up and reset cache before reusing - */ - virtual void cleanup()= 0; }; struct st_table_ref; @@ -45,10 +41,9 @@ virtual ~Subquery_cache_tmptable(); virtual result check_value(Item **value); virtual my_bool put_value(Item *value); - virtual void cleanup(); + +private: void init(); - -private: bool make_equalities(); /* tmp table parameters */ === modified file 'sql/table.cc' --- a/sql/table.cc 2010-05-27 17:41:38 +0000 +++ b/sql/table.cc 2010-06-05 19:57:27 +0000 @@ -5187,10 +5187,16 @@ key_part_info->store_length= key_part_info->length; if ((*reg_field)->real_maybe_null()) + { key_part_info->store_length+= HA_KEY_NULL_LENGTH; + keyinfo->key_length+= HA_KEY_NULL_LENGTH; + } if ((*reg_field)->type() == MYSQL_TYPE_BLOB || (*reg_field)->real_type() == MYSQL_TYPE_VARCHAR) + { key_part_info->store_length+= HA_KEY_BLOB_LENGTH; + keyinfo->key_length+= HA_KEY_BLOB_LENGTH; // ??? + } key_part_info->type= (uint8) (*reg_field)->key_type(); key_part_info->key_type =

1 0

[Maria-developers] what can go wrong with this code?
by MARK CALLAGHAN 05 Jun '10

05 Jun '10

>From dispatch_command() in sql_parse.cc net_end_statement() is called after ha_autocommit_or_rollback() but before close_thread_tables(). What can go wrong in the call to close_thread_tables() after the response to the client? Commit or rollback was done before a response was sent to the client. /* If commit fails, we should be able to reset the OK status. */ thd->main_da.can_overwrite_status= TRUE; ha_autocommit_or_rollback(thd, thd->is_error()); thd->main_da.can_overwrite_status= FALSE; thd->transaction.stmt.reset(); net_end_statement(thd); query_cache_end_of_result(thd); thd->proc_info= "closing tables"; /* Free tables */ close_thread_tables(thd); -- Mark Callaghan mdcallag(a)gmail.com

2 1

Re: [Maria-developers] [Bug 314570] Re: update is not changing internal auto increment value
by Sergei Golubchik 05 Jun '10

05 Jun '10

Hi, Michael! On Jun 04, Michael Widenius wrote: > > hi! > > >>>>> "Sergei" == Sergei <sergii(a)pisem.net> writes: > > Sergei> ** Changed in: maria > Sergei> Importance: Undecided => Low > > Sergei> -- > Sergei> update is not changing internal auto increment value > Sergei> https://bugs.launchpad.net/bugs/314570 > > Why low ? > > Looks like a serious issue that we should get Percona to fix at once! Because Heikki said it's not a bug, but intentional InnoDB behavior. I'm not sure we should fix it at all. Heikki is certainly not fixing it. Regards, Sergei

2 1

[Maria-developers] MariaDB 5.2.1 Release Notes and Changelog
by Daniel Bartholomew 04 Jun '10

04 Jun '10

All, MariaDB 5.2.1 is getting closer to being released so I've started filling out the Release Notes and Changelog pages: http://askmonty.org/wiki/Manual:MariaDB_5.2.1_Release_Notes http://askmonty.org/wiki/Manual:MariaDB_5.2.1_Changelog On the documentation TODO list for this release is a page on the OQGraph storage engine for the manual. Any volunteers? :) (I'll get to it next week, but if someone wants to put something up right away I wouldn't object.) Thanks. -- Daniel Bartholomew Monty Program - http://askmonty.org

1 0

Re: [Maria-developers] [Commits] Rev 2802: few small MySQL bugs/issues that impact the engines, as discussed in the SE summit in http://bazaar.launchpad.net/~maria-captains/maria/5.2/
by Sergei Golubchik 03 Jun '10

03 Jun '10

Hi, Monty! Thanks for the review! See my replies below. On Jun 03, Michael Widenius wrote: > > > At http://bazaar.launchpad.net/~maria-captains/maria/5.2/ > > ------------------------------------------------------------ > > revno: 2802 > > > few small MySQL bugs/issues that impact the engines, as discussed in the SE summit > > * remove handler::index_read_last() > > * create handler::keyread_read_time() (was get_index_only_read_time() in opt_range.cc) > > * ha_show_status() allows engine's show_status() to fail > > * remove HTON_FLUSH_AFTER_RENAME > > * fix key_cmp_if_same() to work for floats and doubles > > * set table->status in the server, don't force engines to do it > > * increment status vars in the server, don't force engines to do it > > > +++ b/mysql-test/r/status_user.result 2010-06-01 22:39:29 +0000 > > @@ -100,8 +100,8 @@ Handler_commit 19 > > Handler_delete 1 > > Handler_discover 0 > > Handler_prepare 18 > > -Handler_read_first 1 > > -Handler_read_key 8 > > +Handler_read_first 0 > > +Handler_read_key 3 > > Any explanation why this change happened (as the test didn't change > and I can't understand how the values could suddently be less now). This change is correct. Before my commit, calls were counted in the handler, say, in the index_first() and index_next(). And ha_innobase::rnd_next() is implemented by calling index_first/index_next. So, innodb was incrementing Handler_read_first and Handler_read_next for table scans (and of course it was incrementing Handler_read_rnd_next too - double counting). It was wrong - first, it was double counting. Second, Handler_* should count handler calls as done by mysql, not expose internal implementation of the engine. For example, mi_rfirst() calls mi_rnext() internally, but we don't count it as Handler_read_next. The same should be true for any engine, even if it mixes implementation levels. > By the way, it would be nice if the file comments would be part of the > commit email (as I assume you documented this issue there). I have not :( But I will, when I recommit. > > +++ b/mysql-test/r/partition_pruning.result 2010-06-01 22:39:29 +0000 > > @@ -2373,7 +2373,7 @@ flush status; > > update t1 set a=100 where a+1=5+1; > > show status like 'Handler_read_rnd_next'; > > Variable_name Value > > -Handler_read_rnd_next 10 > > +Handler_read_rnd_next 19 > > Any explanation why this change happened (as the test didn't change) > Is it because we don't anymore count rows read in 'show' commands? This is questionable change, I wanted to discuss it. ha_partition::index_next (for example) calls underlying engine's file->ha_index_next(), not file->index_next(). After my change Handler_read_key_next is incremented for both ha_partition::index_next and file->index_next(). Double counting. Before my change when a partition was pruned, Handler_read_key* counters were not incremented at all (as ha_partition::index_read did not call file->index_read() at all). Now it is incremented - that's why the numbers are increased. Possible solutions: * do not increment Handler_read* counters for ha_partition methods, only count calls to the underlying engines. * do not increment Handler_read* counters for underlying engines - only count calls from the upper layer into the handler, this is logical but counters won't show partition pruning or handler call overhead caused by many partitions. this can be solved by adding special set of counters Handler_partition_read_* (or something). > > === modified file 'sql/handler.cc' > > --- a/sql/handler.cc 2010-06-01 19:52:20 +0000 > > +++ b/sql/handler.cc 2010-06-01 22:39:29 +0000 > > @@ -2131,8 +2125,6 @@ int handler::read_first_row(uchar * buf, > > register int error; > > DBUG_ENTER("handler::read_first_row"); > > > > - ha_statistic_increment(&SSV::ha_read_first_count); > > The above is wrong; We are later calling 'index_first()' in this > function, not ha_index_first(), so we miss one increment (which was > shown in the test cases). Note that we do also call rnd_next() in > this function, without any counting of rows so we need to fix other > things in this function too! That's fine, the counter is incremented in ha_read_first_row() wrapper. If anything, the old code was wrong as it was incrementing ha_read_first_count twice (once here and once in index_first). > Simplest solution is to change to call ha_index_first / ha_rnd_next() > in this function. This will also fix the 'table->status' variable that > your are not counting anymore. > This should be ok as we very seldom use 'handler::read_first_row()' see above. I think it's ok to just increment ha_read_first_count in the wrapper. Especially because read_first_row() is rarely used. > Note that we should do same change in other functions that are calling > handler functions directly: > > handler::read_range_first > - This calls index_first() and index_read_map() > get_auto_increment() > - This calls index_last() and index_read_map() > index_read_idx_map() > - This calls index_read_map() > - Note that we can't trivially change this to call ha_index_read_map() > as we increment things statistics in ha_index_read_idx_map() > - We need to update table->status in this function! Yes. But read_range_first() for example has no dedicated counter, so either it increments Handler_read_key* counters in the default implementation, or it increments nothing at all when any engine provides its own implementation :( > > +/* > > + Calculate cost of 'index only' scan for given index and number of records. > > + > > + SYNOPSIS > > + handler->keyread_read_time() > > + param parameters structure > > + records #of records to read > > + keynr key to read > > + > > + NOTES > > + It is assumed that we will read trough the whole key range and that all > > + key blocks are half full (normally things are much better). It is also > > + assumed that each time we read the next key from the index, the handler > > + performs a random seek, thus the cost is proportional to the number of > > + blocks read. > > +*/ > > + > > +double handler::keyread_read_time(uint index, uint ranges, ha_rows rows) > > +{ > > + double read_time; > > + uint keys_per_block= (stats.block_size/2/ > > + (table->key_info[index].key_length + ref_length) + 1); > > + read_time=((double) (rows+keys_per_block-1)/ (double) keys_per_block); > > + return read_time; > > +} > > Do we really need the 'ranges' argument ? > (It's always '1' in the current code and you are not using it) I don't know :) I've copied it from the handler::read_time(), just to have the interface the same for consistency. After all - logically - if the read_time() may depend on the number of ranges, keyread_read_time() certainly can do too. > > === modified file 'sql/key.cc' > > --- a/sql/key.cc 2008-10-10 10:01:01 +0000 > > +++ b/sql/key.cc 2010-06-01 22:39:29 +0000 > > @@ -278,8 +278,10 @@ bool key_cmp_if_same(TABLE *table,const > > key++; > > store_length--; > > } > > - if (key_part->key_part_flag & (HA_BLOB_PART | HA_VAR_LENGTH_PART | > > - HA_BIT_PART)) > > + if ((key_part->key_part_flag & (HA_BLOB_PART | HA_VAR_LENGTH_PART | > > + HA_BIT_PART)) || > > + key_part->type == HA_KEYTYPE_FLOAT || > > + key_part->type == HA_KEYTYPE_DOUBLE) > > { > > if (key_part->field->key_cmp(key, key_part->length)) > > return 1; > > I understand that for float and double there is some extraordinary > cases where memcmp() is not same as =, but who has had a problem with > this? there was a bug report in mysql bugdb. http://bugs.mysql.com/bug.php?id=44372 > As a separate note, I think it would be better to add to key_part_flag > HA_NO_CMP_WITH_MEMCMP for key_parts of type FLOAT or DOUBLE > when we open the table. This would simplify this test a bit. I'll try to > > === modified file 'sql/table.h' > > --- a/sql/table.h 2010-06-01 19:52:20 +0000 > > +++ b/sql/table.h 2010-06-01 22:39:29 +0000 > > @@ -13,6 +13,8 @@ > > along with this program; if not, write to the Free Software > > Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ > > > > +#ifndef SQL_TABLE_INCLUDED > > +#define SQL_TABLE_INCLUDED > > Do we really need this one as it's automaticly included by mysql_priv.h ? > Anyway, it should probably be MYSQL_TABLE_H to be similar our other defines. This is actually unrelated change, I tried to include table.h to handler.h (to solve the problem of inline handler methods needing TABLE) and had to add include guards. later I solved the problem differently but kept the guards as they're a good thing anyway. As for the name of the guard, it's new (~1 yr old) MySQL style. As I personally don't care about the name of the guards, as long as they all use a consistent style, I use the MySQL naming style here. > > === modified file 'storage/myisam/ha_myisam.cc' > > <cut> > > > int ha_myisam::index_next(uchar *buf) > > { > > DBUG_ASSERT(inited==INDEX); > > - ha_statistic_increment(&SSV::ha_read_next_count); > > int error=mi_rnext(file,buf,active_index); > > table->status=error ? STATUS_NOT_FOUND: 0; > > you should probably remove the setting of table->status here Neither updating table->status not ha_statistic_increment() can hurt here, and as you have seen I've not updated any other engine at all. I've only did it in MyISAM to check that the change works, the code compiles, test results don't change, and so on. But I'll remove table->status updates from MyISAM. > The whole function can the be changed to: > > return mi_rnext(file,buf,active_index); > > Same goes for all other instances of setting table->status in this file > Regards, Sergei

1 0

[Maria-developers] Time to remove sql_select.cc:find_best() ?
by Timour Katchaounov 02 Jun '10

02 Jun '10

Hi, I was looking today at some optimizer code, and bumped again into sql_select.cc:find_best(). We have been using the greedy optimizer for years, and this function has been dead code for a while, isn't it time to remove it? The less code, the better. Timour

2 3

Re: [Maria-developers] sphinx vs mariadb
by Andrew Aksyonoff 02 Jun '10

02 Jun '10

Hello Kristian, Thursday, May 27, 2010, 1:20:59 PM, you wrote: KN> [Cc:ed maria-developers@ for general interest, hope that's ok] That's fine. Seems you bcc-ed though. ;) >> Mine was 1.10. Downgrading to 1.9.6 did the trick, thanks. KN> Ok, good, at some point we can get someone to help sort out what KN> the problem might be with 1.10. Or at least add some check that'd *clearly* complain about improper version. Fighting with 1.10 was.. emotional. >> 1. Use prebuilt searchd binary, sphinx.conf file and test index KN> My idea is that mysql-test-run.pl will look for an already KN> installed searchd and indexer binary (in eg. SPHINXSEARCH_INDEXER, KN> SPHINXSEARCH_SEARCD, and maybe $PATH). If not found, sphinxse That's also good. For some reason I though everything should be self contained (ie. work immediately out of bzr clone). KN> tests will be skipped, if found mysql-test-run.pl will generate a KN> simple .conf and start the daemon for the test. There is already a Hmm, why *generate* that? I'd just bundle .conf and source .xml data for indexer. Maybe prebuilt .sp* indexes too. Indexes are binary but test ones can be kept tiny. -- Best regards, Andrew mailto:shodan@shodan.ru

2 3

[Maria-developers] Updated (by Guest): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 01 Jun '10

01 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 60 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Tue, 01 Jun 2010, 14:20)=-=- Status updated. --- /tmp/wklog.116.old.32652 2010-06-01 14:20:15.000000000 +0000 +++ /tmp/wklog.116.new.32652 2010-06-01 14:20:15.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 01 Jun '10

01 Jun '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 60 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Tue, 01 Jun 2010, 14:20)=-=- Status updated. --- /tmp/wklog.116.old.32652 2010-06-01 14:20:15.000000000 +0000 +++ /tmp/wklog.116.new.32652 2010-06-01 14:20:15.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] bzr commit into file:///home/tsk/mprog/src/5.3-mwl89/ branch (timour:2793)
by timour＠askmonty.org 01 Jun '10

01 Jun '10

#At file:///home/tsk/mprog/src/5.3-mwl89/ based on revid:timour@askmonty.org-20100527131347-unr62oupctbp912x 2793 timour(a)askmonty.org 2010-06-01 MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation Phase 2: Changed the code-generation for subquery materialization to be performed in runtime memory for each (re)execution, instead of in statement memory (once per prepared statement). - Item_in_subselect::setup_engine() no longer wraps materialization related objects to be created in statement memory. - Merged subselect_hash_sj_engine::init_permanent and subselect_hash_sj_engine::init_runtime into subselect_hash_sj_engine::init, which is called for each (re)execution. - Fixed deletion of the temp table accordingly. modified: sql/item_subselect.cc sql/item_subselect.h === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-05-27 13:13:47 +0000 +++ b/sql/item_subselect.cc 2010-06-01 11:57:35 +0000 @@ -148,6 +148,7 @@ void Item_in_subselect::cleanup() Item_subselect::~Item_subselect() { delete engine; + engine= NULL; } Item_subselect::trans_res @@ -2090,82 +2091,62 @@ void Item_in_subselect::update_used_tabl bool Item_in_subselect::setup_engine() { - subselect_hash_sj_engine *new_engine= NULL; - bool res= FALSE; + subselect_hash_sj_engine *mat_engine= NULL; + subselect_single_select_engine *select_engine; DBUG_ENTER("Item_in_subselect::setup_engine"); + /* + The select (IN=>EXISTS) engine is pre-created already at parse time, and + is stored in statment memory (preserved across PS executions). + */ + DBUG_ASSERT(engine->engine_type() == subselect_engine::SINGLE_SELECT_ENGINE); + select_engine= (subselect_single_select_engine*) engine; - if (engine->engine_type() == subselect_engine::SINGLE_SELECT_ENGINE) - { - /* Create/initialize objects in permanent memory. */ - subselect_single_select_engine *old_engine; - Query_arena *arena= thd->stmt_arena, backup; - - old_engine= (subselect_single_select_engine*) engine; - - if (arena->is_conventional()) - arena= 0; - else - thd->set_n_backup_active_arena(arena, &backup); - - if (!(new_engine= new subselect_hash_sj_engine(thd, this, - old_engine)) || - new_engine->init_permanent(&old_engine->join->fields_list)) - { - Item_subselect::trans_res trans_res; - /* - If for some reason we cannot use materialization for this IN predicate, - delete all materialization-related objects, and apply the IN=>EXISTS - transformation. - */ - delete new_engine; - new_engine= NULL; - exec_method= NOT_TRANSFORMED; - if (left_expr->cols() == 1) - trans_res= single_value_in_to_exists_transformer(old_engine->join, - &eq_creator); - else - trans_res= row_value_in_to_exists_transformer(old_engine->join); - /* - The IN=>EXISTS transformation above injects new predicates into the - WHERE and HAVING clauses. Since the subquery was already optimized, - below we force its reoptimization with the new injected conditions - by the first call to subselect_single_select_engine::exec(). - This is the only case of lazy subquery optimization in the server. - */ - DBUG_ASSERT(old_engine->join->optimized); - old_engine->join->optimized= false; - res= (trans_res != Item_subselect::RES_OK); - } - if (new_engine) - engine= new_engine; - - if (arena) - thd->restore_active_arena(arena, &backup); - } - else - { - DBUG_ASSERT(engine->engine_type() == subselect_engine::HASH_SJ_ENGINE); - new_engine= (subselect_hash_sj_engine*) engine; - } + /* Create/initialize execution objects. */ + if (!(mat_engine= new subselect_hash_sj_engine(thd, this, select_engine))) + DBUG_RETURN(TRUE); - /* Initilizations done in runtime memory, repeated for each execution. */ - if (new_engine) + if (mat_engine->init(&select_engine->join->fields_list)) { + Item_subselect::trans_res trans_res; + /* + If for some reason we cannot use materialization for this IN predicate, + delete all materialization-related objects, and apply the IN=>EXISTS + transformation. + */ + delete mat_engine; + mat_engine= NULL; + exec_method= NOT_TRANSFORMED; + + if (left_expr->cols() == 1) + trans_res= single_value_in_to_exists_transformer(select_engine->join, + &eq_creator); + else + trans_res= row_value_in_to_exists_transformer(select_engine->join); /* - Reset the LIMIT 1 set in Item_exists_subselect::fix_length_and_dec. - TODO: - Currently we set the subquery LIMIT to infinity, and this is correct - because we forbid at parse time LIMIT inside IN subqueries (see - Item_in_subselect::test_limit). However, once we allow this, here - we should set the correct limit if given in the query. + The IN=>EXISTS transformation above injects new predicates into the + WHERE and HAVING clauses. Since the subquery was already optimized, + below we force its reoptimization with the new injected conditions + by the first call to subselect_single_select_engine::exec(). + This is the only case of lazy subquery optimization in the server. */ - unit->global_parameters->select_limit= NULL; - if ((res= new_engine->init_runtime())) - DBUG_RETURN(res); + DBUG_ASSERT(select_engine->join->optimized); + select_engine->join->optimized= false; + DBUG_RETURN(trans_res != Item_subselect::RES_OK); } - DBUG_RETURN(res); + /* + Reset the "LIMIT 1" set in Item_exists_subselect::fix_length_and_dec. + TODO: + Currently we set the subquery LIMIT to infinity, and this is correct + because we forbid at parse time LIMIT inside IN subqueries (see + Item_in_subselect::test_limit). However, once we allow this, here + we should set the correct limit if given in the query. + */ + unit->global_parameters->select_limit= NULL; + + engine= mat_engine; + DBUG_RETURN(FALSE); } @@ -3680,14 +3661,14 @@ bitmap_init_memroot(MY_BITMAP *map, uint @retval FALSE otherwise */ -bool subselect_hash_sj_engine::init_permanent(List<Item> *tmp_columns) +bool subselect_hash_sj_engine::init(List<Item> *tmp_columns) { select_union *result_sink; /* Options to create_tmp_table. */ ulonglong tmp_create_options= thd->options | TMP_TABLE_ALL_COLUMNS; /* | TMP_TABLE_FORCE_MYISAM; TIMOUR: force MYISAM */ - DBUG_ENTER("subselect_hash_sj_engine::init_permanent"); + DBUG_ENTER("subselect_hash_sj_engine::init"); if (bitmap_init_memroot(&non_null_key_parts, tmp_columns->elements, thd->mem_root) || @@ -3762,6 +3743,17 @@ bool subselect_hash_sj_engine::init_perm !(lookup_engine= make_unique_engine())) DBUG_RETURN(TRUE); + /* + Repeat name resolution for 'cond' since cond is not part of any + clause of the query, and it is not 'fixed' during JOIN::prepare. + */ + if (semi_join_conds && !semi_join_conds->fixed && + semi_join_conds->fix_fields(thd, (Item**)&semi_join_conds)) + DBUG_RETURN(TRUE); + /* Let our engine reuse this query plan for materialization. */ + materialize_join= materialize_engine->join; + materialize_join->change_result(result); + DBUG_RETURN(FALSE); } @@ -3907,30 +3899,6 @@ subselect_hash_sj_engine::make_unique_en } -/** - Initialize members of the engine that need to be re-initilized at each - execution. - - @retval TRUE if a memory allocation error occurred - @retval FALSE if success -*/ - -bool subselect_hash_sj_engine::init_runtime() -{ - /* - Repeat name resolution for 'cond' since cond is not part of any - clause of the query, and it is not 'fixed' during JOIN::prepare. - */ - if (semi_join_conds && !semi_join_conds->fixed && - semi_join_conds->fix_fields(thd, (Item**)&semi_join_conds)) - return TRUE; - /* Let our engine reuse this query plan for materialization. */ - materialize_join= materialize_engine->join; - materialize_join->change_result(result); - return FALSE; -} - - subselect_hash_sj_engine::~subselect_hash_sj_engine() { delete lookup_engine; @@ -3967,6 +3935,13 @@ void subselect_hash_sj_engine::cleanup() count_null_only_columns= 0; strategy= UNDEFINED; materialize_engine->cleanup(); + /* + Restore the original Item_in_subselect engine. This engine is created once + at parse time and stored across executions, while all other materialization + related engines are created and chosen for each execution. + */ + ((Item_in_subselect *) item)->engine= materialize_engine; + if (lookup_engine_type == TABLE_SCAN_ENGINE || lookup_engine_type == ROWID_MERGE_ENGINE) { @@ -3983,6 +3958,9 @@ void subselect_hash_sj_engine::cleanup() DBUG_ASSERT(lookup_engine->engine_type() == UNIQUESUBQUERY_ENGINE); lookup_engine->cleanup(); result->cleanup(); /* Resets the temp table as well. */ + DBUG_ASSERT(tmp_table); + free_tmp_table(thd, tmp_table); + tmp_table= NULL; } === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-05-27 13:13:47 +0000 +++ b/sql/item_subselect.h 2010-06-01 11:57:35 +0000 @@ -802,8 +802,7 @@ public: } ~subselect_hash_sj_engine(); - bool init_permanent(List<Item> *tmp_columns); - bool init_runtime(); + bool init(List<Item> *tmp_columns); void cleanup(); int prepare(); int exec();

1 0

[Maria-developers] Rev 2789: Subquery cache for pre-review (MWL#66) in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 31 May '10

31 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2789 revision-id: sanja(a)askmonty.org-20100531212554-oal32d5v360l6cul parent: sergii(a)pisem.net-20100510134608-oyi2vznyghgcrt0x committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Tue 2010-06-01 00:25:54 +0300 message: Subquery cache for pre-review (MWL#66) === modified file 'libmysqld/Makefile.am' --- a/libmysqld/Makefile.am 2010-03-20 12:01:47 +0000 +++ b/libmysqld/Makefile.am 2010-05-31 21:25:54 +0000 @@ -80,7 +80,8 @@ sql_tablespace.cc \ rpl_injector.cc my_user.c partition_info.cc \ sql_servers.cc event_parse_data.cc opt_table_elimination.cc \ - multi_range_read.cc opt_index_cond_pushdown.cc + multi_range_read.cc opt_index_cond_pushdown.cc \ + sql_subquery_cache.cc libmysqld_int_a_SOURCES= $(libmysqld_sources) nodist_libmysqld_int_a_SOURCES= $(libmysqlsources) $(sqlsources) === modified file 'mysql-test/r/index_merge_myisam.result' --- a/mysql-test/r/index_merge_myisam.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/index_merge_myisam.result 2010-05-31 21:25:54 +0000 @@ -1419,19 +1419,19 @@ # select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='index_merge=off,index_merge_union=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='index_merge_union=on'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,index_merge_sort_union=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=off,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=off,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=4; ERROR 42000: Variable 'optimizer_switch' can't be set to the value of '4' set optimizer_switch=NULL; @@ -1458,21 +1458,21 @@ set optimizer_switch='index_merge=off,index_merge_union=off,default'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; select @@global.optimizer_switch; @@global.optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set @@global.optimizer_switch=default; select @@global.optimizer_switch; @@global.optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on # # Check index_merge's @@optimizer_switch flags # select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (a int, b int, c int, filler char(100), @@ -1582,5 +1582,5 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on drop table t0, t1; === modified file 'mysql-test/r/myisam_mrr.result' --- a/mysql-test/r/myisam_mrr.result 2010-03-11 21:43:31 +0000 +++ b/mysql-test/r/myisam_mrr.result 2010-05-31 21:25:54 +0000 @@ -394,7 +394,7 @@ # - engine_condition_pushdown does not affect ICP select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (a int, b int, key(a)); === added file 'mysql-test/r/subquery_cache.result' --- a/mysql-test/r/subquery_cache.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/subquery_cache.result 2010-05-31 21:25:54 +0000 @@ -0,0 +1,591 @@ +set optimizer_switch='subquery_cache=on'; +flush status; +create table t1 (a int, b int); +insert into t1 values (1,2),(3,4),(1,2),(3,4),(3,4),(4,5),(4,5),(5,6),(5,6),(4,5); +create table t2 (c int, d int); +insert into t2 values (2,3),(3,4),(5,6); +#single value subquery test +select a, (select d from t2 where b=c) + 1 from t1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 6 +Subquery_cache_miss 4 +#single value subquery test (PS) +prepare stmt1 from 'select a, (select d from t2 where b=c) + 1 from t1'; +execute stmt1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 12 +Subquery_cache_miss 8 +execute stmt1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 18 +Subquery_cache_miss 12 +deallocate prepare stmt1; +#single value subquery test (SP) +CREATE PROCEDURE p1() select a, (select d from t2 where b=c) + 1 from t1; +call p1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +call p1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +drop procedure p1; +#IN subquery test +flush status; +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 0 +select a, b , b in (select d from t2) as SUBS from t1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 6 +Subquery_cache_miss 4 +insert into t1 values (7,8),(9,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +7 8 0 +9 NULL NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 12 +Subquery_cache_miss 10 +insert into t2 values (8,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +7 8 NULL +9 NULL NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 18 +Subquery_cache_miss 16 +#IN subquery tesy (PS) +delete from t1 where a > 6; +delete from t2 where c > 6; +prepare stmt1 from 'select a, b , b in (select d from t2) as SUBS from t1'; +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 24 +Subquery_cache_miss 20 +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 30 +Subquery_cache_miss 24 +insert into t1 values (7,8),(9,NULL); +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 36 +Subquery_cache_miss 30 +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 42 +Subquery_cache_miss 36 +insert into t2 values (8,NULL); +execute stmt1; +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 48 +Subquery_cache_miss 42 +execute stmt1; +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 54 +Subquery_cache_miss 48 +deallocate prepare stmt1; +#IN subquery tesy (SP) +delete from t1 where a > 6; +delete from t2 where c > 6; +CREATE PROCEDURE p1() select a, b , b in (select d from t2) as SUBS from t1; +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 60 +Subquery_cache_miss 52 +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 66 +Subquery_cache_miss 56 +insert into t1 values (7,8),(9,NULL); +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 72 +Subquery_cache_miss 62 +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 78 +Subquery_cache_miss 68 +insert into t2 values (8,NULL); +call p1(); +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 84 +Subquery_cache_miss 74 +call p1(); +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 90 +Subquery_cache_miss 80 +drop procedure p1; +# test of simple exists +select a, b , exists (select * from t2 where b=d) as SUBS from t1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +# test of prepared statement exists +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 96 +Subquery_cache_miss 86 +prepare stmt1 from 'select a, b , exists (select * from t2 where b=d) as SUBS from t1'; +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 102 +Subquery_cache_miss 92 +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 108 +Subquery_cache_miss 98 +deallocate prepare stmt1; +# test of stored procedure exists +CREATE PROCEDURE p1() select a, b , exists (select * from t2 where b=d) as SUBS from t1; +call p1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +call p1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +drop procedure p1; +#clean up +drop table t1,t2; +test different types +#int +CREATE TABLE t1 ( a int, b int); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +a +1 +3 +DROP TABLE t1; +#char +CREATE TABLE t1 ( a char(1), b char (1)); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#decimal +CREATE TABLE t1 ( a decimal(3,1), b decimal(3,1)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +a +1.0 +3.0 +DROP TABLE t1; +#date +CREATE TABLE t1 ( a date, b date); +INSERT INTO t1 VALUES('1000-01-01','1000-01-01'),('2000-02-01','2000-02-01'),('3000-03-03','3000-03-03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-01'); +a +1000-01-01 +3000-03-03 +DROP TABLE t1; +#datetime +CREATE TABLE t1 ( a datetime, b datetime); +INSERT INTO t1 VALUES('1000-01-01 01:01:01','1000-01-01 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('3000-03-03 03:03:03','3000-03-03 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +a +1000-01-01 01:01:01 +3000-03-03 03:03:03 +DROP TABLE t1; +#time +CREATE TABLE t1 ( a time, b time); +INSERT INTO t1 VALUES('01:01:01','01:01:01'),('02:02:02','02:02:02'),('03:03:03','03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '02:02:02'); +a +01:01:01 +03:03:03 +DROP TABLE t1; +#timestamp +CREATE TABLE t1 ( a timestamp, b timestamp); +INSERT INTO t1 VALUES('2000-02-02 01:01:01','2000-02-02 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('2000-02-02 03:03:03','2000-02-02 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +a +2000-02-02 01:01:01 +2000-02-02 03:03:03 +DROP TABLE t1; +#bit +CREATE TABLE t1 ( a bit(20), b bit(20)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a+0 FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +a+0 +1 +3 +DROP TABLE t1; +#enum +CREATE TABLE t1 ( a enum('1','2','3'), b enum('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#set +CREATE TABLE t1 ( a set('1','2','3'), b set('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#blob +CREATE TABLE t1 ( a blob, b blob); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#geometry +CREATE TABLE t1 ( a geometry, b geometry); +INSERT INTO t1 VALUES(POINT(1,1),POINT(1,1)),(POINT(2,2),POINT(2,2)),(POINT(3,3),POINT(3,3)); +SELECT astext(a) FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = POINT(2,2)); +astext(a) +POINT(1 1) +POINT(3 3) +DROP TABLE t1; +#uncacheable queries test (random and side effect) +flush status; +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (2), (4), (1), (3); +select a, a in (select a from t1) from t1 as ext; +a a in (select a from t1) +2 1 +4 1 +1 1 +3 1 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 4 +select a, a in (select a from t1 where -1 < rand()) from t1 as ext; +a a in (select a from t1 where -1 < rand()) +2 1 +4 1 +1 1 +3 1 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 4 +select a, a in (select a from t1 where -1 < benchmark(a,100)) from t1 as ext; +a a in (select a from t1 where -1 < benchmark(a,100)) +2 1 +4 1 +1 1 +3 1 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 4 +drop table t1; +set optimizer_switch='subquery_cache=default'; === modified file 'mysql-test/r/subselect3.result' --- a/mysql-test/r/subselect3.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3.result 2010-05-31 21:25:54 +0000 @@ -105,6 +105,7 @@ Handler_read_rnd_next 5 delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; oref a Z @@ -123,6 +124,7 @@ select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; Z No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1. +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; create table t1 (a int, b int, primary key (a)); insert into t1 values (1,1), (3,1),(100,1); === modified file 'mysql-test/r/subselect3_jcl6.result' --- a/mysql-test/r/subselect3_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3_jcl6.result 2010-05-31 21:25:54 +0000 @@ -109,6 +109,7 @@ Handler_read_rnd_next 5 delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; oref a Z @@ -127,6 +128,7 @@ select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; Z No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1. +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; create table t1 (a int, b int, primary key (a)); insert into t1 values (1,1), (3,1),(100,1); === modified file 'mysql-test/r/subselect_no_mat.result' --- a/mysql-test/r/subselect_no_mat.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_mat.result 2010-05-31 21:25:54 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='materialization=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_no_opts.result' --- a/mysql-test/r/subselect_no_opts.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_opts.result 2010-05-31 21:25:54 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='materialization=off,semijoin=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_no_semijoin.result' --- a/mysql-test/r/subselect_no_semijoin.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_semijoin.result 2010-05-31 21:25:54 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='semijoin=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_sj.result' --- a/mysql-test/r/subselect_sj.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect_sj.result 2010-05-31 21:25:54 +0000 @@ -202,39 +202,39 @@ select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; drop table t0, t1, t2; drop table t10, t11, t12; === modified file 'mysql-test/r/subselect_sj_jcl6.result' --- a/mysql-test/r/subselect_sj_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect_sj_jcl6.result 2010-05-31 21:25:54 +0000 @@ -206,39 +206,39 @@ select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; drop table t0, t1, t2; drop table t10, t11, t12; === added file 'mysql-test/t/subquery_cache.test' --- a/mysql-test/t/subquery_cache.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/subquery_cache.test 2010-05-31 21:25:54 +0000 @@ -0,0 +1,204 @@ + +set optimizer_switch='subquery_cache=on'; +flush status; + +create table t1 (a int, b int); +insert into t1 values (1,2),(3,4),(1,2),(3,4),(3,4),(4,5),(4,5),(5,6),(5,6),(4,5); +create table t2 (c int, d int); +insert into t2 values (2,3),(3,4),(5,6); + +--echo #single value subquery test +select a, (select d from t2 where b=c) + 1 from t1; + +show status like "subquery_cache%"; + +--echo #single value subquery test (PS) +prepare stmt1 from 'select a, (select d from t2 where b=c) + 1 from t1'; +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; +deallocate prepare stmt1; + +--echo #single value subquery test (SP) +CREATE PROCEDURE p1() select a, (select d from t2 where b=c) + 1 from t1; + +call p1; +call p1; + +drop procedure p1; + +--echo #IN subquery test +flush status; + +show status like "subquery_cache%"; +select a, b , b in (select d from t2) as SUBS from t1; +show status like "subquery_cache%"; + +insert into t1 values (7,8),(9,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +show status like "subquery_cache%"; + +insert into t2 values (8,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +show status like "subquery_cache%"; + +--echo #IN subquery tesy (PS) +delete from t1 where a > 6; +delete from t2 where c > 6; + +prepare stmt1 from 'select a, b , b in (select d from t2) as SUBS from t1'; +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; + +insert into t1 values (7,8),(9,NULL); +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; + +insert into t2 values (8,NULL); +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; + +deallocate prepare stmt1; + + +--echo #IN subquery tesy (SP) +delete from t1 where a > 6; +delete from t2 where c > 6; + +CREATE PROCEDURE p1() select a, b , b in (select d from t2) as SUBS from t1; + +call p1(); +show status like "subquery_cache%"; +call p1(); +show status like "subquery_cache%"; + +insert into t1 values (7,8),(9,NULL); +call p1(); +show status like "subquery_cache%"; +call p1(); +show status like "subquery_cache%"; + +insert into t2 values (8,NULL); +call p1(); +show status like "subquery_cache%"; +call p1(); +show status like "subquery_cache%"; + +drop procedure p1; + + +--echo # test of simple exists +select a, b , exists (select * from t2 where b=d) as SUBS from t1; + +--echo # test of prepared statement exists +show status like "subquery_cache%"; +prepare stmt1 from 'select a, b , exists (select * from t2 where b=d) as SUBS from t1'; +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; +deallocate prepare stmt1; + +--echo # test of stored procedure exists +CREATE PROCEDURE p1() select a, b , exists (select * from t2 where b=d) as SUBS from t1; +call p1; +call p1; +drop procedure p1; + +--echo #clean up +drop table t1,t2; + +--echo test different types +--echo #int +CREATE TABLE t1 ( a int, b int); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +DROP TABLE t1; + +--echo #char +CREATE TABLE t1 ( a char(1), b char (1)); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #decimal +CREATE TABLE t1 ( a decimal(3,1), b decimal(3,1)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +DROP TABLE t1; + +--echo #date +CREATE TABLE t1 ( a date, b date); +INSERT INTO t1 VALUES('1000-01-01','1000-01-01'),('2000-02-01','2000-02-01'),('3000-03-03','3000-03-03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-01'); +DROP TABLE t1; + +--echo #datetime +CREATE TABLE t1 ( a datetime, b datetime); +INSERT INTO t1 VALUES('1000-01-01 01:01:01','1000-01-01 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('3000-03-03 03:03:03','3000-03-03 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +DROP TABLE t1; + +--echo #time +CREATE TABLE t1 ( a time, b time); +INSERT INTO t1 VALUES('01:01:01','01:01:01'),('02:02:02','02:02:02'),('03:03:03','03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '02:02:02'); +DROP TABLE t1; + +--echo #timestamp +CREATE TABLE t1 ( a timestamp, b timestamp); +INSERT INTO t1 VALUES('2000-02-02 01:01:01','2000-02-02 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('2000-02-02 03:03:03','2000-02-02 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +DROP TABLE t1; + +--echo #bit +CREATE TABLE t1 ( a bit(20), b bit(20)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a+0 FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +DROP TABLE t1; + +--echo #enum +CREATE TABLE t1 ( a enum('1','2','3'), b enum('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #set +CREATE TABLE t1 ( a set('1','2','3'), b set('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #blob +CREATE TABLE t1 ( a blob, b blob); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #geometry +CREATE TABLE t1 ( a geometry, b geometry); +INSERT INTO t1 VALUES(POINT(1,1),POINT(1,1)),(POINT(2,2),POINT(2,2)),(POINT(3,3),POINT(3,3)); +SELECT astext(a) FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = POINT(2,2)); +DROP TABLE t1; + + +--echo #uncacheable queries test (random and side effect) +flush status; +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (2), (4), (1), (3); +select a, a in (select a from t1) from t1 as ext; +show status like "subquery_cache%"; +select a, a in (select a from t1 where -1 < rand()) from t1 as ext; +show status like "subquery_cache%"; +select a, a in (select a from t1 where -1 < benchmark(a,100)) from t1 as ext; +show status like "subquery_cache%"; +drop table t1; + +set optimizer_switch='subquery_cache=default'; === modified file 'mysql-test/t/subselect3.test' --- a/mysql-test/t/subselect3.test 2010-03-20 12:01:47 +0000 +++ b/mysql-test/t/subselect3.test 2010-05-31 21:25:54 +0000 @@ -98,10 +98,12 @@ delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; show status like '%Handler_read%'; select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; === modified file 'sql/CMakeLists.txt' --- a/sql/CMakeLists.txt 2010-03-20 12:01:47 +0000 +++ b/sql/CMakeLists.txt 2010-05-31 21:25:54 +0000 @@ -78,7 +78,7 @@ rpl_rli.cc rpl_mi.cc sql_servers.cc sql_connect.cc scheduler.cc sql_profile.cc event_parse_data.cc opt_table_elimination.cc - ds_mrr.cc + ds_mrr.cc sql_subquery_cache.cc ${PROJECT_SOURCE_DIR}/sql/sql_yacc.cc ${PROJECT_SOURCE_DIR}/sql/sql_yacc.h ${PROJECT_SOURCE_DIR}/include/mysqld_error.h === modified file 'sql/Makefile.am' --- a/sql/Makefile.am 2010-03-20 12:01:47 +0000 +++ b/sql/Makefile.am 2010-05-31 21:25:54 +0000 @@ -80,7 +80,7 @@ event_data_objects.h event_scheduler.h \ sql_partition.h partition_info.h partition_element.h \ contributors.h sql_servers.h \ - multi_range_read.h + multi_range_read.h sql_subquery_cache.h mysqld_SOURCES = sql_lex.cc sql_handler.cc sql_partition.cc \ item.cc item_sum.cc item_buff.cc item_func.cc \ @@ -130,7 +130,7 @@ sql_servers.cc event_parse_data.cc \ opt_table_elimination.cc \ multi_range_read.cc \ - opt_index_cond_pushdown.cc + opt_index_cond_pushdown.cc sql_subquery_cache.cc nodist_mysqld_SOURCES = mini_client_errors.c pack.c client.c my_time.c my_user.c === modified file 'sql/item.cc' --- a/sql/item.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item.cc 2010-05-31 21:25:54 +0000 @@ -28,6 +28,9 @@ const String my_null_string("NULL", 4, default_charset_info); +static int save_field_in_field(Field *from,my_bool * null_value, + Field *to, bool no_conversions); + /****************************************************************************/ /* Hybrid_type_traits {_real} */ @@ -2273,6 +2276,13 @@ str->append(str_value); } +void Item_bool_cache::print(String *str, enum_query_type query_type) +{ + if (null_value) + str->append("NULL", 4); + else + Item_int::print(str, query_type); +} Item_uint::Item_uint(const char *str_arg, uint length): Item_int(str_arg, length) @@ -3646,12 +3656,17 @@ resolved_item->db_name : ""); const char *table_name= (resolved_item->table_name ? resolved_item->table_name : ""); + DBUG_ENTER("mark_as_dependent"); + DBUG_PRINT("enter", ("Field '%s.%s.%s in select %d resolved in %d", + db_name, table_name, + resolved_item->field_name, current->select_number, + last->select_number)); /* store pointer on SELECT_LEX from which item is dependent */ if (mark_item) mark_item->depended_from= last; if (current->mark_as_dependent(thd, last, /** resolved_item psergey-thu **/mark_item)) - return TRUE; + DBUG_RETURN(TRUE); if (thd->lex->describe & DESCRIBE_EXTENDED) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, @@ -3661,7 +3676,7 @@ resolved_item->field_name, current->select_number, last->select_number); } - return FALSE; + DBUG_RETURN(FALSE); } @@ -3698,6 +3713,7 @@ resolving) */ SELECT_LEX *previous_select= current_sel; + for (; previous_select->outer_select() != last_select; previous_select= previous_select->outer_select()) { @@ -3726,6 +3742,7 @@ mark_as_dependent(thd, last_select, current_sel, resolved_item, dependent); } + return; } @@ -4098,6 +4115,9 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); return 0; } } @@ -4113,7 +4133,9 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); - + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* A reference to a view field had been found and we substituted it instead of this Item (find_field_in_tables @@ -4215,6 +4237,10 @@ mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, rf, rf); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); + return 0; } else @@ -4222,6 +4248,9 @@ mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, this, (Item_ident*)*reference); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); if (last_checked_context->select_lex->having_fix_field) { Item_ref *rf; @@ -5082,39 +5111,48 @@ /** + Saves one Fields of an Item of in other Field + + @param from Field to copy value from + @param null_value reference on item null_value to set it if it is needed + @param to Field to cope value to + @param no_conversions how to deal with NULL value (see + set_field_to_null_with_conversions()) + + @retval FALSE OK + @retval TRUE Error +*/ + +static int save_field_in_field(Field *from, my_bool *null_value, + Field *to, bool no_conversions) +{ + int res; + if (from->is_null()) + { + (*null_value)= 1; + res= set_field_to_null_with_conversions(to, no_conversions); + } + else + { + to->set_notnull(); + res= field_conv(to, from); + (*null_value)= 0; + } + return res; +} + +/** Set a field's value from a item. */ void Item_field::save_org_in_field(Field *to) { - if (field->is_null()) - { - null_value=1; - set_field_to_null_with_conversions(to, 1); - } - else - { - to->set_notnull(); - field_conv(to,field); - null_value=0; - } + save_field_in_field(field, &null_value, to, TRUE); } int Item_field::save_in_field(Field *to, bool no_conversions) { - int res; - if (result_field->is_null()) - { - null_value=1; - res= set_field_to_null_with_conversions(to, no_conversions); - } - else - { - to->set_notnull(); - res= field_conv(to,result_field); - null_value=0; - } - return res; + return save_field_in_field(result_field, &null_value, to, no_conversions); } @@ -5973,6 +6011,9 @@ refer_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* view reference found, we substituted it instead of this Item, so can quit @@ -6023,6 +6064,9 @@ thd->change_item_tree(reference, fld); mark_as_dependent(thd, last_checked_context->select_lex, thd->lex->current_select, fld, fld); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of @@ -6046,6 +6090,9 @@ DBUG_ASSERT(*ref && (*ref)->fixed); mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, this, this); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + ref); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of @@ -6312,7 +6359,8 @@ int Item_ref::save_in_field(Field *to, bool no_conversions) { int res; - DBUG_ASSERT(!result_field); + if (result_field) + return save_field_in_field(result_field, &null_value, to, no_conversions); res= (*ref)->save_in_field(to, no_conversions); null_value= (*ref)->null_value; return res; === modified file 'sql/item.h' --- a/sql/item.h 2010-03-20 12:01:47 +0000 +++ b/sql/item.h 2010-05-31 21:25:54 +0000 @@ -1922,8 +1922,31 @@ virtual void print(String *str, enum_query_type query_type); Item_num *neg (); uint decimal_precision() const { return max_length; } - bool check_partition_func_processor(uchar *bool_arg) { return FALSE;} - bool check_vcol_func_processor(uchar *arg) { return FALSE;} +}; + + +/** + Item represent TRUE/FALSE/NULL for subquery values +*/ + +class Item_bool_cache: public Item_int +{ +public: + Item_bool_cache(): Item_int(0, 1) + { + unsigned_flag= maybe_null= null_value= TRUE; + name= (char *)"bool chache"; + } + Item_bool_cache(my_bool val, my_bool null): Item_int(val, 1) + { + unsigned_flag= maybe_null= TRUE; + null_value= null; + name= (char *)"bool chache"; + } + Item *clone_item() { return new Item_bool_cache(value, null_value); } + uint decimal_precision() const { return 1; } + virtual void print(String *str, enum_query_type query_type); + void set(my_bool val, my_bool null) {value= test(val); null_value= null;} }; @@ -3146,7 +3169,8 @@ example(0), used_table_map(0), cached_field(0), cached_field_type(MYSQL_TYPE_STRING), value_cached(0) { - fixed= 1; + fixed= 1; + maybe_null= 1; null_value= 1; } Item_cache(enum_field_types field_type_arg): @@ -3154,6 +3178,7 @@ value_cached(0) { fixed= 1; + maybe_null= 1; null_value= 1; } === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.cc 2010-05-31 21:25:54 +0000 @@ -1736,6 +1736,15 @@ used_tables_cache|= args[1]->used_tables(); not_null_tables_cache|= args[1]->not_null_tables(); const_item_cache&= args[1]->const_item(); + DBUG_ASSERT(scache == NULL); + if (args[0]->cols() ==1 && + thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE && + !(sub->engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) + { + sub->depends_on.push_front((Item**)&cache); + scache= new Subquery_cache_tmptable(thd, sub->depends_on, &result); + } fixed= 1; return FALSE; } @@ -1744,10 +1753,26 @@ longlong Item_in_optimizer::val_int() { bool tmp; + DBUG_ENTER("Item_in_optimizer::val_int"); + DBUG_ASSERT(fixed == 1); cache->store(args[0]); cache->cache_value(); - + + /* check if result is in the cache */ + if (scache) + { + Subquery_cache_tmptable::result res; + Item *cached_value; + res= scache->check_value(&cached_value); + if (res == Subquery_cache_tmptable::HIT) + { + tmp= cached_value->val_int(); + null_value= cached_value->null_value; + DBUG_RETURN(tmp); + } + } + if (cache->null_value) { /* @@ -1818,11 +1843,18 @@ for (uint i= 0; i < ncols; i++) item_subs->set_cond_guard_var(i, TRUE); } - return 0; + DBUG_RETURN(0); } tmp= args[1]->val_bool_result(); null_value= args[1]->null_value; - return tmp; + + /* put result in the cache */ + if (scache) + { + result.set(tmp, null_value); + scache->put_value(&result); + } + DBUG_RETURN(tmp); } @@ -1839,6 +1871,11 @@ Item_bool_func::cleanup(); if (!save_cache) cache= 0; + if (scache) + { + delete scache; + scache= 0; + } DBUG_VOID_RETURN; } === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.h 2010-05-31 21:25:54 +0000 @@ -215,6 +215,7 @@ class Item_cache; +class Subquery_cache; #define UNKNOWN ((my_bool)-1) @@ -237,6 +238,10 @@ { protected: Item_cache *cache; + /* Subquery cache */ + Subquery_cache *scache; + /* result representation for the subquery cache */ + Item_bool_cache result; bool save_cache; /* Stores the value of "NULL IN (SELECT ...)" for uncorrelated subqueries: @@ -247,7 +252,7 @@ my_bool result_for_null_param; public: Item_in_optimizer(Item *a, Item_in_subselect *b): - Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), + Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), scache(NULL), save_cache(0), result_for_null_param(UNKNOWN) {} bool fix_fields(THD *, Item **); === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.cc 2010-05-31 21:25:54 +0000 @@ -34,11 +34,10 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), - engine(0), old_engine(0), used_tables_cache(0), have_to_be_excluded(0), - const_item_cache(1), - inside_first_fix_fields(0), done_first_fix_fields(FALSE), - eliminated(FALSE), - engine_changed(0), changed(0), is_correlated(FALSE) + engine(0), old_engine(0), scache(0), used_tables_cache(0), + have_to_be_excluded(0), const_item_cache(1), inside_first_fix_fields(0), + done_first_fix_fields(FALSE), eliminated(FALSE), engine_changed(0), + changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -116,6 +115,12 @@ } if (engine) engine->cleanup(); + depends_on.empty(); + if (scache) + { + delete scache; + scache= 0; + } reset(); value_assigned= 0; DBUG_VOID_RETURN; @@ -148,6 +153,8 @@ Item_subselect::~Item_subselect() { delete engine; + if (scache) + delete scache; } Item_subselect::trans_res @@ -746,9 +753,22 @@ void Item_singlerow_subselect::fix_length_and_dec() { + DBUG_ENTER("Item_singlerow_subselect::fix_length_and_dec"); if ((max_columns= engine->cols()) == 1) { + DBUG_PRINT("info", ("one, elements: %u flag %u", + (uint)depends_on.elements, + (uint)test(thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE))); engine->fix_length_and_dec(row= &value); + if (depends_on.elements && + optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && + !(engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) + { + DBUG_ASSERT(scache == NULL); + scache= new Subquery_cache_tmptable(thd, depends_on, value); + DBUG_PRINT("info", ("cache: 0x%lx", (ulong) scache)); + } } else { @@ -765,6 +785,7 @@ */ if (engine->no_tables()) maybe_null= engine->may_be_null(); + DBUG_VOID_RETURN; } uint Item_singlerow_subselect::cols() @@ -797,77 +818,206 @@ exec(); } +/** + Checks subquery cache for value + + @retval NULL nothing found + @retval reference on item representing value found in the cache +*/ + +Item *Item_subselect::check_cache() +{ + DBUG_ENTER("Item_subselect::check_cache"); + if (scache) + { + Subquery_cache_tmptable::result res; + Item *cached_value; + res= scache->check_value(&cached_value); + if (res == Subquery_cache_tmptable::HIT) + DBUG_RETURN(cached_value); + } + DBUG_RETURN(NULL); +} + double Item_singlerow_subselect::val_real() { + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_real"); DBUG_ASSERT(fixed == 1); - if (!exec() && !value->null_value) + + if ((cached_value = check_cache())) + { + double res= cached_value->val_real(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_real(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_real()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } longlong Item_singlerow_subselect::val_int() { + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_int"); DBUG_ASSERT(fixed == 1); - if (!exec() && !value->null_value) + + if ((cached_value = check_cache())) + { + longlong res= cached_value->val_int(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_int(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_int()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } String *Item_singlerow_subselect::val_str(String *str) { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_str"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + String *res= cached_value->val_str(str); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_str(str); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_str(str)); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } my_decimal *Item_singlerow_subselect::val_decimal(my_decimal *decimal_value) { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_decimal"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_decimal *res= cached_value->val_decimal(decimal_value); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_decimal(decimal_value); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_decimal(decimal_value)); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } bool Item_singlerow_subselect::val_bool() { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_bool"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + bool res= cached_value->val_bool(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_bool(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_bool()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } @@ -952,33 +1102,79 @@ void Item_exists_subselect::fix_length_and_dec() { + DBUG_ENTER("Item_exists_subselect::fix_length_and_dec"); decimals= 0; max_length= 1; max_columns= engine->cols(); /* We need only 1 row to determine existence */ unit->global_parameters->select_limit= new Item_int((int32) 1); + if (substype() == EXISTS_SUBS && depends_on.elements && + optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && + !(engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) + { + DBUG_ASSERT(scache == NULL); + scache= new Subquery_cache_tmptable(thd, depends_on, &result); + DBUG_PRINT("info", ("cache: 0x%lx", (ulong) scache)); + } + DBUG_VOID_RETURN; } double Item_exists_subselect::val_real() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_int"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + double res= cached_value->val_real(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) { reset(); - return 0; - } - return (double) value; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN((double) value); } longlong Item_exists_subselect::val_int() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_real"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + longlong res= cached_value->val_int(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + DBUG_ASSERT(fixed == 1); if (exec()) { reset(); - return 0; - } - return value; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN(value); } @@ -997,11 +1193,32 @@ String *Item_exists_subselect::val_str(String *str) { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_str"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + String *res= cached_value->val_str(str); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) + { reset(); + str->set((ulonglong)0,&my_charset_bin); + DBUG_RETURN(str); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + str->set((ulonglong)value,&my_charset_bin); - return str; + DBUG_RETURN(str); } @@ -1020,23 +1237,61 @@ my_decimal *Item_exists_subselect::val_decimal(my_decimal *decimal_value) { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_decvimal"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_decimal *res= cached_value->val_decimal(decimal_value); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) + { reset(); + int2my_decimal(E_DEC_FATAL_ERROR, 0, 0, decimal_value); + DBUG_RETURN(decimal_value); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + int2my_decimal(E_DEC_FATAL_ERROR, value, 0, decimal_value); - return decimal_value; + DBUG_RETURN(decimal_value); } bool Item_exists_subselect::val_bool() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_real"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_bool res= cached_value->val_bool(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) { reset(); - return 0; - } - return value != 0; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN(value != 0); } === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.h 2010-05-31 21:25:54 +0000 @@ -27,6 +27,7 @@ class subselect_hash_sj_engine; class Item_bool_func2; class Cached_item; +class Subquery_cache; /* base class for subselects */ @@ -57,6 +58,10 @@ subselect_engine *engine; /* old engine if engine was changed */ subselect_engine *old_engine; + /* subquery cache */ + Subquery_cache *scache; + /* null consrtant for caching */ + Item_null const_null_value; /* cache of used external tables */ table_map used_tables_cache; /* allowed number of columns (1 for single value subqueries) */ @@ -67,7 +72,7 @@ bool have_to_be_excluded; /* cache of constant state */ bool const_item_cache; - + bool inside_first_fix_fields; bool done_first_fix_fields; public: @@ -88,13 +93,21 @@ */ List<Ref_to_outside> upper_refs; st_select_lex *parent_select; - - /* + + /** + List of references on items subquery depends on (externally resolved); + + @note We can't store direct links on Items because it could be + substituted with other item (for example for grouping). + */ + List<Item*> depends_on; + + /* TRUE<=>Table Elimination has made it redundant to evaluate this select (and so it is not part of QEP, etc) - */ + */ bool eliminated; - + /* changed engine indicator */ bool engine_changed; /* subquery is transformed */ @@ -178,6 +191,8 @@ return trace_unsupported_by_check_vcol_func_processor("subselect"); } + Item *check_cache(); + /** Get the SELECT_LEX structure associated with this Item. @return the SELECT_LEX structure associated with this Item @@ -202,6 +217,7 @@ { protected: Item_cache *value, **row; + public: Item_singlerow_subselect(st_select_lex *select_lex); Item_singlerow_subselect() :Item_subselect(), value(0), row (0) {} @@ -268,6 +284,8 @@ { protected: bool value; /* value of this item (boolean: exists/not-exists) */ + /* result representation for the subquery cache */ + Item_bool_cache result; public: Item_exists_subselect(st_select_lex *select_lex); === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_sum.cc 2010-05-31 21:25:54 +0000 @@ -319,6 +319,7 @@ if (aggr_level >= 0) { ref_by= ref; + thd->lex->current_select->register_dependency_item(aggr_sel, ref); /* Add the object to the list of registered objects assigned to aggr_sel */ if (!aggr_sel->inner_sum_func_list) next= this; === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-03-20 12:01:47 +0000 +++ b/sql/mysql_priv.h 2010-05-31 21:25:54 +0000 @@ -568,12 +568,13 @@ #define OPTIMIZER_SWITCH_SEMIJOIN 256 #define OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE 512 #define OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN 1024 +#define OPTIMIZER_SWITCH_SUBQUERY_CACHE (1<<11) #ifdef DBUG_OFF -# define OPTIMIZER_SWITCH_LAST 2048 +# define OPTIMIZER_SWITCH_LAST (1<<12) #else -# define OPTIMIZER_SWITCH_TABLE_ELIMINATION 2048 -# define OPTIMIZER_SWITCH_LAST 4096 +# define OPTIMIZER_SWITCH_TABLE_ELIMINATION (1<<12) +# define OPTIMIZER_SWITCH_LAST (1<<13) #endif #ifdef DBUG_OFF @@ -588,7 +589,8 @@ OPTIMIZER_SWITCH_MATERIALIZATION | \ OPTIMIZER_SWITCH_SEMIJOIN | \ OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE|\ - OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN) + OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN|\ + OPTIMIZER_SWITCH_SUBQUERY_CACHE) #else # define OPTIMIZER_SWITCH_DEFAULT (OPTIMIZER_SWITCH_INDEX_MERGE | \ OPTIMIZER_SWITCH_INDEX_MERGE_UNION | \ @@ -601,7 +603,8 @@ OPTIMIZER_SWITCH_MATERIALIZATION | \ OPTIMIZER_SWITCH_SEMIJOIN | \ OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE|\ - OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN) + OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN|\ + OPTIMIZER_SWITCH_SUBQUERY_CACHE) #endif /* @@ -936,6 +939,7 @@ #ifdef MYSQL_SERVER #include "sql_servers.h" #include "opt_range.h" +#include "sql_subquery_cache.h" #ifdef HAVE_QUERY_CACHE struct Query_cache_query_flags @@ -1269,6 +1273,10 @@ Item *having, ORDER *proc_param, ulonglong select_type, select_result *result, SELECT_LEX_UNIT *unit, SELECT_LEX *select_lex); + +struct st_join_table *create_index_lookup_join_tab(TABLE *table); +int join_read_key2(THD *thd, struct st_join_table *tab, TABLE *table, + struct st_table_ref *table_ref); void free_underlaid_joins(THD *thd, SELECT_LEX *select); bool mysql_explain_union(THD *thd, SELECT_LEX_UNIT *unit, select_result *result); @@ -1288,6 +1296,7 @@ bool table_cant_handle_bit_fields, bool make_copy_field, uint convert_blob_length); +bool open_tmp_table(TABLE *table); void sp_prepare_create_field(THD *thd, Create_field *sql_field); int prepare_create_field(Create_field *sql_field, uint *blob_columns, === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2010-03-20 12:01:47 +0000 +++ b/sql/mysqld.cc 2010-05-31 21:25:54 +0000 @@ -305,6 +305,7 @@ "firstmatch","loosescan","materialization", "semijoin", "partial_match_rowid_merge", "partial_match_table_scan", + "subquery_cache", #ifndef DBUG_OFF "table_elimination", #endif @@ -325,6 +326,7 @@ sizeof("semijoin") - 1, sizeof("partial_match_rowid_merge") - 1, sizeof("partial_match_table_scan") - 1, + sizeof("subquery_cache") - 1, #ifndef DBUG_OFF sizeof("table_elimination") - 1, #endif @@ -404,8 +406,9 @@ static const char *optimizer_switch_str="index_merge=on,index_merge_union=on," "index_merge_sort_union=on," "index_merge_intersection=on," - "index_condition_pushdown=on" -#ifndef DBUG_OFF + "index_condition_pushdown=on," + "subquery_cache=on" +#ifndef DBUG_OFF ",table_elimination=on"; #else ; @@ -5872,7 +5875,9 @@ OPT_RECORD_RND_BUFFER, OPT_DIV_PRECINCREMENT, OPT_RELAY_LOG_SPACE_LIMIT, OPT_RELAY_LOG_PURGE, OPT_SLAVE_NET_TIMEOUT, OPT_SLAVE_COMPRESSED_PROTOCOL, OPT_SLOW_LAUNCH_TIME, - OPT_SLAVE_TRANS_RETRIES, OPT_READONLY, OPT_ROWID_MERGE_BUFF_SIZE, + OPT_SLAVE_TRANS_RETRIES, + OPT_SUBQUERY_CACHE, + OPT_READONLY, OPT_ROWID_MERGE_BUFF_SIZE, OPT_DEBUGGING, OPT_DEBUG_FLUSH, OPT_SORT_BUFFER, OPT_TABLE_OPEN_CACHE, OPT_TABLE_DEF_CACHE, OPT_THREAD_CONCURRENCY, OPT_THREAD_CACHE_SIZE, @@ -7164,7 +7169,7 @@ {"optimizer_switch", OPT_OPTIMIZER_SWITCH, "optimizer_switch=option=val[,option=val...], where option={index_merge, " "index_merge_union, index_merge_sort_union, index_merge_intersection, " - "index_condition_pushdown" + "index_condition_pushdown, subquery_cache" #ifndef DBUG_OFF ", table_elimination" #endif @@ -7868,6 +7873,8 @@ {"Ssl_version", (char*) &show_ssl_get_version, SHOW_FUNC}, #endif /* HAVE_OPENSSL */ {"Syncs", (char*) &my_sync_count, SHOW_LONG_NOFLUSH}, + {"Subquery_cache_hit", (char*) &subquery_cache_hit, SHOW_LONG}, + {"Subquery_cache_miss", (char*) &subquery_cache_miss, SHOW_LONG}, {"Table_locks_immediate", (char*) &locks_immediate, SHOW_LONG}, {"Table_locks_waited", (char*) &locks_waited, SHOW_LONG}, #ifdef HAVE_MMAP @@ -8006,6 +8013,7 @@ abort_loop= select_thread_in_use= signal_thread_in_use= 0; ready_to_exit= shutdown_in_progress= grant_option= 0; aborted_threads= aborted_connects= 0; + subquery_cache_miss= subquery_cache_hit= 0; delayed_insert_threads= delayed_insert_writes= delayed_rows_in_use= 0; delayed_insert_errors= thread_created= 0; specialflag= 0; === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_base.cc 2010-05-31 21:25:54 +0000 @@ -8062,6 +8062,10 @@ if (*conds) { thd->where="where clause"; + DBUG_EXECUTE("where", + print_where(*conds, + "WHERE in setup_conds", + QT_ORDINARY);); if ((!(*conds)->fixed && (*conds)->fix_fields(thd, conds)) || (*conds)->check_cols(1)) goto err_no_arena; === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.cc 2010-05-31 21:25:54 +0000 @@ -3020,6 +3020,7 @@ table_charset= 0; precomputed_group_by= 0; bit_fields_as_long= 0; + skip_create_table= 0; DBUG_VOID_RETURN; } === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.h 2010-05-31 21:25:54 +0000 @@ -2786,12 +2786,17 @@ that MEMORY tables cannot index BIT columns. */ bool bit_fields_as_long; + /* + Whether to create or postpone actual creation of this temporary table. + TRUE <=> create_tmp_table will create only the TABLE structure. + */ + bool skip_create_table; TMP_TABLE_PARAM() :copy_field(0), group_parts(0), group_length(0), group_null_parts(0), convert_blob_length(0), schema_table(0), precomputed_group_by(0), force_copy_fields(0), - bit_fields_as_long(0) + bit_fields_as_long(0), skip_create_table(0) {} ~TMP_TABLE_PARAM() { === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.cc 2010-05-31 21:25:54 +0000 @@ -1829,6 +1829,52 @@ } +/** + Registers reference on items on which the subqueries depends + + @param last pointer to last st_select_lex struct, before + which all st_select_lex have to be marked as + dependent + @param dependency reference on the item on which all this + subqueries depends + +*/ + +void st_select_lex::register_dependency_item(st_select_lex *last, + Item **dependency) +{ + SELECT_LEX *s= this; + DBUG_ENTER("st_select_lex::register_dependency_item"); + DBUG_ASSERT(this != last); + DBUG_ASSERT(*dependency); + do + { + /* check duplicates */ + List_iterator_fast<Item*> li(s->master_unit()->item->depends_on); + Item **dep; + while ((dep= li++)) + { + if ((*dep)->eq(*dependency, FALSE)) + { + DBUG_PRINT("info", ("dependency %s already present", + ((*dependency)->name ? + (*dependency)->name : + "<no name>"))); + DBUG_VOID_RETURN; + } + } + + s->master_unit()->item->depends_on.push_back(dependency); + DBUG_PRINT("info", ("depends_on: Select: %d added: %s", + s->select_number, + ((*dependency)->name ? + (*dependency)->name : + "<no name>"))); + } while ((s= s->outer_select()) != last && s != 0); + DBUG_VOID_RETURN; +} + + /* st_select_lex_node::mark_as_dependent mark all st_select_lex struct from this to 'last' as dependent @@ -1843,7 +1889,7 @@ bool st_select_lex::mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency) { - + DBUG_ENTER("st_select_lex::mark_as_dependent"); DBUG_ASSERT(this != last); /* @@ -1872,11 +1918,11 @@ Item_subselect *subquery_expr= s->master_unit()->item; if (subquery_expr && subquery_expr->mark_as_dependent(thd, last, dependency)) - return TRUE; + DBUG_RETURN(TRUE); } while ((s= s->outer_select()) != last && s != 0); is_correlated= TRUE; this->master_unit()->item->is_correlated= TRUE; - return FALSE; + DBUG_RETURN(FALSE); } bool st_select_lex_node::set_braces(bool value) { return 1; } === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.h 2010-05-31 21:25:54 +0000 @@ -748,6 +748,7 @@ } bool mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency); + void register_dependency_item(st_select_lex *last, Item **dependency); bool set_braces(bool value); bool inc_in_sum_expr(); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-10 13:46:08 +0000 +++ b/sql/sql_select.cc 2010-05-31 21:25:54 +0000 @@ -151,7 +151,6 @@ static int join_read_system(JOIN_TAB *tab); static int join_read_const(JOIN_TAB *tab); static int join_read_key(JOIN_TAB *tab); -static int join_read_key2(JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref); static void join_read_key_unlock_row(st_join_table *tab); static int join_read_always_key(JOIN_TAB *tab); static int join_read_last_key(JOIN_TAB *tab); @@ -5209,7 +5208,7 @@ 'join->best_positions' contains a complete optimal extension of the current partial QEP. */ - DBUG_EXECUTE("opt", print_plan(join, join->tables, + DBUG_EXECUTE("opt", print_plan(join, n_tables, record_count, read_time, read_time, "optimal");); DBUG_RETURN(FALSE); @@ -7625,6 +7624,40 @@ /** + Creates and fills JOIN_TAB for index look up in temporary table + + @param table The table where to look up + + @return JOIN_TAB object or NULL in case of error +*/ + +JOIN_TAB *create_index_lookup_join_tab(TABLE *table) +{ + JOIN_TAB *tab; + DBUG_ENTER("create_index_lookup_join_tab"); + + if (!((tab= new JOIN_TAB))) + DBUG_RETURN(NULL); + tab->read_record.table= table; + tab->read_record.file=table->file; + /*tab->read_record.unlock_row= rr_unlock_row;*/ + tab->next_select=0; + tab->sorted= 1; + + table->status= STATUS_NO_RECORD; + tab->read_first_record= join_read_key; + /*tab->read_record.unlock_row= join_read_key_unlock_row;*/ + tab->read_record.read_record= join_no_more_records; + if (table->covering_keys.is_set(tab->ref.key) && + !table->no_keyread) + { + table->key_read=1; + table->file->extra(HA_EXTRA_KEYREAD); + } + DBUG_RETURN(tab); +} + +/** Give error if we some tables are done with a full join. This is used by multi_table_update and multi_table_delete when running @@ -10778,6 +10811,7 @@ case Item::REF_ITEM: case Item::NULL_ITEM: case Item::VARBIN_ITEM: + case Item::CACHE_ITEM: if (make_copy_field) { DBUG_ASSERT(((Item_result_field*)item)->result_field); @@ -11552,7 +11586,8 @@ &param->recinfo, select_options)) goto err; } - if (open_tmp_table(table)) + DBUG_PRINT("info", ("skip_create_table: %d", (int)param->skip_create_table)); + if (!param->skip_create_table && open_tmp_table(table)) goto err; thd->mem_root= mem_root_save; @@ -11700,16 +11735,17 @@ bool open_tmp_table(TABLE *table) { int error; + DBUG_ENTER("open_tmp_table"); if ((error= table->file->ha_open(table, table->s->table_name.str, O_RDWR, HA_OPEN_TMP_TABLE | HA_OPEN_INTERNAL_TABLE))) { table->file->print_error(error,MYF(0)); /* purecov: inspected */ table->db_stat=0; - return(1); + DBUG_RETURN(1); } (void) table->file->extra(HA_EXTRA_QUICK); /* Faster */ - return(0); + DBUG_RETURN(0); } @@ -12540,7 +12576,8 @@ else { /* Do index lookup in the materialized table */ - if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + if ((res= join_read_key2(join_tab->join->thd, join_tab, + sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); @@ -13323,61 +13360,62 @@ static int join_read_key(JOIN_TAB *tab) { - return join_read_key2(tab, tab->table, &tab->ref); + return join_read_key2(tab->join->thd, tab, tab->table, &tab->ref); } -/* +/* eq_ref access handler but generalized a bit to support TABLE and TABLE_REF not from the join_tab. See join_read_key for detailed synopsis. */ -static int -join_read_key2(JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref) +int join_read_key2(THD *thd, JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref) { int error; + DBUG_ENTER("join_read_key2"); if (!table->file->inited) { table->file->ha_index_init(table_ref->key, tab->sorted); } /* TODO: Why don't we do "Late NULLs Filtering" here? */ - if (cmp_buffer_with_ref(tab->join->thd, table, table_ref) || + if (cmp_buffer_with_ref(thd, table, table_ref) || (table->status & (STATUS_GARBAGE | STATUS_NO_PARENT | STATUS_NULL_ROW))) { if (table_ref->key_err) { table->status=STATUS_NOT_FOUND; - return -1; + DBUG_RETURN(-1); } /* Moving away from the current record. Unlock the row in the handler if it did not match the partial WHERE. */ - if (tab->ref.has_record && tab->ref.use_count == 0) + if (table_ref->has_record ) + if (table_ref->use_count == 0) { tab->read_record.file->unlock_row(); - tab->ref.has_record= FALSE; + table_ref->has_record= FALSE; } error=table->file->ha_index_read_map(table->record[0], table_ref->key_buff, make_prev_keypart_map(table_ref->key_parts), HA_READ_KEY_EXACT); if (error && error != HA_ERR_KEY_NOT_FOUND && error != HA_ERR_END_OF_FILE) - return report_error(table, error); + DBUG_RETURN(report_error(table, error)); if (! error) { - tab->ref.has_record= TRUE; - tab->ref.use_count= 1; + table_ref->has_record= TRUE; + table_ref->use_count= 1; } } else if (table->status == 0) { - DBUG_ASSERT(tab->ref.has_record); - tab->ref.use_count++; + DBUG_ASSERT(table_ref->has_record); + table_ref->use_count++; } table->null_row=0; - return table->status ? -1 : 0; + DBUG_RETURN(table->status ? -1 : 0); } === added file 'sql/sql_subquery_cache.cc' --- a/sql/sql_subquery_cache.cc 1970-01-01 00:00:00 +0000 +++ b/sql/sql_subquery_cache.cc 2010-05-31 21:25:54 +0000 @@ -0,0 +1,360 @@ + +#include "mysql_priv.h" +#include "sql_select.h" + +ulonglong subquery_cache_miss, subquery_cache_hit; + +/** + Creates structures which we need for index look up + + @retval FALSE OK + @retval TRUE Error +*/ + +static my_bool createtmp_table_search_structures(THD *thd, + TABLE *table, + List_iterator_fast<Item> &li, + TABLE_REF **ref) +{ + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + TABLE_REF *tab_ref; + KEY *tmp_key; /* The only index on the temporary table. */ + Item *item; + uint tmp_key_parts; /* Number of keyparts in tmp_key. */ + uint i; + + DBUG_ENTER("createtmp_table_search_structures"); + + tmp_key= table->key_info; + tmp_key_parts= tmp_key->key_parts; + + if (!(tab_ref= (TABLE_REF*) thd->alloc(sizeof(TABLE_REF)))) + DBUG_RETURN(TRUE); + + tab_ref->key= 0; /* The only temp table index. */ + tab_ref->key_length= tmp_key->key_length; + if (!(tab_ref->key_buff= + (uchar*) thd->calloc(ALIGN_SIZE(tmp_key->key_length) * 2)) || + !(tab_ref->key_copy= + (store_key**) thd->alloc((sizeof(store_key*) * + (tmp_key_parts + 1)))) || + !(tab_ref->items= + (Item**) thd->alloc(sizeof(Item*) * tmp_key_parts))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + + tab_ref->key_buff2=tab_ref->key_buff+ALIGN_SIZE(tmp_key->key_length); + tab_ref->key_err=1; + tab_ref->null_rejecting= 1; + tab_ref->disable_cache= FALSE; + tab_ref->has_record= 0; + + KEY_PART_INFO *cur_key_part= tmp_key->key_part; + store_key **ref_key= tab_ref->key_copy; + uchar *cur_ref_buff= tab_ref->key_buff; + + for (i= 0; i < tmp_key_parts; i++, cur_key_part++, ref_key++) + { + item= li++; + DBUG_ASSERT(item); + tab_ref->items[i]= item; + int null_count= test(cur_key_part->field->real_maybe_null()); + *ref_key= new store_key_item(thd, cur_key_part->field, + /* TODO: + the NULL byte is taken into account in + cur_key_part->store_length, so instead of + cur_ref_buff + test(maybe_null), we could + use that information instead. + */ + cur_ref_buff + null_count, + null_count ? tab_ref->key_buff : 0, + cur_key_part->length, tab_ref->items[i]); + cur_ref_buff+= cur_key_part->store_length; + } + *ref_key= NULL; /* End marker. */ + tab_ref->key_err= 1; + tab_ref->key_parts= tmp_key_parts; + *ref= tab_ref; + + DBUG_RETURN(FALSE); +} + + +Subquery_cache_tmptable::Subquery_cache_tmptable(THD *thd, + List<Item*> &dependance, + Item *value) + :cache_table(NULL), table_thd(thd), list(&dependance), val(value), + equalities(NULL), inited (0) +{ + DBUG_ENTER("Subquery_cache_tmptable::Subquery_cache_tmptable"); + DBUG_VOID_RETURN; +}; + + +/** + Creates equalities expression. + + @note For some type of fields index lookup do not return failure but set + pointer on the next record. To check exact match we use expression like: + field1=value1 and field2=value2 ... + + @retval FALSE OK + @retval TRUE Error +*/ + +bool Subquery_cache_tmptable::make_equalities() +{ + List<Item> args; + List_iterator_fast<Item*> li(*list); + Item **ref; + Name_resolution_context *cn= NULL; + DBUG_ENTER("Subquery_cache_tmptable::make_equalities"); + + for (uint i= 1 /* skip result filed */; (ref= li++); i++) + { + Field *fld= cache_table->field[i]; + /* Only some field types should be checked after lookup */ + if (fld->type() == MYSQL_TYPE_VARCHAR || + fld->type() == MYSQL_TYPE_TINY_BLOB || + fld->type() == MYSQL_TYPE_MEDIUM_BLOB || + fld->type() == MYSQL_TYPE_LONG_BLOB || + fld->type() == MYSQL_TYPE_BLOB || + fld->type() == MYSQL_TYPE_VAR_STRING || + fld->type() == MYSQL_TYPE_STRING || + fld->type() == MYSQL_TYPE_NEWDECIMAL || + fld->type() == MYSQL_TYPE_DECIMAL) + { + if (!cn) + { + // dummy resolution context + cn= new Name_resolution_context(); + cn->init(); + } + args.push_front(new Item_func_eq(new Item_ref(cn, ref, "", "", FALSE), + new Item_field(fld))); + } + } + if (args.elements == 1) + equalities= args.head(); + else + equalities= new Item_cond_and(args); + + DBUG_RETURN(equalities->fix_fields(table_thd, &equalities)); +} + + +/** + Enumerates all fields in field number order. + + @param arg reference on current field number + + @return field number +*/ + +static uint field_enumerator(uchar *arg) +{ + return ((uint*)arg)[0]++; +} + +/** + Initializes temporary table and index for this cache +*/ + +void Subquery_cache_tmptable::init() +{ + List_iterator_fast<Item*> li(*list); + List_iterator_fast<Item> li_items(items); + Item **item; + uint field_counter; + DBUG_ENTER("Subquery_cache_tmptable::init"); + DBUG_ASSERT(!inited); + inited= TRUE; + + if (!(ULONGLONG_MAX >> (list->elements + 1))) + { + DBUG_PRINT("info", ("Too many dependencies")); + DBUG_VOID_RETURN; + } + + cache_table= NULL; + while ((item= li++)) + { + DBUG_ASSERT(item); + DBUG_ASSERT(*item); + DBUG_ASSERT((*item)->fixed); + items.push_back((*item)); + } + + cache_table_param.init(); + /* dependance items and result */ + cache_table_param.field_count= list->elements + 1; + /* postpone table creation to index description */ + cache_table_param.skip_create_table= 1; + + + items.push_front(val); + if (!(cache_table= create_tmp_table(table_thd, &cache_table_param, + items, (ORDER*) NULL, + FALSE, FALSE, + ((table_thd->options | + TMP_TABLE_ALL_COLUMNS) & + ~(OPTION_BIG_TABLES | + TMP_TABLE_FORCE_MYISAM)), + HA_POS_ERROR, + (char *)"subquery-cache-table"))) + { + DBUG_PRINT("error", ("create_tmp_table failed, caching switched off")); + DBUG_VOID_RETURN; + } + + if (cache_table->s->db_type() != heap_hton) + { + DBUG_PRINT("error", ("we need only heap table")); + goto error; + } + + /* first field in the table is result value, so we skip it */ + li_items++; + field_counter=1; + + if (cache_table->alloc_keys(1) || + (cache_table->add_tmp_key(0, items.elements - 1, + &field_enumerator, + (uchar*)&field_counter) < 0) || + createtmp_table_search_structures(table_thd, cache_table, li_items, + &tab_ref) || + !(tab= create_index_lookup_join_tab(cache_table))) + { + DBUG_PRINT("error", ("creating index failed")); + goto error; + } + cache_table->s->keys= 1; + cache_table->s->uniques= 1; + + if (open_tmp_table(cache_table)) + { + DBUG_PRINT("error", ("Opening (creating) temporary table failed")); + goto error; + } + + if (!(chached_result= new Item_field(cache_table->field[0]))) + { + DBUG_PRINT("error", ("Creating Item_field failed")); + goto error; + } + + if (make_equalities()) + { + DBUG_PRINT("error", ("Creating equalities failed")); + goto error; + } + + DBUG_VOID_RETURN; + +error: + /* switch off cache */ + free_tmp_table(table_thd, cache_table); + cache_table= NULL; + DBUG_VOID_RETURN; +} + + +Subquery_cache_tmptable::~Subquery_cache_tmptable() +{ + if (cache_table) + free_tmp_table(table_thd, cache_table); +} + + +/** + Checks if current key present in the cache and returns value if it is true + + @param value assigned Item with value from the cache if key + is found + @return result of the key lookup +*/ + +Subquery_cache::result Subquery_cache_tmptable::check_value(Item **value) +{ + int res; + DBUG_ENTER("Subquery_cache_tmptable::check_value"); + + /* + We delay cache initialization to get item references which should be + used at the moment of query execution. I.e. we store reference on item + reference at the moment of class creation but for table creation and + index supply structures (join_tab) we need real Items which used at the + moment of execution so we can resolve reference only at this point. + */ + if (!inited) + init(); + + if (cache_table) + { + DBUG_PRINT("info", ("status: %u has_record %u", + (uint)cache_table->status, (uint)tab_ref->has_record)); + if ((res= join_read_key2(table_thd, tab, cache_table, tab_ref)) == 1) + DBUG_RETURN(ERROR); + if (res || (equalities && !equalities->val_int())) + { + subquery_cache_miss++; + DBUG_RETURN(MISS); + } + + subquery_cache_hit++; + *value= chached_result; + DBUG_RETURN(Subquery_cache::HIT); + } + DBUG_RETURN(Subquery_cache::MISS); +} + + +/** + Puts given value in the cache + + @param value Value to put in the cache + + @retval FALSE OK + @retval TRUE Error +*/ + +my_bool Subquery_cache_tmptable::put_value(Item *value) +{ + int error; + DBUG_ENTER("Subquery_cache_tmptable::put_value"); + DBUG_ASSERT(inited); + + if (!cache_table) + { + DBUG_PRINT("info", ("No table so behave as we successfully put value")); + DBUG_RETURN(FALSE); + } + + *(items.head_ref())= value; + fill_record(table_thd, cache_table->field, items, 1); + if (table_thd->is_error()) + goto err;; + + if ((error= cache_table->file->ha_write_row(cache_table->record[0]))) + { + /* create_myisam_from_heap will generate error if needed */ + if (cache_table->file->is_fatal_error(error, HA_CHECK_DUP) && + create_internal_tmp_table_from_heap(table_thd, cache_table, + cache_table_param.start_recinfo, + &cache_table_param.recinfo, + error, 1)) + goto err; + } + cache_table->status= 0; /* cache_table->record contains an existed record */ + tab_ref->has_record= TRUE; /* the same as above */ + DBUG_PRINT("info", ("has_record: TRUE status: 0")); + + DBUG_RETURN(FALSE); + +err: + free_tmp_table(table_thd, cache_table); + cache_table= NULL; + DBUG_RETURN(TRUE); +} === added file 'sql/sql_subquery_cache.h' --- a/sql/sql_subquery_cache.h 1970-01-01 00:00:00 +0000 +++ b/sql/sql_subquery_cache.h 2010-05-31 21:25:54 +0000 @@ -0,0 +1,74 @@ +#ifndef _SQL_SUBQUERY_CACHE_H_ +#define _SQL_SUBQUERY_CACHE_H_ + +/** + Interface for subquery cache +*/ + +extern ulonglong subquery_cache_miss, subquery_cache_hit; + +class Subquery_cache :public Sql_alloc +{ +public: + enum result {ERROR, HIT, MISS}; + + Subquery_cache(){}; + virtual ~Subquery_cache() {}; + /** + Checks presence of the key (taken from cache owner) and if found return + it via value parameter + */ + virtual result check_value(Item **value)= 0; + /** + Puts value into this cache (key should be taken from cache owner) + */ + virtual my_bool put_value(Item *value)= 0; +}; + +struct st_table_ref; +struct st_join_table; +//class Item_cache; +class Item_field; + +/** + Implementation of subquery cache over temporary table +*/ + +class Subquery_cache_tmptable :public Subquery_cache +{ +public: + Subquery_cache_tmptable(THD *thd, List<Item*> &dependance, Item *value); + virtual ~Subquery_cache_tmptable(); + virtual result check_value(Item **value); + virtual my_bool put_value(Item *value); + +private: + void init(); + bool make_equalities(); + + /* tmp table parameters */ + TMP_TABLE_PARAM cache_table_param; + /* temporary table to store this cache */ + TABLE *cache_table; + /* Thread handler for the temporary table */ + THD *table_thd; + /* tab_ref for index search */ + struct st_table_ref *tab_ref; + /* cache of subquery value to avoid evaluating it twice */ + //Item_cache *value_cache; + /* JOIN_TAB for index lookup */ + st_join_table *tab; + /* Chached result */ + Item_field *chached_result; + /* List of references to items */ + List<Item*> *list; + /* List of items */ + List<Item> items; + /* Value Item example */ + Item *val; + /* Expression to check after index lookup */ + Item *equalities; + /* set if structures are inited */ + bool inited; +}; +#endif === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-20 12:01:47 +0000 +++ b/sql/table.cc 2010-05-31 21:25:54 +0000 @@ -20,6 +20,7 @@ #include "sql_trigger.h" #include <m_ctype.h> #include "my_md5.h" +#include "my_bit.h" /* INFORMATION_SCHEMA name */ LEX_STRING INFORMATION_SCHEMA_NAME= {C_STRING_WITH_LEN("information_schema")}; @@ -5096,6 +5097,115 @@ file->column_bitmaps_signal(); } + +/** + @brief + Allocate space for keys + + @param key_count number of keys to allocate. + + @details + Allocate space enough to fit 'key_count' keys for this table. + + @return FALSE space was successfully allocated. + @return TRUE an error occur. +*/ + +bool TABLE::alloc_keys(uint key_count) +{ + DBUG_ASSERT(!s->keys); + key_info= s->key_info= (KEY*) alloc_root(&mem_root, sizeof(KEY)*key_count); + max_keys= key_count; + return !(key_info); +} + + +/** + @brief Adds one key to a temporary table. + + @param key key number. + @param key_parts number of fields in the key + @param next_field_no function which returns field numbers which + should be included in the key + @param arg above function argement + + @return <0 an error occur. + @return >=0 number of newly added key. +*/ + +bool TABLE::add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg) +{ + DBUG_ASSERT(key < max_keys); + + char buf[NAME_CHAR_LEN]; + KEY* keyinfo; + Field **reg_field; + uint i; + bool key_start= TRUE; + KEY_PART_INFO* key_part_info= + (KEY_PART_INFO*) alloc_root(&mem_root, sizeof(KEY_PART_INFO)*key_parts); + if (!key_part_info) + return TRUE; + keyinfo= key_info + key; + keyinfo->key_part= key_part_info; + keyinfo->usable_key_parts= keyinfo->key_parts = key_parts; + keyinfo->key_length=0; + keyinfo->algorithm= HA_KEY_ALG_UNDEF; + keyinfo->flags= HA_GENERATED_KEY; + sprintf(buf, "key%i", key); + if (!(keyinfo->name= strdup_root(&mem_root, buf))) + return TRUE; + keyinfo->rec_per_key= (ulong*) alloc_root(&mem_root, + sizeof(ulong)*key_parts); + if (!keyinfo->rec_per_key) + return TRUE; + bzero(keyinfo->rec_per_key, sizeof(ulong)*key_parts); + for (i= 0; i < key_parts; i++) + { + reg_field= field + next_field_no(arg); + if (key_start) + (*reg_field)->key_start.set_bit(key); + key_start= FALSE; + (*reg_field)->part_of_key.set_bit(key); + (*reg_field)->flags|= PART_KEY_FLAG; + key_part_info->null_bit= (*reg_field)->null_bit; + key_part_info->null_offset= (uint) ((*reg_field)->null_ptr - + (uchar*) record[0]); + key_part_info->field= *reg_field; + key_part_info->offset= (*reg_field)->offset(record[0]); + key_part_info->length= (uint16) (*reg_field)->pack_length(); + keyinfo->key_length+= key_part_info->length; + /* TODO: + The below method of computing the key format length of the + key part is a copy/paste from opt_range.cc, and table.cc. + This should be factored out, e.g. as a method of Field. + In addition it is not clear if any of the Field::*_length + methods is supposed to compute the same length. If so, it + might be reused. + */ + key_part_info->store_length= key_part_info->length; + + if ((*reg_field)->real_maybe_null()) + key_part_info->store_length+= HA_KEY_NULL_LENGTH; + if ((*reg_field)->type() == MYSQL_TYPE_BLOB || + (*reg_field)->real_type() == MYSQL_TYPE_VARCHAR) + key_part_info->store_length+= HA_KEY_BLOB_LENGTH; + + key_part_info->type= (uint8) (*reg_field)->key_type(); + key_part_info->key_type = + ((ha_base_keytype) key_part_info->type == HA_KEYTYPE_TEXT || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT1 || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT2) ? + 0 : FIELDFLAG_BINARY; + key_part_info++; + } + set_if_bigger(s->max_key_length, keyinfo->key_length); + s->keys++; + return FALSE; +} + + /** @brief Check if this is part of a MERGE table with attached children. === modified file 'sql/table.h' --- a/sql/table.h 2010-03-20 12:01:47 +0000 +++ b/sql/table.h 2010-05-31 21:25:54 +0000 @@ -781,6 +781,7 @@ uint temp_pool_slot; /* Used by intern temp tables */ uint status; /* What's in record[0] */ uint db_stat; /* mode of file as in handler.h */ + uint max_keys; /* Size of allocated key_info array. */ /* number of select if it is derived table */ uint derived_select_number; int current_lock; /* Type of lock on table */ @@ -913,6 +914,9 @@ */ inline bool needs_reopen_or_name_lock() { return s->version != refresh_version; } + bool alloc_keys(uint key_count); + bool add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg); bool is_children_attached(void); }; === modified file 'storage/maria/ha_maria.cc' --- a/storage/maria/ha_maria.cc 2010-03-20 12:01:47 +0000 +++ b/storage/maria/ha_maria.cc 2010-05-31 21:25:54 +0000 @@ -995,6 +995,8 @@ { MARIA_HA *tmp= file; file= 0; + if (!tmp) + return 0; return maria_close(tmp); }

1 0

[Maria-developers] Rev 2789: Subquery cache for pre-review (MWL#66) in file:///home/bell/maria/bzr/work-maria-5.3-scache2/
by sanja＠askmonty.org 31 May '10

31 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache2/ ------------------------------------------------------------ revno: 2789 revision-id: sanja(a)askmonty.org-20100531212240-qwphnvofu9f0l06l parent: sergii(a)pisem.net-20100510134608-oyi2vznyghgcrt0x committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache2 timestamp: Tue 2010-06-01 00:22:40 +0300 message: Subquery cache for pre-review (MWL#66) === modified file 'libmysqld/Makefile.am' --- a/libmysqld/Makefile.am 2010-03-20 12:01:47 +0000 +++ b/libmysqld/Makefile.am 2010-05-31 21:22:40 +0000 @@ -80,7 +80,8 @@ sql_tablespace.cc \ rpl_injector.cc my_user.c partition_info.cc \ sql_servers.cc event_parse_data.cc opt_table_elimination.cc \ - multi_range_read.cc opt_index_cond_pushdown.cc + multi_range_read.cc opt_index_cond_pushdown.cc \ + sql_subquery_cache.cc libmysqld_int_a_SOURCES= $(libmysqld_sources) nodist_libmysqld_int_a_SOURCES= $(libmysqlsources) $(sqlsources) === modified file 'mysql-test/r/index_merge_myisam.result' --- a/mysql-test/r/index_merge_myisam.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/index_merge_myisam.result 2010-05-31 21:22:40 +0000 @@ -1419,19 +1419,19 @@ # select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='index_merge=off,index_merge_union=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='index_merge_union=on'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,index_merge_sort_union=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=off,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=off,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=4; ERROR 42000: Variable 'optimizer_switch' can't be set to the value of '4' set optimizer_switch=NULL; @@ -1458,21 +1458,21 @@ set optimizer_switch='index_merge=off,index_merge_union=off,default'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; select @@global.optimizer_switch; @@global.optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set @@global.optimizer_switch=default; select @@global.optimizer_switch; @@global.optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on # # Check index_merge's @@optimizer_switch flags # select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (a int, b int, c int, filler char(100), @@ -1582,5 +1582,5 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on drop table t0, t1; === modified file 'mysql-test/r/myisam_mrr.result' --- a/mysql-test/r/myisam_mrr.result 2010-03-11 21:43:31 +0000 +++ b/mysql-test/r/myisam_mrr.result 2010-05-31 21:22:40 +0000 @@ -394,7 +394,7 @@ # - engine_condition_pushdown does not affect ICP select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (a int, b int, key(a)); === modified file 'mysql-test/r/subselect3.result' --- a/mysql-test/r/subselect3.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3.result 2010-05-31 21:22:40 +0000 @@ -105,6 +105,7 @@ Handler_read_rnd_next 5 delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; oref a Z @@ -123,6 +124,7 @@ select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; Z No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1. +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; create table t1 (a int, b int, primary key (a)); insert into t1 values (1,1), (3,1),(100,1); === modified file 'mysql-test/r/subselect3_jcl6.result' --- a/mysql-test/r/subselect3_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3_jcl6.result 2010-05-31 21:22:40 +0000 @@ -109,6 +109,7 @@ Handler_read_rnd_next 5 delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; oref a Z @@ -127,6 +128,7 @@ select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; Z No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1. +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; create table t1 (a int, b int, primary key (a)); insert into t1 values (1,1), (3,1),(100,1); === modified file 'mysql-test/r/subselect_no_mat.result' --- a/mysql-test/r/subselect_no_mat.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_mat.result 2010-05-31 21:22:40 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='materialization=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_no_opts.result' --- a/mysql-test/r/subselect_no_opts.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_opts.result 2010-05-31 21:22:40 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='materialization=off,semijoin=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_no_semijoin.result' --- a/mysql-test/r/subselect_no_semijoin.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_semijoin.result 2010-05-31 21:22:40 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='semijoin=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_sj.result' --- a/mysql-test/r/subselect_sj.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect_sj.result 2010-05-31 21:22:40 +0000 @@ -202,39 +202,39 @@ select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; drop table t0, t1, t2; drop table t10, t11, t12; === modified file 'mysql-test/r/subselect_sj_jcl6.result' --- a/mysql-test/r/subselect_sj_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect_sj_jcl6.result 2010-05-31 21:22:40 +0000 @@ -206,39 +206,39 @@ select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; drop table t0, t1, t2; drop table t10, t11, t12; === modified file 'mysql-test/t/subselect3.test' --- a/mysql-test/t/subselect3.test 2010-03-20 12:01:47 +0000 +++ b/mysql-test/t/subselect3.test 2010-05-31 21:22:40 +0000 @@ -98,10 +98,12 @@ delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; show status like '%Handler_read%'; select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; === modified file 'sql/CMakeLists.txt' --- a/sql/CMakeLists.txt 2010-03-20 12:01:47 +0000 +++ b/sql/CMakeLists.txt 2010-05-31 21:22:40 +0000 @@ -78,7 +78,7 @@ rpl_rli.cc rpl_mi.cc sql_servers.cc sql_connect.cc scheduler.cc sql_profile.cc event_parse_data.cc opt_table_elimination.cc - ds_mrr.cc + ds_mrr.cc sql_subquery_cache.cc ${PROJECT_SOURCE_DIR}/sql/sql_yacc.cc ${PROJECT_SOURCE_DIR}/sql/sql_yacc.h ${PROJECT_SOURCE_DIR}/include/mysqld_error.h === modified file 'sql/Makefile.am' --- a/sql/Makefile.am 2010-03-20 12:01:47 +0000 +++ b/sql/Makefile.am 2010-05-31 21:22:40 +0000 @@ -80,7 +80,7 @@ event_data_objects.h event_scheduler.h \ sql_partition.h partition_info.h partition_element.h \ contributors.h sql_servers.h \ - multi_range_read.h + multi_range_read.h sql_subquery_cache.h mysqld_SOURCES = sql_lex.cc sql_handler.cc sql_partition.cc \ item.cc item_sum.cc item_buff.cc item_func.cc \ @@ -130,7 +130,7 @@ sql_servers.cc event_parse_data.cc \ opt_table_elimination.cc \ multi_range_read.cc \ - opt_index_cond_pushdown.cc + opt_index_cond_pushdown.cc sql_subquery_cache.cc nodist_mysqld_SOURCES = mini_client_errors.c pack.c client.c my_time.c my_user.c === modified file 'sql/item.cc' --- a/sql/item.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item.cc 2010-05-31 21:22:40 +0000 @@ -28,6 +28,9 @@ const String my_null_string("NULL", 4, default_charset_info); +static int save_field_in_field(Field *from,my_bool * null_value, + Field *to, bool no_conversions); + /****************************************************************************/ /* Hybrid_type_traits {_real} */ @@ -2273,6 +2276,13 @@ str->append(str_value); } +void Item_bool_cache::print(String *str, enum_query_type query_type) +{ + if (null_value) + str->append("NULL", 4); + else + Item_int::print(str, query_type); +} Item_uint::Item_uint(const char *str_arg, uint length): Item_int(str_arg, length) @@ -3646,12 +3656,17 @@ resolved_item->db_name : ""); const char *table_name= (resolved_item->table_name ? resolved_item->table_name : ""); + DBUG_ENTER("mark_as_dependent"); + DBUG_PRINT("enter", ("Field '%s.%s.%s in select %d resolved in %d", + db_name, table_name, + resolved_item->field_name, current->select_number, + last->select_number)); /* store pointer on SELECT_LEX from which item is dependent */ if (mark_item) mark_item->depended_from= last; if (current->mark_as_dependent(thd, last, /** resolved_item psergey-thu **/mark_item)) - return TRUE; + DBUG_RETURN(TRUE); if (thd->lex->describe & DESCRIBE_EXTENDED) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, @@ -3661,7 +3676,7 @@ resolved_item->field_name, current->select_number, last->select_number); } - return FALSE; + DBUG_RETURN(FALSE); } @@ -3698,6 +3713,7 @@ resolving) */ SELECT_LEX *previous_select= current_sel; + for (; previous_select->outer_select() != last_select; previous_select= previous_select->outer_select()) { @@ -3726,6 +3742,7 @@ mark_as_dependent(thd, last_select, current_sel, resolved_item, dependent); } + return; } @@ -4098,6 +4115,9 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); return 0; } } @@ -4113,7 +4133,9 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); - + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* A reference to a view field had been found and we substituted it instead of this Item (find_field_in_tables @@ -4215,6 +4237,10 @@ mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, rf, rf); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); + return 0; } else @@ -4222,6 +4248,9 @@ mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, this, (Item_ident*)*reference); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); if (last_checked_context->select_lex->having_fix_field) { Item_ref *rf; @@ -5082,39 +5111,48 @@ /** + Saves one Fields of an Item of in other Field + + @param from Field to copy value from + @param null_value reference on item null_value to set it if it is needed + @param to Field to cope value to + @param no_conversions how to deal with NULL value (see + set_field_to_null_with_conversions()) + + @retval FALSE OK + @retval TRUE Error +*/ + +static int save_field_in_field(Field *from, my_bool *null_value, + Field *to, bool no_conversions) +{ + int res; + if (from->is_null()) + { + (*null_value)= 1; + res= set_field_to_null_with_conversions(to, no_conversions); + } + else + { + to->set_notnull(); + res= field_conv(to, from); + (*null_value)= 0; + } + return res; +} + +/** Set a field's value from a item. */ void Item_field::save_org_in_field(Field *to) { - if (field->is_null()) - { - null_value=1; - set_field_to_null_with_conversions(to, 1); - } - else - { - to->set_notnull(); - field_conv(to,field); - null_value=0; - } + save_field_in_field(field, &null_value, to, TRUE); } int Item_field::save_in_field(Field *to, bool no_conversions) { - int res; - if (result_field->is_null()) - { - null_value=1; - res= set_field_to_null_with_conversions(to, no_conversions); - } - else - { - to->set_notnull(); - res= field_conv(to,result_field); - null_value=0; - } - return res; + return save_field_in_field(result_field, &null_value, to, no_conversions); } @@ -5973,6 +6011,9 @@ refer_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* view reference found, we substituted it instead of this Item, so can quit @@ -6023,6 +6064,9 @@ thd->change_item_tree(reference, fld); mark_as_dependent(thd, last_checked_context->select_lex, thd->lex->current_select, fld, fld); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of @@ -6046,6 +6090,9 @@ DBUG_ASSERT(*ref && (*ref)->fixed); mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, this, this); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + ref); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of @@ -6312,7 +6359,8 @@ int Item_ref::save_in_field(Field *to, bool no_conversions) { int res; - DBUG_ASSERT(!result_field); + if (result_field) + return save_field_in_field(result_field, &null_value, to, no_conversions); res= (*ref)->save_in_field(to, no_conversions); null_value= (*ref)->null_value; return res; === modified file 'sql/item.h' --- a/sql/item.h 2010-03-20 12:01:47 +0000 +++ b/sql/item.h 2010-05-31 21:22:40 +0000 @@ -1922,8 +1922,31 @@ virtual void print(String *str, enum_query_type query_type); Item_num *neg (); uint decimal_precision() const { return max_length; } - bool check_partition_func_processor(uchar *bool_arg) { return FALSE;} - bool check_vcol_func_processor(uchar *arg) { return FALSE;} +}; + + +/** + Item represent TRUE/FALSE/NULL for subquery values +*/ + +class Item_bool_cache: public Item_int +{ +public: + Item_bool_cache(): Item_int(0, 1) + { + unsigned_flag= maybe_null= null_value= TRUE; + name= (char *)"bool chache"; + } + Item_bool_cache(my_bool val, my_bool null): Item_int(val, 1) + { + unsigned_flag= maybe_null= TRUE; + null_value= null; + name= (char *)"bool chache"; + } + Item *clone_item() { return new Item_bool_cache(value, null_value); } + uint decimal_precision() const { return 1; } + virtual void print(String *str, enum_query_type query_type); + void set(my_bool val, my_bool null) {value= test(val); null_value= null;} }; @@ -3146,7 +3169,8 @@ example(0), used_table_map(0), cached_field(0), cached_field_type(MYSQL_TYPE_STRING), value_cached(0) { - fixed= 1; + fixed= 1; + maybe_null= 1; null_value= 1; } Item_cache(enum_field_types field_type_arg): @@ -3154,6 +3178,7 @@ value_cached(0) { fixed= 1; + maybe_null= 1; null_value= 1; } === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.cc 2010-05-31 21:22:40 +0000 @@ -1736,6 +1736,15 @@ used_tables_cache|= args[1]->used_tables(); not_null_tables_cache|= args[1]->not_null_tables(); const_item_cache&= args[1]->const_item(); + DBUG_ASSERT(scache == NULL); + if (args[0]->cols() ==1 && + thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE && + !(sub->engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) + { + sub->depends_on.push_front((Item**)&cache); + scache= new Subquery_cache_tmptable(thd, sub->depends_on, &result); + } fixed= 1; return FALSE; } @@ -1744,10 +1753,26 @@ longlong Item_in_optimizer::val_int() { bool tmp; + DBUG_ENTER("Item_in_optimizer::val_int"); + DBUG_ASSERT(fixed == 1); cache->store(args[0]); cache->cache_value(); - + + /* check if result is in the cache */ + if (scache) + { + Subquery_cache_tmptable::result res; + Item *cached_value; + res= scache->check_value(&cached_value); + if (res == Subquery_cache_tmptable::HIT) + { + tmp= cached_value->val_int(); + null_value= cached_value->null_value; + DBUG_RETURN(tmp); + } + } + if (cache->null_value) { /* @@ -1818,11 +1843,18 @@ for (uint i= 0; i < ncols; i++) item_subs->set_cond_guard_var(i, TRUE); } - return 0; + DBUG_RETURN(0); } tmp= args[1]->val_bool_result(); null_value= args[1]->null_value; - return tmp; + + /* put result in the cache */ + if (scache) + { + result.set(tmp, null_value); + scache->put_value(&result); + } + DBUG_RETURN(tmp); } @@ -1839,6 +1871,11 @@ Item_bool_func::cleanup(); if (!save_cache) cache= 0; + if (scache) + { + delete scache; + scache= 0; + } DBUG_VOID_RETURN; } === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.h 2010-05-31 21:22:40 +0000 @@ -215,6 +215,7 @@ class Item_cache; +class Subquery_cache; #define UNKNOWN ((my_bool)-1) @@ -237,6 +238,10 @@ { protected: Item_cache *cache; + /* Subquery cache */ + Subquery_cache *scache; + /* result representation for the subquery cache */ + Item_bool_cache result; bool save_cache; /* Stores the value of "NULL IN (SELECT ...)" for uncorrelated subqueries: @@ -247,7 +252,7 @@ my_bool result_for_null_param; public: Item_in_optimizer(Item *a, Item_in_subselect *b): - Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), + Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), scache(NULL), save_cache(0), result_for_null_param(UNKNOWN) {} bool fix_fields(THD *, Item **); === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.cc 2010-05-31 21:22:40 +0000 @@ -34,11 +34,10 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), - engine(0), old_engine(0), used_tables_cache(0), have_to_be_excluded(0), - const_item_cache(1), - inside_first_fix_fields(0), done_first_fix_fields(FALSE), - eliminated(FALSE), - engine_changed(0), changed(0), is_correlated(FALSE) + engine(0), old_engine(0), scache(0), used_tables_cache(0), + have_to_be_excluded(0), const_item_cache(1), inside_first_fix_fields(0), + done_first_fix_fields(FALSE), eliminated(FALSE), engine_changed(0), + changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -116,6 +115,12 @@ } if (engine) engine->cleanup(); + depends_on.empty(); + if (scache) + { + delete scache; + scache= 0; + } reset(); value_assigned= 0; DBUG_VOID_RETURN; @@ -148,6 +153,8 @@ Item_subselect::~Item_subselect() { delete engine; + if (scache) + delete scache; } Item_subselect::trans_res @@ -746,9 +753,22 @@ void Item_singlerow_subselect::fix_length_and_dec() { + DBUG_ENTER("Item_singlerow_subselect::fix_length_and_dec"); if ((max_columns= engine->cols()) == 1) { + DBUG_PRINT("info", ("one, elements: %u flag %u", + (uint)depends_on.elements, + (uint)test(thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE))); engine->fix_length_and_dec(row= &value); + if (depends_on.elements && + optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && + !(engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) + { + DBUG_ASSERT(scache == NULL); + scache= new Subquery_cache_tmptable(thd, depends_on, value); + DBUG_PRINT("info", ("cache: 0x%lx", (ulong) scache)); + } } else { @@ -765,6 +785,7 @@ */ if (engine->no_tables()) maybe_null= engine->may_be_null(); + DBUG_VOID_RETURN; } uint Item_singlerow_subselect::cols() @@ -797,77 +818,206 @@ exec(); } +/** + Checks subquery cache for value + + @retval NULL nothing found + @retval reference on item representing value found in the cache +*/ + +Item *Item_subselect::check_cache() +{ + DBUG_ENTER("Item_subselect::check_cache"); + if (scache) + { + Subquery_cache_tmptable::result res; + Item *cached_value; + res= scache->check_value(&cached_value); + if (res == Subquery_cache_tmptable::HIT) + DBUG_RETURN(cached_value); + } + DBUG_RETURN(NULL); +} + double Item_singlerow_subselect::val_real() { + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_real"); DBUG_ASSERT(fixed == 1); - if (!exec() && !value->null_value) + + if ((cached_value = check_cache())) + { + double res= cached_value->val_real(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_real(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_real()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } longlong Item_singlerow_subselect::val_int() { + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_int"); DBUG_ASSERT(fixed == 1); - if (!exec() && !value->null_value) + + if ((cached_value = check_cache())) + { + longlong res= cached_value->val_int(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_int(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_int()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } String *Item_singlerow_subselect::val_str(String *str) { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_str"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + String *res= cached_value->val_str(str); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_str(str); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_str(str)); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } my_decimal *Item_singlerow_subselect::val_decimal(my_decimal *decimal_value) { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_decimal"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_decimal *res= cached_value->val_decimal(decimal_value); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_decimal(decimal_value); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_decimal(decimal_value)); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } bool Item_singlerow_subselect::val_bool() { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_bool"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + bool res= cached_value->val_bool(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_bool(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_bool()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } @@ -952,33 +1102,79 @@ void Item_exists_subselect::fix_length_and_dec() { + DBUG_ENTER("Item_exists_subselect::fix_length_and_dec"); decimals= 0; max_length= 1; max_columns= engine->cols(); /* We need only 1 row to determine existence */ unit->global_parameters->select_limit= new Item_int((int32) 1); + if (substype() == EXISTS_SUBS && depends_on.elements && + optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && + !(engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) + { + DBUG_ASSERT(scache == NULL); + scache= new Subquery_cache_tmptable(thd, depends_on, &result); + DBUG_PRINT("info", ("cache: 0x%lx", (ulong) scache)); + } + DBUG_VOID_RETURN; } double Item_exists_subselect::val_real() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_int"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + double res= cached_value->val_real(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) { reset(); - return 0; - } - return (double) value; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN((double) value); } longlong Item_exists_subselect::val_int() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_real"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + longlong res= cached_value->val_int(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + DBUG_ASSERT(fixed == 1); if (exec()) { reset(); - return 0; - } - return value; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN(value); } @@ -997,11 +1193,32 @@ String *Item_exists_subselect::val_str(String *str) { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_str"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + String *res= cached_value->val_str(str); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) + { reset(); + str->set((ulonglong)0,&my_charset_bin); + DBUG_RETURN(str); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + str->set((ulonglong)value,&my_charset_bin); - return str; + DBUG_RETURN(str); } @@ -1020,23 +1237,61 @@ my_decimal *Item_exists_subselect::val_decimal(my_decimal *decimal_value) { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_decvimal"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_decimal *res= cached_value->val_decimal(decimal_value); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) + { reset(); + int2my_decimal(E_DEC_FATAL_ERROR, 0, 0, decimal_value); + DBUG_RETURN(decimal_value); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + int2my_decimal(E_DEC_FATAL_ERROR, value, 0, decimal_value); - return decimal_value; + DBUG_RETURN(decimal_value); } bool Item_exists_subselect::val_bool() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_real"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_bool res= cached_value->val_bool(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) { reset(); - return 0; - } - return value != 0; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN(value != 0); } === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.h 2010-05-31 21:22:40 +0000 @@ -27,6 +27,7 @@ class subselect_hash_sj_engine; class Item_bool_func2; class Cached_item; +class Subquery_cache; /* base class for subselects */ @@ -57,6 +58,10 @@ subselect_engine *engine; /* old engine if engine was changed */ subselect_engine *old_engine; + /* subquery cache */ + Subquery_cache *scache; + /* null consrtant for caching */ + Item_null const_null_value; /* cache of used external tables */ table_map used_tables_cache; /* allowed number of columns (1 for single value subqueries) */ @@ -67,7 +72,7 @@ bool have_to_be_excluded; /* cache of constant state */ bool const_item_cache; - + bool inside_first_fix_fields; bool done_first_fix_fields; public: @@ -88,13 +93,21 @@ */ List<Ref_to_outside> upper_refs; st_select_lex *parent_select; - - /* + + /** + List of references on items subquery depends on (externally resolved); + + @note We can't store direct links on Items because it could be + substituted with other item (for example for grouping). + */ + List<Item*> depends_on; + + /* TRUE<=>Table Elimination has made it redundant to evaluate this select (and so it is not part of QEP, etc) - */ + */ bool eliminated; - + /* changed engine indicator */ bool engine_changed; /* subquery is transformed */ @@ -178,6 +191,8 @@ return trace_unsupported_by_check_vcol_func_processor("subselect"); } + Item *check_cache(); + /** Get the SELECT_LEX structure associated with this Item. @return the SELECT_LEX structure associated with this Item @@ -202,6 +217,7 @@ { protected: Item_cache *value, **row; + public: Item_singlerow_subselect(st_select_lex *select_lex); Item_singlerow_subselect() :Item_subselect(), value(0), row (0) {} @@ -268,6 +284,8 @@ { protected: bool value; /* value of this item (boolean: exists/not-exists) */ + /* result representation for the subquery cache */ + Item_bool_cache result; public: Item_exists_subselect(st_select_lex *select_lex); === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_sum.cc 2010-05-31 21:22:40 +0000 @@ -319,6 +319,7 @@ if (aggr_level >= 0) { ref_by= ref; + thd->lex->current_select->register_dependency_item(aggr_sel, ref); /* Add the object to the list of registered objects assigned to aggr_sel */ if (!aggr_sel->inner_sum_func_list) next= this; === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-03-20 12:01:47 +0000 +++ b/sql/mysql_priv.h 2010-05-31 21:22:40 +0000 @@ -568,12 +568,13 @@ #define OPTIMIZER_SWITCH_SEMIJOIN 256 #define OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE 512 #define OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN 1024 +#define OPTIMIZER_SWITCH_SUBQUERY_CACHE (1<<11) #ifdef DBUG_OFF -# define OPTIMIZER_SWITCH_LAST 2048 +# define OPTIMIZER_SWITCH_LAST (1<<12) #else -# define OPTIMIZER_SWITCH_TABLE_ELIMINATION 2048 -# define OPTIMIZER_SWITCH_LAST 4096 +# define OPTIMIZER_SWITCH_TABLE_ELIMINATION (1<<12) +# define OPTIMIZER_SWITCH_LAST (1<<13) #endif #ifdef DBUG_OFF @@ -588,7 +589,8 @@ OPTIMIZER_SWITCH_MATERIALIZATION | \ OPTIMIZER_SWITCH_SEMIJOIN | \ OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE|\ - OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN) + OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN|\ + OPTIMIZER_SWITCH_SUBQUERY_CACHE) #else # define OPTIMIZER_SWITCH_DEFAULT (OPTIMIZER_SWITCH_INDEX_MERGE | \ OPTIMIZER_SWITCH_INDEX_MERGE_UNION | \ @@ -601,7 +603,8 @@ OPTIMIZER_SWITCH_MATERIALIZATION | \ OPTIMIZER_SWITCH_SEMIJOIN | \ OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE|\ - OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN) + OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN|\ + OPTIMIZER_SWITCH_SUBQUERY_CACHE) #endif /* @@ -936,6 +939,7 @@ #ifdef MYSQL_SERVER #include "sql_servers.h" #include "opt_range.h" +#include "sql_subquery_cache.h" #ifdef HAVE_QUERY_CACHE struct Query_cache_query_flags @@ -1269,6 +1273,10 @@ Item *having, ORDER *proc_param, ulonglong select_type, select_result *result, SELECT_LEX_UNIT *unit, SELECT_LEX *select_lex); + +struct st_join_table *create_index_lookup_join_tab(TABLE *table); +int join_read_key2(THD *thd, struct st_join_table *tab, TABLE *table, + struct st_table_ref *table_ref); void free_underlaid_joins(THD *thd, SELECT_LEX *select); bool mysql_explain_union(THD *thd, SELECT_LEX_UNIT *unit, select_result *result); @@ -1288,6 +1296,7 @@ bool table_cant_handle_bit_fields, bool make_copy_field, uint convert_blob_length); +bool open_tmp_table(TABLE *table); void sp_prepare_create_field(THD *thd, Create_field *sql_field); int prepare_create_field(Create_field *sql_field, uint *blob_columns, === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2010-03-20 12:01:47 +0000 +++ b/sql/mysqld.cc 2010-05-31 21:22:40 +0000 @@ -305,6 +305,7 @@ "firstmatch","loosescan","materialization", "semijoin", "partial_match_rowid_merge", "partial_match_table_scan", + "subquery_cache", #ifndef DBUG_OFF "table_elimination", #endif @@ -325,6 +326,7 @@ sizeof("semijoin") - 1, sizeof("partial_match_rowid_merge") - 1, sizeof("partial_match_table_scan") - 1, + sizeof("subquery_cache") - 1, #ifndef DBUG_OFF sizeof("table_elimination") - 1, #endif @@ -404,8 +406,9 @@ static const char *optimizer_switch_str="index_merge=on,index_merge_union=on," "index_merge_sort_union=on," "index_merge_intersection=on," - "index_condition_pushdown=on" -#ifndef DBUG_OFF + "index_condition_pushdown=on," + "subquery_cache=on" +#ifndef DBUG_OFF ",table_elimination=on"; #else ; @@ -5872,7 +5875,9 @@ OPT_RECORD_RND_BUFFER, OPT_DIV_PRECINCREMENT, OPT_RELAY_LOG_SPACE_LIMIT, OPT_RELAY_LOG_PURGE, OPT_SLAVE_NET_TIMEOUT, OPT_SLAVE_COMPRESSED_PROTOCOL, OPT_SLOW_LAUNCH_TIME, - OPT_SLAVE_TRANS_RETRIES, OPT_READONLY, OPT_ROWID_MERGE_BUFF_SIZE, + OPT_SLAVE_TRANS_RETRIES, + OPT_SUBQUERY_CACHE, + OPT_READONLY, OPT_ROWID_MERGE_BUFF_SIZE, OPT_DEBUGGING, OPT_DEBUG_FLUSH, OPT_SORT_BUFFER, OPT_TABLE_OPEN_CACHE, OPT_TABLE_DEF_CACHE, OPT_THREAD_CONCURRENCY, OPT_THREAD_CACHE_SIZE, @@ -7164,7 +7169,7 @@ {"optimizer_switch", OPT_OPTIMIZER_SWITCH, "optimizer_switch=option=val[,option=val...], where option={index_merge, " "index_merge_union, index_merge_sort_union, index_merge_intersection, " - "index_condition_pushdown" + "index_condition_pushdown, subquery_cache" #ifndef DBUG_OFF ", table_elimination" #endif @@ -7868,6 +7873,8 @@ {"Ssl_version", (char*) &show_ssl_get_version, SHOW_FUNC}, #endif /* HAVE_OPENSSL */ {"Syncs", (char*) &my_sync_count, SHOW_LONG_NOFLUSH}, + {"Subquery_cache_hit", (char*) &subquery_cache_hit, SHOW_LONG}, + {"Subquery_cache_miss", (char*) &subquery_cache_miss, SHOW_LONG}, {"Table_locks_immediate", (char*) &locks_immediate, SHOW_LONG}, {"Table_locks_waited", (char*) &locks_waited, SHOW_LONG}, #ifdef HAVE_MMAP @@ -8006,6 +8013,7 @@ abort_loop= select_thread_in_use= signal_thread_in_use= 0; ready_to_exit= shutdown_in_progress= grant_option= 0; aborted_threads= aborted_connects= 0; + subquery_cache_miss= subquery_cache_hit= 0; delayed_insert_threads= delayed_insert_writes= delayed_rows_in_use= 0; delayed_insert_errors= thread_created= 0; specialflag= 0; === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_base.cc 2010-05-31 21:22:40 +0000 @@ -8062,6 +8062,10 @@ if (*conds) { thd->where="where clause"; + DBUG_EXECUTE("where", + print_where(*conds, + "WHERE in setup_conds", + QT_ORDINARY);); if ((!(*conds)->fixed && (*conds)->fix_fields(thd, conds)) || (*conds)->check_cols(1)) goto err_no_arena; === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.cc 2010-05-31 21:22:40 +0000 @@ -3020,6 +3020,7 @@ table_charset= 0; precomputed_group_by= 0; bit_fields_as_long= 0; + skip_create_table= 0; DBUG_VOID_RETURN; } === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.h 2010-05-31 21:22:40 +0000 @@ -2786,12 +2786,17 @@ that MEMORY tables cannot index BIT columns. */ bool bit_fields_as_long; + /* + Whether to create or postpone actual creation of this temporary table. + TRUE <=> create_tmp_table will create only the TABLE structure. + */ + bool skip_create_table; TMP_TABLE_PARAM() :copy_field(0), group_parts(0), group_length(0), group_null_parts(0), convert_blob_length(0), schema_table(0), precomputed_group_by(0), force_copy_fields(0), - bit_fields_as_long(0) + bit_fields_as_long(0), skip_create_table(0) {} ~TMP_TABLE_PARAM() { === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.cc 2010-05-31 21:22:40 +0000 @@ -1829,6 +1829,52 @@ } +/** + Registers reference on items on which the subqueries depends + + @param last pointer to last st_select_lex struct, before + which all st_select_lex have to be marked as + dependent + @param dependency reference on the item on which all this + subqueries depends + +*/ + +void st_select_lex::register_dependency_item(st_select_lex *last, + Item **dependency) +{ + SELECT_LEX *s= this; + DBUG_ENTER("st_select_lex::register_dependency_item"); + DBUG_ASSERT(this != last); + DBUG_ASSERT(*dependency); + do + { + /* check duplicates */ + List_iterator_fast<Item*> li(s->master_unit()->item->depends_on); + Item **dep; + while ((dep= li++)) + { + if ((*dep)->eq(*dependency, FALSE)) + { + DBUG_PRINT("info", ("dependency %s already present", + ((*dependency)->name ? + (*dependency)->name : + "<no name>"))); + DBUG_VOID_RETURN; + } + } + + s->master_unit()->item->depends_on.push_back(dependency); + DBUG_PRINT("info", ("depends_on: Select: %d added: %s", + s->select_number, + ((*dependency)->name ? + (*dependency)->name : + "<no name>"))); + } while ((s= s->outer_select()) != last && s != 0); + DBUG_VOID_RETURN; +} + + /* st_select_lex_node::mark_as_dependent mark all st_select_lex struct from this to 'last' as dependent @@ -1843,7 +1889,7 @@ bool st_select_lex::mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency) { - + DBUG_ENTER("st_select_lex::mark_as_dependent"); DBUG_ASSERT(this != last); /* @@ -1872,11 +1918,11 @@ Item_subselect *subquery_expr= s->master_unit()->item; if (subquery_expr && subquery_expr->mark_as_dependent(thd, last, dependency)) - return TRUE; + DBUG_RETURN(TRUE); } while ((s= s->outer_select()) != last && s != 0); is_correlated= TRUE; this->master_unit()->item->is_correlated= TRUE; - return FALSE; + DBUG_RETURN(FALSE); } bool st_select_lex_node::set_braces(bool value) { return 1; } === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.h 2010-05-31 21:22:40 +0000 @@ -748,6 +748,7 @@ } bool mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency); + void register_dependency_item(st_select_lex *last, Item **dependency); bool set_braces(bool value); bool inc_in_sum_expr(); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-10 13:46:08 +0000 +++ b/sql/sql_select.cc 2010-05-31 21:22:40 +0000 @@ -151,7 +151,6 @@ static int join_read_system(JOIN_TAB *tab); static int join_read_const(JOIN_TAB *tab); static int join_read_key(JOIN_TAB *tab); -static int join_read_key2(JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref); static void join_read_key_unlock_row(st_join_table *tab); static int join_read_always_key(JOIN_TAB *tab); static int join_read_last_key(JOIN_TAB *tab); @@ -5209,7 +5208,7 @@ 'join->best_positions' contains a complete optimal extension of the current partial QEP. */ - DBUG_EXECUTE("opt", print_plan(join, join->tables, + DBUG_EXECUTE("opt", print_plan(join, n_tables, record_count, read_time, read_time, "optimal");); DBUG_RETURN(FALSE); @@ -7625,6 +7624,40 @@ /** + Creates and fills JOIN_TAB for index look up in temporary table + + @param table The table where to look up + + @return JOIN_TAB object or NULL in case of error +*/ + +JOIN_TAB *create_index_lookup_join_tab(TABLE *table) +{ + JOIN_TAB *tab; + DBUG_ENTER("create_index_lookup_join_tab"); + + if (!((tab= new JOIN_TAB))) + DBUG_RETURN(NULL); + tab->read_record.table= table; + tab->read_record.file=table->file; + /*tab->read_record.unlock_row= rr_unlock_row;*/ + tab->next_select=0; + tab->sorted= 1; + + table->status= STATUS_NO_RECORD; + tab->read_first_record= join_read_key; + /*tab->read_record.unlock_row= join_read_key_unlock_row;*/ + tab->read_record.read_record= join_no_more_records; + if (table->covering_keys.is_set(tab->ref.key) && + !table->no_keyread) + { + table->key_read=1; + table->file->extra(HA_EXTRA_KEYREAD); + } + DBUG_RETURN(tab); +} + +/** Give error if we some tables are done with a full join. This is used by multi_table_update and multi_table_delete when running @@ -10778,6 +10811,7 @@ case Item::REF_ITEM: case Item::NULL_ITEM: case Item::VARBIN_ITEM: + case Item::CACHE_ITEM: if (make_copy_field) { DBUG_ASSERT(((Item_result_field*)item)->result_field); @@ -11552,7 +11586,8 @@ &param->recinfo, select_options)) goto err; } - if (open_tmp_table(table)) + DBUG_PRINT("info", ("skip_create_table: %d", (int)param->skip_create_table)); + if (!param->skip_create_table && open_tmp_table(table)) goto err; thd->mem_root= mem_root_save; @@ -11700,16 +11735,17 @@ bool open_tmp_table(TABLE *table) { int error; + DBUG_ENTER("open_tmp_table"); if ((error= table->file->ha_open(table, table->s->table_name.str, O_RDWR, HA_OPEN_TMP_TABLE | HA_OPEN_INTERNAL_TABLE))) { table->file->print_error(error,MYF(0)); /* purecov: inspected */ table->db_stat=0; - return(1); + DBUG_RETURN(1); } (void) table->file->extra(HA_EXTRA_QUICK); /* Faster */ - return(0); + DBUG_RETURN(0); } @@ -12540,7 +12576,8 @@ else { /* Do index lookup in the materialized table */ - if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + if ((res= join_read_key2(join_tab->join->thd, join_tab, + sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); @@ -13323,61 +13360,62 @@ static int join_read_key(JOIN_TAB *tab) { - return join_read_key2(tab, tab->table, &tab->ref); + return join_read_key2(tab->join->thd, tab, tab->table, &tab->ref); } -/* +/* eq_ref access handler but generalized a bit to support TABLE and TABLE_REF not from the join_tab. See join_read_key for detailed synopsis. */ -static int -join_read_key2(JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref) +int join_read_key2(THD *thd, JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref) { int error; + DBUG_ENTER("join_read_key2"); if (!table->file->inited) { table->file->ha_index_init(table_ref->key, tab->sorted); } /* TODO: Why don't we do "Late NULLs Filtering" here? */ - if (cmp_buffer_with_ref(tab->join->thd, table, table_ref) || + if (cmp_buffer_with_ref(thd, table, table_ref) || (table->status & (STATUS_GARBAGE | STATUS_NO_PARENT | STATUS_NULL_ROW))) { if (table_ref->key_err) { table->status=STATUS_NOT_FOUND; - return -1; + DBUG_RETURN(-1); } /* Moving away from the current record. Unlock the row in the handler if it did not match the partial WHERE. */ - if (tab->ref.has_record && tab->ref.use_count == 0) + if (table_ref->has_record ) + if (table_ref->use_count == 0) { tab->read_record.file->unlock_row(); - tab->ref.has_record= FALSE; + table_ref->has_record= FALSE; } error=table->file->ha_index_read_map(table->record[0], table_ref->key_buff, make_prev_keypart_map(table_ref->key_parts), HA_READ_KEY_EXACT); if (error && error != HA_ERR_KEY_NOT_FOUND && error != HA_ERR_END_OF_FILE) - return report_error(table, error); + DBUG_RETURN(report_error(table, error)); if (! error) { - tab->ref.has_record= TRUE; - tab->ref.use_count= 1; + table_ref->has_record= TRUE; + table_ref->use_count= 1; } } else if (table->status == 0) { - DBUG_ASSERT(tab->ref.has_record); - tab->ref.use_count++; + DBUG_ASSERT(table_ref->has_record); + table_ref->use_count++; } table->null_row=0; - return table->status ? -1 : 0; + DBUG_RETURN(table->status ? -1 : 0); } === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-20 12:01:47 +0000 +++ b/sql/table.cc 2010-05-31 21:22:40 +0000 @@ -20,6 +20,7 @@ #include "sql_trigger.h" #include <m_ctype.h> #include "my_md5.h" +#include "my_bit.h" /* INFORMATION_SCHEMA name */ LEX_STRING INFORMATION_SCHEMA_NAME= {C_STRING_WITH_LEN("information_schema")}; @@ -5096,6 +5097,115 @@ file->column_bitmaps_signal(); } + +/** + @brief + Allocate space for keys + + @param key_count number of keys to allocate. + + @details + Allocate space enough to fit 'key_count' keys for this table. + + @return FALSE space was successfully allocated. + @return TRUE an error occur. +*/ + +bool TABLE::alloc_keys(uint key_count) +{ + DBUG_ASSERT(!s->keys); + key_info= s->key_info= (KEY*) alloc_root(&mem_root, sizeof(KEY)*key_count); + max_keys= key_count; + return !(key_info); +} + + +/** + @brief Adds one key to a temporary table. + + @param key key number. + @param key_parts number of fields in the key + @param next_field_no function which returns field numbers which + should be included in the key + @param arg above function argement + + @return <0 an error occur. + @return >=0 number of newly added key. +*/ + +bool TABLE::add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg) +{ + DBUG_ASSERT(key < max_keys); + + char buf[NAME_CHAR_LEN]; + KEY* keyinfo; + Field **reg_field; + uint i; + bool key_start= TRUE; + KEY_PART_INFO* key_part_info= + (KEY_PART_INFO*) alloc_root(&mem_root, sizeof(KEY_PART_INFO)*key_parts); + if (!key_part_info) + return TRUE; + keyinfo= key_info + key; + keyinfo->key_part= key_part_info; + keyinfo->usable_key_parts= keyinfo->key_parts = key_parts; + keyinfo->key_length=0; + keyinfo->algorithm= HA_KEY_ALG_UNDEF; + keyinfo->flags= HA_GENERATED_KEY; + sprintf(buf, "key%i", key); + if (!(keyinfo->name= strdup_root(&mem_root, buf))) + return TRUE; + keyinfo->rec_per_key= (ulong*) alloc_root(&mem_root, + sizeof(ulong)*key_parts); + if (!keyinfo->rec_per_key) + return TRUE; + bzero(keyinfo->rec_per_key, sizeof(ulong)*key_parts); + for (i= 0; i < key_parts; i++) + { + reg_field= field + next_field_no(arg); + if (key_start) + (*reg_field)->key_start.set_bit(key); + key_start= FALSE; + (*reg_field)->part_of_key.set_bit(key); + (*reg_field)->flags|= PART_KEY_FLAG; + key_part_info->null_bit= (*reg_field)->null_bit; + key_part_info->null_offset= (uint) ((*reg_field)->null_ptr - + (uchar*) record[0]); + key_part_info->field= *reg_field; + key_part_info->offset= (*reg_field)->offset(record[0]); + key_part_info->length= (uint16) (*reg_field)->pack_length(); + keyinfo->key_length+= key_part_info->length; + /* TODO: + The below method of computing the key format length of the + key part is a copy/paste from opt_range.cc, and table.cc. + This should be factored out, e.g. as a method of Field. + In addition it is not clear if any of the Field::*_length + methods is supposed to compute the same length. If so, it + might be reused. + */ + key_part_info->store_length= key_part_info->length; + + if ((*reg_field)->real_maybe_null()) + key_part_info->store_length+= HA_KEY_NULL_LENGTH; + if ((*reg_field)->type() == MYSQL_TYPE_BLOB || + (*reg_field)->real_type() == MYSQL_TYPE_VARCHAR) + key_part_info->store_length+= HA_KEY_BLOB_LENGTH; + + key_part_info->type= (uint8) (*reg_field)->key_type(); + key_part_info->key_type = + ((ha_base_keytype) key_part_info->type == HA_KEYTYPE_TEXT || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT1 || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT2) ? + 0 : FIELDFLAG_BINARY; + key_part_info++; + } + set_if_bigger(s->max_key_length, keyinfo->key_length); + s->keys++; + return FALSE; +} + + /** @brief Check if this is part of a MERGE table with attached children. === modified file 'sql/table.h' --- a/sql/table.h 2010-03-20 12:01:47 +0000 +++ b/sql/table.h 2010-05-31 21:22:40 +0000 @@ -781,6 +781,7 @@ uint temp_pool_slot; /* Used by intern temp tables */ uint status; /* What's in record[0] */ uint db_stat; /* mode of file as in handler.h */ + uint max_keys; /* Size of allocated key_info array. */ /* number of select if it is derived table */ uint derived_select_number; int current_lock; /* Type of lock on table */ @@ -913,6 +914,9 @@ */ inline bool needs_reopen_or_name_lock() { return s->version != refresh_version; } + bool alloc_keys(uint key_count); + bool add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg); bool is_children_attached(void); }; === modified file 'storage/maria/ha_maria.cc' --- a/storage/maria/ha_maria.cc 2010-03-20 12:01:47 +0000 +++ b/storage/maria/ha_maria.cc 2010-05-31 21:22:40 +0000 @@ -995,6 +995,8 @@ { MARIA_HA *tmp= file; file= 0; + if (!tmp) + return 0; return maria_close(tmp); }

1 0

[Maria-developers] IRC log of hingo and cafuego about the deb packages still named "mysql"
by Henrik Ingo 31 May '10

31 May '10

---------- Forwarded Message ---------- Subject: IRC log of hingo and cafuego about the deb packages still named "mysql" Date: Friday 28 May 2010 From: Henrik Ingo <hingo(a)askmonty.org> To: maria-developers(a)lists.launchpad.net Archiving this here for later reference. The background is that current MariaDB packaging (which is based on ourdelta) still has a few packages called mysql-something instead of mariadb-something. This works for ourdelta, but since you obviously cannot have 2 identically named packages in the same repository, this is a showstopper for getting MariaDB into Debian. My gut feeling of the below is that this is a bug in apt. If there is one package called mysql-common (which is one of the problematic packages) and one called mariadb-common that Provides: mysql-common, then if user chooses to install mariadb-* and uninstall mysql-*, apt should be happy and let the user do that. [12:54:43] <hingo> cafuego? [12:55:24] <evil_steve> is the six foot something dutch guy in the corner nursing the fruitiest, girliest drink in the place. [12:56:20] <capitol> ^^ [13:03:57] <cafuego> hingo: yes? [13:04:16] <hingo> cafuego: You are the one doing the ourdelta deb packages? [13:04:30] <cafuego> Yup [13:08:30] <-- monk-eeee (~monk- eeee(a)c220-237-92-67.kelvn3.qld.optusnet.com.au) has quit (Quit: Computer has gone to sleep) [13:13:06] --> monk-eeee (~monk- eeee(a)c220-237-92-67.kelvn3.qld.optusnet.com.au) has joined #ourdelta [13:14:42] <hingo> Oh sorry, I drifted off... [13:15:27] <hingo> So, I was looking into some old emails and returned to the fact we ship a "mysql-common" package with MariaDB. [13:16:00] <hingo> Arjen says this is because some other debian package is "hard-coded" to depend on that, but he is never able to remember more details. Do you? [13:17:01] <hingo> Details such as 1) do you remember which packages in debian break if we rename it to mariadb-common and 2) since packages do "depends" and "provides", how is it even possible to depend on the package name so that it cannot be solved with a provides:? [13:17:11] <hingo> cafuego ^ [13:37:32] <-- monk-eeee (~monk- eeee(a)c220-237-92-67.kelvn3.qld.optusnet.com.au) has quit (Quit: Computer has gone to sleep) [13:43:18] <cafuego> hingo: ummm... i think it was perl-dbi or somesuch [13:44:16] <cafuego> hingo: The problem was that Provides can't be versioned, so the distro pkg always wins. [13:44:43] <hingo> cafuego: Ok, that makes more sense. [13:45:01] <cafuego> hingo: A friend suggested sticking an empty mysql-common package in and upping the epoch on that so the distro always loses :-) [13:45:38] <hingo> cafuego: So perl-dbi depends always on a specific version. [13:45:55] <cafuego> hingo: No, but it always grabs the newest version. [13:46:09] <cafuego> I think, let me check [13:46:36] <cafuego> wrong pkg [13:47:38] <cafuego> libdbd-mysql-perl [13:48:13] <cafuego> On my box that has a versioned depend on libmysqlclient15off (>= 5.0.27-1) [13:48:25] <hingo> cafuego: yes, but that is perl-dbi in commonspeak :-) [13:48:34] <cafuego> ;-) [13:48:53] <cafuego> So unless I stick libmysqlclient15off in my pkg it'll always keep the distro version. [13:49:01] <cafuego> a provides won't do it :-( [13:49:16] <hingo> cafuego: So not actually mysql-common as such, just that file? [13:49:41] <cafuego> Yeah I think so. It's been a while since I worked in it. [13:49:45] <cafuego> s/in/on/ [13:50:15] <cafuego> The client won't install without libdbd-mysql-perl and libdbd-mysql-perl won't install wtihout libmysqlclient15off [13:50:43] --> monk-eeee (~monk- eeee(a)c220-237-92-67.kelvn3.qld.optusnet.com.au) has joined #ourdelta [13:51:03] <hingo> cafuego: So why is the package name relevant at all then? it depends on a file name of a library. Can't the package name be called anything? [13:51:15] <cafuego> hingo: sorry? [13:51:27] <cafuego> hingo: I'm only talking pkg names here. [13:51:41] <hingo> the package name is mysql-common [13:51:54] <cafuego> So with my maria 5.1.42-mariadb68 package [13:53:01] <hingo> mysql-common_5.1.42-mariadb68_all.deb [13:53:22] <cafuego> mariadb-client-5.1 depends on libdbd-mysql-perl (which is provided by the distro, and thus I can't edit its depends) depends on libmysqlclient16 (>= 5.1.21-1) [13:54:02] <cafuego> if I create libmariadbclient16 witha Provides: libmysqlclient16 the distro will NOT install that in preference [13:54:12] <hingo> Ok, I get the libmysqlclient packages. I was speaking about mysql-common in http://mirror.ourdelta.org/deb/dists/lenny/mariadb-ourdelta/ [13:55:52] <cafuego> AH yep. So something depends on libmysqlclient15off [13:57:00] <cafuego> I don't think I have a lenny box handy :-/ [13:57:18] <hingo> But libmysqlclient15off is a separate package? [13:57:22] <cafuego> yes [13:57:43] <cafuego> that's the name of the pkg provided by the distro, that would override a Provides in the maria packages [13:59:38] <hingo> cafuego: I'm confused. mariadb-common does not contain libmysqlclient15off or any other libmysqlclient. [13:59:39] <cafuego> There's 134 packages in Lenny that depend on libmysqlclient15off. [14:00:02] <hingo> I mean of course mysql-common from the mariadb repo. [14:00:47] <cafuego> I swear I had a good reason at the time ;-) [14:01:38] <cafuego> Oh that's right. [14:01:40] <hingo> Ok. I can see how there could be a similar reason as for libmysqlclient* problems. You kind of answered my second question anyway. [14:01:56] <cafuego> Packages depend on libmysqlclient15off and libmysqlclient15off depends on mysql-common. [14:02:09] <hingo> ok. [14:02:24] <hingo> Yes, of course. [14:02:44] <cafuego> libmysqlclient15off has a versioned depend, so a provides line in mariadb-common doesn't override that [14:03:21] <cafuego> I think I got stuck in circular depependency land and yelled at my machine a lot. Then I decided to just not rename everything :-) [14:03:36] <hingo> Hmm... I bet that versioned depend isn't really necessary. It's just a .cnf file there... [14:04:08] <cafuego> hingo: probably, but it's a distro pkg so I can't change it without actually providing a package of that name anyway. [14:04:25] <hingo> cafuego: No, of course. [14:04:40] <cafuego> So it went into the "currently unfixable" basket [14:04:45] <cafuego> Well [14:05:25] <cafuego> It's easily fixable as long as I don't expect users to want to simply 'aptitude upgrade', but instead download depdns and manually install them with dpkg. [14:05:50] <hingo> cafuego: Btw, do you have an idea why RPM based systems avoid this same problem? For them it works with a Provides? [14:06:11] <hingo> cafuego: No we of course want to support apt-get/aptitude. [14:06:33] <hingo> cafuego: We are looking into getting MariaDB into Debian itself, but then we cannot have 2 packages with the same name. [14:07:18] <cafuego> Ah yes. [14:07:46] <cafuego> Well, in *theory* they shouldn't have that awful depend in squeeze [14:07:47] <hingo> This smells apt bug to me actually. -- Henrik Ingo Project Manager and COO, Monty Program Ab hingo(a)askmonty.org, skype:henrik.ingo, +358405697354 http://askmonty.org/wiki/index.php/About_Us What's up with MariaDB? http://askmonty.org/wiki/index.php/MariaDB ------------------------------------------------------- -- email: henrik.ingo(a)avoinelama.fi tel: +358-40-5697354 www: www.avoinelama.fi/~hingo book: www.openlife.cc

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 35 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg ------------------------------------------------------------ -=-=(View All Progress Notes, 32 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 35 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg ------------------------------------------------------------ -=-=(View All Progress Notes, 32 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 35 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg ------------------------------------------------------------ -=-=(View All Progress Notes, 32 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 35 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Help Alexi debug+fix some test problems in the patch. Worked 4 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg ------------------------------------------------------------ -=-=(View All Progress Notes, 32 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 7 ESTIMATE.......: 9 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Wrote patch that allows to test SphinxSE in mysql-test-run, using external Sphinx daemon. Worked 7 hours and estimate 9 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 28 May 2010, 07:49)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.4369 2010-05-28 07:49:13.000000000 +0000 +++ /tmp/wklog.42.new.4369 2010-05-28 07:49:13.000000000 +0000 @@ -49,6 +49,10 @@ might be possible to pre-generate the necessary data/index files and store them in the source tree. +I pushed a proof-of-concept patch for this here: + + lp:~knielsen/maria/5.2-sphinxse + Here is a sample test case using this: --source include/have_sphinx.inc -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. I pushed a proof-of-concept patch for this here: lp:~knielsen/maria/5.2-sphinxse Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 7 ESTIMATE.......: 9 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:49)=-=- Wrote patch that allows to test SphinxSE in mysql-test-run, using external Sphinx daemon. Worked 7 hours and estimate 9 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 28 May 2010, 07:49)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.4369 2010-05-28 07:49:13.000000000 +0000 +++ /tmp/wklog.42.new.4369 2010-05-28 07:49:13.000000000 +0000 @@ -49,6 +49,10 @@ might be possible to pre-generate the necessary data/index files and store them in the source tree. +I pushed a proof-of-concept patch for this here: + + lp:~knielsen/maria/5.2-sphinxse + Here is a sample test case using this: --source include/have_sphinx.inc -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. I pushed a proof-of-concept patch for this here: lp:~knielsen/maria/5.2-sphinxse Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 60 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 31 May '10

31 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 60 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Mon, 31 May 2010, 06:48)=-=- Finish first architecture draft (changed my mind a number of times before I was satisfied). Write up architecture in worklog. Fix remaining test failures in proof-of-concept patch + implement xtradb part. Run some benchmarks on proof-of-concept implementation. Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours). -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1-converting]
by Sergei 30 May '10

30 May '10

Name: 5.1 => 5.1-converting -- lp:maria/5.1 https://code.launchpad.net/~maria-captains/maria/5.1-converting Your team Maria developers is subscribed to branch lp:maria/5.1. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1-converting/+edit-subsc…

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 07:49)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.4369 2010-05-28 07:49:13.000000000 +0000 +++ /tmp/wklog.42.new.4369 2010-05-28 07:49:13.000000000 +0000 @@ -49,6 +49,10 @@ might be possible to pre-generate the necessary data/index files and store them in the source tree. +I pushed a proof-of-concept patch for this here: + + lp:~knielsen/maria/5.2-sphinxse + Here is a sample test case using this: --source include/have_sphinx.inc -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. I pushed a proof-of-concept patch for this here: lp:~knielsen/maria/5.2-sphinxse Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 07:49)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.4369 2010-05-28 07:49:13.000000000 +0000 +++ /tmp/wklog.42.new.4369 2010-05-28 07:49:13.000000000 +0000 @@ -49,6 +49,10 @@ might be possible to pre-generate the necessary data/index files and store them in the source tree. +I pushed a proof-of-concept patch for this here: + + lp:~knielsen/maria/5.2-sphinxse + Here is a sample test case using this: --source include/have_sphinx.inc -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. I pushed a proof-of-concept patch for this here: lp:~knielsen/maria/5.2-sphinxse Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 06:31)=-=- High-Level Specification modified. --- /tmp/wklog.42.old.32746 2010-05-28 06:31:24.000000000 +0000 +++ /tmp/wklog.42.new.32746 2010-05-28 06:31:24.000000000 +0000 @@ -1 +1,63 @@ +Code +---- + +Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into +the MariaDB tree. + +It is a plugin, so it can be added to the tree just by including the +sub-directory storage/sphinx/. + +The Sphinx plugin is already of some maturity, having been used with MySQL for +some time. + + +Testing +------- + +To get testing in the mysql-test-run framework, some extensions are needed. + +To use the Sphinx storage engine, the external Sphinx search daemon needs to +be running with some data directory containing indexed data. It also needs to +be allocated a port. + +This is the indended approach: + +1. Testing will use an external Sphinx setup installed on the machine. Sphinx +binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), +or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER +and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx +binaries can not be found, then Sphinx tests will be disabled (using some +--source include/have_sphinx.inc in the test cases). + +2. The mysql-test-run framework will install Sphinx search data and start/stop +the Sphinx search daemon for the test cases, similarly how it is done for the +other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with +options --console, --config, and --pidfile. + +3. The mysql-test-run framework will generate a Sphinx config file from a +template in mysql-test/suite/sphinx/my.cnf. This config file will allocate +ports and data directories appropriate for avoiding conflicts between multiple +simultaneous mysql-test-run executions. The Sphinx config file is sufficiently +similar to MySQL my.cnf that we can use the existing framework for generating +config file, with just a slightly modified variant of the code writing the +file to disk. + +4. The mysql-test-run framework will pre-load the mysql database with tables +and data for Sphinx to index. It will then run the `indexer` program to +generate the indexes, and then start the `searchd` daemon. These three steps +must be done in order, as each step depends on the previous. ALTERNATIVE: it +might be possible to pre-generate the necessary data/index files and store +them in the source tree. + +Here is a sample test case using this: + +--source include/have_sphinx.inc +--source include/have_sphinxse.inc + +--replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT +eval create table ts ( id int unsigned not null, w int not null, q varchar(255) +not null, index(q) ) engine=sphinx +connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; +select * from ts where q='test'; +drop table ts; -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree HIGH-LEVEL SPECIFICATION: Code ---- Andrew Aksyonoff from Sphinx is helping to integrate the SphinxSE plugin into the MariaDB tree. It is a plugin, so it can be added to the tree just by including the sub-directory storage/sphinx/. The Sphinx plugin is already of some maturity, having been used with MySQL for some time. Testing ------- To get testing in the mysql-test-run framework, some extensions are needed. To use the Sphinx storage engine, the external Sphinx search daemon needs to be running with some data directory containing indexed data. It also needs to be allocated a port. This is the indended approach: 1. Testing will use an external Sphinx setup installed on the machine. Sphinx binaries will be searched in typical locations (eg. /usr/bin, /usr/local/bin), or can be specified explicitly in the environment with SPHINXSEARCH_INDEXER and SPHINXSEARCH_SEARCHD for the two required binaries. If the external Sphinx binaries can not be found, then Sphinx tests will be disabled (using some --source include/have_sphinx.inc in the test cases). 2. The mysql-test-run framework will install Sphinx search data and start/stop the Sphinx search daemon for the test cases, similarly how it is done for the other servers mysqld, ndbd, etc. We will run the Sphinx search daemon with options --console, --config, and --pidfile. 3. The mysql-test-run framework will generate a Sphinx config file from a template in mysql-test/suite/sphinx/my.cnf. This config file will allocate ports and data directories appropriate for avoiding conflicts between multiple simultaneous mysql-test-run executions. The Sphinx config file is sufficiently similar to MySQL my.cnf that we can use the existing framework for generating config file, with just a slightly modified variant of the code writing the file to disk. 4. The mysql-test-run framework will pre-load the mysql database with tables and data for Sphinx to index. It will then run the `indexer` program to generate the indexes, and then start the `searchd` daemon. These three steps must be done in order, as each step depends on the previous. ALTERNATIVE: it might be possible to pre-generate the necessary data/index files and store them in the source tree. Here is a sample test case using this: --source include/have_sphinx.inc --source include/have_sphinxse.inc --replace_result $SPHINXSEARCH_PORT SPHINXSEARCH_PORT eval create table ts ( id int unsigned not null, w int not null, q varchar(255) not null, index(q) ) engine=sphinx connection="sphinx://127.0.0.1:$SPHINXSEARCH_PORT/*"; select * from ts where q='test'; drop table ts; ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 06:07)=-=- Version updated. --- /tmp/wklog.42.old.32184 2010-05-28 06:07:00.000000000 +0000 +++ /tmp/wklog.42.new.32184 2010-05-28 06:07:00.000000000 +0000 @@ -1 +1 @@ -9.x +Server-5.2 -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: 9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add Sphinx storage engine to MariaDB (42)
by worklog-noreply＠askmonty.org 28 May '10

28 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add Sphinx storage engine to MariaDB CREATION DATE..: Mon, 10 Aug 2009, 23:57 SUPERVISOR.....: Monty IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 42 (http://askmonty.org/worklog/?tid=42) VERSION........: 9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 16 PROGRESS NOTES: -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Category updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-BackLog +Server-Sprint -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Version updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Maria-2.0 +9.x -=-=(Knielsen - Fri, 28 May 2010, 06:06)=-=- Status updated. --- /tmp/wklog.42.old.32171 2010-05-28 06:06:23.000000000 +0000 +++ /tmp/wklog.42.new.32171 2010-05-28 06:06:23.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Guest - Tue, 15 Sep 2009, 02:25)=-=- no Reported zero hours worked. Estimate unchanged. -=-=(Guest - Tue, 15 Sep 2009, 02:24)=-=- Version updated. --- /tmp/wklog.42.old.13241 2009-09-15 02:24:07.000000000 +0300 +++ /tmp/wklog.42.new.13241 2009-09-15 02:24:07.000000000 +0300 @@ -1 +1 @@ -Connector/.NET-5.1 +Maria-2.0 DESCRIPTION: Add the Sphinx storage engine to the MariaDB tree ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 2794: cleunup in file:///home/bell/maria/bzr/work-maria-5.3-scache/
by sanja＠askmonty.org 27 May '10

27 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache/ ------------------------------------------------------------ revno: 2794 revision-id: sanja(a)askmonty.org-20100527182744-1tu96cgyiaodzs32 parent: sanja(a)askmonty.org-20100527174138-7c3guoyiyjh7dhrk committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache timestamp: Thu 2010-05-27 21:27:44 +0300 message: cleunup === modified file 'sql/sql_subquery_cache.cc' --- a/sql/sql_subquery_cache.cc 2010-05-27 17:41:38 +0000 +++ b/sql/sql_subquery_cache.cc 2010-05-27 18:27:44 +0000 @@ -148,7 +148,6 @@ void Subquery_cache_tmptable::init() { - ulonglong keymap; List_iterator_fast<Item*> li(*list); List_iterator_fast<Item> li_items(items); Item **item; @@ -198,14 +197,6 @@ goto error; } - /* makes all bits set for keys */ - keymap= 1 << (items.elements); /* + 1 - 1 */ - if (!keymap) - keymap= ULONGLONG_MAX; - else - keymap--; - keymap&=~1; - li_items++; field_counter=1; if (cache_table->alloc_keys(1) ||

1 0

[Maria-developers] Rev 2793: Fixed some of test. in file:///home/bell/maria/bzr/work-maria-5.3-scache/
by sanja＠askmonty.org 27 May '10

27 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache/ ------------------------------------------------------------ revno: 2793 revision-id: sanja(a)askmonty.org-20100527174138-7c3guoyiyjh7dhrk parent: sanja(a)askmonty.org-20100525182914-z3zeviggq9026x1n committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache timestamp: Thu 2010-05-27 20:41:38 +0300 message: Fixed some of test. fised problem with saving reference in field. Igor's routines added === modified file 'mysql-test/r/index_merge_myisam.result' --- a/mysql-test/r/index_merge_myisam.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/index_merge_myisam.result 2010-05-27 17:41:38 +0000 @@ -1419,19 +1419,19 @@ # select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='index_merge=off,index_merge_union=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='index_merge_union=on'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,index_merge_sort_union=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=off,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=off,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=4; ERROR 42000: Variable 'optimizer_switch' can't be set to the value of '4' set optimizer_switch=NULL; @@ -1458,21 +1458,21 @@ set optimizer_switch='index_merge=off,index_merge_union=off,default'; select @@optimizer_switch; @@optimizer_switch -index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=off,index_merge_union=off,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; select @@global.optimizer_switch; @@global.optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set @@global.optimizer_switch=default; select @@global.optimizer_switch; @@global.optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on # # Check index_merge's @@optimizer_switch flags # select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (a int, b int, c int, filler char(100), @@ -1582,5 +1582,5 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on drop table t0, t1; === modified file 'mysql-test/r/subquery_cache.result' --- a/mysql-test/r/subquery_cache.result 2010-05-25 12:54:57 +0000 +++ b/mysql-test/r/subquery_cache.result 2010-05-27 17:41:38 +0000 @@ -587,4 +587,5 @@ Variable_name Value Subquery_cache_hit 0 Subquery_cache_miss 4 +drop table t1; set optimizer_switch='subquery_cache=default'; === modified file 'mysql-test/r/subselect3.result' --- a/mysql-test/r/subselect3.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3.result 2010-05-27 17:41:38 +0000 @@ -105,6 +105,7 @@ Handler_read_rnd_next 5 delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; oref a Z @@ -123,6 +124,7 @@ select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; Z No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1. +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; create table t1 (a int, b int, primary key (a)); insert into t1 values (1,1), (3,1),(100,1); === modified file 'mysql-test/r/subselect3_jcl6.result' --- a/mysql-test/r/subselect3_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3_jcl6.result 2010-05-27 17:41:38 +0000 @@ -109,6 +109,7 @@ Handler_read_rnd_next 5 delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; oref a Z @@ -127,6 +128,7 @@ select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; Z No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1. +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; create table t1 (a int, b int, primary key (a)); insert into t1 values (1,1), (3,1),(100,1); === modified file 'mysql-test/r/subselect_no_mat.result' --- a/mysql-test/r/subselect_no_mat.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_mat.result 2010-05-27 17:41:38 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='materialization=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_no_opts.result' --- a/mysql-test/r/subselect_no_opts.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_opts.result 2010-05-27 17:41:38 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='materialization=off,semijoin=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_no_semijoin.result' --- a/mysql-test/r/subselect_no_semijoin.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_semijoin.result 2010-05-27 17:41:38 +0000 @@ -1,6 +1,6 @@ show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='semijoin=off'; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t11,t12; set @save_optimizer_switch=@@optimizer_switch; @@ -4826,4 +4826,4 @@ set optimizer_switch=default; show variables like 'optimizer_switch'; Variable_name Value -optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on === modified file 'mysql-test/r/subselect_sj.result' --- a/mysql-test/r/subselect_sj.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect_sj.result 2010-05-27 17:41:38 +0000 @@ -202,39 +202,39 @@ select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; drop table t0, t1, t2; drop table t10, t11, t12; === modified file 'mysql-test/r/subselect_sj_jcl6.result' --- a/mysql-test/r/subselect_sj_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect_sj_jcl6.result 2010-05-27 17:41:38 +0000 @@ -206,39 +206,39 @@ select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,semijoin=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=on,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,semijoin=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch='default,materialization=off,loosescan=off'; select @@optimizer_switch; @@optimizer_switch -index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on +index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_condition_pushdown=on,firstmatch=on,loosescan=off,materialization=off,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on set optimizer_switch=default; drop table t0, t1, t2; drop table t10, t11, t12; === modified file 'mysql-test/t/subquery_cache.test' --- a/mysql-test/t/subquery_cache.test 2010-05-25 12:54:57 +0000 +++ b/mysql-test/t/subquery_cache.test 2010-05-27 17:41:38 +0000 @@ -199,5 +199,6 @@ show status like "subquery_cache%"; select a, a in (select a from t1 where -1 < benchmark(a,100)) from t1 as ext; show status like "subquery_cache%"; +drop table t1; set optimizer_switch='subquery_cache=default'; === modified file 'mysql-test/t/subselect3.test' --- a/mysql-test/t/subselect3.test 2010-03-20 12:01:47 +0000 +++ b/mysql-test/t/subselect3.test 2010-05-27 17:41:38 +0000 @@ -98,10 +98,12 @@ delete from t2; insert into t2 values (NULL, 0),(NULL, 0), (NULL, 0), (NULL, 0); +set optimizer_switch='subquery_cache=off'; flush status; select oref, a, a in (select a from t1 where oref=t2.oref) Z from t2; show status like '%Handler_read%'; select 'No key lookups, seq reads: 29= 5 reads from t2 + 4 * 6 reads from t1.' Z; +set @@optimizer_switch=@save_optimizer_switch; drop table t1, t2; === modified file 'sql/item.cc' --- a/sql/item.cc 2010-05-24 17:29:56 +0000 +++ b/sql/item.cc 2010-05-27 17:41:38 +0000 @@ -28,6 +28,9 @@ const String my_null_string("NULL", 4, default_charset_info); +static int save_field_in_field(Field *from,my_bool * null_value, + Field *to, bool no_conversions); + /****************************************************************************/ /* Hybrid_type_traits {_real} */ @@ -5107,40 +5110,36 @@ } +static int save_field_in_field(Field *from, my_bool *null_value, + Field *to, bool no_conversions) +{ + int res; + if (from->is_null()) + { + (*null_value)= 1; + res= set_field_to_null_with_conversions(to, no_conversions); + } + else + { + to->set_notnull(); + res= field_conv(to, from); + (*null_value)= 0; + } + return res; +} + /** Set a field's value from a item. */ void Item_field::save_org_in_field(Field *to) { - if (field->is_null()) - { - null_value=1; - set_field_to_null_with_conversions(to, 1); - } - else - { - to->set_notnull(); - field_conv(to,field); - null_value=0; - } + save_field_in_field(field, &null_value, to, TRUE); } int Item_field::save_in_field(Field *to, bool no_conversions) { - int res; - if (result_field->is_null()) - { - null_value=1; - res= set_field_to_null_with_conversions(to, no_conversions); - } - else - { - to->set_notnull(); - res= field_conv(to,result_field); - null_value=0; - } - return res; + return save_field_in_field(result_field, &null_value, to, no_conversions); } @@ -6347,7 +6346,8 @@ int Item_ref::save_in_field(Field *to, bool no_conversions) { int res; - DBUG_ASSERT(!result_field); + if (result_field) + return save_field_in_field(result_field, &null_value, to, no_conversions); res= (*ref)->save_in_field(to, no_conversions); null_value= (*ref)->null_value; return res; === modified file 'sql/item.h' --- a/sql/item.h 2010-05-24 17:29:56 +0000 +++ b/sql/item.h 2010-05-27 17:41:38 +0000 @@ -1143,11 +1143,6 @@ { return Field::GEOM_GEOMETRY; }; String *check_well_formed_result(String *str, bool send_error= 0); bool eq_by_collation(Item *item, bool binary_cmp, CHARSET_INFO *cs); - - /** - Used to get reference on real item (not Item_ref) - */ - virtual Item **unref(Item **my_ref) { return my_ref; }; }; @@ -2507,11 +2502,6 @@ { return trace_unsupported_by_check_vcol_func_processor("ref"); } - - /** - Used to get reference on real item (not Item_ref) - */ - virtual Item **unref(Item **my_ref) {return (*ref)->unref(ref); }; }; === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-05-24 17:29:56 +0000 +++ b/sql/sql_lex.cc 2010-05-27 17:41:38 +0000 @@ -1847,7 +1847,6 @@ DBUG_ENTER("st_select_lex::register_dependency_item"); DBUG_ASSERT(this != last); DBUG_ASSERT(*dependency); - dependency= (*dependency)->unref(dependency); do { /* check duplicates */ === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-24 17:29:56 +0000 +++ b/sql/sql_select.cc 2010-05-27 17:41:38 +0000 @@ -13390,7 +13390,8 @@ Moving away from the current record. Unlock the row in the handler if it did not match the partial WHERE. */ - if (table_ref->has_record && table_ref->use_count == 0) + if (table_ref->has_record ) + if (table_ref->use_count == 0) { tab->read_record.file->unlock_row(); table_ref->has_record= FALSE; === modified file 'sql/sql_subquery_cache.cc' --- a/sql/sql_subquery_cache.cc 2010-05-25 10:45:36 +0000 +++ b/sql/sql_subquery_cache.cc 2010-05-27 17:41:38 +0000 @@ -139,12 +139,20 @@ DBUG_RETURN(equalities->fix_fields(table_thd, &equalities)); } + +static uint field_enumerator(uchar *arg) +{ + return ((uint*)arg)[0]++; +} + + void Subquery_cache_tmptable::init() { ulonglong keymap; List_iterator_fast<Item*> li(*list); List_iterator_fast<Item> li_items(items); Item **item; + uint field_counter; DBUG_ENTER("Subquery_cache_tmptable::init"); DBUG_ASSERT(!inited); inited= TRUE; @@ -199,8 +207,11 @@ keymap&=~1; li_items++; + field_counter=1; if (cache_table->alloc_keys(1) || - (cache_table->add_tmp_key(keymap, "cache-table-key") < 0) || + (cache_table->add_tmp_key(0, items.elements - 1, + &field_enumerator, + (uchar*)&field_counter) < 0) || createtmp_table_search_structures(table_thd, cache_table, li_items, &tab_ref) || !(tab= create_index_lookup_join_tab(cache_table))) === modified file 'sql/table.cc' --- a/sql/table.cc 2010-05-24 17:29:56 +0000 +++ b/sql/table.cc 2010-05-27 17:41:38 +0000 @@ -5114,7 +5114,7 @@ bool TABLE::alloc_keys(uint key_count) { DBUG_ASSERT(!s->keys); - key_info= s->key_info= (KEY*) my_malloc(sizeof(KEY)*key_count, MYF(0)); + key_info= s->key_info= (KEY*) alloc_root(&mem_root, sizeof(KEY)*key_count); max_keys= key_count; return !(key_info); } @@ -5123,52 +5123,51 @@ /** @brief Adds one key to a temporary table. - @param key_parts bitmap of fields that take a part in the key. - @param key_name name of the key - - @details - Creates a key for this table from fields which corresponds the bits set to 1 - in the 'key_parts' bitmap. The 'key_name' name is given to the newly created - key. + @param key key number. + @param key_parts number of fields in the key + @param next_field_no function which returns field numbers which + should be included in the key + @param arg above function argement @return <0 an error occur. @return >=0 number of newly added key. */ -int TABLE::add_tmp_key(ulonglong key_parts, const char *key_name) +bool TABLE::add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg) { - DBUG_ASSERT(s->keys< max_keys); + DBUG_ASSERT(key < max_keys); + char buf[NAME_CHAR_LEN]; KEY* keyinfo; Field **reg_field; uint i; bool key_start= TRUE; - uint key_part_count= my_count_bits(key_parts); KEY_PART_INFO* key_part_info= - (KEY_PART_INFO*) my_malloc(sizeof(KEY_PART_INFO)* key_part_count, MYF(0)); + (KEY_PART_INFO*) alloc_root(&mem_root, sizeof(KEY_PART_INFO)*key_parts); if (!key_part_info) - return -1; - keyinfo= key_info + s->keys; - keyinfo->key_part=key_part_info; - keyinfo->usable_key_parts=keyinfo->key_parts= key_part_count; + return TRUE; + keyinfo= key_info + key; + keyinfo->key_part= key_part_info; + keyinfo->usable_key_parts= keyinfo->key_parts = key_parts; keyinfo->key_length=0; keyinfo->algorithm= HA_KEY_ALG_UNDEF; - keyinfo->name= (char *)key_name; keyinfo->flags= HA_GENERATED_KEY; - keyinfo->rec_per_key= (ulong*)my_malloc(sizeof(ulong)*key_part_count, MYF(0)); + sprintf(buf, "key%i", key); + if (!(keyinfo->name= strdup_root(&mem_root, buf))) + return TRUE; + keyinfo->rec_per_key= (ulong*) alloc_root(&mem_root, + sizeof(ulong)*key_parts); if (!keyinfo->rec_per_key) - return -1; - bzero(keyinfo->rec_per_key, sizeof(ulong)*key_part_count); - for (i= 0, reg_field=field ; - *reg_field; - i++, reg_field++) + return TRUE; + bzero(keyinfo->rec_per_key, sizeof(ulong)*key_parts); + for (i= 0; i < key_parts; i++) { - if (!(key_parts & (1 << i))) - continue; + reg_field= field + next_field_no(arg); if (key_start) - (*reg_field)->key_start.set_bit(s->keys); + (*reg_field)->key_start.set_bit(key); key_start= FALSE; - (*reg_field)->part_of_key.set_bit(s->keys); + (*reg_field)->part_of_key.set_bit(key); (*reg_field)->flags|= PART_KEY_FLAG; key_part_info->null_bit= (*reg_field)->null_bit; key_part_info->null_offset= (uint) ((*reg_field)->null_ptr - @@ -5202,7 +5201,8 @@ key_part_info++; } set_if_bigger(s->max_key_length, keyinfo->key_length); - return ++s->keys - 1; + s->keys++; + return FALSE; } === modified file 'sql/table.h' --- a/sql/table.h 2010-05-24 17:29:56 +0000 +++ b/sql/table.h 2010-05-27 17:41:38 +0000 @@ -914,9 +914,10 @@ */ inline bool needs_reopen_or_name_lock() { return s->version != refresh_version; } + bool alloc_keys(uint key_count); + bool add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg); bool is_children_attached(void); - bool alloc_keys(uint key_count); - int add_tmp_key(ulonglong key_parts, const char *key_name); }; enum enum_schema_table_state

1 0

[Maria-developers] bzr commit into file:///home/tsk/mprog/src/5.3-mwl89/ branch (timour:2792)
by timour＠askmonty.org 27 May '10

27 May '10

#At file:///home/tsk/mprog/src/5.3-mwl89/ based on revid:psergey@askmonty.org-20100503154606-z7v6errebcv9gax1 2792 timour(a)askmonty.org 2010-05-27 MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation Phase 1: Implement recursive bottom-up optimization of subqueires instead of lazy optimization. The patch implements a preparatory phase for MWL#89, which is a prerequisite to implement a cost-based choice between both strategies. The patch passes the complete regression test. The main change is implemented by the method: JOIN::optimize_materialized_in_subqueries(). All other changes were required to correct problems resulting from changing the order of optimization. Most of these problems followed the same pattern - there are some shared structures between a subquery and its parent query. Depending on which one is optimized first (parent or child query), these shared strucutres may get different values, thus resulting in an inconsistent query plan. modified: mysql-test/r/subselect_mat.result sql/item_subselect.cc sql/item_subselect.h sql/sql_class.cc sql/sql_class.h sql/sql_select.cc sql/sql_select.h === modified file 'mysql-test/r/subselect_mat.result' --- a/mysql-test/r/subselect_mat.result 2010-04-05 21:15:15 +0000 +++ b/mysql-test/r/subselect_mat.result 2010-05-27 13:13:47 +0000 @@ -1139,7 +1139,7 @@ insert into t1 values (5); explain select min(a1) from t1 where 7 in (select b1 from t2 group by b1); id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Select tables optimized away -2 SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found +2 SUBQUERY NULL NULL NULL NULL NULL NULL NULL no matching row in const table select min(a1) from t1 where 7 in (select b1 from t2 group by b1); min(a1) set @@optimizer_switch='default,materialization=off'; @@ -1153,7 +1153,7 @@ set @@optimizer_switch='default,semijoin explain select min(a1) from t1 where 7 in (select b1 from t2); id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Select tables optimized away -2 SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found +2 SUBQUERY NULL NULL NULL NULL NULL NULL NULL no matching row in const table select min(a1) from t1 where 7 in (select b1 from t2); min(a1) set @@optimizer_switch='default,materialization=off'; === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-04-02 14:27:06 +0000 +++ b/sql/item_subselect.cc 2010-05-27 13:13:47 +0000 @@ -2110,7 +2110,7 @@ bool Item_in_subselect::setup_engine() if (!(new_engine= new subselect_hash_sj_engine(thd, this, old_engine)) || - new_engine->init_permanent(unit->get_unit_column_types())) + new_engine->init_permanent(&old_engine->join->fields_list)) { Item_subselect::trans_res trans_res; /* @@ -2126,6 +2126,15 @@ bool Item_in_subselect::setup_engine() &eq_creator); else trans_res= row_value_in_to_exists_transformer(old_engine->join); + /* + The IN=>EXISTS transformation above injects new predicates into the + WHERE and HAVING clauses. Since the subquery was already optimized, + below we force its reoptimization with the new injected conditions + by the first call to subselect_single_select_engine::exec(). + This is the only case of lazy subquery optimization in the server. + */ + DBUG_ASSERT(old_engine->join->optimized); + old_engine->join->optimized= false; res= (trans_res != Item_subselect::RES_OK); } if (new_engine) @@ -3673,6 +3682,7 @@ bitmap_init_memroot(MY_BITMAP *map, uint bool subselect_hash_sj_engine::init_permanent(List<Item> *tmp_columns) { + select_union *result_sink; /* Options to create_tmp_table. */ ulonglong tmp_create_options= thd->options | TMP_TABLE_ALL_COLUMNS; /* | TMP_TABLE_FORCE_MYISAM; TIMOUR: force MYISAM */ @@ -3706,15 +3716,16 @@ bool subselect_hash_sj_engine::init_perm DBUG_RETURN(TRUE); } */ - if (!(result= new select_materialize_with_stats)) + if (!(result_sink= new select_materialize_with_stats)) DBUG_RETURN(TRUE); - if (((select_union*) result)->create_result_table( - thd, tmp_columns, TRUE, tmp_create_options, - "materialized subselect", TRUE)) + if (result_sink->create_result_table(thd, tmp_columns, TRUE, + tmp_create_options, + "materialized subselect", TRUE)) DBUG_RETURN(TRUE); - tmp_table= ((select_union*) result)->table; + tmp_table= result_sink->table; + result= result_sink; /* If the subquery has blobs, or the total key lenght is bigger than @@ -3882,7 +3893,6 @@ subselect_hash_sj_engine::make_unique_en cur_ref_buff + test(maybe_null), we could use that information instead. */ - cur_ref_buff + null_count, null_count ? cur_ref_buff : 0, cur_key_part->length, tab->ref.items[i]); @@ -3908,11 +3918,6 @@ subselect_hash_sj_engine::make_unique_en bool subselect_hash_sj_engine::init_runtime() { /* - Create and optimize the JOIN that will be used to materialize - the subquery if not yet created. - */ - materialize_engine->prepare(); - /* Repeat name resolution for 'cond' since cond is not part of any clause of the query, and it is not 'fixed' during JOIN::prepare. */ @@ -3935,6 +3940,16 @@ subselect_hash_sj_engine::~subselect_has } +int subselect_hash_sj_engine::prepare() +{ + /* + Create and optimize the JOIN that will be used to materialize + the subquery if not yet created. + */ + return materialize_engine->prepare(); +} + + /** Cleanup performed after each PS execution. @@ -3996,9 +4011,8 @@ int subselect_hash_sj_engine::exec() the subquery predicate. */ thd->lex->current_select= materialize_engine->select_lex; - if ((res= materialize_join->optimize())) - goto err; /* purecov: inspected */ - DBUG_ASSERT(!is_materialized); /* We should materialize only once. */ + /* The subquery should be optimized, and materialized only once. */ + DBUG_ASSERT(materialize_join->optimized && !is_materialized); materialize_join->exec(); if ((res= test(materialize_join->error || thd->is_fatal_error))) goto err; @@ -4909,7 +4923,7 @@ bool subselect_rowid_merge_engine::parti /* If there is a non-NULL key, it must be the first key in the keys array. */ DBUG_ASSERT(!non_null_key || (non_null_key && merge_keys[0] == non_null_key)); - /* The prioryty queue for keys must be empty. */ + /* The priority queue for keys must be empty. */ DBUG_ASSERT(!pq.elements); /* All data accesses during execution are via handler::ha_rnd_pos() */ === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.h 2010-05-27 13:13:47 +0000 @@ -805,7 +805,7 @@ public: bool init_permanent(List<Item> *tmp_columns); bool init_runtime(); void cleanup(); - int prepare() { return 0; } /* Override virtual function in base class. */ + int prepare(); int exec(); virtual void print(String *str, enum_query_type query_type); uint cols() === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-04-05 21:15:15 +0000 +++ b/sql/sql_class.cc 2010-05-27 13:13:47 +0000 @@ -2948,8 +2948,15 @@ create_result_table(THD *thd_arg, Listelements; tmp_table_param.bit_fields_as_long= bit_fields_as_long; + /* + TIMOUR: + Setting this parameter here limits the use of this class only for + materialized subqueries. + */ + tmp_table_param.materialized_subquery= true; if (! (table= create_tmp_table(thd_arg, &tmp_table_param, *column_types, (ORDER*) 0, is_union_distinct, 1, @@ -3034,6 +3041,7 @@ void TMP_TABLE_PARAM::init() table_charset= 0; precomputed_group_by= 0; bit_fields_as_long= 0; + materialized_subquery= 0; DBUG_VOID_RETURN; } === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-04-05 21:15:15 +0000 +++ b/sql/sql_class.h 2010-05-27 13:13:47 +0000 @@ -2772,6 +2772,7 @@ public: uint convert_blob_length; CHARSET_INFO *table_charset; bool schema_table; + bool materialized_subquery; /* True if GROUP BY and its aggregate functions are already computed by a table access method (e.g. by loose index scan). In this case @@ -2790,8 +2791,8 @@ public: TMP_TABLE_PARAM() :copy_field(0), group_parts(0), group_length(0), group_null_parts(0), convert_blob_length(0), - schema_table(0), precomputed_group_by(0), force_copy_fields(0), - bit_fields_as_long(0) + schema_table(0), materialized_subquery(0), precomputed_group_by(0), + force_copy_fields(0), bit_fields_as_long(0) {} ~TMP_TABLE_PARAM() { === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-03 15:46:06 +0000 +++ b/sql/sql_select.cc 2010-05-27 13:13:47 +0000 @@ -714,6 +714,7 @@ JOIN::optimize() { ulonglong select_opts_for_readinfo; uint no_jbuf_after; + int res; DBUG_ENTER("JOIN::optimize"); // to prevent double initialization on EXPLAIN @@ -723,6 +724,10 @@ JOIN::optimize() thd_proc_info(thd, "optimizing"); + /* Optimize recursively all IN subqueries of this query. */ + if ((res= optimize_materialized_in_subqueries())) + DBUG_RETURN(res); + /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ if (convert_join_subqueries_to_semijoins(this)) DBUG_RETURN(1); /* purecov: inspected */ @@ -848,7 +853,6 @@ JOIN::optimize() */ if (tables_list && implicit_grouping) { - int res; /* opt_sum_query() returns HA_ERR_KEY_NOT_FOUND if no rows match to the WHERE conditions, @@ -1277,7 +1281,6 @@ JOIN::optimize() if (setup_subquery_materialization()) DBUG_RETURN(1); - int res; if ((res= rewrite_to_index_subquery_engine(this)) != -1) DBUG_RETURN(res); /* @@ -2413,8 +2416,9 @@ err: Setup for execution all subqueries of a query, for which the optimizer chose hash semi-join. - @details Iterate over all subqueries of the query, and if they are under an - IN predicate, and the optimizer chose to compute it via hash semi-join: + @details Iterate over all immediate child subqueries of the query, and if + they are under an IN predicate, and the optimizer chose to compute it via + hash semi-join: - try to initialize all data structures needed for the materialized execution of the IN predicate, - if this fails, then perform the IN=>EXISTS transformation which was @@ -2454,6 +2458,51 @@ bool JOIN::setup_subquery_materializatio } +/** + Optimize all immediate children IN subqueries of this join. + + @note + This method must be called in the very beginning of JOIN::optimize(). + As a result all children subqueries are optimized recursively before + their parent. +*/ + +int +JOIN::optimize_materialized_in_subqueries() +{ + int res; + for (SELECT_LEX_UNIT *un= select_lex->first_inner_unit(); un; + un= un->next_unit()) + { + for (SELECT_LEX *sl= un->first_select(); sl; sl= sl->next_select()) + { + Item_subselect *subquery_predicate= sl->master_unit()->item; + if (subquery_predicate && + subquery_predicate->substype() == Item_subselect::IN_SUBS && + ((Item_in_subselect*) subquery_predicate)->exec_method == + Item_in_subselect::MATERIALIZATION + // @todo TIMOUR: + // Think also how to pre-optimize for IN_TO_EXISTS because we still + // call the optimizer in subselect_single_select_engine::exec() + ) + { + JOIN *subquery_join= sl->join; + if (subquery_join) + { + SELECT_LEX *save_select= thd->lex->current_select; + thd->lex->current_select= subquery_predicate->get_select_lex(); + res= subquery_join->optimize(); + thd->lex->current_select= save_select; + if (res) + return res; + } + } + } + } + return 0; +} + + /***************************************************************************** Create JOIN_TABS, make a guess about the table types, Approximate how many records will be used in each table @@ -11142,7 +11191,27 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA continue; // Some kindf of const item } if (type == Item::SUM_FUNC_ITEM) - ((Item_sum *) item)->result_field= new_field; + { + Item_sum *agg_item= (Item_sum *) item; + /* + Update the result field only if it has never been set, or if the + created temporary table is not to be used for subquery + materialization. + + The reason is that for subqueries that require materialization as part + of their plan, we create the 'external' temporary table needed for IN + execution, after the 'internal' temporary table needed for grouping. + Since both the external and the internal temporary tables are created + for the same list of SELECT fields of the subquery, setting + 'result_field' for each temp table creation overrides the previous + value of result field. + + The condition below prevents the creation of the external temp table + to override the 'result_field' that was set for the internal temp table. + */ + if (!agg_item->result_field || !param->materialized_subquery) + agg_item->result_field= new_field; + } tmp_from_field++; reclength+=new_field->pack_length(); if (!(new_field->flags & NOT_NULL_FLAG)) @@ -18881,6 +18950,8 @@ bool JOIN::change_result(select_result * { DBUG_ENTER("JOIN::change_result"); result= res; + if (tmp_join) + tmp_join->result= res; if (!procedure && (result->prepare(fields_list, select_lex->master_unit()) || result->prepare2())) { === modified file 'sql/sql_select.h' --- a/sql/sql_select.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_select.h 2010-05-27 13:13:47 +0000 @@ -1717,6 +1717,7 @@ private: */ bool implicit_grouping; bool make_simple_join(JOIN *join, TABLE *tmp_table); + int optimize_materialized_in_subqueries(); };

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (igor:2788)
by Igor Babaev 26 May '10

26 May '10

#At lp:maria/5.2 based on revid:psergey@askmonty.org-20100329200940-9ikx6gpww0gtsx00 2788 Igor Babaev 2010-05-26 MWL#106: Backport optimizations for derived tables and views. The main consolidated patch. added: mysql-test/r/derived_view.result mysql-test/t/derived_view.test modified: mysql-test/r/derived.result mysql-test/r/explain.result mysql-test/r/func_str.result mysql-test/r/index_merge_myisam.result mysql-test/r/information_schema.result mysql-test/r/innodb_lock_wait_timeout_1.result mysql-test/r/innodb_mysql.result mysql-test/r/lock_multi_bug38499.result mysql-test/r/myisam_mrr.result mysql-test/r/ps.result mysql-test/r/ps_ddl.result mysql-test/r/strict.result mysql-test/r/subselect.result mysql-test/r/subselect3.result mysql-test/r/subselect3_jcl6.result mysql-test/r/subselect_no_mat.result mysql-test/r/subselect_no_opts.result mysql-test/r/subselect_no_semijoin.result mysql-test/r/table_elim.result mysql-test/r/view.result mysql-test/r/view_grant.result mysql-test/t/lock_multi_bug38499.test sql/field.cc sql/field.h sql/handler.cc sql/item.cc sql/item.h sql/item_cmpfunc.cc sql/item_cmpfunc.h sql/item_func.cc sql/item_func.h sql/item_subselect.cc sql/item_subselect.h sql/mysql_priv.h sql/opt_range.cc sql/opt_subselect.cc sql/opt_sum.cc sql/records.cc sql/sp_head.cc sql/sql_acl.cc sql/sql_base.cc sql/sql_bitmap.h sql/sql_cache.cc sql/sql_class.cc sql/sql_class.h sql/sql_cursor.cc sql/sql_delete.cc sql/sql_derived.cc sql/sql_help.cc sql/sql_insert.cc sql/sql_join_cache.cc sql/sql_lex.cc sql/sql_lex.h sql/sql_list.h sql/sql_load.cc sql/sql_olap.cc sql/sql_parse.cc sql/sql_prepare.cc sql/sql_select.cc sql/sql_select.h sql/sql_show.cc sql/sql_table.cc sql/sql_union.cc sql/sql_update.cc sql/sql_view.cc sql/sql_yacc.yy sql/table.cc sql/table.h === modified file 'mysql-test/r/derived.result' --- a/mysql-test/r/derived.result 2009-07-11 18:44:29 +0000 +++ b/mysql-test/r/derived.result 2010-05-26 20:18:18 +0000 @@ -57,9 +57,8 @@ a b a b 3 c 3 c explain select * from t1 as x1, (select * from t1) as x2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY x1 ALL NULL NULL NULL NULL 4 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 Using join buffer -2 DERIVED t1 ALL NULL NULL NULL NULL 4 +1 SIMPLE x1 ALL NULL NULL NULL NULL 4 +1 SIMPLE t1 ALL NULL NULL NULL NULL 4 Using join buffer drop table if exists t2,t3; select * from (select 1) as a; 1 @@ -91,7 +90,7 @@ a b 2 b explain select * from (select * from t1 union select * from t1) a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 3 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 8 2 DERIVED t1 ALL NULL NULL NULL NULL 4 3 UNION t1 ALL NULL NULL NULL NULL 4 NULL UNION RESULT <union2,3> ALL NULL NULL NULL NULL NULL @@ -113,9 +112,8 @@ a b 3 c explain select * from (select t1.*, t2.a as t2a from t1,t2 where t1.a=t2.a) t1; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t2 system NULL NULL NULL NULL 1 -2 DERIVED t1 ALL NULL NULL NULL NULL 4 Using where +1 SIMPLE t2 system NULL NULL NULL NULL 1 +1 SIMPLE t1 ALL NULL NULL NULL NULL 4 Using where drop table t1, t2; create table t1(a int not null, t char(8), index(a)); SELECT * FROM (SELECT * FROM t1) as b ORDER BY a ASC LIMIT 0,20; @@ -142,8 +140,7 @@ a t 20 20 explain select count(*) from t1 as tt1, (select * from t1) as tt2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Select tables optimized away -2 DERIVED t1 ALL NULL NULL NULL NULL 10000 +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Select tables optimized away drop table t1; SELECT * FROM (SELECT (SELECT * FROM (SELECT 1 as a) as a )) as b; (SELECT * FROM (SELECT 1 as a) as a ) @@ -172,30 +169,30 @@ insert into t1 values (NULL, 'a', 1), (N insert into t2 values (1, 100), (1, 101), (1, 102), (2, 100), (2, 103), (2, 104), (3, 101), (3, 102), (3, 105); SELECT STRAIGHT_JOIN d.pla_id, m2.mat_id FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; pla_id mat_id -100 1 -101 1 102 1 -103 2 +101 1 +100 1 104 2 +103 2 105 3 SELECT STRAIGHT_JOIN d.pla_id, m2.test FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; pla_id test -100 1 -101 1 102 1 -103 2 +101 1 +100 1 104 2 +103 2 105 3 explain SELECT STRAIGHT_JOIN d.pla_id, m2.mat_id FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY m2 ALL NULL NULL NULL NULL 9 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 6 Using where; Using join buffer +1 PRIMARY <derived2> ref key0 key0 7 test.m2.matintnum 2 Using where 2 DERIVED mp ALL NULL NULL NULL NULL 9 Using temporary; Using filesort 2 DERIVED m1 eq_ref PRIMARY PRIMARY 3 test.mp.mat_id 1 explain SELECT STRAIGHT_JOIN d.pla_id, m2.test FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY m2 ALL NULL NULL NULL NULL 9 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 6 Using where; Using join buffer +1 PRIMARY <derived2> ref key0 key0 7 test.m2.matintnum 2 Using where 2 DERIVED mp ALL NULL NULL NULL NULL 9 Using temporary; Using filesort 2 DERIVED m1 eq_ref PRIMARY PRIMARY 3 test.mp.mat_id 1 drop table t1,t2; @@ -230,9 +227,8 @@ count(*) 2 explain select count(*) from t1 INNER JOIN (SELECT A.E1, A.E2, A.E3 FROM t1 AS A WHERE A.E3 = (SELECT MAX(B.E3) FROM t1 AS B WHERE A.E2 = B.E2)) AS THEMAX ON t1.E1 = THEMAX.E2 AND t1.E1 = t1.E2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 -1 PRIMARY t1 eq_ref PRIMARY PRIMARY 4 THEMAX.E2 1 Using where -2 DERIVED A ALL NULL NULL NULL NULL 2 Using where +1 SIMPLE A ALL NULL NULL NULL NULL 2 Using where +1 SIMPLE t1 eq_ref PRIMARY PRIMARY 4 test.A.E2 1 Using where 3 DEPENDENT SUBQUERY B ALL NULL NULL NULL NULL 2 Using where drop table t1; create table t1 (a int); @@ -245,8 +241,8 @@ a a 2 2 explain select * from ( select * from t1 union select * from t1) a,(select * from t1 union select * from t1) b; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 -1 PRIMARY <derived4> ALL NULL NULL NULL NULL 2 Using join buffer +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 +1 PRIMARY <derived4> ALL NULL NULL NULL NULL 4 Using join buffer 4 DERIVED t1 ALL NULL NULL NULL NULL 2 5 UNION t1 ALL NULL NULL NULL NULL 2 NULL UNION RESULT <union4,5> ALL NULL NULL NULL NULL NULL @@ -311,7 +307,7 @@ a 7.0000 b 3.5000 explain SELECT s.name, AVG(s.val) AS median FROM (SELECT x.name, x.val FROM t1 x, t1 y WHERE x.name=y.name GROUP BY x.name, x.val HAVING SUM(y.val <= x.val) >= COUNT(*)/2 AND SUM(y.val >= x.val) >= COUNT(*)/2) AS s GROUP BY s.name; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 3 Using temporary; Using filesort +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 289 Using temporary; Using filesort 2 DERIVED x ALL NULL NULL NULL NULL 17 Using temporary; Using filesort 2 DERIVED y ALL NULL NULL NULL NULL 17 Using where; Using join buffer drop table t1; @@ -322,8 +318,7 @@ id select_type table type possible_keys 1 SIMPLE t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index explain select a from (select a from t2 where a>1) tt; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index +1 SIMPLE t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index drop table t2; CREATE TABLE `t1` ( `itemid` int(11) NOT NULL default '0', `grpid` varchar(15) NOT NULL default '', `vendor` int(11) NOT NULL default '0', `date_` date NOT NULL default '0000-00-00', `price` decimal(12,2) NOT NULL default '0.00', PRIMARY KEY (`itemid`,`grpid`,`vendor`,`date_`), KEY `itemid` (`itemid`,`vendor`), KEY `itemid_2` (`itemid`,`date_`)); insert into t1 values (128, 'rozn', 2, curdate(), 10), === added file 'mysql-test/r/derived_view.result' --- a/mysql-test/r/derived_view.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/derived_view.result 2010-05-26 20:18:18 +0000 @@ -0,0 +1,570 @@ +drop table if exists t1,t2; +drop view if exists v1,v2,v3,v4; +create table t1(f1 int, f11 int); +create table t2(f2 int, f22 int); +insert into t1 values(1,1),(2,2),(3,3),(5,5),(9,9),(7,7); +insert into t1 values(17,17),(13,13),(11,11),(15,15),(19,19); +insert into t2 values(1,1),(3,3),(2,2),(4,4),(8,8),(6,6); +insert into t2 values(12,12),(14,14),(10,10),(18,18),(16,16); +Tests: +for merged derived tables +explain for simple derived +explain select * from (select * from t1) tt; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 +select * from (select * from t1) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +9 9 +7 7 +17 17 +13 13 +11 11 +15 15 +19 19 +explain for multitable derived +explain extended select * from (select * from t1 join t2 on f1=f2) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t1` join `test`.`t2` where (`test`.`t2`.`f2` = `test`.`t1`.`f1`) +select * from (select * from t1 join t2 on f1=f2) tt; +f1 f11 f2 f22 +1 1 1 1 +3 3 3 3 +2 2 2 2 +explain for derived with where +explain extended +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f11` = 2) and (`test`.`t1`.`f1` in (2,3))) +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +f1 f11 +2 2 +join of derived +explain extended +select * from (select * from t1 where f1 in (2,3)) tt join +(select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` join `test`.`t1` where ((`test`.`t1`.`f1` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` in (1,2)) and (`test`.`t1`.`f1` in (2,3))) +select * from (select * from t1 where f1 in (2,3)) tt join +(select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; +f1 f11 f1 f11 +2 2 2 2 +flush status; +explain extended +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f11` = 2) and (`test`.`t1`.`f1` in (2,3))) +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +f1 f11 +2 2 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 12 +for merged views +create view v1 as select * from t1; +create view v2 as select * from t1 join t2 on f1=f2; +create view v3 as select * from t1 where f1 in (2,3); +create view v4 as select * from t2 where f2 in (2,3); +explain for simple views +explain extended select * from v1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` +select * from v1; +f1 f11 +1 1 +2 2 +3 3 +5 5 +9 9 +7 7 +17 17 +13 13 +11 11 +15 15 +19 19 +explain for multitable views +explain extended select * from v2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t1` join `test`.`t2` where (`test`.`t2`.`f2` = `test`.`t1`.`f1`) +select * from v2; +f1 f11 f2 f22 +1 1 1 1 +3 3 3 3 +2 2 2 2 +explain for views with where +explain extended select * from v3 where f11 in (1,3); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f11` in (1,3)) and (`test`.`t1`.`f1` in (2,3))) +select * from v3 where f11 in (1,3); +f1 f11 +3 3 +explain for joined views +explain extended +select * from v3 join v4 on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t1` join `test`.`t2` where ((`test`.`t2`.`f2` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` in (2,3)) and (`test`.`t1`.`f1` in (2,3))) +select * from v3 join v4 on f1=f2; +f1 f11 f2 f22 +3 3 3 3 +2 2 2 2 +flush status; +explain extended select * from v4 where f2 in (1,3); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` where ((`test`.`t2`.`f2` in (1,3)) and (`test`.`t2`.`f2` in (2,3))) +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from v4 where f2 in (1,3); +f2 f22 +3 3 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 12 +for materialized derived tables +explain for simple derived +explain extended select * from (select * from t1 group by f1) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` group by `test`.`t1`.`f1`) `tt` +select * from (select * from t1 having f1=f1) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +9 9 +7 7 +17 17 +13 13 +11 11 +15 15 +19 19 +explain showing created indexes +explain extended +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 100.00 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 100.00 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`tt`.`f2` AS `f2`,`tt`.`f22` AS `f22` from `test`.`t1` join (select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` group by `test`.`t2`.`f2`) `tt` where (`tt`.`f2` = `test`.`t1`.`f1`) +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +explain showing late materialization +flush status; +explain select * from t1 join (select * from t2 group by f2) tt on f1=f2; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 Using temporary; Using filesort +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 11 +Handler_read_next 3 +Handler_read_prev 0 +Handler_read_rnd 11 +Handler_read_rnd_next 36 +for materialized views +drop view v1,v2,v3; +create view v1 as select * from t1 group by f1; +create view v2 as select * from t2 group by f2; +create view v3 as select t1.f1,t1.f11 from t1 join t1 as t11 where t1.f1=t11.f1 +having t1.f1<100; +explain for simple derived +explain extended select * from v1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`v1` +select * from v1; +f1 f11 +1 1 +2 2 +3 3 +5 5 +7 7 +9 9 +11 11 +13 13 +15 15 +17 17 +19 19 +explain showing created indexes +explain extended select * from t1 join v2 on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 100.00 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 100.00 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`v2`.`f2` AS `f2`,`v2`.`f22` AS `f22` from `test`.`t1` join `test`.`v2` where (`v2`.`f2` = `test`.`t1`.`f1`) +select * from t1 join v2 on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +explain extended +select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 100.00 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 10 100.00 Using where +1 PRIMARY <derived3> ref key0 key0 5 test.t1.f1 10 100.00 Using where +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 +3 DERIVED t11 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t11 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`v31`.`f1` AS `f1`,`v31`.`f11` AS `f11`,`v3`.`f1` AS `f1`,`v3`.`f11` AS `f11` from `test`.`t1` join `test`.`v3` `v31` join `test`.`v3` where ((`v31`.`f1` = `test`.`t1`.`f1`) and (`v3`.`f1` = `test`.`t1`.`f1`)) +flush status; +select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +f1 f11 f1 f11 f1 f11 +1 1 1 1 1 1 +2 2 2 2 2 2 +3 3 3 3 3 3 +5 5 5 5 5 5 +9 9 9 9 9 9 +7 7 7 7 7 7 +17 17 17 17 17 17 +13 13 13 13 13 13 +11 11 11 11 11 11 +15 15 15 15 15 15 +19 19 19 19 19 19 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 22 +Handler_read_next 22 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 60 +explain showing late materialization +flush status; +explain select * from t1 join v2 on f1=f2; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 Using temporary; Using filesort +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from t1 join v2 on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 11 +Handler_read_next 3 +Handler_read_prev 0 +Handler_read_rnd 11 +Handler_read_rnd_next 36 +explain extended select * from v1 join v4 on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 11 100.00 Using where +1 PRIMARY <derived2> ref key0 key0 5 test.t2.f2 2 100.00 Using where +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`v1` join `test`.`t2` where ((`v1`.`f1` = `test`.`t2`.`f2`) and (`test`.`t2`.`f2` in (2,3))) +select * from v1 join v4 on f1=f2; +f1 f11 f2 f22 +3 3 3 3 +2 2 2 2 +merged derived in merged derived +explain extended select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7)) +select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2) zz; +f1 f11 +3 3 +5 5 +materialized derived in merged derived +explain extended select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) +select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; +f1 f11 +3 3 +5 5 +merged derived in materialized derived +explain extended select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `zz`.`f1` AS `f1`,`zz`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7)) group by `test`.`t1`.`f1`) `zz` +select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; +f1 f11 +3 3 +5 5 +materialized derived in materialized derived +explain extended select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `zz`.`f1` AS `f1`,`zz`.`f11` AS `f11` from (select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) group by `tt`.`f1`) `zz` +select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; +f1 f11 +3 3 +5 5 +mat in merged derived join mat in merged derived +explain extended select * from +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z +on x.f1 = z.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE <derived3> ALL key0 NULL NULL NULL 11 100.00 Using where +1 SIMPLE <derived5> ref key0 key0 5 tt.f1 2 100.00 Using where +5 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11`,`tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` join (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where ((`tt`.`f1` = `tt`.`f1`) and (`tt`.`f1` > 2) and (`tt`.`f1` > 2)) +flush status; +select * from +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z +on x.f1 = z.f1; +f1 f11 f1 f11 +3 3 3 3 +5 5 5 5 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 2 +Handler_read_next 2 +Handler_read_prev 0 +Handler_read_rnd 8 +Handler_read_rnd_next 39 +flush status; +merged in merged derived join merged in merged derived +explain extended select * from +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) z +on x.f1 = z.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` join `test`.`t1` where ((`test`.`t1`.`f1` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7) and (`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7)) +select * from +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) z +on x.f1 = z.f1; +f1 f11 f1 f11 +3 3 3 3 +5 5 5 5 +materialized in materialized derived join +materialized in materialized derived +explain extended select * from +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z +on x.f1 = z.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL key0 NULL NULL NULL 11 100.00 +1 PRIMARY <derived4> ref key0 key0 5 x.f1 2 100.00 Using where +4 DERIVED <derived5> ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +5 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +2 DERIVED <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `x`.`f1` AS `f1`,`x`.`f11` AS `f11`,`z`.`f1` AS `f1`,`z`.`f11` AS `f11` from (select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) group by `tt`.`f1`) `x` join (select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) group by `tt`.`f1`) `z` where (`z`.`f1` = `x`.`f1`) +select * from +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z +on x.f1 = z.f1; +f1 f11 f1 f11 +3 3 3 3 +5 5 5 5 +merged view in materialized derived +explain extended +select * from (select * from v4 group by 1) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f2` AS `f2`,`tt`.`f22` AS `f22` from (select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` where (`test`.`t2`.`f2` in (2,3)) group by 1) `tt` +select * from (select * from v4 group by 1) tt; +f2 f22 +2 2 +3 3 +materialized view in merged derived +explain extended +select * from ( select * from v1 where f1 < 7) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`v1` where (`v1`.`f1` < 7) +select * from ( select * from v1 where f1 < 7) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +merged view in a merged view in a merged derived +create view v6 as select * from v4 where f2 < 7; +explain extended select * from (select * from v6) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` where ((`test`.`t2`.`f2` < 7) and (`test`.`t2`.`f2` in (2,3))) +select * from (select * from v6) tt; +f2 f22 +3 3 +2 2 +materialized view in a merged view in a materialized derived +create view v7 as select * from v1; +explain extended select * from (select * from v7 group by 1) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED <derived4> ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +4 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`v1` group by 1) `tt` +select * from (select * from v7 group by 1) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +7 7 +9 9 +11 11 +13 13 +15 15 +17 17 +19 19 +join of above two +explain extended select * from v6 join v7 on f2=f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE <derived5> ref key0 key0 5 test.t2.f2 2 100.00 Using where +5 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22`,`v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`t2` join `test`.`v1` where ((`v1`.`f1` = `test`.`t2`.`f2`) and (`test`.`t2`.`f2` < 7) and (`test`.`t2`.`f2` in (2,3))) +select * from v6 join v7 on f2=f1; +f2 f22 f1 f11 +3 3 3 3 +2 2 2 2 +test two keys +explain select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 Using where +1 PRIMARY xx ALL NULL NULL NULL NULL 11 Using where; Using join buffer +2 DERIVED t2 ALL NULL NULL NULL NULL 11 Using temporary; Using filesort +select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; +f1 f11 f2 f22 f1 f11 +1 1 1 1 1 1 +2 2 2 2 2 2 +3 3 3 3 3 3 +TODO: Add test with 64 tables mergeable view to test fall back to +materialization on tables > MAX_TABLES merge +drop table t1,t2; +drop view v1,v2,v3,v4,v6,v7; === modified file 'mysql-test/r/explain.result' --- a/mysql-test/r/explain.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/explain.result 2010-05-26 20:18:18 +0000 @@ -102,7 +102,7 @@ INSERT INTO t2 VALUES (),(),(); EXPLAIN SELECT 1 FROM (SELECT 1 FROM t2,t1 WHERE b < c GROUP BY 1 LIMIT 1) AS d2; id select_type table type possible_keys key key_len ref rows Extra -X X X X X X X X X const row not found +X X X X X X X X X X X X X X X X X X X X X X X X X X X Range checked for each record (index map: 0xFFFFFFFFFF) DROP TABLE t2; @@ -114,7 +114,7 @@ INSERT INTO t2 VALUES (1),(2); EXPLAIN EXTENDED SELECT 1 FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: @@ -122,7 +122,7 @@ Note 1003 select 1 AS `1` from (select c EXPLAIN EXTENDED SELECT 1 FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: @@ -132,7 +132,7 @@ prepare s1 from FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1'; execute s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: @@ -142,14 +142,14 @@ prepare s1 from FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1'; execute s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: Note 1003 select 1 AS `1` from (select count(distinct `test`.`t1`.`a`) AS `COUNT(DISTINCT t1.a)` from `test`.`t1` join `test`.`t2` group by `test`.`t1`.`a`) `s1` execute s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: === modified file 'mysql-test/r/func_str.result' --- a/mysql-test/r/func_str.result 2009-12-04 15:36:58 +0000 +++ b/mysql-test/r/func_str.result 2010-05-26 20:18:18 +0000 @@ -2549,14 +2549,12 @@ create table t1(f1 tinyint default null) insert into t1 values (-1),(null); explain select 1 as a from t1,(select decode(f1,f1) as b from t1) a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY t1 ALL NULL NULL NULL NULL 2 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 Using join buffer -2 DERIVED t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 Using join buffer explain select 1 as a from t1,(select encode(f1,f1) as b from t1) a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY t1 ALL NULL NULL NULL NULL 2 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 Using join buffer -2 DERIVED t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 Using join buffer drop table t1; # # Bug#49141: Encode function is significantly slower in 5.1 compared to 5.0 === modified file 'mysql-test/r/index_merge_myisam.result' --- a/mysql-test/r/index_merge_myisam.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/index_merge_myisam.result 2010-05-26 20:18:18 +0000 @@ -285,8 +285,7 @@ id select_type table type possible_keys NULL UNION RESULT <union1,2> ALL NULL NULL NULL NULL NULL explain select * from (select * from t1 where key1 = 3 or key2 =3) as Z where key8 >5; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index_merge i1,i2 i1,i2 4,4 NULL 2 Using union(i1,i2); Using where; Using index +1 SIMPLE t1 ALL i1,i2,i8 NULL NULL NULL 1024 Using where create table t3 like t0; insert into t3 select * from t0; alter table t3 add key9 int not null, add index i9(key9); === modified file 'mysql-test/r/information_schema.result' --- a/mysql-test/r/information_schema.result 2010-03-15 11:51:23 +0000 +++ b/mysql-test/r/information_schema.result 2010-05-26 20:18:18 +0000 @@ -1285,8 +1285,7 @@ id select_type table type possible_keys 1 SIMPLE tables ALL NULL NULL NULL NULL NULL Open_frm_only; Scanned all databases; Using filesort explain select * from (select table_name from information_schema.tables) as a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found -2 DERIVED tables ALL NULL NULL NULL NULL NULL Skip_open_table; Scanned all databases +1 SIMPLE tables ALL NULL NULL NULL NULL NULL Skip_open_table; Scanned all databases drop view v1; create table t1 (f1 int(11)); create table t2 (f1 int(11), f2 int(11)); === modified file 'mysql-test/r/innodb_lock_wait_timeout_1.result' --- a/mysql-test/r/innodb_lock_wait_timeout_1.result 2009-11-12 11:43:33 +0000 +++ b/mysql-test/r/innodb_lock_wait_timeout_1.result 2010-05-26 20:18:18 +0000 @@ -104,7 +104,7 @@ id 1 select_type PRIMARY table <derived2> type ALL -possible_keys NULL +possible_keys key0 key NULL key_len NULL ref NULL @@ -308,7 +308,7 @@ id 1 select_type PRIMARY table <derived2> type ALL -possible_keys NULL +possible_keys key0 key NULL key_len NULL ref NULL === modified file 'mysql-test/r/innodb_mysql.result' --- a/mysql-test/r/innodb_mysql.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/innodb_mysql.result 2010-05-26 20:18:18 +0000 @@ -1731,8 +1731,8 @@ EXPLAIN SELECT 1 FROM (SELECT COUNT(DISTINCT c1) FROM t1 WHERE c2 IN (1, 1) AND c3 = 2 GROUP BY c2) x; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index c3,c2 c2 10 NULL 5 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +2 DERIVED t1 index_merge c3,c2 c3,c2 5,10 NULL 1 Using intersect(c3,c2); Using where; Using filesort DROP TABLE t1; CREATE TABLE t1 (c1 REAL, c2 REAL, c3 REAL, KEY (c3), KEY (c2, c3)) ENGINE=InnoDB; @@ -1745,8 +1745,8 @@ EXPLAIN SELECT 1 FROM (SELECT COUNT(DISTINCT c1) FROM t1 WHERE c2 IN (1, 1) AND c3 = 2 GROUP BY c2) x; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index c3,c2 c2 18 NULL 5 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +2 DERIVED t1 index_merge c3,c2 c3,c2 9,18 NULL 1 Using intersect(c3,c2); Using where; Using filesort DROP TABLE t1; CREATE TABLE t1 (c1 DECIMAL(12,2), c2 DECIMAL(12,2), c3 DECIMAL(12,2), KEY (c3), KEY (c2, c3)) @@ -1760,8 +1760,8 @@ EXPLAIN SELECT 1 FROM (SELECT COUNT(DISTINCT c1) FROM t1 WHERE c2 IN (1, 1) AND c3 = 2 GROUP BY c2) x; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index c3,c2 c2 14 NULL 5 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +2 DERIVED t1 index_merge c3,c2 c3,c2 7,14 NULL 1 Using intersect(c3,c2); Using where; Using filesort DROP TABLE t1; End of 5.1 tests drop table if exists t1, t2, t3; === modified file 'mysql-test/r/lock_multi_bug38499.result' --- a/mysql-test/r/lock_multi_bug38499.result 2009-08-28 21:49:16 +0000 +++ b/mysql-test/r/lock_multi_bug38499.result 2010-05-26 20:18:18 +0000 @@ -2,7 +2,9 @@ SET @odl_sync_frm = @@global.sync_frm; SET @@global.sync_frm = OFF; DROP TABLE IF EXISTS t1; CREATE TABLE t1( a INT, b INT ); +CREATE TABLE t2( a INT, b INT ); INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3), (4, 4); +INSERT INTO t2 VALUES (1, 1), (2, 2), (3, 3), (4, 4); # 1. test regular tables # 1.1. test altering of columns that multiupdate doesn't use # 1.1.1. normal mode @@ -18,5 +20,5 @@ ALTER TABLE t1 ADD COLUMN a INT; # 2.2. test altering of columns that multiupdate uses # 2.2.1. normal mode # 2.2.2. PS mode -DROP TABLE t1; +DROP TABLE t1,t2; SET @@global.sync_frm = @odl_sync_frm; === modified file 'mysql-test/r/myisam_mrr.result' --- a/mysql-test/r/myisam_mrr.result 2010-03-11 21:43:31 +0000 +++ b/mysql-test/r/myisam_mrr.result 2010-05-26 20:18:18 +0000 @@ -347,7 +347,7 @@ GROUP BY t2.pk ); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL NULL Impossible WHERE -2 SUBQUERY t2 ALL int_key int_key 5 3 33.33 Using index condition; Using filesort +2 SUBQUERY t2 ALL int_key int_key 5 const 3 33.33 Using index condition; Using filesort Warnings: Note 1003 select min(`test`.`t1`.`pk`) AS `MIN(t1.pk)` from `test`.`t1` where 0 DROP TABLE t1, t2; === modified file 'mysql-test/r/ps.result' --- a/mysql-test/r/ps.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/ps.result 2010-05-26 20:18:18 +0000 @@ -156,7 +156,6 @@ prepare stmt1 from @stmt ; execute stmt1 ; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables -6 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table 5 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 4 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables @@ -164,7 +163,6 @@ id select_type table type possible_keys execute stmt1 ; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables -6 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table 5 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 4 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables @@ -172,7 +170,6 @@ id select_type table type possible_keys explain SELECT (SELECT SUM(c1 + c12 + 0.0) FROM t2 where (t1.c2 - 0e-3) = t2.c2 GROUP BY t1.c15 LIMIT 1) as scalar_s, exists (select 1.0e+0 from t2 where t2.c3 * 9.0000000000 = t1.c4) as exists_s, c5 * 4 in (select c6 + 0.3e+1 from t2) as in_s, (c7 - 4, c8 - 4) in (select c9 + 4.0, c10 + 40e-1 from t2) as in_row_s FROM t1, (select c25 x, c32 y from t2) tt WHERE x * 1 = c25; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables -6 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table 5 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 4 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables === modified file 'mysql-test/r/ps_ddl.result' --- a/mysql-test/r/ps_ddl.result 2010-01-16 07:44:24 +0000 +++ b/mysql-test/r/ps_ddl.result 2010-05-26 20:18:18 +0000 @@ -1507,12 +1507,12 @@ create view v_27690_1 as select A.a, A.b execute stmt; a b a b 1 1 1 1 -2 2 1 1 -1 1 1 1 -2 2 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 +1 1 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 call p_verify_reprepare_count(1); SUCCESS @@ -1520,12 +1520,12 @@ SUCCESS execute stmt; a b a b 1 1 1 1 -2 2 1 1 -1 1 1 1 -2 2 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 +1 1 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 call p_verify_reprepare_count(0); SUCCESS === modified file 'mysql-test/r/strict.result' --- a/mysql-test/r/strict.result 2009-06-11 16:21:32 +0000 +++ b/mysql-test/r/strict.result 2010-05-26 20:18:18 +0000 @@ -1107,6 +1107,8 @@ Warnings: Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date +Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date +Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date drop table t1; create table t1 (col1 char(3), col2 integer); insert into t1 (col1) values (cast(1000 as char(3))); === modified file 'mysql-test/r/subselect.result' --- a/mysql-test/r/subselect.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect.result 2010-05-26 20:18:18 +0000 @@ -46,13 +46,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -201,11 +201,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -365,9 +364,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM @@ -1339,7 +1338,7 @@ a explain extended select * from t2 where t2.a in (select a from t1); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where (`test`.`t1`.`a` = `test`.`t2`.`a`) select * from t2 where t2.a in (select a from t1 where t1.b <> 30); @@ -1349,7 +1348,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) select * from t2 where t2.a in (select t1.a from t1,t3 where t1.b=t3.a); @@ -1360,7 +1359,7 @@ explain extended select * from t2 where id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index 1 PRIMARY t3 index a a 5 NULL 3 100.00 Using index -1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 116 100.61 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 11 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1` join `test`.`t3`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` = `test`.`t3`.`a`)) insert into t1 values (3,31); @@ -1376,7 +1375,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) drop table t0, t1, t2, t3; === modified file 'mysql-test/r/subselect3.result' --- a/mysql-test/r/subselect3.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3.result 2010-05-26 20:18:18 +0000 @@ -879,7 +879,7 @@ Level Code Message Note 1276 Field or reference 'test.t1.a' of SELECT #3 was resolved in SELECT #2 Note 1276 Field or reference 'test.t1.c' of SELECT #3 was resolved in SELECT #2 Error 1054 Unknown column 'c' in 'field list' -Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `t1`.`c`) AS `(SELECT COUNT(a) FROM +Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `test`.`t1`.`c`) AS `(SELECT COUNT(a) FROM (SELECT COUNT(b) FROM t1) AS x GROUP BY c )` from `test`.`t1` group by `test`.`t1`.`b`) `y` DROP TABLE t1; @@ -1105,9 +1105,8 @@ a set @@optimizer_switch=default; explain select * from (select a from t0) X where a in (select a from t1); id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 -1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(<derived2>) -2 DERIVED t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(t0) drop table t0, t1; create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); === modified file 'mysql-test/r/subselect3_jcl6.result' --- a/mysql-test/r/subselect3_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3_jcl6.result 2010-05-26 20:18:18 +0000 @@ -883,7 +883,7 @@ Level Code Message Note 1276 Field or reference 'test.t1.a' of SELECT #3 was resolved in SELECT #2 Note 1276 Field or reference 'test.t1.c' of SELECT #3 was resolved in SELECT #2 Error 1054 Unknown column 'c' in 'field list' -Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `t1`.`c`) AS `(SELECT COUNT(a) FROM +Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `test`.`t1`.`c`) AS `(SELECT COUNT(a) FROM (SELECT COUNT(b) FROM t1) AS x GROUP BY c )` from `test`.`t1` group by `test`.`t1`.`b`) `y` DROP TABLE t1; @@ -1110,9 +1110,8 @@ a set @@optimizer_switch=default; explain select * from (select a from t0) X where a in (select a from t1); id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 -1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(<derived2>); Using join buffer -2 DERIVED t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(t0); Using join buffer drop table t0, t1; create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); === modified file 'mysql-test/r/subselect_no_mat.result' --- a/mysql-test/r/subselect_no_mat.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_mat.result 2010-05-26 20:18:18 +0000 @@ -50,13 +50,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -205,11 +205,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -369,9 +368,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM @@ -1343,7 +1342,7 @@ a explain extended select * from t2 where t2.a in (select a from t1); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where (`test`.`t1`.`a` = `test`.`t2`.`a`) select * from t2 where t2.a in (select a from t1 where t1.b <> 30); @@ -1353,7 +1352,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) select * from t2 where t2.a in (select t1.a from t1,t3 where t1.b=t3.a); @@ -1364,7 +1363,7 @@ explain extended select * from t2 where id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index 1 PRIMARY t3 index a a 5 NULL 3 100.00 Using index -1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 116 100.61 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 11 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1` join `test`.`t3`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` = `test`.`t3`.`a`)) insert into t1 values (3,31); @@ -1380,7 +1379,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) drop table t0, t1, t2, t3; === modified file 'mysql-test/r/subselect_no_opts.result' --- a/mysql-test/r/subselect_no_opts.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_opts.result 2010-05-26 20:18:18 +0000 @@ -50,13 +50,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -205,11 +205,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -369,9 +368,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM === modified file 'mysql-test/r/subselect_no_semijoin.result' --- a/mysql-test/r/subselect_no_semijoin.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_semijoin.result 2010-05-26 20:18:18 +0000 @@ -50,13 +50,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -205,11 +205,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -369,9 +368,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM === modified file 'mysql-test/r/table_elim.result' --- a/mysql-test/r/table_elim.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/table_elim.result 2010-05-26 20:18:18 +0000 @@ -117,58 +117,58 @@ t2 where id=f.id); This should use one table: explain select id from v1 where id=2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY f const PRIMARY PRIMARY 4 const 1 Using index +1 SIMPLE f const PRIMARY PRIMARY 4 const 1 Using index This should use one table: explain extended select id from v1 where id in (1,2,3,4); id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index +1 SIMPLE f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index Warnings: -Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` where (`f`.`id` in (1,2,3,4)) This should use facts and a1 tables: explain extended select id from v1 where attr1 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index +1 SIMPLE a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index Warnings: -Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t1` `a1` where ((`f`.`id` = `a1`.`id`) and (`a1`.`attr1` between 12 and 14)) This should use facts, a2 and its subquery: explain extended select id from v1 where attr2 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using where; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using index +1 SIMPLE a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using where; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using index 3 DEPENDENT SUBQUERY t2 ref PRIMARY PRIMARY 4 test.a2.id 2 100.00 Using index Warnings: -Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t2` `a2` where ((`f`.`id` = `a2`.`id`) and (`a2`.`attr2` between 12 and 14) and (`a2`.`fromdate` = (select max(`test`.`t2`.`fromdate`) AS `MAX(fromdate)` from `test`.`t2` where (`test`.`t2`.`id` = `a2`.`id`)))) This should use one table: explain select id from v2 where id=2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY f const PRIMARY PRIMARY 4 const 1 Using index +1 SIMPLE f const PRIMARY PRIMARY 4 const 1 Using index This should use one table: explain extended select id from v2 where id in (1,2,3,4); id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index +1 SIMPLE f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index Warnings: -Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` where (`f`.`id` in (1,2,3,4)) This should use facts and a1 tables: explain extended select id from v2 where attr1 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index +1 SIMPLE a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index Warnings: -Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t1` `a1` where ((`f`.`id` = `a1`.`id`) and (`a1`.`attr1` between 12 and 14)) This should use facts, a2 and its subquery: explain extended select id from v2 where attr2 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using where; Using index +1 SIMPLE a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using where; Using index 3 DEPENDENT SUBQUERY t2 ref PRIMARY PRIMARY 4 test.f.id 2 100.00 Using index Warnings: -Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t2` `a2` where ((`f`.`id` = `a2`.`id`) and (`a2`.`attr2` between 12 and 14) and (`a2`.`fromdate` = (select max(`test`.`t2`.`fromdate`) AS `MAX(fromdate)` from `test`.`t2` where (`test`.`t2`.`id` = `f`.`id`)))) drop view v1, v2; drop table t0, t1, t2; === modified file 'mysql-test/r/view.result' --- a/mysql-test/r/view.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/view.result 2010-05-26 20:18:18 +0000 @@ -117,7 +117,7 @@ c 12 explain extended select c from v5; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> ALL NULL NULL NULL NULL 5 100.00 +1 SIMPLE <derived3> ALL NULL NULL NULL NULL 5 100.00 3 DERIVED t1 ALL NULL NULL NULL NULL 5 100.00 Warnings: Note 1003 select (`v2`.`c` + 1) AS `c` from `test`.`v2` @@ -237,7 +237,7 @@ a 3 explain select * from v1; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 3 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 6 2 DERIVED t1 ALL NULL NULL NULL NULL 6 Using temporary select * from t1; a @@ -302,7 +302,7 @@ a+1 4 explain select * from v1; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 2 DERIVED t1 ALL NULL NULL NULL NULL 4 Using filesort drop view v1; drop table t1; === modified file 'mysql-test/r/view_grant.result' --- a/mysql-test/r/view_grant.result 2009-10-16 11:12:21 +0000 +++ b/mysql-test/r/view_grant.result 2010-05-26 20:18:18 +0000 @@ -110,7 +110,7 @@ show create view mysqltest.v1; ERROR 42000: SHOW VIEW command denied to user 'mysqltest_1'@'localhost' for table 'v1' explain select c from mysqltest.v2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 2 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table show create view mysqltest.v2; ERROR 42000: SHOW VIEW command denied to user 'mysqltest_1'@'localhost' for table 'v2' @@ -131,7 +131,7 @@ View Create View character_set_client co v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `mysqltest`.`v1` AS select (`mysqltest`.`t1`.`a` + 1) AS `c`,(`mysqltest`.`t1`.`b` + 1) AS `d` from `mysqltest`.`t1` latin1 latin1_swedish_ci explain select c from mysqltest.v2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 2 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table show create view mysqltest.v2; View Create View character_set_client collation_connection @@ -144,7 +144,7 @@ View Create View character_set_client co v3 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `mysqltest`.`v3` AS select (`mysqltest`.`t2`.`a` + 1) AS `c`,(`mysqltest`.`t2`.`b` + 1) AS `d` from `mysqltest`.`t2` latin1 latin1_swedish_ci explain select c from mysqltest.v4; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 2 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table show create view mysqltest.v4; View Create View character_set_client collation_connection === added file 'mysql-test/t/derived_view.test' --- a/mysql-test/t/derived_view.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/derived_view.test 2010-05-26 20:18:18 +0000 @@ -0,0 +1,217 @@ +--disable_warnings +drop table if exists t1,t2; +drop view if exists v1,v2,v3,v4; +--enable_warnings +create table t1(f1 int, f11 int); +create table t2(f2 int, f22 int); +insert into t1 values(1,1),(2,2),(3,3),(5,5),(9,9),(7,7); +insert into t1 values(17,17),(13,13),(11,11),(15,15),(19,19); +insert into t2 values(1,1),(3,3),(2,2),(4,4),(8,8),(6,6); +insert into t2 values(12,12),(14,14),(10,10),(18,18),(16,16); + +--echo Tests: + +--echo for merged derived tables +--echo explain for simple derived +explain select * from (select * from t1) tt; +select * from (select * from t1) tt; +--echo explain for multitable derived +explain extended select * from (select * from t1 join t2 on f1=f2) tt; +select * from (select * from t1 join t2 on f1=f2) tt; +--echo explain for derived with where +explain extended + select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +--echo join of derived +explain extended + select * from (select * from t1 where f1 in (2,3)) tt join + (select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; +select * from (select * from t1 where f1 in (2,3)) tt join + (select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; + +flush status; +explain extended + select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +show status like 'Handler_read%'; +flush status; +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +show status like 'Handler_read%'; + +--echo for merged views +create view v1 as select * from t1; +create view v2 as select * from t1 join t2 on f1=f2; +create view v3 as select * from t1 where f1 in (2,3); +create view v4 as select * from t2 where f2 in (2,3); +--echo explain for simple views +explain extended select * from v1; +select * from v1; +--echo explain for multitable views +explain extended select * from v2; +select * from v2; +--echo explain for views with where +explain extended select * from v3 where f11 in (1,3); +select * from v3 where f11 in (1,3); +--echo explain for joined views +explain extended + select * from v3 join v4 on f1=f2; +select * from v3 join v4 on f1=f2; + +flush status; +explain extended select * from v4 where f2 in (1,3); +show status like 'Handler_read%'; +flush status; +select * from v4 where f2 in (1,3); +show status like 'Handler_read%'; + +--echo for materialized derived tables +--echo explain for simple derived +explain extended select * from (select * from t1 group by f1) tt; +select * from (select * from t1 having f1=f1) tt; +--echo explain showing created indexes +explain extended + select * from t1 join (select * from t2 group by f2) tt on f1=f2; +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +--echo explain showing late materialization +flush status; +explain select * from t1 join (select * from t2 group by f2) tt on f1=f2; +show status like 'Handler_read%'; +flush status; +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +show status like 'Handler_read%'; + +--echo for materialized views +drop view v1,v2,v3; +create view v1 as select * from t1 group by f1; +create view v2 as select * from t2 group by f2; +create view v3 as select t1.f1,t1.f11 from t1 join t1 as t11 where t1.f1=t11.f1 + having t1.f1<100; +--echo explain for simple derived +explain extended select * from v1; +select * from v1; +--echo explain showing created indexes +explain extended select * from t1 join v2 on f1=f2; +select * from t1 join v2 on f1=f2; +explain extended + select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +flush status; +select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +show status like 'Handler_read%'; +--echo explain showing late materialization +flush status; +explain select * from t1 join v2 on f1=f2; +show status like 'Handler_read%'; +flush status; +select * from t1 join v2 on f1=f2; +show status like 'Handler_read%'; + +explain extended select * from v1 join v4 on f1=f2; +select * from v1 join v4 on f1=f2; + +--echo merged derived in merged derived +explain extended select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2) zz; +select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2) zz; + +--echo materialized derived in merged derived +explain extended select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; +select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; + +--echo merged derived in materialized derived +explain extended select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; +select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; + +--echo materialized derived in materialized derived +explain extended select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; +select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; + +--echo mat in merged derived join mat in merged derived +explain extended select * from + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z + on x.f1 = z.f1; + +flush status; +select * from + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z + on x.f1 = z.f1; +show status like 'Handler_read%'; +flush status; + +--echo merged in merged derived join merged in merged derived +explain extended select * from + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) z + on x.f1 = z.f1; + +select * from + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) z + on x.f1 = z.f1; + +--echo materialized in materialized derived join +--echo materialized in materialized derived +explain extended select * from + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z + on x.f1 = z.f1; + +select * from + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z + on x.f1 = z.f1; + +--echo merged view in materialized derived +explain extended +select * from (select * from v4 group by 1) tt; +select * from (select * from v4 group by 1) tt; + +--echo materialized view in merged derived +explain extended +select * from ( select * from v1 where f1 < 7) tt; +select * from ( select * from v1 where f1 < 7) tt; + +--echo merged view in a merged view in a merged derived +create view v6 as select * from v4 where f2 < 7; +explain extended select * from (select * from v6) tt; +select * from (select * from v6) tt; + +--echo materialized view in a merged view in a materialized derived +create view v7 as select * from v1; +explain extended select * from (select * from v7 group by 1) tt; +select * from (select * from v7 group by 1) tt; + +--echo join of above two +explain extended select * from v6 join v7 on f2=f1; +select * from v6 join v7 on f2=f1; + +--echo test two keys +explain select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; +select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; + + +--echo TODO: Add test with 64 tables mergeable view to test fall back to +--echo materialization on tables > MAX_TABLES merge +drop table t1,t2; +drop view v1,v2,v3,v4,v6,v7; === modified file 'mysql-test/t/lock_multi_bug38499.test' --- a/mysql-test/t/lock_multi_bug38499.test 2009-08-28 21:49:16 +0000 +++ b/mysql-test/t/lock_multi_bug38499.test 2010-05-26 20:18:18 +0000 @@ -16,7 +16,9 @@ connect (writer,localhost,root,,); DROP TABLE IF EXISTS t1; --enable_warnings CREATE TABLE t1( a INT, b INT ); +CREATE TABLE t2( a INT, b INT ); INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3), (4, 4); +INSERT INTO t2 VALUES (1, 1), (2, 2), (3, 3), (4, 4); --echo # 1. test regular tables --echo # 1.1. test altering of columns that multiupdate doesn't use @@ -28,7 +30,7 @@ while ($i) { --dec $i --connection writer - send UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0; + send UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0; --connection locker ALTER TABLE t1 ADD COLUMN (c INT); @@ -41,7 +43,7 @@ while ($i) { --echo # 1.1.2. PS mode --connection writer -PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0'; +PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0'; let $i = 100; while ($i) { @@ -75,7 +77,7 @@ while ($i) { UPDATE t1 SET a=b; --connection writer ---send UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0; +--send UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0; --connection locker --error 0,ER_CANT_DROP_FIELD_OR_KEY @@ -100,7 +102,7 @@ while ($i) { UPDATE t1 SET a=b; --connection writer - PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0'; + PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0'; --send EXECUTE stmt --connection locker @@ -210,7 +212,7 @@ while ($i) { } --enable_query_log --connection default -DROP TABLE t1; +DROP TABLE t1,t2; # Close connections === modified file 'sql/field.cc' --- a/sql/field.cc 2010-03-20 12:01:47 +0000 +++ b/sql/field.cc 2010-05-26 20:18:18 +0000 @@ -10458,3 +10458,27 @@ Field::set_datetime_warning(MYSQL_ERROR: field_name); } } + + +/* + @brief + Return possible keys for a field + + @details + Return bit map of keys over this field which can be used by the range + optimizer. For a field of a generic table such keys are all keys that starts + from this field. For a field of a materialized derived table/view such keys + are all keys in which this field takes a part. This is less restrictive as + keys for a materialized derived table/view are generated on the fly from + present fields, thus the case when a field for the beginning of a key is + absent is impossible. + + @return map of possible keys +*/ + +key_map Field::get_possible_keys() +{ + DBUG_ASSERT(table->pos_in_table_list); + return (table->pos_in_table_list->is_materialized_derived() ? + part_of_key : key_start); +} === modified file 'sql/field.h' --- a/sql/field.h 2010-03-15 11:51:23 +0000 +++ b/sql/field.h 2010-05-26 20:18:18 +0000 @@ -582,6 +582,9 @@ public: DBUG_ASSERT(0); return GEOM_GEOMETRY; } + + key_map get_possible_keys(); + /* Hash value */ virtual void hash(ulong *nr, ulong *nr2); friend bool reopen_table(THD *,struct st_table *,bool); === modified file 'sql/handler.cc' --- a/sql/handler.cc 2010-03-20 12:01:47 +0000 +++ b/sql/handler.cc 2010-05-26 20:18:18 +0000 @@ -2480,8 +2480,9 @@ int handler::update_auto_increment() void handler::column_bitmaps_signal() { DBUG_ENTER("column_bitmaps_signal"); - DBUG_PRINT("info", ("read_set: 0x%lx write_set: 0x%lx", (long) table->read_set, - (long) table->write_set)); + if (table) + DBUG_PRINT("info", ("read_set: 0x%lx write_set: 0x%lx", + (long) table->read_set, (long) table->write_set)); DBUG_VOID_RETURN; } === modified file 'sql/item.cc' --- a/sql/item.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item.cc 2010-05-26 20:18:18 +0000 @@ -711,6 +711,23 @@ bool Item_field::register_field_in_bitma return 0; } + +/* + Mark field in write_map + + NOTES + This is used by UPDATE to register underlying fields of used view fields. +*/ + +bool Item_field::register_field_in_write_map(uchar *arg) +{ + TABLE *table= (TABLE *) arg; + if (field->table == table || !table) + bitmap_set_bit(field->table->write_set, field->field_index); + return 0; +} + + bool Item::check_cols(uint c) { if (c != 1) @@ -2202,6 +2219,10 @@ table_map Item_field::used_tables() cons return (depended_from ? OUTER_REF_TABLE_BIT : field->table->map); } +table_map Item_field::all_used_tables() const +{ + return (depended_from ? OUTER_REF_TABLE_BIT : field->table->map); +} void Item_field::fix_after_pullout(st_select_lex *new_parent, Item **ref) { @@ -2454,7 +2475,7 @@ my_decimal *Item_float::val_decimal(my_d void Item_string::print(String *str, enum_query_type query_type) { - if (query_type == QT_ORDINARY && is_cs_specified()) + if (query_type != QT_IS && is_cs_specified()) { str->append('_'); str->append(collation.collation->csname); @@ -2462,7 +2483,7 @@ void Item_string::print(String *str, enu str->append('\''); - if (query_type == QT_ORDINARY || + if (query_type != QT_IS || my_charset_same(str_value.charset(), system_charset_info)) { str_value.print(str); @@ -3945,6 +3966,34 @@ resolve_ref_in_select_and_group(THD *thd } +/* + @brief + Whether a table belongs to an outer select. + + @param table table to check + @param select current select + + @details + Try to find select the table belongs to by ascending the derived tables chain. +*/ + +static +bool is_outer_table(TABLE_LIST *table, SELECT_LEX *select) +{ + DBUG_ASSERT(table->select_lex != select); + TABLE_LIST *tl; + + for (tl= select->master_unit()->derived; + tl && tl->is_merged_derived(); + select= tl->select_lex, tl= select->master_unit()->derived) + { + if (tl->select_lex == table->select_lex) + return FALSE; + } + return TRUE; +} + + /** Resolve the name of an outer select column reference. @@ -4382,7 +4431,8 @@ bool Item_field::fix_fields(THD *thd, It if (!outer_fixed && cached_table && cached_table->select_lex && context->select_lex && - cached_table->select_lex != context->select_lex) + cached_table->select_lex != context->select_lex && + is_outer_table(cached_table, context->select_lex)) { int ret; if ((ret= fix_outer_field(thd, &from_field, reference)) < 0) @@ -5786,8 +5836,9 @@ public: st_select_lex *sel; for (sel= current_select; sel; sel= sel->outer_select()) { + List_iterator<TABLE_LIST> li(sel->leaf_tables); TABLE_LIST *tbl; - for (tbl= sel->leaf_tables; tbl; tbl= tbl->next_leaf) + while ((tbl= li++)) { if (tbl->table == item->field->table) { @@ -7506,6 +7557,8 @@ Item_result Item_type_holder::result_typ enum_field_types Item_type_holder::get_real_type(Item *item) { + if (item->type() == REF_ITEM) + item= item->real_item(); switch(item->type()) { case FIELD_ITEM: === modified file 'sql/item.h' --- a/sql/item.h 2010-03-20 12:01:47 +0000 +++ b/sql/item.h 2010-05-26 20:18:18 +0000 @@ -778,6 +778,7 @@ public: class Field_enumerator) */ virtual table_map used_tables() const { return (table_map) 0L; } + virtual table_map all_used_tables() const { return used_tables(); } /* Return table map of tables that can't be NULL tables (tables that are used in a context where if they would contain a NULL row generated @@ -934,8 +935,12 @@ public: virtual bool reset_query_id_processor(uchar *query_id_arg) { return 0; } virtual bool is_expensive_processor(uchar *arg) { return 0; } virtual bool register_field_in_read_map(uchar *arg) { return 0; } + virtual bool register_field_in_write_map(uchar *arg) { return 0; } virtual bool enumerate_field_refs_processor(uchar *arg) { return 0; } virtual bool mark_as_eliminated_processor(uchar *arg) { return 0; } + virtual bool view_used_tables_processor(uchar *arg) { return 0; } + virtual bool eval_not_null_tables(uchar *opt_arg) { return 0; } + /* The next function differs from the previous one that a bitmap to be updated is passed as uchar *arg. @@ -1143,6 +1148,12 @@ public: { return Field::GEOM_GEOMETRY; }; String *check_well_formed_result(String *str, bool send_error= 0); bool eq_by_collation(Item *item, bool binary_cmp, CHARSET_INFO *cs); + table_map view_used_tables(TABLE_LIST *view) + { + view->view_used_tables= 0; + walk(&Item::view_used_tables_processor, 0, (uchar *) view); + return view->view_used_tables; + } }; @@ -1616,6 +1627,7 @@ public: int save_in_field(Field *field,bool no_conversions); void save_org_in_field(Field *field); table_map used_tables() const; + table_map all_used_tables() const; enum Item_result result_type () const { return field->result_type(); @@ -1645,6 +1657,7 @@ public: bool add_field_to_set_processor(uchar * arg); bool find_item_in_field_list_processor(uchar *arg); bool register_field_in_read_map(uchar *arg); + bool register_field_in_write_map(uchar *arg); bool register_field_in_bitmap(uchar *arg); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} bool vcol_in_partition_func_processor(uchar *bool_arg); @@ -2515,11 +2528,14 @@ public: */ class Item_direct_view_ref :public Item_direct_ref { + TABLE_LIST *view; public: Item_direct_view_ref(Name_resolution_context *context_arg, Item **item, - const char *table_name_arg, - const char *field_name_arg) - :Item_direct_ref(context_arg, item, table_name_arg, field_name_arg) {} + const char *table_name_arg, + const char *field_name_arg, + TABLE_LIST *view_arg) + :Item_direct_ref(context_arg, item, table_name_arg, field_name_arg), + view(view_arg) {} /* Constructor need to process subselect with temporary tables (see Item) */ Item_direct_view_ref(THD *thd, Item_direct_ref *item) :Item_direct_ref(thd, item) {} @@ -2533,6 +2549,24 @@ public: return item; } virtual Ref_Type ref_type() { return VIEW_REF; } + table_map used_tables() const + { + return depended_from ? + OUTER_REF_TABLE_BIT : + (view->merged ? (*ref)->used_tables() : view->table->map); + } + bool walk(Item_processor processor, bool walk_subquery, uchar *arg) + { + return (*ref)->walk(processor, walk_subquery, arg) || + (this->*processor)(arg); + } + bool view_used_tables_processor(uchar *arg) + { + TABLE_LIST *view_arg= (TABLE_LIST *) arg; + if (view_arg == view) + view_arg->view_used_tables|= (*ref)->used_tables(); + return 0; + } }; @@ -2885,6 +2919,17 @@ public: value. */ +/* + Cached_item_XXX objects are not exactly caches. They do the following: + + Each Cached_item_XXX object has + - its source item + - saved value of the source item + - cmp() method that compares the saved value with the current value of the + source item, and if they were not equal saves item's value into the saved + value. +*/ + class Cached_item :public Sql_alloc { public: === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.cc 2010-05-26 20:18:18 +0000 @@ -2204,6 +2204,16 @@ bool Item_func_between::fix_fields(THD * thd->lex->current_select->between_count++; + + return 0; +} + + +bool Item_func_between::eval_not_null_tables(uchar *opt_arg) +{ + if (Item_func_opt_neg::eval_not_null_tables(NULL)) + return 1; + /* not_null_tables_cache == union(T1(e),T1(e1),T1(e2)) */ if (pred_level && !negated) return 0; @@ -2212,9 +2222,8 @@ bool Item_func_between::fix_fields(THD * not_null_tables_cache= (args[0]->not_null_tables() | (args[1]->not_null_tables() & args[2]->not_null_tables())); - return 0; -} +} void Item_func_between::fix_length_and_dec() @@ -2575,13 +2584,22 @@ Item_func_if::fix_fields(THD *thd, Item if (Item_func::fix_fields(thd, ref)) return 1; + return 0; +} + + +bool +Item_func_if::eval_not_null_tables(uchar *opt_arg) +{ + if (Item_func::eval_not_null_tables(NULL)) + return 1; + not_null_tables_cache= (args[1]->not_null_tables() & args[2]->not_null_tables()); return 0; } - void Item_func_if::fix_length_and_dec() { @@ -3761,11 +3779,22 @@ bool Item_func_in::nulls_in_row() bool Item_func_in::fix_fields(THD *thd, Item **ref) { - Item **arg, **arg_end; if (Item_func_opt_neg::fix_fields(thd, ref)) return 1; + return 0; +} + + +bool +Item_func_in::eval_not_null_tables(uchar *opt_arg) +{ + Item **arg, **arg_end; + + if (Item_func_opt_neg::eval_not_null_tables(NULL)) + return 1; + /* not_null_tables_cache == union(T1(e),union(T1(ei))) */ if (pred_level && negated) return 0; @@ -4186,7 +4215,6 @@ Item_cond::fix_fields(THD *thd, Item **r */ while ((item=li++)) { - table_map tmp_table_map; while (item->type() == Item::COND_ITEM && ((Item_cond*) item)->functype() == functype() && !((Item_cond*) item)->list.is_empty()) @@ -4208,11 +4236,12 @@ Item_cond::fix_fields(THD *thd, Item **r and_tables_cache= (table_map) 0; else { - tmp_table_map= item->not_null_tables(); + table_map tmp_table_map= item->not_null_tables(); not_null_tables_cache|= tmp_table_map; and_tables_cache&= tmp_table_map; const_item_cache= FALSE; - } + } + with_sum_func= with_sum_func || item->with_sum_func; with_subselect|= item->with_subselect; if (item->maybe_null) @@ -4226,6 +4255,28 @@ Item_cond::fix_fields(THD *thd, Item **r } +bool +Item_cond::eval_not_null_tables(uchar *opt_arg) +{ + Item *item; + List_iterator<Item> li(list); + and_tables_cache= ~(table_map) 0; + while ((item=li++)) + { + table_map tmp_table_map; + if (item->const_item()) + and_tables_cache= (table_map) 0; + else + { + tmp_table_map= item->not_null_tables(); + not_null_tables_cache|= tmp_table_map; + and_tables_cache&= tmp_table_map; + } + } + return 0; +} + + void Item_cond::fix_after_pullout(st_select_lex *new_parent, Item **ref) { List_iterator<Item> li(list); === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.h 2010-05-26 20:18:18 +0000 @@ -631,6 +631,7 @@ public: bool is_bool_func() { return 1; } CHARSET_INFO *compare_collation() { return cmp_collation.collation; } uint decimal_precision() const { return 1; } + bool eval_not_null_tables(uchar *opt_arg); }; @@ -730,6 +731,7 @@ public: void fix_length_and_dec(); uint decimal_precision() const; const char *func_name() const { return "if"; } + bool eval_not_null_tables(uchar *opt_arg); }; @@ -1256,6 +1258,7 @@ public: bool nulls_in_row(); bool is_bool_func() { return 1; } CHARSET_INFO *compare_collation() { return cmp_collation.collation; } + bool eval_not_null_tables(uchar *opt_arg); }; class cmp_item_row :public cmp_item @@ -1510,6 +1513,7 @@ public: bool subst_argument_checker(uchar **arg) { return TRUE; } Item *compile(Item_analyzer analyzer, uchar **arg_p, Item_transformer transformer, uchar *arg_t); + bool eval_not_null_tables(uchar *opt_arg); }; === modified file 'sql/item_func.cc' --- a/sql/item_func.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_func.cc 2010-05-26 20:18:18 +0000 @@ -192,7 +192,6 @@ Item_func::fix_fields(THD *thd, Item **r with_sum_func= with_sum_func || item->with_sum_func; used_tables_cache|= item->used_tables(); - not_null_tables_cache|= item->not_null_tables(); const_item_cache&= item->const_item(); with_subselect|= item->with_subselect; } @@ -206,6 +205,21 @@ Item_func::fix_fields(THD *thd, Item **r } +bool +Item_func::eval_not_null_tables(uchar *opt_arg) +{ + Item **arg,**arg_end; + if (arg_count) + { + for (arg=args, arg_end=args+arg_count; arg != arg_end ; arg++) + { + not_null_tables_cache|= (*arg)->not_null_tables(); + } + } + return FALSE; +} + + void Item_func::fix_after_pullout(st_select_lex *new_parent, Item **ref) { Item **arg,**arg_end; @@ -3895,6 +3909,20 @@ bool Item_func_set_user_var::fix_fields( entry->collation.set(args[0]->collation.collation, DERIVATION_IMPLICIT); collation.set(entry->collation.collation, DERIVATION_IMPLICIT); cached_result_type= args[0]->result_type(); + { + /* + When this function is used in a derived table/view force the derived + table to be materialized to preserve possible side-effect of setting a + user variable. + */ + SELECT_LEX_UNIT *unit= thd->lex->current_select->master_unit(); + TABLE_LIST *derived; + for (derived= unit->derived; + derived; + derived= derived->select_lex->master_unit()->derived) + derived->set_materialized_derived(); + } + return FALSE; } === modified file 'sql/item_func.h' --- a/sql/item_func.h 2010-03-20 12:01:47 +0000 +++ b/sql/item_func.h 2010-05-26 20:18:18 +0000 @@ -181,6 +181,7 @@ public: Item_transformer transformer, uchar *arg_t); void traverse_cond(Cond_traverser traverser, void * arg, traverse_order order); + bool eval_not_null_tables(uchar *opt_arg); // bool is_expensive_processor(uchar *arg); // virtual bool is_expensive() { return 0; } inline double fix_result(double value) @@ -1617,14 +1618,7 @@ public: void fix_length_and_dec() { decimals=0; max_length=1; maybe_null=1;} bool check_vcol_func_processor(uchar *int_arg) { -#if 0 - DBUG_ENTER("Item_func_is_free_lock::check_vcol_func_processor"); - DBUG_PRINT("info", - ("check_vcol_func_processor returns TRUE: unsupported function")); - DBUG_RETURN(TRUE); -#else return trace_unsupported_by_check_vcol_func_processor(func_name()); -#endif } }; === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.cc 2010-05-26 20:18:18 +0000 @@ -2894,6 +2894,9 @@ int subselect_uniquesubquery_engine::exe DBUG_RETURN(0); } + if (!tab->preread_init_done && tab->preread_init()) + DBUG_RETURN(1); + if (null_keypart) DBUG_RETURN(scan_table()); @@ -3026,7 +3029,7 @@ subselect_uniquesubquery_engine::~subsel int subselect_indexsubquery_engine::exec() { - DBUG_ENTER("subselect_indexsubquery_engine::exec"); + DBUG_ENTER("subselect_indexsubquery_engine"); int error; bool null_finding= 0; TABLE *table= tab->table; @@ -3057,6 +3060,9 @@ int subselect_indexsubquery_engine::exec DBUG_RETURN(0); } + if (!tab->preread_init_done && tab->preread_init()) + DBUG_RETURN(1); + if (null_keypart) DBUG_RETURN(scan_table()); @@ -3158,10 +3164,13 @@ void subselect_uniquesubquery_engine::ex } -table_map subselect_engine::calc_const_tables(TABLE_LIST *table) +table_map subselect_engine::calc_const_tables(List<TABLE_LIST> &list) { table_map map= 0; - for (; table; table= table->next_leaf) + List_iterator<TABLE_LIST> ti(list); + TABLE_LIST *table; + //for (; table; table= table->next_leaf) + while ((table= ti++)) { TABLE *tbl= table->table; if (tbl && tbl->const_table) @@ -3173,14 +3182,13 @@ table_map subselect_engine::calc_const_t table_map subselect_single_select_engine::upper_select_const_tables() { - return calc_const_tables((TABLE_LIST *) select_lex->outer_select()-> - leaf_tables); + return calc_const_tables(select_lex->outer_select()->leaf_tables); } table_map subselect_union_engine::upper_select_const_tables() { - return calc_const_tables((TABLE_LIST *) unit->outer_select()->leaf_tables); + return calc_const_tables(unit->outer_select()->leaf_tables); } @@ -3711,7 +3719,7 @@ bool subselect_hash_sj_engine::init_perm if (((select_union*) result)->create_result_table( thd, tmp_columns, TRUE, tmp_create_options, - "materialized subselect", TRUE)) + "materialized subselect", TRUE, TRUE)) DBUG_RETURN(TRUE); tmp_table= ((select_union*) result)->table; === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.h 2010-05-26 20:18:18 +0000 @@ -531,6 +531,7 @@ public: virtual bool may_be_null() { return maybe_null; }; virtual table_map upper_select_const_tables()= 0; static table_map calc_const_tables(TABLE_LIST *); + static table_map calc_const_tables(List<TABLE_LIST> &list); virtual void print(String *str, enum_query_type query_type)= 0; virtual bool change_result(Item_subselect *si, select_result_interceptor *result)= 0; === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-03-20 12:01:47 +0000 +++ b/sql/mysql_priv.h 2010-05-26 20:18:18 +0000 @@ -62,12 +62,15 @@ class Parser_state; QT_ORDINARY -- ordinary SQL query. QT_IS -- SQL query to be shown in INFORMATION_SCHEMA (in utf8 and without - character set introducers). + character set introducers). + QT_VIEW_INTERNAL -- view internal representation (like QT_ORDINARY except + ORDER BY clause) */ enum enum_query_type { QT_ORDINARY, - QT_IS + QT_IS, + QT_VIEW_INTERNAL }; /* TODO convert all these three maps to Bitmap classes */ @@ -511,7 +514,6 @@ protected: #define OPTION_PROFILING (ULL(1) << 33) - /** Maximum length of time zone name that we support (Time zone name is char(64) in db). mysqlbinlog needs it. @@ -1276,11 +1278,9 @@ int mysql_explain_select(THD *thd, SELEC select_result *result); bool mysql_union(THD *thd, LEX *lex, select_result *result, SELECT_LEX_UNIT *unit, ulong setup_tables_done_option); -bool mysql_handle_derived(LEX *lex, bool (*processor)(THD *thd, - LEX *lex, - TABLE_LIST *table)); -bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *t); -bool mysql_derived_filling(THD *thd, LEX *lex, TABLE_LIST *t); +bool mysql_handle_derived(LEX *lex, uint phases); +bool mysql_handle_single_derived(LEX *lex, TABLE_LIST *derived, uint phases); +bool mysql_handle_list_of_derived(LEX *lex, TABLE_LIST *dt_list, uint phases); Field *create_tmp_field(THD *thd, TABLE *table,Item *item, Item::Type type, Item ***copy_func, Field **from_field, Field **def_field, @@ -1288,6 +1288,17 @@ Field *create_tmp_field(THD *thd, TABLE bool table_cant_handle_bit_fields, bool make_copy_field, uint convert_blob_length); +bool open_tmp_table(TABLE *table); +#if defined(WITH_MARIA_STORAGE_ENGINE) && defined(USE_MARIA_FOR_TMP_TABLES) +#define TMP_ENGINE_HTON maria_hton +#else +#define TMP_ENGINE_HTON myisam_hton +#endif +bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, + ENGINE_COLUMNDEF *start_recinfo, + ENGINE_COLUMNDEF **recinfo, + ulonglong options); + void sp_prepare_create_field(THD *thd, Create_field *sql_field); int prepare_create_field(Create_field *sql_field, uint *blob_columns, @@ -1600,17 +1611,21 @@ bool get_key_map_from_key_list(key_map * bool insert_fields(THD *thd, Name_resolution_context *context, const char *db_name, const char *table_name, List_iterator<Item> *it, bool any_privileges); +void make_leaves_list(List<TABLE_LIST> &list, TABLE_LIST *tables, + bool full_table_list, TABLE_LIST *boundary); bool setup_tables(THD *thd, Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, bool select_insert); + List<TABLE_LIST> &leaves, bool select_insert, + bool full_table_list); bool setup_tables_and_check_access(THD *thd, Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, + List<TABLE_LIST> &leaves, bool select_insert, ulong want_access_first, - ulong want_access); + ulong want_access, + bool full_table_list); int setup_wild(THD *thd, TABLE_LIST *tables, List<Item> &fields, List<Item> *sum_func_list, uint wild_num); bool setup_fields(THD *thd, Item** ref_pointer_array, @@ -1629,7 +1644,7 @@ inline bool setup_fields_with_no_wrap(TH thd->lex->select_lex.no_wrap_view_item= FALSE; return res; } -int setup_conds(THD *thd, TABLE_LIST *tables, TABLE_LIST *leaves, +int setup_conds(THD *thd, TABLE_LIST *tables, List<TABLE_LIST> &leaves, COND **conds); int setup_ftfuncs(SELECT_LEX* select); int init_ftfuncs(THD *thd, SELECT_LEX* select, bool no_order); @@ -1651,7 +1666,8 @@ inline int open_and_lock_tables(THD *thd /* simple open_and_lock_tables without derived handling for single table */ TABLE *open_n_lock_single_table(THD *thd, TABLE_LIST *table_l, thr_lock_type lock_type); -bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags); +bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags, + uint dt_phases); int lock_tables(THD *thd, TABLE_LIST *tables, uint counter, bool *need_reopen); int decide_logging_format(THD *thd, TABLE_LIST *tables); TABLE *open_temporary_table(THD *thd, const char *path, const char *db, @@ -1680,6 +1696,7 @@ void remove_db_from_cache(const char *db void flush_tables(); bool is_equal(const LEX_STRING *a, const LEX_STRING *b); char *make_default_log_name(char *buff,const char* log_ext); +void unfix_fields(List<Item> &items); #ifdef WITH_PARTITION_STORAGE_ENGINE uint fast_alter_partition_table(THD *thd, TABLE *table, @@ -2528,7 +2545,7 @@ Item * all_any_subquery_creator(Item *le inline void setup_table_map(TABLE *table, TABLE_LIST *table_list, uint tablenr) { table->used_fields= 0; - table->const_table= 0; + table_list->reset_const_table(); table->null_row= 0; table->status= STATUS_NO_RECORD; table->maybe_null= table_list->outer_join; @@ -2544,6 +2561,14 @@ inline void setup_table_map(TABLE *table table->force_index_order= table->force_index_group= 0; table->covering_keys= table->s->keys_for_keyread; table->merge_keys.clear_all(); + TABLE_LIST *orig= table_list->select_lex ? + table_list->select_lex->master_unit()->derived : 0; + if (!orig || !orig->is_merged_derived()) + { + /* Tables merged from derived were set up already.*/ + table->covering_keys= table->s->keys_for_keyread; + table->merge_keys.clear_all(); + } } === modified file 'sql/opt_range.cc' --- a/sql/opt_range.cc 2010-03-20 12:01:47 +0000 +++ b/sql/opt_range.cc 2010-05-26 20:18:18 +0000 @@ -7450,7 +7450,7 @@ ha_rows check_quick_select(PARAM *param, SEL_ARG_RANGE_SEQ seq; RANGE_SEQ_IF seq_if = {sel_arg_range_seq_init, sel_arg_range_seq_next, 0, 0}; handler *file= param->table->file; - ha_rows rows; + ha_rows rows= HA_POS_ERROR; uint keynr= param->real_keynr[idx]; DBUG_ENTER("check_quick_select"); @@ -7490,8 +7490,13 @@ ha_rows check_quick_select(PARAM *param, *mrr_flags |= HA_MRR_USE_DEFAULT_IMPL; *bufsize= param->thd->variables.mrr_buff_size; - rows= file->multi_range_read_info_const(keynr, &seq_if, (void*)&seq, 0, - bufsize, mrr_flags, cost); + /* + Skip materialized derived table/view result table from MRR check as + they aren't contain any data yet. + */ + if (param->table->pos_in_table_list->is_non_derived()) + rows= file->multi_range_read_info_const(keynr, &seq_if, (void*)&seq, 0, + bufsize, mrr_flags, cost); if (rows != HA_POS_ERROR) { param->table->quick_rows[keynr]=rows; === modified file 'sql/opt_subselect.cc' --- a/sql/opt_subselect.cc 2010-03-15 19:52:58 +0000 +++ b/sql/opt_subselect.cc 2010-05-26 20:18:18 +0000 @@ -154,9 +154,9 @@ int check_and_do_in_subquery_rewrites(JO !join->having && !select_lex->with_sum_func && // 4 thd->thd_marker.emb_on_expr_nest && // 5 select_lex->outer_select()->join && // 6 - select_lex->master_unit()->first_select()->leaf_tables && // 7 + select_lex->master_unit()->first_select()->leaf_tables.elements && // 7 in_subs->exec_method == Item_in_subselect::NOT_TRANSFORMED && // 8 - select_lex->outer_select()->leaf_tables && // 9 + select_lex->outer_select()->leaf_tables.elements && // 9 !((join->select_options | // 10 select_lex->outer_select()->join->select_options) // 10 & SELECT_STRAIGHT_JOIN)) // 10 @@ -212,9 +212,9 @@ int check_and_do_in_subquery_rewrites(JO if (optimizer_flag(thd, OPTIMIZER_SWITCH_MATERIALIZATION) && in_subs && // 1 !select_lex->is_part_of_union() && // 2 - select_lex->master_unit()->first_select()->leaf_tables && // 3 + select_lex->master_unit()->first_select()->leaf_tables.elements && // 3 thd->lex->sql_command == SQLCOM_SELECT && // * - select_lex->outer_select()->leaf_tables && // 3A + select_lex->outer_select()->leaf_tables.elements && // 3A subquery_types_allow_materialization(in_subs) && // psergey-todo: duplicated_subselect_card_check: where it's done? (in_subs->is_top_level_item() || @@ -391,11 +391,26 @@ bool convert_join_subqueries_to_semijoin Item_in_subselect **in_subq; Item_in_subselect **in_subq_end; THD *thd= join->thd; + TABLE_LIST *tbl; + List_iterator<TABLE_LIST> ti(join->select_lex->leaf_tables); DBUG_ENTER("convert_join_subqueries_to_semijoins"); if (join->sj_subselects.elements() == 0) DBUG_RETURN(FALSE); + for (in_subq= join->sj_subselects.front(), + in_subq_end= join->sj_subselects.back(); + in_subq != in_subq_end; + in_subq++) + { + SELECT_LEX *subq_sel= (*in_subq)->get_select_lex(); + if (subq_sel->handle_derived(thd->lex, DT_OPTIMIZE)) + DBUG_RETURN(1); + if (subq_sel->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + subq_sel->update_used_tables(); + } + /* First, convert child join's subqueries. We proceed bottom-up here */ for (in_subq= join->sj_subselects.front(), in_subq_end= join->sj_subselects.back(); @@ -422,11 +437,12 @@ bool convert_join_subqueries_to_semijoin // Temporary measure: disable semi-joins when they are together with outer // joins. - for (TABLE_LIST *tbl= join->select_lex->leaf_tables; tbl; tbl=tbl->next_leaf) + while ((tbl= ti++)) { TABLE_LIST *embedding= tbl->embedding; - if (tbl->on_expr || (tbl->embedding && !(embedding->sj_on_expr && - !embedding->embedding))) + if (tbl->on_expr || + (embedding && embedding->outer_join && + !(embedding->sj_on_expr && !embedding->embedding))) { in_subq= join->sj_subselects.front(); arena= thd->activate_stmt_arena_if_needed(&backup); @@ -737,7 +753,7 @@ static bool convert_subq_to_sj(JOIN *par st_select_lex *subq_lex= subq_pred->unit->first_select(); nested_join->join_list.empty(); List_iterator_fast<TABLE_LIST> li(subq_lex->top_join_list); - TABLE_LIST *tl, *last_leaf; + TABLE_LIST *tl; while ((tl= li++)) { tl->embedding= sj_nest; @@ -752,17 +768,15 @@ static bool convert_subq_to_sj(JOIN *par NOTE: We actually insert them at the front! That's because the order is reversed in this list. */ - for (tl= parent_lex->leaf_tables; tl->next_leaf; tl= tl->next_leaf) ; - tl->next_leaf= subq_lex->leaf_tables; - last_leaf= tl; + parent_lex->leaf_tables.concat(&subq_lex->leaf_tables); /* Same as above for next_local chain (a theory: a next_local chain always starts with ::leaf_tables because view's tables are inserted after the view) */ - for (tl= parent_lex->leaf_tables; tl->next_local; tl= tl->next_local) ; - tl->next_local= subq_lex->leaf_tables; + for (tl= parent_lex->leaf_tables.head(); tl->next_local; tl= tl->next_local) ; + tl->next_local= subq_lex->leaf_tables.head(); /* A theory: no need to re-connect the next_global chain */ @@ -776,7 +790,8 @@ static bool convert_subq_to_sj(JOIN *par /* n. Adjust the parent_join->tables counter */ uint table_no= parent_join->tables; /* n. Walk through child's tables and adjust table->map */ - for (tl= subq_lex->leaf_tables; tl; tl= tl->next_leaf, table_no++) + List_iterator_fast<TABLE_LIST> si(subq_lex->leaf_tables); + while ((tl= si++)) { tl->table->tablenr= table_no; tl->table->map= ((table_map)1) << table_no; @@ -786,6 +801,7 @@ static bool convert_subq_to_sj(JOIN *par emb && emb->select_lex == old_sl; emb= emb->embedding) emb->select_lex= parent_join->select_lex; + table_no++; } parent_join->tables += subq_lex->join->tables; @@ -872,7 +888,8 @@ static bool convert_subq_to_sj(JOIN *par { /* Inject into the WHERE */ parent_join->conds= and_items(parent_join->conds, sj_nest->sj_on_expr); - parent_join->conds->fix_fields(parent_join->thd, &parent_join->conds); + if (!parent_join->conds->fixed) + parent_join->conds->fix_fields(parent_join->thd, &parent_join->conds); parent_join->select_lex->where= parent_join->conds; } @@ -1424,6 +1441,7 @@ void advance_sj_state(JOIN *join, table_ TABLE_LIST *emb_sj_nest; POSITION *pos= join->positions + idx; remaining_tables &= ~new_join_tab->table->map; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; pos->prefix_cost.convert_from_cost(*current_read_time); pos->prefix_record_count= *current_record_count; @@ -1593,7 +1611,8 @@ void advance_sj_state(JOIN *join, table_ optimize_wo_join_buffering(join, pos->first_loosescan_table, idx, remaining_tables, TRUE, //first_alt - pos->first_loosescan_table + n_tables, + disable_jbuf ? join->tables : + pos->first_loosescan_table + n_tables, &reopt_rec_count, &reopt_cost, &sj_inner_fanout); /* @@ -1734,8 +1753,8 @@ void advance_sj_state(JOIN *join, table_ /* Need to re-run best-access-path as we prefix_rec_count has changed */ for (i= first_tab + mat_info->tables; i <= idx; i++) { - best_access_path(join, join->positions[i].table, rem_tables, i, FALSE, - prefix_rec_count, &curpos, &dummy); + best_access_path(join, join->positions[i].table, rem_tables, i, + disable_jbuf, prefix_rec_count, &curpos, &dummy); prefix_rec_count *= curpos.records_read; prefix_cost += curpos.read_time; } @@ -2031,6 +2050,7 @@ at_sjmat_pos(const JOIN *join, table_map void fix_semijoin_strategies_for_picked_join_order(JOIN *join) { uint table_count=join->tables; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; uint tablenr; table_map remaining_tables= 0; table_map handled_tabs= 0; @@ -2092,8 +2112,9 @@ void fix_semijoin_strategies_for_picked_ join->cur_sj_inner_tables= 0; for (i= first + sjm->tables; i <= tablenr; i++) { - best_access_path(join, join->best_positions[i].table, rem_tables, i, FALSE, - prefix_rec_count, join->best_positions + i, &dummy); + best_access_path(join, join->best_positions[i].table, rem_tables, i, + disable_jbuf, prefix_rec_count, + join->best_positions + i, &dummy); prefix_rec_count *= join->best_positions[i].records_read; rem_tables &= ~join->best_positions[i].table->table->map; } === modified file 'sql/opt_sum.cc' --- a/sql/opt_sum.cc 2010-01-04 17:54:42 +0000 +++ b/sql/opt_sum.cc 2010-05-26 20:18:18 +0000 @@ -74,10 +74,12 @@ static int maxmin_in_range(bool max_fl, # Multiplication of number of rows in all tables */ -static ulonglong get_exact_record_count(TABLE_LIST *tables) +static ulonglong get_exact_record_count(List<TABLE_LIST> &tables) { ulonglong count= 1; - for (TABLE_LIST *tl= tables; tl; tl= tl->next_leaf) + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(tables); + while ((tl= ti++)) { ha_rows tmp= tl->table->file->records(); if ((tmp == HA_POS_ERROR)) @@ -110,9 +112,11 @@ static ulonglong get_exact_record_count( HA_ERR_... if a deadlock or a lock wait timeout happens, for example */ -int opt_sum_query(TABLE_LIST *tables, List<Item> &all_fields,COND *conds) +int opt_sum_query(List<TABLE_LIST> &tables, List<Item> &all_fields,COND *conds) { List_iterator_fast<Item> it(all_fields); + List_iterator<TABLE_LIST> ti(tables); + TABLE_LIST *tl; int const_result= 1; bool recalc_const_item= 0; ulonglong count= 1; @@ -120,7 +124,7 @@ int opt_sum_query(TABLE_LIST *tables, Li table_map removed_tables= 0, outer_tables= 0, used_tables= 0; table_map where_tables= 0; Item *item; - int error; + int error= 0; if (conds) where_tables= conds->used_tables(); @@ -129,7 +133,7 @@ int opt_sum_query(TABLE_LIST *tables, Li Analyze outer join dependencies, and, if possible, compute the number of returned rows. */ - for (TABLE_LIST *tl= tables; tl; tl= tl->next_leaf) + while ((tl= ti++)) { TABLE_LIST *embedded; for (embedded= tl ; embedded; embedded= embedded->embedding) @@ -170,6 +174,14 @@ int opt_sum_query(TABLE_LIST *tables, Li is_exact_count= FALSE; count= 1; // ensure count != 0 } + else if (tl->is_materialized_derived()) + { + /* + Can't remove a derived table as it's number of rows is just an + estimate. + */ + return 0; + } else { error= tl->table->file->info(HA_STATUS_VARIABLE | HA_STATUS_NO_LOCK); === modified file 'sql/records.cc' --- a/sql/records.cc 2010-02-01 06:14:12 +0000 +++ b/sql/records.cc 2010-05-26 20:18:18 +0000 @@ -286,7 +286,8 @@ void end_read_record(READ_RECORD *info) if (info->table) { filesort_free_buffers(info->table,0); - (void) info->file->extra(HA_EXTRA_NO_CACHE); + if (info->table->created) + (void) info->file->extra(HA_EXTRA_NO_CACHE); if (info->read_record != rr_quick) // otherwise quick_range does it (void) info->file->ha_index_or_rnd_end(); info->table=0; === modified file 'sql/sp_head.cc' --- a/sql/sp_head.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sp_head.cc 2010-05-26 20:18:18 +0000 @@ -2821,6 +2821,9 @@ int sp_instr::exec_open_and_lock_tables( result= -1; else result= 0; + /* Prepare all derived tables/views to catch possible errors. */ + if (!result) + result= mysql_handle_derived(thd->lex, DT_PREPARE) ? -1 : 0; return result; } === modified file 'sql/sql_acl.cc' --- a/sql/sql_acl.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sql_acl.cc 2010-05-26 20:18:18 +0000 @@ -3003,7 +3003,8 @@ int mysql_table_grant(THD *thd, TABLE_LI class LEX_COLUMN *column; List_iterator <LEX_COLUMN> column_iter(columns); - if (open_and_lock_tables(thd, table_list)) + if (open_and_lock_tables(thd, table_list) || + mysql_handle_derived(thd->lex, DT_PREPARE)) DBUG_RETURN(TRUE); while ((column = column_iter++)) === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_base.cc 2010-05-26 20:18:18 +0000 @@ -2998,6 +2998,7 @@ TABLE *open_table(THD *thd, TABLE_LIST * table->fulltext_searched= 0; table->file->ft_handler= 0; table->reginfo.impossible_range= 0; + table->created= TRUE; /* Catch wrong handling of the auto_increment_field_not_null. */ DBUG_ASSERT(!table->auto_increment_field_not_null); table->auto_increment_field_not_null= FALSE; @@ -5044,9 +5045,10 @@ int open_and_lock_tables_derived(THD *th close_tables_for_reopen(thd, &tables); } if (derived && - (mysql_handle_derived(thd->lex, &mysql_derived_prepare) || - (thd->fill_derived_tables() && - mysql_handle_derived(thd->lex, &mysql_derived_filling)))) + (mysql_handle_derived(thd->lex, DT_INIT))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + if (thd->prepare_derived_at_open && derived && + (mysql_handle_derived(thd->lex, DT_PREPARE))) DBUG_RETURN(TRUE); /* purecov: inspected */ DBUG_RETURN(0); } @@ -5062,6 +5064,7 @@ int open_and_lock_tables_derived(THD *th flags - bitmap of flags to modify how the tables will be open: MYSQL_LOCK_IGNORE_FLUSH - open table even if someone has done a flush or namelock on it. + dt_phases - set of flags to pass to the mysql_handle_derived RETURN FALSE - ok @@ -5072,13 +5075,14 @@ int open_and_lock_tables_derived(THD *th data from the tables. */ -bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags) +bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags, + uint dt_phases) { uint counter; DBUG_ENTER("open_normal_and_derived_tables"); DBUG_ASSERT(!thd->fill_derived_tables()); if (open_tables(thd, &tables, &counter, flags) || - mysql_handle_derived(thd->lex, &mysql_derived_prepare)) + mysql_handle_derived(thd->lex, dt_phases)) DBUG_RETURN(TRUE); /* purecov: inspected */ DBUG_RETURN(0); } @@ -5714,9 +5718,7 @@ find_field_in_view(THD *thd, TABLE_LIST Field_iterator_view field_it; field_it.set(table_list); Query_arena *arena= 0, backup; - - DBUG_ASSERT(table_list->schema_table_reformed || - (ref != 0 && table_list->view != 0)); + for (; !field_it.end_of_fields(); field_it.next()) { if (!my_strcasecmp(system_charset_info, field_it.name(), name)) @@ -5735,6 +5737,8 @@ find_field_in_view(THD *thd, TABLE_LIST if (!item) DBUG_RETURN(0); + if (!ref) + DBUG_RETURN((Field*) view_ref_found); /* *ref != NULL means that *ref contains the item that we need to replace. If the item was aliased by the user, set the alias to @@ -6134,6 +6138,8 @@ find_field_in_table_ref(THD *thd, TABLE_ Field *field_to_set= NULL; if (fld == view_ref_found) { + if (!ref) + DBUG_RETURN(fld); Item *it= (*ref)->real_item(); if (it->type() == Item::FIELD_ITEM) field_to_set= ((Item_field*)it)->field; @@ -6141,6 +6147,8 @@ find_field_in_table_ref(THD *thd, TABLE_ { if (thd->mark_used_columns == MARK_COLUMNS_READ) it->walk(&Item::register_field_in_read_map, 1, (uchar *) 0); + else + it->walk(&Item::register_field_in_write_map, 1, (uchar *) 0); } } else @@ -6280,7 +6288,9 @@ find_field_in_tables(THD *thd, Item_iden find_field_in_table even in the case of information schema tables when table_ref->field_translation != NULL. */ - if (table_ref->table && !table_ref->view) + if (table_ref->table && + (!table_ref->is_merged_derived() || + (!table_ref->is_multitable() && table_ref->merged_for_insert))) found= find_field_in_table(thd, table_ref->table, name, length, TRUE, &(item->cached_field_index)); else @@ -6298,7 +6308,8 @@ find_field_in_tables(THD *thd, Item_iden Only views fields should be marked as dependent, not an underlying fields. */ - if (!table_ref->belong_to_view) + if (!table_ref->belong_to_view && + !table_ref->belong_to_derived) { SELECT_LEX *current_sel= thd->lex->current_select; SELECT_LEX *last_select= table_ref->select_lex; @@ -6884,6 +6895,10 @@ mark_common_columns(THD *thd, TABLE_LIST */ if (nj_col_2 && (!using_fields ||is_using_column_1)) { + /* + Create non-fixed fully qualified field and let fix_fields to + resolve it. + */ Item *item_1= nj_col_1->create_item(thd); Item *item_2= nj_col_2->create_item(thd); Field *field_1= nj_col_1->field(); @@ -7548,27 +7563,36 @@ bool setup_fields(THD *thd, Item **ref_p make_leaves_list() list pointer to pointer on list first element tables table list + full_table_list whether to include tables from mergeable derived table/view. + we need them for checks for INSERT/UPDATE statements only. RETURN pointer on pointer to next_leaf of last element */ -TABLE_LIST **make_leaves_list(TABLE_LIST **list, TABLE_LIST *tables) +void make_leaves_list(List<TABLE_LIST> &list, TABLE_LIST *tables, + bool full_table_list, TABLE_LIST *boundary) + { for (TABLE_LIST *table= tables; table; table= table->next_local) { - if (table->merge_underlying_list) - { - DBUG_ASSERT(table->view && - table->effective_algorithm == VIEW_ALGORITHM_MERGE); - list= make_leaves_list(list, table->merge_underlying_list); + if (table == boundary) + full_table_list= !full_table_list; + if (full_table_list && table->is_merged_derived()) + { + SELECT_LEX *select_lex= table->get_single_select(); + /* + It's safe to use select_lex->leaf_tables because all derived + tables/views were already prepared and has their leaf_tables + set properly. + */ + make_leaves_list(list, select_lex->get_table_list(), + full_table_list, boundary); } else { - *list= table; - list= &table->next_leaf; + list.push_back(table); } } - return list; } /* @@ -7583,6 +7607,7 @@ TABLE_LIST **make_leaves_list(TABLE_LIST leaves List of join table leaves list (select_lex->leaf_tables) refresh It is onle refresh for subquery select_insert It is SELECT ... INSERT command + full_table_list a parameter to pass to the make_leaves_list function NOTE Check also that the 'used keys' and 'ignored keys' exists and set up the @@ -7601,9 +7626,13 @@ TABLE_LIST **make_leaves_list(TABLE_LIST bool setup_tables(THD *thd, Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, bool select_insert) + List<TABLE_LIST> &leaves, bool select_insert, + bool full_table_list) { uint tablenr= 0; + List_iterator<TABLE_LIST> ti(leaves); + TABLE_LIST *table_list; + DBUG_ENTER("setup_tables"); DBUG_ASSERT ((select_insert && !tables->next_name_resolution_table) || !tables || @@ -7615,40 +7644,57 @@ bool setup_tables(THD *thd, Name_resolut TABLE_LIST *first_select_table= (select_insert ? tables->next_local: 0); - if (!(*leaves)) - make_leaves_list(leaves, tables); - - TABLE_LIST *table_list; - for (table_list= *leaves; - table_list; - table_list= table_list->next_leaf, tablenr++) - { - TABLE *table= table_list->table; - table->pos_in_table_list= table_list; - if (first_select_table && - table_list->top_table() == first_select_table) - { - /* new counting for SELECT of INSERT ... SELECT command */ - first_select_table= 0; - tablenr= 0; + SELECT_LEX *select_lex= select_insert ? &thd->lex->select_lex : + thd->lex->current_select; + if (select_lex->first_cond_optimization) + { + leaves.empty(); + select_lex->leaf_tables_exec.empty(); + make_leaves_list(leaves, tables, full_table_list, first_select_table); + + while ((table_list= ti++)) + { + TABLE *table= table_list->table; + table->pos_in_table_list= table_list; + if (first_select_table && + table_list->top_table() == first_select_table) + { + /* new counting for SELECT of INSERT ... SELECT command */ + first_select_table= 0; + thd->lex->select_lex.insert_tables= tablenr; + tablenr= 0; + } + setup_table_map(table, table_list, tablenr); + if (table_list->process_index_hints(table)) + DBUG_RETURN(1); + tablenr++; } - setup_table_map(table, table_list, tablenr); - if (table_list->process_index_hints(table)) + if (tablenr > MAX_TABLES) + { + my_error(ER_TOO_MANY_TABLES,MYF(0),MAX_TABLES); DBUG_RETURN(1); + } } - if (tablenr > MAX_TABLES) - { - my_error(ER_TOO_MANY_TABLES,MYF(0),MAX_TABLES); - DBUG_RETURN(1); - } + else + { + List_iterator_fast <TABLE_LIST> ti(select_lex->leaf_tables_exec); + select_lex->leaf_tables.empty(); + while ((table_list= ti++)) + { + table_list->table->tablenr= table_list->tablenr_exec; + table_list->table->map= table_list->map_exec; + table_list->table->pos_in_table_list= table_list; + select_lex->leaf_tables.push_back(table_list); + } + } + for (table_list= tables; table_list; table_list= table_list->next_local) { if (table_list->merge_underlying_list) { - DBUG_ASSERT(table_list->view && - table_list->effective_algorithm == VIEW_ALGORITHM_MERGE); + DBUG_ASSERT(table_list->is_merged_derived()); Query_arena *arena= thd->stmt_arena, backup; bool res; if (arena->is_conventional()) @@ -7675,7 +7721,7 @@ bool setup_tables(THD *thd, Name_resolut prepare tables and check access for the view tables SYNOPSIS - setup_tables_and_check_view_access() + setup_tables_and_check_access() thd Thread handler context name resolution contest to setup table list there from_clause Top-level list of table references in the FROM clause @@ -7685,6 +7731,7 @@ bool setup_tables(THD *thd, Name_resolut refresh It is onle refresh for subquery select_insert It is SELECT ... INSERT command want_access what access is needed + full_table_list a parameter to pass to the make_leaves_list function NOTE a wrapper for check_tables that will also check the resulting @@ -7698,33 +7745,32 @@ bool setup_tables_and_check_access(THD * Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, + List<TABLE_LIST> &leaves, bool select_insert, ulong want_access_first, - ulong want_access) + ulong want_access, + bool full_table_list) { - TABLE_LIST *leaves_tmp= NULL; bool first_table= true; if (setup_tables(thd, context, from_clause, tables, - &leaves_tmp, select_insert)) + leaves, select_insert, full_table_list)) return TRUE; - if (leaves) - *leaves= leaves_tmp; - - for (; leaves_tmp; leaves_tmp= leaves_tmp->next_leaf) + List_iterator<TABLE_LIST> ti(leaves); + TABLE_LIST *table_list; + while((table_list= ti++)) { - if (leaves_tmp->belong_to_view && + if (table_list->belong_to_view && check_single_table_access(thd, first_table ? want_access_first : - want_access, leaves_tmp, FALSE)) + want_access, table_list, FALSE)) { tables->hide_view_error(thd); return TRUE; } first_table= 0; } - return FALSE; + return FALSE; } @@ -7860,8 +7906,8 @@ insert_fields(THD *thd, Name_resolution_ information_schema table, or a nested table reference. See the comment for TABLE_LIST. */ - if (!((table && !tables->view && (table->grant.privilege & SELECT_ACL)) || - (tables->view && (tables->grant.privilege & SELECT_ACL))) && + if (!(table && tables->is_non_derived() && (table->grant.privilege & SELECT_ACL) || + (!tables->is_non_derived() && (tables->grant.privilege & SELECT_ACL))) && !any_privileges) { field_iterator.set(tables); @@ -7891,7 +7937,7 @@ insert_fields(THD *thd, Name_resolution_ if (!(item= field_iterator.create_item(thd))) DBUG_RETURN(TRUE); - DBUG_ASSERT(item->fixed); +// DBUG_ASSERT(item->fixed); /* cache the table for the Item_fields inserted by expanding stars */ if (item->type() == Item::FIELD_ITEM && tables->cacheable_table) ((Item_field *)item)->cached_table= tables; @@ -8021,13 +8067,14 @@ insert_fields(THD *thd, Name_resolution_ FALSE if all is OK */ -int setup_conds(THD *thd, TABLE_LIST *tables, TABLE_LIST *leaves, +int setup_conds(THD *thd, TABLE_LIST *tables, List<TABLE_LIST> &leaves, COND **conds) { SELECT_LEX *select_lex= thd->lex->current_select; Query_arena *arena= thd->stmt_arena, backup; TABLE_LIST *table= NULL; // For HP compilers TABLE_LIST *save_emb_on_expr_nest= thd->thd_marker.emb_on_expr_nest; + List_iterator<TABLE_LIST> ti(leaves); /* it_is_update set to TRUE when tables of primary SELECT_LEX (SELECT_LEX which belong to LEX, i.e. most up SELECT) will be updated by @@ -8039,9 +8086,15 @@ int setup_conds(THD *thd, TABLE_LIST *ta bool it_is_update= (select_lex == &thd->lex->select_lex) && thd->lex->which_check_option_applicable(); bool save_is_item_list_lookup= select_lex->is_item_list_lookup; - select_lex->is_item_list_lookup= 0; + TABLE_LIST *derived= select_lex->master_unit()->derived; DBUG_ENTER("setup_conds"); + /* Do not fix conditions for the derived tables that have been merged */ + if (derived && derived->merged) + DBUG_RETURN(0); + + select_lex->is_item_list_lookup= 0; + if (select_lex->conds_processed_with_permanent_arena || arena->is_conventional()) arena= 0; // For easier test @@ -8054,7 +8107,10 @@ int setup_conds(THD *thd, TABLE_LIST *ta for (table= tables; table; table= table->next_local) { - if (table->prepare_where(thd, conds, FALSE)) + if (select_lex == &thd->lex->select_lex && + select_lex->first_cond_optimization && + table->merged_for_insert && + table->prepare_where(thd, conds, FALSE)) goto err_no_arena; } @@ -8072,7 +8128,7 @@ int setup_conds(THD *thd, TABLE_LIST *ta Apply fix_fields() to all ON clauses at all levels of nesting, including the ones inside view definitions. */ - for (table= leaves; table; table= table->next_leaf) + while ((table= ti++)) { TABLE_LIST *embedded; /* The table at the current level of nesting. */ TABLE_LIST *embedding= table; /* The parent nested table reference. */ @@ -9283,6 +9339,27 @@ void close_performance_schema_table(THD thd->restore_backup_open_tables_state(backup); } + +/** + @brief + Remove 'fixed' flag from items in a list + + @param items list of items to un-fix + + @details + This function sets to 0 the 'fixed' flag for items in the 'items' list. + It's needed to force correct marking of views' fields for INSERT/UPDATE + statements. +*/ + +void unfix_fields(List<Item> &fields) +{ + List_iterator<Item> li(fields); + Item *item; + while ((item= li++)) + item->fixed= 0; +} + /** @} (end of group Data_Dictionary) */ === modified file 'sql/sql_bitmap.h' --- a/sql/sql_bitmap.h 2009-08-12 22:34:21 +0000 +++ b/sql/sql_bitmap.h 2010-05-26 20:18:18 +0000 @@ -91,6 +91,10 @@ public: DBUG_ASSERT(sizeof(buffer) >= 4); return (ulonglong) uint4korr(buffer); } + uint bits_set() + { + return bitmap_bits_set(&map); + } }; /* An iterator to quickly walk over bits in unlonglong bitmap. */ @@ -169,5 +173,16 @@ public: public: Iterator(Bitmap<64> &bmp) : Table_map_iterator(bmp.map) {} }; + uint bits_set() + { + //TODO: use my_count_bits() + uint res= 0, i= 0; + for (; i < 64 ; i++) + { + if (map & ((ulonglong)1<<i)) + res++; + } + return res; + } }; === modified file 'sql/sql_cache.cc' --- a/sql/sql_cache.cc 2010-01-29 10:42:31 +0000 +++ b/sql/sql_cache.cc 2010-05-26 20:18:18 +0000 @@ -3477,16 +3477,17 @@ Query_cache::process_and_count_tables(TH } else { - DBUG_PRINT("qcache", ("table: %s db: %s type: %u", - tables_used->table->s->table_name.str, - tables_used->table->s->db.str, - tables_used->table->s->db_type()->db_type)); if (tables_used->derived) { + DBUG_PRINT("qcache", ("table: %s", tables_used->alias)); table_count--; DBUG_PRINT("qcache", ("derived table skipped")); continue; } + DBUG_PRINT("qcache", ("table: %s db: %s type: %u", + tables_used->table->s->table_name.str, + tables_used->table->s->db.str, + tables_used->table->s->db_type()->db_type)); *tables_type|= tables_used->table->file->table_cache_type(); /* === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.cc 2010-05-26 20:18:18 +0000 @@ -771,6 +771,7 @@ THD::THD() thr_lock_owner_init(&main_lock_id, &lock_info); m_internal_handler= NULL; + prepare_derived_at_open= FALSE; } @@ -2946,7 +2947,8 @@ bool select_materialize_with_stats:: create_result_table(THD *thd_arg, List<Item> *column_types, bool is_union_distinct, ulonglong options, - const char *table_alias, bool bit_fields_as_long) + const char *table_alias, bool bit_fields_as_long, + bool create_table) { DBUG_ASSERT(table == 0); tmp_table_param.field_count= column_types->elements; === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.h 2010-05-26 20:18:18 +0000 @@ -1474,6 +1474,9 @@ public: */ TABLE_LIST *emb_on_expr_nest; } thd_marker; + + bool prepare_derived_at_open; + #ifndef MYSQL_CLIENT int binlog_setup_trx_data(); @@ -2810,12 +2813,12 @@ public: class select_union :public select_result_interceptor { -protected: - TMP_TABLE_PARAM tmp_table_param; public: + TMP_TABLE_PARAM tmp_table_param; TABLE *table; + ha_rows records; - select_union() :table(0) { tmp_table_param.init(); } + select_union() :table(0), records(0) { tmp_table_param.init(); } int prepare(List<Item> &list, SELECT_LEX_UNIT *u); bool send_data(List<Item> &items); bool send_eof(); @@ -2823,7 +2826,9 @@ public: virtual bool create_result_table(THD *thd, List<Item> *column_types, bool is_distinct, ulonglong options, - const char *alias, bool bit_fields_as_long); + const char *alias, + bool bit_fields_as_long, + bool create_table); }; /* Base subselect interface class */ @@ -2885,9 +2890,11 @@ protected: public: select_materialize_with_stats() {} - virtual bool create_result_table(THD *thd, List<Item> *column_types, - bool is_distinct, ulonglong options, - const char *alias, bool bit_fields_as_long); + bool create_result_table(THD *thd, List<Item> *column_types, + bool is_distinct, ulonglong options, + const char *alias, + bool bit_fields_as_long, + bool create_table); bool init_result_table(ulonglong select_options); bool send_data(List<Item> &items); void cleanup() @@ -3175,7 +3182,7 @@ public: class multi_update :public select_result_interceptor { TABLE_LIST *all_tables; /* query/update command tables */ - TABLE_LIST *leaves; /* list of leves of join table tree */ + List<TABLE_LIST> *leaves; /* list of leves of join table tree */ TABLE_LIST *update_tables, *table_being_updated; TABLE **tmp_tables, *main_table, *table_to_update; TMP_TABLE_PARAM *tmp_table_param; @@ -3201,7 +3208,7 @@ class multi_update :public select_result bool error_handled; public: - multi_update(TABLE_LIST *ut, TABLE_LIST *leaves_list, + multi_update(TABLE_LIST *ut, List<TABLE_LIST> *leaves_list, List<Item> *fields, List<Item> *values, enum_duplicates handle_duplicates, bool ignore); ~multi_update(); === modified file 'sql/sql_cursor.cc' --- a/sql/sql_cursor.cc 2010-02-17 21:59:41 +0000 +++ b/sql/sql_cursor.cc 2010-05-26 20:18:18 +0000 @@ -715,8 +715,8 @@ bool Select_materialize::send_fields(Lis DBUG_ASSERT(table == 0); if (create_result_table(unit->thd, unit->get_unit_column_types(), FALSE, thd->options | TMP_TABLE_ALL_COLUMNS, "", - FALSE)) - return TRUE; + FALSE, TRUE)) + return TRUE; materialized_cursor= new (&table->mem_root) Materialized_cursor(result, table); === modified file 'sql/sql_delete.cc' --- a/sql/sql_delete.cc 2010-03-10 13:55:40 +0000 +++ b/sql/sql_delete.cc 2010-05-26 20:18:18 +0000 @@ -58,10 +58,18 @@ bool mysql_delete(THD *thd, TABLE_LIST * if (open_and_lock_tables(thd, table_list)) DBUG_RETURN(TRUE); - if (!(table= table_list->table)) + + if (mysql_handle_list_of_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT) || + mysql_handle_list_of_derived(thd->lex, table_list, DT_PREPARE)) + DBUG_RETURN(TRUE); + + if (!(table= table_list->table) || !table->created) { - my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), - table_list->view_db.str, table_list->view_name.str); + if (!table_list->updatable) + my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "DELETE"); + else + my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), + table_list->view_db.str, table_list->view_name.str); DBUG_RETURN(TRUE); } thd_proc_info(thd, "init"); @@ -70,6 +78,11 @@ bool mysql_delete(THD *thd, TABLE_LIST * if (mysql_prepare_delete(thd, table_list, &conds)) DBUG_RETURN(TRUE); + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } /* check ORDER BY even if it can be ignored */ if (order && order->elements) { @@ -384,6 +397,12 @@ cleanup: query_cache_invalidate3(thd, table_list, 1); } + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } + delete select; transactional_table= table->file->has_transactions(); @@ -481,8 +500,8 @@ int mysql_prepare_delete(THD *thd, TABLE if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, table_list, - &select_lex->leaf_tables, FALSE, - DELETE_ACL, SELECT_ACL) || + select_lex->leaf_tables, FALSE, + DELETE_ACL, SELECT_ACL, TRUE) || setup_conds(thd, table_list, select_lex->leaf_tables, conds) || setup_ftfuncs(select_lex)) DBUG_RETURN(TRUE); @@ -540,6 +559,11 @@ int mysql_multi_delete_prepare(THD *thd) TABLE_LIST *target_tbl; DBUG_ENTER("mysql_multi_delete_prepare"); + TABLE_LIST *tables= lex->query_tables; + if (mysql_handle_derived(lex, DT_INIT) || + mysql_handle_list_of_derived(lex, tables, DT_MERGE_FOR_INSERT) || + mysql_handle_list_of_derived(lex, tables, DT_PREPARE)) + DBUG_RETURN(TRUE); /* setup_tables() need for VIEWs. JOIN::prepare() will not do it second time. @@ -549,8 +573,8 @@ int mysql_multi_delete_prepare(THD *thd) if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, lex->query_tables, - &lex->select_lex.leaf_tables, FALSE, - DELETE_ACL, SELECT_ACL)) + lex->select_lex.leaf_tables, FALSE, + DELETE_ACL, SELECT_ACL, TRUE)) DBUG_RETURN(TRUE); @@ -564,16 +588,13 @@ int mysql_multi_delete_prepare(THD *thd) target_tbl; target_tbl= target_tbl->next_local) { + if (!(target_tbl->table= target_tbl->correspondent_table->table)) { - DBUG_ASSERT(target_tbl->correspondent_table->view && - target_tbl->correspondent_table->merge_underlying_list && - target_tbl->correspondent_table->merge_underlying_list-> - next_local); - my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), - target_tbl->correspondent_table->view_db.str, - target_tbl->correspondent_table->view_name.str); - DBUG_RETURN(TRUE); + my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), + target_tbl->correspondent_table->view_db.str, + target_tbl->correspondent_table->view_name.str); + DBUG_RETURN(TRUE); } if (!target_tbl->correspondent_table->updatable || @@ -623,6 +644,12 @@ multi_delete::prepare(List<Item> &values unit= u; do_delete= 1; thd_proc_info(thd, "deleting from main table"); + SELECT_LEX *select_lex= u->first_select(); + if (select_lex->first_cond_optimization) + { + if (select_lex->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + } DBUG_RETURN(0); } === modified file 'sql/sql_derived.cc' --- a/sql/sql_derived.cc 2010-02-17 21:59:41 +0000 +++ b/sql/sql_derived.cc 2010-05-26 20:18:18 +0000 @@ -23,38 +23,79 @@ #include "mysql_priv.h" #include "sql_select.h" +typedef bool (*dt_processor)(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_init(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_optimize(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_merge(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_create(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_fill(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_reinit(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_merge_for_insert(THD *thd, LEX *lex, TABLE_LIST *derived); + + +dt_processor processors[]= +{ + &mysql_derived_init, + &mysql_derived_prepare, + &mysql_derived_optimize, + &mysql_derived_merge, + &mysql_derived_merge_for_insert, + &mysql_derived_create, + &mysql_derived_fill, + &mysql_derived_reinit, +}; /* - Call given derived table processor (preparing or filling tables) + @brief + Run specified phases on all derived tables/views in given LEX. - SYNOPSIS - mysql_handle_derived() - lex LEX for this thread - processor procedure of derived table processing - - RETURN - FALSE OK - TRUE Error -*/ + @param lex LEX for this thread + @param phases phases to run derived tables/views through + @return FALSE OK + @return TRUE Error +*/ bool -mysql_handle_derived(LEX *lex, bool (*processor)(THD*, LEX*, TABLE_LIST*)) +mysql_handle_derived(LEX *lex, uint phases) { bool res= FALSE; - if (lex->derived_tables) + THD *thd= lex->thd; + if (!lex->derived_tables) + return FALSE; + + lex->thd->derived_tables_processing= TRUE; + + for (uint phase= 0; phase < DT_PHASES && !res; phase++) { - lex->thd->derived_tables_processing= TRUE; + uint phase_flag= DT_INIT << phase; + if (phase_flag > phases) + break; + if (!(phases & phase_flag)) + continue; + if (phase_flag >= DT_CREATE && !thd->fill_derived_tables()) + break; + for (SELECT_LEX *sl= lex->all_selects_list; - sl; + sl && !res; sl= sl->next_select_in_list()) { for (TABLE_LIST *cursor= sl->get_table_list(); - cursor; + cursor && !res; cursor= cursor->next_local) { - if ((res= (*processor)(lex->thd, lex, cursor))) - goto out; + uint8 allowed_phases= (cursor->is_merged_derived() ? DT_PHASES_MERGE : + DT_PHASES_MATERIALIZE); + /* + Skip derived tables to which the phase isn't applicable. + TODO: mark derived at the parse time, later set it's type + (merged or materialized) + */ + if ((phase_flag != DT_PREPARE && !(allowed_phases & phase_flag)) || + (cursor->merged_for_insert && phase_flag != DT_REINIT)) + continue; + res= (*processors[phase])(lex->thd, lex, cursor); } if (lex->describe) { @@ -67,30 +108,454 @@ mysql_handle_derived(LEX *lex, bool (*pr } } } -out: + lex->thd->derived_tables_processing= FALSE; + return res; +} + +/* + @brief + Run through phases for the given derived table/view. + + @param lex LEX for this thread + @param derived the derived table to handle + @param phase_map phases to process tables/views through + + @details + + This function process the derived table (view) 'derived' to performs all + actions that are to be done on the table at the phases specified by + phase_map. The processing is carried out starting from the actions + performed at the earlier phases (those having smaller ordinal numbers). + + @note + This function runs specified phases of the derived tables handling on the + given derived table/view. This function is used in the chain of calls: + SELECT_LEX::handle_derived -> + TABLE_LIST::handle_derived -> + mysql_handle_single_derived + This chain of calls implements the bottom-up handling of the derived tables: + i.e. most inner derived tables/views are handled first. This order is + required for the all phases except the merge and the create steps. + For the sake of code simplicity this order is kept for all phases. + + @return FALSE ok + @return TRUE error +*/ + +bool +mysql_handle_single_derived(LEX *lex, TABLE_LIST *derived, uint phases) +{ + bool res= FALSE; + THD *thd= lex->thd; + uint8 allowed_phases= (derived->is_merged_derived() ? DT_PHASES_MERGE : + DT_PHASES_MATERIALIZE); + if (!lex->derived_tables) + return FALSE; + + lex->thd->derived_tables_processing= TRUE; + + for (uint phase= 0; phase < DT_PHASES; phase++) + { + uint phase_flag= DT_INIT << phase; + if (phase_flag > phases) + break; + if (!(phases & phase_flag)) + continue; + /* Skip derived tables to which the phase isn't applicable. */ + if (phase_flag != DT_PREPARE && + !(allowed_phases & phase_flag)) + continue; + if (phase_flag >= DT_CREATE && !thd->fill_derived_tables()) + break; + + if ((res= (*processors[phase])(lex->thd, lex, derived))) + break; + } lex->thd->derived_tables_processing= FALSE; return res; } /** - @brief Create temporary table structure (but do not fill it). + @brief + Run specified phases for derived tables/views in the given list + + @param lex LEX for this thread + @param table_list list of derived tables/view to handle + @param phase_map phases to process tables/views through + + @details + This function runs phases specified by the 'phases_map' on derived + tables/views found in the 'dt_list' with help of the + TABLE_LIST::handle_derived function. + 'lex' is passed as an argument to the TABLE_LIST::handle_derived. + + @return FALSE ok + @return TRUE error +*/ + +bool +mysql_handle_list_of_derived(LEX *lex, TABLE_LIST *table_list, uint phases) +{ + for (TABLE_LIST *tl= table_list; tl; tl= tl->next_local) + { + if (tl->is_view_or_derived() && + tl->handle_derived(lex, phases)) + return TRUE; + } + return FALSE; +} - @param thd Thread handle - @param lex LEX for this thread - @param orig_table_list TABLE_LIST for the upper SELECT - @details +/** + @brief + Merge a derived table/view into the embedding select - This function is called before any command containing derived tables is - executed. Currently the function is used for derived tables, i.e. + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + This function merges the given derived table / view into the parent select + construction. Any derived table/reference to view occurred in the FROM + clause of the embedding select is represented by a TABLE_LIST structure a + pointer to which is passed to the function as in the parameter 'derived'. + This structure contains the number/map, alias, a link to SELECT_LEX of the + derived table and other info. If the 'derived' table is used in a nested join + then additionally the structure contains a reference to the ON expression + for this join. + + The merge process results in elimination of the derived table (or the + reference to a view) such that: + - the FROM list of the derived table/view is wrapped into a nested join + after which the nest is added to the FROM list of the embedding select + - the WHERE condition of the derived table (view) is ANDed with the ON + condition attached to the table. + + @note + Tables are merged into the leaf_tables list, original derived table is removed + from this list also. SELECT_LEX::table_list list is left untouched. + Where expression is merged with derived table's on_expr and can be found after + the merge through the SELECT_LEX::table_list. + + Examples of the derived table/view merge: + + Schema: + Tables: t1(f1), t2(f2), t3(f3) + View v1: SELECT f1 FROM t1 WHERE f1 < 1 + + Example with a view: + Before merge: + + The query (Q1): SELECT f1,f2 FROM t2 LEFT JOIN v1 ON f1 = f2 + + (LEX of the main query) + | + (select_lex) + | + (FROM table list) + | + (join list)= t2, v1 + / \ + / (on_expr)= (f1 = f2) + | + (LEX of the v1 view) + | + (select_lex)= SELECT f1 FROM t1 WHERE f1 < 1 + + + After merge: + + The rewritten query Q1 (Q1'): + SELECT f1,f2 FROM t2 LEFT JOIN (t1) ON ((f1 = f2) and (f1 < 1)) + + (LEX of the main query) + | + (select_lex) + | + (FROM table list) + | + (join list)= t2, (t1) + \ + (on_expr)= (f1 = f2) and (f1 < 1) + + In this example table numbers are assigned as follows: + (outer select): t2 - 1, v1 - 2 + (inner select): t1 - 1 + After the merge table numbers will be: + (outer select): t2 - 1, t1 - 2 + + Example with a derived table: + The query Q2: + SELECT f1,f2 + FROM (SELECT f1 FROM t1, t3 WHERE f1=f3 and f1 < 1) tt, t2 + WHERE f1 = f2 + + Before merge: + (LEX of the main query) + | + (select_lex) + / \ + (FROM table list) (WHERE clause)= (f1 = f2) + | + (join list)= tt, t2 + / \ + / (on_expr)= (empty) + / + (select_lex)= SELECT f1 FROM t1, t3 WHERE f1 = f3 and f1 < 1 + + After merge: + + The rewritten query Q2 (Q2'): + SELECT f1,f2 + FROM (t1, t3) JOIN t2 ON (f1 = f3 and f1 < 1) + WHERE f1 = f2 + + (LEX of the main query) + | + (select_lex) + / \ + (FROM table list) (WHERE clause)= (f1 = f2) + | + (join list)= t2, (t1, t3) + \ + (on_expr)= (f1 = f3 and f1 < 1) + + In this example table numbers are assigned as follows: + (outer select): tt - 1, t2 - 2 + (inner select): t1 - 1, t3 - 2 + After the merge table numbers will be: + (outer select): t1 - 1, t2 - 2, t3 - 3 - - Anonymous derived tables, or - - Named derived tables (aka views) with the @c TEMPTABLE algorithm. - - The table reference, contained in @c orig_table_list, is updated with the - fields of a new temporary table. + @return FALSE if derived table/view were successfully merged. + @return TRUE if an error occur. +*/ + +bool mysql_derived_merge(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + bool res= FALSE; + SELECT_LEX *dt_select= derived->get_single_select(); + table_map map; + uint tablenr; + SELECT_LEX *parent_lex= derived->select_lex; + Query_arena *arena, backup; + + if (derived->merged) + return FALSE; + + arena= thd->activate_stmt_arena_if_needed(&backup); // For easier test + derived->merged= TRUE; + /* + Check whether there is enough free bits in table map to merge subquery. + If not - materialize it. This check isn't cached so when there is a big + and small subqueries, and the bigger one can't be merged it wouldn't + block the smaller one. + */ + if (parent_lex->get_free_table_map(&map, &tablenr)) + { + /* There is no enough table bits, fall back to materialization. */ + derived->change_refs_to_fields(); + derived->set_materialized_derived(); + goto exit_merge; + } + + if (dt_select->leaf_tables.elements + tablenr > MAX_TABLES) + { + /* There is no enough table bits, fall back to materialization. */ + derived->change_refs_to_fields(); + derived->set_materialized_derived(); + goto exit_merge; + } + + if (dt_select->options & OPTION_SCHEMA_TABLE) + parent_lex->options |= OPTION_SCHEMA_TABLE; + + parent_lex->cond_count+= dt_select->cond_count; + if (!derived->get_unit()->prepared) + { + dt_select->leaf_tables.empty(); + make_leaves_list(dt_select->leaf_tables, derived, TRUE, 0); + } + + if (!derived->merged_for_insert) + { derived->nested_join= (NESTED_JOIN*) thd->calloc(sizeof(NESTED_JOIN)); + if (!derived->nested_join) + { + res= TRUE; + goto exit_merge; + } + + /* Merge derived table's subquery in the parent select. */ + if (parent_lex->merge_subquery(derived, dt_select, tablenr, map)) + { + res= TRUE; + goto exit_merge; + } + + /* + exclude select lex so it doesn't show up in explain. + do this only for derived table as for views this is already done. + + From sql_view.cc + Add subqueries units to SELECT into which we merging current view. + unit(->next)* chain starts with subqueries that are used by this + view and continues with subqueries that are used by other views. + We must not add any subquery twice (otherwise we'll form a loop), + to do this we remember in end_unit the first subquery that has + been already added. + */ + derived->get_unit()->exclude_level(); + if (parent_lex->join) + parent_lex->join->tables+= dt_select->join->tables - 1; + } + if (derived->get_unit()->prepared) + { + Item *expr= derived->on_expr; + expr= and_conds(expr, dt_select->join ? dt_select->join->conds : 0); + if (expr && (derived->prep_on_expr || expr != derived->on_expr)) + { + derived->on_expr= expr; + derived->prep_on_expr= expr->copy_andor_structure(thd); + } + if (derived->on_expr && + ((!derived->on_expr->fixed && + derived->on_expr->fix_fields(thd, &derived->on_expr)) || + derived->on_expr->check_cols(1))) + { + res= TRUE; /* purecov: inspected */ + goto exit_merge; + } + // Update used tables cache according to new table map + if (derived->on_expr) + derived->on_expr->update_used_tables(); + } + +exit_merge: + if (arena) + thd->restore_active_arena(arena, &backup); + return res; +} + + +/** + @brief + Merge a view for the embedding INSERT/UPDATE/DELETE + + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + This function substitutes the derived table for the first table from + the query of the derived table thus making it a correct target table for the + INSERT/UPDATE/DELETE statements. As this operation is correct only for + single table views only, for multi table views this function does nothing. + The derived parameter isn't checked to be a view as derived tables aren't + allowed for INSERT/UPDATE/DELETE statements. + + @return FALSE if derived table/view were successfully merged. + @return TRUE if an error occur. +*/ + +bool mysql_derived_merge_for_insert(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + SELECT_LEX *dt_select= derived->get_single_select(); + + if (derived->merged_for_insert) + return FALSE; + if (!derived->is_multitable()) + { + TABLE_LIST *tl=((TABLE_LIST*)dt_select->table_list.first); + TABLE *table= tl->table; + /* preserve old map & tablenr. */ + if (!derived->merged_for_insert && derived->table) + table->set_table_map(derived->table->map, derived->table->tablenr); + + derived->table= table; + derived->schema_table= + ((TABLE_LIST*)dt_select->table_list.first)->schema_table; + if (!derived->merged) + { + Query_arena *arena, backup; + arena= thd->activate_stmt_arena_if_needed(&backup); // For easier test + derived->select_lex->leaf_tables.push_back(tl); + derived->nested_join= (NESTED_JOIN*) thd->calloc(sizeof(NESTED_JOIN)); + if (derived->nested_join) + { + derived->wrap_into_nested_join(tl->select_lex->top_join_list); + derived->get_unit()->exclude_level(); + } + if (arena) + thd->restore_active_arena(arena, &backup); + derived->merged= TRUE; + if (!derived->nested_join) + return TRUE; + } + } + else + { + if (!derived->merged_for_insert && mysql_derived_merge(thd, lex, derived)) + return TRUE; + } + derived->merged_for_insert= TRUE; + + return FALSE; +} + + +/* + @brief + Initialize a derived table/view + + @param thd Thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @detail + Fill info about derived table/view without preparing an + underlying select. Such as: create a field translation for views, mark it as + a multitable if it is and so on. + + @return + false OK + true Error +*/ + + +bool mysql_derived_init(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + SELECT_LEX_UNIT *unit= derived->get_unit(); + DBUG_ENTER("mysql_derived_init"); + + // Skip already prepared views/DT + if (!unit || unit->prepared) + DBUG_RETURN(FALSE); + + DBUG_RETURN(derived->init_derived(thd, TRUE)); +} + + +/* + @brief + Create temporary table structure (but do not fill it) + + @param thd Thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @detail + Prepare underlying select for a derived table/view. To properly resolve + names in the embedding query the TABLE structure is created. Actual table + is created later by the mysql_derived_create function. + + This function is called before any command containing derived table + is executed. All types of derived tables are handled by this function: + - Anonymous derived tables, or + - Named derived tables (aka views). + + The table reference, contained in @c derived, is updated with the + fields of a new temporary table. Derived tables are stored in @c thd->derived_tables and closed by close_thread_tables(). @@ -114,202 +579,359 @@ out: the state of privilege checking (GRANT_INFO struct) is copied as-is to the temporary table. - This function implements a signature called "derived table processor", and - is passed as a function pointer to mysql_handle_derived(). + Only the TABLE structure is created here, actual table is created by the + mysql_derived_create function. @note This function sets @c SELECT_ACL for @c TEMPTABLE views as well as anonymous derived tables, but this is ok since later access checking will distinguish between them. - @see mysql_handle_derived(), mysql_derived_filling(), GRANT_INFO + @see mysql_handle_derived(), mysql_derived_fill(), GRANT_INFO @return false OK true Error */ -bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *orig_table_list) +bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *derived) { - SELECT_LEX_UNIT *unit= orig_table_list->derived; - ulonglong create_options; + SELECT_LEX_UNIT *unit= derived->get_unit(); DBUG_ENTER("mysql_derived_prepare"); bool res= FALSE; - if (unit) - { - SELECT_LEX *first_select= unit->first_select(); - TABLE *table= 0; - select_union *derived_result; - /* prevent name resolving out of derived table */ - for (SELECT_LEX *sl= first_select; sl; sl= sl->next_select()) - sl->context.outer_context= 0; + // Skip already prepared views/DT + if (!unit || unit->prepared) + DBUG_RETURN(FALSE); - if (!(derived_result= new select_union)) - DBUG_RETURN(TRUE); // out of memory + /* It's a target view for an INSERT, create field translation only. */ + if (derived->merged_for_insert) + { + res= derived->create_field_translation(thd); + DBUG_RETURN(res); + } - // st_select_lex_unit::prepare correctly work for single select - if ((res= unit->prepare(thd, derived_result, 0))) - goto exit; + Query_arena *arena= thd->stmt_arena, backup; + if (arena->is_conventional()) + arena= 0; // For easier test + else + thd->set_n_backup_active_arena(arena, &backup); - if ((res= check_duplicate_names(unit->types, 0))) - goto exit; + SELECT_LEX *first_select= unit->first_select(); - create_options= (first_select->options | thd->options | - TMP_TABLE_ALL_COLUMNS); - /* - Temp table is created so that it hounours if UNION without ALL is to be - processed + /* prevent name resolving out of derived table */ + for (SELECT_LEX *sl= first_select; sl; sl= sl->next_select()) + { + sl->context.outer_context= 0; + // Prepare underlying views/DT first. + sl->handle_derived(lex, DT_PREPARE); + } - As 'distinct' parameter we always pass FALSE (0), because underlying - query will control distinct condition by itself. Correct test of - distinct underlying query will be is_union && - !unit->union_distinct->next_select() (i.e. it is union and last distinct - SELECT is last SELECT of UNION). - */ - if ((res= derived_result->create_result_table(thd, &unit->types, FALSE, - create_options, - orig_table_list->alias, - FALSE))) - goto exit; + unit->derived= derived; + + if (!(derived->derived_result= new select_union)) + DBUG_RETURN(TRUE); // out of memory - table= derived_result->table; + // st_select_lex_unit::prepare correctly work for single select + if ((res= unit->prepare(thd, derived->derived_result, 0))) + goto exit; + + if ((res= check_duplicate_names(unit->types, 0))) + goto exit; + + /* + Check whether we can merge this derived table into main select. + Depending on the result field translation will or will not + be created. + */ + if (derived->init_derived(thd, FALSE)) + goto exit; + + /* + Temp table is created so that it hounours if UNION without ALL is to be + processed + + As 'distinct' parameter we always pass FALSE (0), because underlying + query will control distinct condition by itself. Correct test of + distinct underlying query will be is_union && + !unit->union_distinct->next_select() (i.e. it is union and last distinct + SELECT is last SELECT of UNION). + */ + if (derived->derived_result->create_result_table(thd, &unit->types, FALSE, + (first_select->options | + thd->options | + TMP_TABLE_ALL_COLUMNS), + derived->alias, + FALSE, FALSE)) + goto exit; + + derived->table= derived->derived_result->table; + if (derived->is_derived() && derived->is_merged_derived()) + first_select->mark_as_belong_to_derived(derived); exit: - /* Hide "Unknown column" or "Unknown function" error */ - if (orig_table_list->view) - { - if (thd->is_error() && + /* Hide "Unknown column" or "Unknown function" error */ + if (derived->view) + { + if (thd->is_error() && (thd->main_da.sql_errno() == ER_BAD_FIELD_ERROR || - thd->main_da.sql_errno() == ER_FUNC_INEXISTENT_NAME_COLLISION || - thd->main_da.sql_errno() == ER_SP_DOES_NOT_EXIST)) - { - thd->clear_error(); - my_error(ER_VIEW_INVALID, MYF(0), orig_table_list->db, - orig_table_list->table_name); - } - } - - /* - if it is preparation PS only or commands that need only VIEW structure - then we do not need real data and we can skip execution (and parameters - is not defined, too) - */ - if (res) + thd->main_da.sql_errno() == ER_FUNC_INEXISTENT_NAME_COLLISION || + thd->main_da.sql_errno() == ER_SP_DOES_NOT_EXIST)) { - if (table) - free_tmp_table(thd, table); - delete derived_result; + thd->clear_error(); + my_error(ER_VIEW_INVALID, MYF(0), derived->db, + derived->table_name); } + } + + /* + if it is preparation PS only or commands that need only VIEW structure + then we do not need real data and we can skip execution (and parameters + is not defined, too) + */ + if (res) + { + if (derived->table) + free_tmp_table(thd, derived->table); + delete derived->derived_result; + } + else + { + TABLE *table= derived->table; + table->derived_select_number= first_select->select_number; + table->s->tmp_table= NON_TRANSACTIONAL_TMP_TABLE; +#ifndef NO_EMBEDDED_ACCESS_CHECKS + if (derived->referencing_view) + table->grant= derived->grant; else { - if (!thd->fill_derived_tables()) - { - delete derived_result; - derived_result= NULL; - } - orig_table_list->derived_result= derived_result; - orig_table_list->table= table; - orig_table_list->table_name= table->s->table_name.str; - orig_table_list->table_name_length= table->s->table_name.length; - table->derived_select_number= first_select->select_number; - table->s->tmp_table= NON_TRANSACTIONAL_TMP_TABLE; -#ifndef NO_EMBEDDED_ACCESS_CHECKS - if (orig_table_list->referencing_view) - table->grant= orig_table_list->grant; - else - table->grant.privilege= SELECT_ACL; -#endif - orig_table_list->db= (char *)""; - orig_table_list->db_length= 0; - // Force read of table stats in the optimizer - table->file->info(HA_STATUS_VARIABLE); - /* Add new temporary table to list of open derived tables */ - table->next= thd->derived_tables; - thd->derived_tables= table; + table->grant.privilege= SELECT_ACL; + if (derived->is_derived()) + derived->grant.privilege= SELECT_ACL; } +#endif + /* Add new temporary table to list of open derived tables */ + table->next= thd->derived_tables; + thd->derived_tables= table; } - else if (orig_table_list->merge_underlying_list) - orig_table_list->set_underlying_merge(); + if (arena) + thd->restore_active_arena(arena, &backup); DBUG_RETURN(res); } -/* - fill derived table +/** + @brief + Runs optimize phase for a derived table/view. + + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + Runs optimize phase for given 'derived' derived table/view. + If optimizer finds out that it's of the type "SELECT a_constant" then this + functions also materializes it. - SYNOPSIS - mysql_derived_filling() - thd Thread handle - lex LEX for this thread - unit node that contains all SELECT's for derived tables - orig_table_list TABLE_LIST for the upper SELECT - - IMPLEMENTATION - Derived table is resolved with temporary table. It is created based on the - queries defined. After temporary table is filled, if this is not EXPLAIN, - then the entire unit / node is deleted. unit is deleted if UNION is used - for derived table and node is deleted is it is a simple SELECT. - If you use this function, make sure it's not called at prepare. - Due to evaluation of LIMIT clause it can not be used at prepared stage. - - RETURN - FALSE OK - TRUE Error + @return FALSE ok. + @return TRUE if an error occur. */ -bool mysql_derived_filling(THD *thd, LEX *lex, TABLE_LIST *orig_table_list) +bool mysql_derived_optimize(THD *thd, LEX *lex, TABLE_LIST *derived) { - TABLE *table= orig_table_list->table; - SELECT_LEX_UNIT *unit= orig_table_list->derived; + SELECT_LEX_UNIT *unit= derived->get_unit(); + SELECT_LEX *first_select= unit->first_select(); + SELECT_LEX *save_current_select= lex->current_select; + bool res= FALSE; - /*check that table creation pass without problem and it is derived table */ - if (table && unit) + if (unit->optimized && !unit->uncacheable && !unit->describe) + return FALSE; + lex->current_select= first_select; + + if (unit->is_union()) + { + // optimize union without execution + res= unit->optimize(); + } + else if (unit->derived) { - SELECT_LEX *first_select= unit->first_select(); - select_union *derived_result= orig_table_list->derived_result; - SELECT_LEX *save_current_select= lex->current_select; - if (unit->is_union()) + if (!derived->is_merged_derived()) { - // execute union without clean up - res= unit->exec(); + unit->optimized= TRUE; + if ((res= first_select->join->optimize())) + goto err; } - else + } + /* + Materialize derived tables/views of the "SELECT a_constant" type. + Such tables should be materialized at the optimization phase for + correct constant evaluation. + */ + if (!res && derived->fill_me && !derived->merged_for_insert) + { + if (derived->is_merged_derived()) { - unit->set_limit(first_select); - if (unit->select_limit_cnt == HA_POS_ERROR) - first_select->options&= ~OPTION_FOUND_ROWS; - - lex->current_select= first_select; - res= mysql_select(thd, &first_select->ref_pointer_array, - (TABLE_LIST*) first_select->table_list.first, - first_select->with_wild, - first_select->item_list, first_select->where, - (first_select->order_list.elements+ - first_select->group_list.elements), - (ORDER *) first_select->order_list.first, - (ORDER *) first_select->group_list.first, - first_select->having, (ORDER*) NULL, - (first_select->options | thd->options | - SELECT_NO_UNLOCK), - derived_result, unit, first_select); + derived->change_refs_to_fields(); + derived->set_materialized_derived(); } + if ((res= mysql_derived_create(thd, lex, derived))) + goto err; + if ((res= mysql_derived_fill(thd, lex, derived))) + goto err; + } +err: + lex->current_select= save_current_select; + return res; +} - if (!res) - { - /* - Here we entirely fix both TABLE_LIST and list of SELECT's as - there were no derived tables - */ - if (derived_result->flush()) - res= TRUE; - if (!lex->describe) - unit->cleanup(); - } - else - unit->cleanup(); - lex->current_select= save_current_select; +/** + @brief + Actually create result table for a materialized derived table/view. + + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + This function actually creates the result table for given 'derived' + table/view, but it doesn't fill it. + 'thd' and 'lex' parameters are not used by this function. + + @return FALSE ok. + @return TRUE if an error occur. +*/ + +bool mysql_derived_create(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + TABLE *table= derived->table; + SELECT_LEX_UNIT *unit= derived->get_unit(); + + if (table->created) + return FALSE; + select_union *result= (select_union*)unit->result; + if (table->s->db_type() == TMP_ENGINE_HTON) + { + if (create_internal_tmp_table(table, result->tmp_table_param.keyinfo, + result->tmp_table_param.start_recinfo, + &result->tmp_table_param.recinfo, + (unit->first_select()->options | + thd->options | TMP_TABLE_ALL_COLUMNS))) + return(TRUE); + } + if (open_tmp_table(table)) + return TRUE; + table->file->extra(HA_EXTRA_WRITE_CACHE); + table->file->extra(HA_EXTRA_IGNORE_DUP_KEY); + return FALSE; +} + + +/* + @brief + Execute subquery of a materialized derived table/view and fill the result + table. + + @param thd Thread handle + @param lex LEX for this thread + @param derived reference to the derived table. + + @details + Execute subquery of given 'derived' table/view and fill the result + table. After result table is filled, if this is not the EXPLAIN statement, + the entire unit / node is deleted. unit is deleted if UNION is used + for derived table and node is deleted is it is a simple SELECT. + 'lex' is unused and 'thd' is passed as an argument to an underlying function. + + @note + If you use this function, make sure it's not called at prepare. + Due to evaluation of LIMIT clause it can not be used at prepared stage. + + @return FALSE OK + @return TRUE Error +*/ + +bool mysql_derived_fill(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + TABLE *table= derived->table; + SELECT_LEX_UNIT *unit= derived->get_unit(); + bool res= FALSE; + + if (unit->executed && !unit->uncacheable && !unit->describe) + return FALSE; + /*check that table creation passed without problems. */ + DBUG_ASSERT(table && table->created); + SELECT_LEX *first_select= unit->first_select(); + select_union *derived_result= derived->derived_result; + SELECT_LEX *save_current_select= lex->current_select; + if (unit->is_union()) + { + // execute union without clean up + res= unit->exec(); + } + else + { + unit->set_limit(first_select); + if (unit->select_limit_cnt == HA_POS_ERROR) + first_select->options&= ~OPTION_FOUND_ROWS; + + lex->current_select= first_select; + res= mysql_select(thd, &first_select->ref_pointer_array, + (TABLE_LIST*) first_select->table_list.first, + first_select->with_wild, + first_select->item_list, first_select->where, + (first_select->order_list.elements+ + first_select->group_list.elements), + (ORDER *) first_select->order_list.first, + (ORDER *) first_select->group_list.first, + first_select->having, (ORDER*) NULL, + (first_select->options | thd->options | + SELECT_NO_UNLOCK), + derived_result, unit, first_select); + } + + if (!res) + { + if (derived_result->flush()) + res= TRUE; + unit->executed= TRUE; } + if (res || !lex->describe) + unit->cleanup(); + lex->current_select= save_current_select; + return res; } + + +/** + @brief + Re-initialize given derived table/view for the next execution. + + @param thd thread handle + @param lex LEX for this thread + @param derived reference to the derived table. + + @details + Re-initialize given 'derived' table/view for the next execution. + All underlying views/derived tables are recursively reinitialized prior + to re-initialization of given derived table. + 'thd' and 'lex' are passed as arguments to called functions. + + @return FALSE OK + @return TRUE Error +*/ + +bool mysql_derived_reinit(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + st_select_lex_unit *unit= derived->get_unit(); + + if (derived->table) + derived->merged_for_insert= FALSE; + unit->unclean(); + unit->types.empty(); + /* for derived tables & PS (which can't be reset by Item_subquery) */ + unit->reinit_exec_mechanism(); + unit->set_thd(thd); + return FALSE; +} === modified file 'sql/sql_help.cc' --- a/sql/sql_help.cc 2009-10-19 17:14:48 +0000 +++ b/sql/sql_help.cc 2010-05-26 20:18:18 +0000 @@ -628,7 +628,7 @@ bool mysqld_help(THD *thd, const char *m Protocol *protocol= thd->protocol; SQL_SELECT *select; st_find_field used_fields[array_elements(init_used_fields)]; - TABLE_LIST *leaves= 0; + List<TABLE_LIST> leaves; TABLE_LIST tables[4]; List<String> topics_list, categories_list, subcategories_list; String name, description, example; @@ -667,7 +667,7 @@ bool mysqld_help(THD *thd, const char *m thd->lex->select_lex.context.first_name_resolution_table= &tables[0]; if (setup_tables(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, - tables, &leaves, FALSE)) + tables, leaves, FALSE, FALSE)) goto error; memcpy((char*) used_fields, (char*) init_used_fields, sizeof(used_fields)); if (init_fields(thd, tables, used_fields, array_elements(used_fields))) === modified file 'sql/sql_insert.cc' --- a/sql/sql_insert.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sql_insert.cc 2010-05-26 20:18:18 +0000 @@ -124,7 +124,7 @@ bool check_view_single_update(List<Item> { it.init(*values); while ((item= it++)) - tables|= item->used_tables(); + tables|= item->view_used_tables(view); } /* Convert to real table bits */ @@ -140,6 +140,11 @@ bool check_view_single_update(List<Item> if (view->check_single_table(&tbl, tables, view) || tbl == 0) goto error; + /* + A buffer for the insert values was allocated for the merged view. + Use it. + */ + //tbl->table->insert_values= view->table->insert_values; view->table= tbl->table; *map= tables; @@ -243,6 +248,10 @@ static int check_insert_fields(THD *thd, */ table_list->next_local= 0; context->resolve_in_table_list_only(table_list); + /* 'Unfix' fields to allow correct marking by the setup_fields function. */ + if (table_list->is_view()) + unfix_fields(fields); + res= setup_fields(thd, 0, fields, MARK_COLUMNS_WRITE, 0, 0); /* Restore the current context. */ @@ -252,7 +261,7 @@ static int check_insert_fields(THD *thd, if (res) return -1; - if (table_list->effective_algorithm == VIEW_ALGORITHM_MERGE) + if (table_list->is_view() && table_list->is_merged_derived()) { if (check_view_single_update(fields, fields_and_values_from_different_maps ? @@ -341,7 +350,8 @@ static int check_update_fields(THD *thd, if (setup_fields(thd, 0, update_fields, MARK_COLUMNS_WRITE, 0, 0)) return -1; - if (insert_table_list->effective_algorithm == VIEW_ALGORITHM_MERGE && + if (insert_table_list->is_view() && + insert_table_list->is_merged_derived() && check_view_single_update(update_fields, &update_values, insert_table_list, map)) return -1; @@ -641,6 +651,12 @@ bool mysql_insert(THD *thd,TABLE_LIST *t table_list->table_name); DBUG_RETURN(TRUE); } + /* + mark the table_list as a target for insert, to skip the DT/view prepare phase + for correct access rights checks + TODO: remove this hack + */ + table_list->skip_prepare_derived= TRUE; if (table_list->lock_type == TL_WRITE_DELAYED) { @@ -652,6 +668,7 @@ bool mysql_insert(THD *thd,TABLE_LIST *t if (open_and_lock_tables(thd, table_list)) DBUG_RETURN(TRUE); } + lock_type= table_list->lock_type; thd_proc_info(thd, "init"); @@ -1010,6 +1027,12 @@ bool mysql_insert(THD *thd,TABLE_LIST *t ::my_ok(thd, (ulong) thd->row_count_func, id, buff); } thd->abort_on_warning= 0; + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } + DBUG_RETURN(FALSE); abort: @@ -1138,6 +1161,11 @@ static bool mysql_prepare_insert_check_t bool insert_into_view= (table_list->view != 0); DBUG_ENTER("mysql_prepare_insert_check_table"); + if (!table_list->updatable) + { + my_error(ER_NON_INSERTABLE_TABLE, MYF(0), table_list->alias, "INSERT"); + DBUG_RETURN(TRUE); + } /* first table in list is the one we'll INSERT into, requires INSERT_ACL. all others require SELECT_ACL only. the ACL requirement below is for @@ -1148,14 +1176,16 @@ static bool mysql_prepare_insert_check_t if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, table_list, - &thd->lex->select_lex.leaf_tables, - select_insert, INSERT_ACL, SELECT_ACL)) + thd->lex->select_lex.leaf_tables, + select_insert, INSERT_ACL, SELECT_ACL, + TRUE)) DBUG_RETURN(TRUE); if (insert_into_view && !fields.elements) { thd->lex->empty_field_list_on_rset= 1; - if (!table_list->table) + if (!thd->lex->select_lex.leaf_tables.head()->table || + table_list->is_multitable()) { my_error(ER_VIEW_NO_INSERT_FIELD_LIST, MYF(0), table_list->view_db.str, table_list->view_name.str); @@ -1246,6 +1276,12 @@ bool mysql_prepare_insert(THD *thd, TABL /* INSERT should have a SELECT or VALUES clause */ DBUG_ASSERT (!select_insert || !values); + if (mysql_handle_derived(thd->lex, DT_INIT)) + DBUG_RETURN(TRUE); + if (table_list->handle_derived(thd->lex, DT_MERGE_FOR_INSERT)) + DBUG_RETURN(TRUE); + if (mysql_handle_list_of_derived(thd->lex, table_list, DT_PREPARE)) + DBUG_RETURN(TRUE); /* For subqueries in VALUES() we should not see the table in which we are inserting (for INSERT ... SELECT this is done by changing table_list, @@ -2913,9 +2949,9 @@ bool mysql_insert_select_prepare(THD *th { LEX *lex= thd->lex; SELECT_LEX *select_lex= &lex->select_lex; - TABLE_LIST *first_select_leaf_table; DBUG_ENTER("mysql_insert_select_prepare"); + /* Statement-based replication of INSERT ... SELECT ... LIMIT is not safe as order of rows is not defined, so in mixed mode we go to row-based. @@ -2941,21 +2977,37 @@ bool mysql_insert_select_prepare(THD *th &select_lex->where, TRUE, FALSE, FALSE)) DBUG_RETURN(TRUE); + DBUG_ASSERT(select_lex->leaf_tables.elements != 0); + List_iterator<TABLE_LIST> ti(select_lex->leaf_tables); + TABLE_LIST *table; + uint insert_tables; + + if (select_lex->first_cond_optimization) + { + /* Back up leaf_tables list. */ + Query_arena *arena= thd->stmt_arena, backup; + arena= thd->activate_stmt_arena_if_needed(&backup); // For easier test + + insert_tables= select_lex->insert_tables; + while ((table= ti++) && insert_tables--) + { + select_lex->leaf_tables_exec.push_back(table); + table->tablenr_exec= table->table->tablenr; + table->map_exec= table->table->map; + } + if (arena) + thd->restore_active_arena(arena, &backup); + } + ti.rewind(); /* exclude first table from leaf tables list, because it belong to INSERT */ - DBUG_ASSERT(select_lex->leaf_tables != 0); - lex->leaf_tables_insert= select_lex->leaf_tables; /* skip all leaf tables belonged to view where we are insert */ - for (first_select_leaf_table= select_lex->leaf_tables->next_leaf; - first_select_leaf_table && - first_select_leaf_table->belong_to_view && - first_select_leaf_table->belong_to_view == - lex->leaf_tables_insert->belong_to_view; - first_select_leaf_table= first_select_leaf_table->next_leaf) - {} - select_lex->leaf_tables= first_select_leaf_table; + insert_tables= select_lex->insert_tables; + while ((table= ti++) && insert_tables--) + ti.remove(); + DBUG_RETURN(FALSE); } @@ -3169,7 +3221,7 @@ void select_insert::cleanup() select_insert::~select_insert() { DBUG_ENTER("~select_insert"); - if (table) + if (table && table->created) { table->next_number_field=0; table->auto_increment_field_not_null= FALSE; === modified file 'sql/sql_join_cache.cc' --- a/sql/sql_join_cache.cc 2010-03-07 15:41:45 +0000 +++ b/sql/sql_join_cache.cc 2010-05-26 20:18:18 +0000 @@ -2370,6 +2370,8 @@ JOIN_CACHE_BKA::init_join_matching_recor init_mrr_buff(); + if (!join_tab->preread_init_done && join_tab->preread_init()) + return NESTED_LOOP_ERROR; /* Prepare to iterate over keys from the join buffer and to get matching candidates obtained with MMR handler functions. === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.cc 2010-05-26 20:18:18 +0000 @@ -23,6 +23,7 @@ #include <hash.h> #include "sp.h" #include "sp_head.h" +#include "sql_select.h" /* We are using pointer to this variable for distinguishing between assignment @@ -317,7 +318,6 @@ void lex_start(THD *thd) lex->derived_tables= 0; lex->lock_option= TL_READ; lex->safe_to_cache_query= 1; - lex->leaf_tables_insert= 0; lex->parsing_options.reset(); lex->empty_field_list_on_rset= 0; lex->select_lex.select_number= 1; @@ -1590,6 +1590,7 @@ void st_select_lex_unit::init_query() item_list.empty(); describe= 0; found_rows_for_union= 0; + derived= 0; } void st_select_lex::init_query() @@ -1598,7 +1599,8 @@ void st_select_lex::init_query() table_list.empty(); top_join_list.empty(); join_list= &top_join_list; - embedding= leaf_tables= 0; + embedding= 0; + leaf_tables.empty(); item_list.empty(); join= 0; having= prep_having= where= prep_where= 0; @@ -2060,9 +2062,27 @@ void st_select_lex::print_order(String * { if (order->counter_used) { - char buffer[20]; - size_t length= my_snprintf(buffer, 20, "%d", order->counter); - str->append(buffer, (uint) length); + if (query_type != QT_VIEW_INTERNAL) + { + char buffer[20]; + size_t length= my_snprintf(buffer, 20, "%d", order->counter); + str->append(buffer, (uint) length); + } + else + { + /* replace numeric reference with expression */ + if (order->item[0]->type() == Item::INT_ITEM && + order->item[0]->basic_const_item()) + { + char buffer[20]; + size_t length= my_snprintf(buffer, 20, "%d", order->counter); + str->append(buffer, (uint) length); + /* make it expression instead of integer constant */ + str->append(STRING_WITH_LEN("+0")); + } + else + (*order->item)->print(str, query_type); + } } else (*order->item)->print(str, query_type); @@ -2264,22 +2284,6 @@ bool st_lex::can_be_merged() /* find non VIEW subqueries/unions */ bool selects_allow_merge= select_lex.next_select() == 0; - if (selects_allow_merge) - { - for (SELECT_LEX_UNIT *tmp_unit= select_lex.first_inner_unit(); - tmp_unit; - tmp_unit= tmp_unit->next_unit()) - { - if (tmp_unit->first_select()->parent_lex == this && - (tmp_unit->item == 0 || - (tmp_unit->item->place() != IN_WHERE && - tmp_unit->item->place() != IN_ON))) - { - selects_allow_merge= 0; - break; - } - } - } return (selects_allow_merge && select_lex.group_list.elements == 0 && @@ -2909,7 +2913,11 @@ static void fix_prepare_info_in_table_li tbl->prep_on_expr= tbl->on_expr; tbl->on_expr= tbl->on_expr->copy_andor_structure(thd); } - fix_prepare_info_in_table_list(thd, tbl->merge_underlying_list); + if (tbl->is_view_or_derived() && tbl->is_merged_derived()) + { + SELECT_LEX *sel= tbl->get_single_select(); + fix_prepare_info_in_table_list(thd, sel->get_table_list()); + } } } @@ -3024,6 +3032,384 @@ bool st_select_lex::add_index_hint (THD str, length)); } + +/** + @brief Process all derived tables/views of the SELECT. + + @param lex LEX of this thread + @param phase phases to run derived tables/views through + + @details + This function runs specified 'phases' on all tables from the + table_list of this select. + + @return FALSE ok. + @return TRUE an error occur. +*/ + +bool st_select_lex::handle_derived(struct st_lex *lex, uint phases) +{ + for (TABLE_LIST *cursor= (TABLE_LIST*) table_list.first; + cursor; + cursor= cursor->next_local) + { + if (cursor->is_view_or_derived() && cursor->handle_derived(lex, phases)) + return TRUE; + } + return FALSE; +} + + +/** + @brief + Returns first unoccupied table map and table number + + @param map [out] return found map + @param tablenr [out] return found tablenr + + @details + Returns first unoccupied table map and table number in this select. + Map and table are returned in *'map' and *'tablenr' accordingly. + + @retrun TRUE no free table map/table number + @return FALSE found free table map/table number +*/ + +bool st_select_lex::get_free_table_map(table_map *map, uint *tablenr) +{ + *map= 0; + *tablenr= 0; + TABLE_LIST *tl; + if (!join) + { + (*map)= 1<<1; + (*tablenr)++; + return FALSE; + } + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (tl->table->map > *map) + *map= tl->table->map; + if (tl->table->tablenr > *tablenr) + *tablenr= tl->table->tablenr; + } + (*map)<<= 1; + (*tablenr)++; + if (*tablenr >= MAX_TABLES) + return TRUE; + return FALSE; +} + + +/** + @brief + Append given table to the leaf_tables list. + + @param link Offset to which list in table structure to use + @param table Table to append + + @details + Append given 'table' to the leaf_tables list using the 'link' offset. + If the 'table' is linked with other tables through next_leaf/next_local + chains then whole list will be appended. +*/ + +void st_select_lex::append_table_to_list(TABLE_LIST *TABLE_LIST::*link, + TABLE_LIST *table) +{ + TABLE_LIST *tl; + for (tl= leaf_tables.head(); tl->*link; tl= tl->*link); + tl->*link= table; +} + +/* + @brief + Remove given table from the leaf_tables list. + + @param link Offset to which list in table structure to use + @param table Table to remove + + @details + Remove 'table' from the leaf_tables list using the 'link' offset. +*/ + +void st_select_lex::remove_table_from_list(TABLE_LIST *table) +{ + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (tl == table) + { + ti.remove(); + break; + } + } +} + + +/** + @brief + Assigns new table maps to tables in the leaf_tables list + + @param derived Derived table to take initial table map from + @param map table map to begin with + @param tablenr table number to begin with + @param parent_lex new parent select_lex + + @details + Assign new table maps/table numbers to all tables in the leaf_tables list. + 'map'/'tablenr' are used for the first table and shifted to left/ + increased for each consequent table in the leaf_tables list. + If the 'derived' table is given then it's table map/number is used for the + first table in the list and 'map'/'tablenr' are used for the second and + all consequent tables. + The 'parent_lex' is set as the new parent select_lex for all tables in the + list. +*/ + +void st_select_lex::remap_tables(TABLE_LIST *derived, table_map map, + uint tablenr, SELECT_LEX *parent_lex) +{ + bool first_table= TRUE; + TABLE_LIST *tl; + table_map first_map; + uint first_tablenr; + + if (derived && derived->table) + { + first_map= derived->table->map; + first_tablenr= derived->table->tablenr; + } + else + { + first_map= map; + map<<= 1; + first_tablenr= tablenr++; + } + /* + Assign table bit/table number. + To the first table of the subselect the table bit/tablenr of the + derived table is assigned. The rest of tables are getting bits + sequentially, starting from the provided table map/tablenr. + */ + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (first_table) + { + first_table= FALSE; + tl->table->set_table_map(first_map, first_tablenr); + } + else + { + tl->table->set_table_map(map, tablenr); + tablenr++; + map<<= 1; + } + SELECT_LEX *old_sl= tl->select_lex; + tl->select_lex= parent_lex; + for(TABLE_LIST *emb= tl->embedding; + emb && emb->select_lex == old_sl; + emb= emb->embedding) + emb->select_lex= parent_lex; + } +} + +/** + @brief + Merge a subquery into this select. + + @param derived derived table of the subquery to be merged + @param subq_select select_lex of the subquery + @param map table map for assigning to merged tables from subquery + @param table_no table number for assigning to merged tables from subquery + + @details + This function merges a subquery into its parent select. In short the + merge operation appends the subquery FROM table list to the parent's + FROM table list. In more details: + .) the top_join_list of the subquery is wrapped into a join_nest + and attached to 'derived' + .) subquery's leaf_tables list is merged with the leaf_tables + list of this select_lex + .) the table maps and table numbers of the tables merged from + the subquery are adjusted to reflect their new binding to + this select + + @return TRUE an error occur + @return FALSE ok +*/ + +bool SELECT_LEX::merge_subquery(TABLE_LIST *derived, SELECT_LEX *subq_select, + uint table_no, table_map map) +{ + derived->wrap_into_nested_join(subq_select->top_join_list); + /* Reconnect the next_leaf chain. */ + leaf_tables.concat(&subq_select->leaf_tables); + + ftfunc_list->concat(subq_select->ftfunc_list); + if (join) + { + Item_in_subselect **in_subq; + Item_in_subselect **in_subq_end; + for (in_subq= subq_select->join->sj_subselects.front(), + in_subq_end= subq_select->join->sj_subselects.back(); + in_subq != in_subq_end; + in_subq++) + { + join->sj_subselects.append(join->thd->mem_root, *in_subq); + (*in_subq)->emb_on_expr_nest= derived; + } + } + /* + Remove merged table from chain. + When merge_subquery is called at a subquery-to-semijoin transformation + the derived isn't in the leaf_tables list, so in this case the call of + remove_table_from_list does not cause any actions. + */ + remove_table_from_list(derived); + + /* Walk through child's tables and adjust table map, tablenr, + * parent_lex */ + subq_select->remap_tables(derived, map, table_no, this); + return FALSE; +} + + +/** + @brief + Mark tables from the leaf_tables list as belong to a derived table. + + @param derived tables will be marked as belonging to this derived + + @details + Run through the leaf_list and mark all tables as belonging to the 'derived'. +*/ + +void SELECT_LEX::mark_as_belong_to_derived(TABLE_LIST *derived) +{ + /* Mark tables as belonging to this DT */ + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + tl->skip_temporary= 1; + tl->belong_to_derived= derived; + } +} + + +/** + @brief + Update used_tables cache for this select + + @details + This function updates used_tables cache of ON expressions of all tables + in the leaf_tables list and of the conds expression (if any). +*/ + +void SELECT_LEX::update_used_tables() +{ + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (tl->on_expr) + { + tl->on_expr->update_used_tables(); + tl->on_expr->walk(&Item::eval_not_null_tables, 0, NULL); + } + TABLE_LIST *embedding= tl->embedding; + while (embedding) + { + if (embedding->on_expr && + embedding->nested_join->join_list.head() == tl) + { + embedding->on_expr->update_used_tables(); + embedding->on_expr->walk(&Item::eval_not_null_tables, 0, NULL); + } + tl= embedding; + embedding= tl->embedding; + } + } + if (join->conds) + { + join->conds->update_used_tables(); + join->conds->walk(&Item::eval_not_null_tables, 0, NULL); + } +} + +/** + @brief + Increase estimated number of records for a derived table/view + + @param records number of records to increase estimate by + + @details + This function increases estimated number of records by the 'records' + for the derived table to which this select belongs to. +*/ + +void SELECT_LEX::increase_derived_records(uint records) +{ + SELECT_LEX_UNIT *unit= master_unit(); + DBUG_ASSERT(unit->derived); + + select_union *result= (select_union*)unit->result; + result->records+= records; +} + + +/** + @brief + Mark select's derived table as a const one. + + @param empty Whether select has an empty result set + + @details + Mark derived table/view of this select as a constant one (to + materialize it at the optimization phase) unless this select belongs to a + union. Estimated number of rows is incremented if this select has non empty + result set. +*/ + +void SELECT_LEX::mark_const_derived(bool empty) +{ + TABLE_LIST *derived= master_unit()->derived; + if (!join->thd->lex->describe && derived) + { + if (!empty) + increase_derived_records(1); + if (!master_unit()->is_union() && !derived->is_merged_derived()) + derived->fill_me= TRUE; + } +} + +bool st_select_lex::save_leaf_tables(THD *thd) +{ + Query_arena *arena= thd->stmt_arena, backup; + if (arena->is_conventional()) + arena= 0; + else + thd->set_n_backup_active_arena(arena, &backup); + + List_iterator_fast<TABLE_LIST> li(leaf_tables); + TABLE_LIST *table; + while ((table= li++)) + { + if (leaf_tables_exec.push_back(table)) + return 1; + table->tablenr_exec= table->table->tablenr; + table->map_exec= table->table->map; + } + if (arena) + thd->restore_active_arena(arena, &backup); + + return 0; +} + /** A routine used by the parser to decide whether we are specifying a full partitioning or if only partitions to add or to split. === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.h 2010-05-26 20:18:18 +0000 @@ -469,6 +469,11 @@ public: friend bool mysql_new_select(struct st_lex *lex, bool move_down); friend bool mysql_make_view(THD *thd, File_parser *parser, TABLE_LIST *table, uint flags); + friend bool mysql_derived_prepare(THD *thd, st_lex *lex, + TABLE_LIST *orig_table_list); + friend bool mysql_derived_merge(THD *thd, st_lex *lex, + TABLE_LIST *orig_table_list); + friend bool TABLE_LIST::init_derived(THD *thd, bool init_view); private: void fast_exclude(); }; @@ -487,13 +492,12 @@ class st_select_lex_unit: public st_sele protected: TABLE_LIST result_table_list; select_union *union_result; - TABLE *table; /* temporary table using for appending UNION results */ - - select_result *result; ulonglong found_rows_for_union; bool saved_error; public: + TABLE *table; /* temporary table using for appending UNION results */ + select_result *result; bool prepared, // prepare phase already performed for UNION (unit) optimized, // optimize phase already performed for UNION (unit) executed, // already executed @@ -520,6 +524,11 @@ public: ha_rows select_limit_cnt, offset_limit_cnt; /* not NULL if unit used in subselect, point to subselect item */ Item_subselect *item; + /* + TABLE_LIST representing this union in the embedding select. Used for + derived tables/views handling. + */ + TABLE_LIST *derived; /* thread handler */ THD *thd; /* @@ -549,6 +558,7 @@ public: /* UNION methods */ bool prepare(THD *thd, select_result *result, ulong additional_options); + bool optimize(); bool exec(); bool cleanup(); inline void unclean() { cleaned= 0; } @@ -610,8 +620,15 @@ public: Beginning of the list of leaves in a FROM clause, where the leaves inlcude all base tables including view tables. The tables are connected by TABLE_LIST::next_leaf, so leaf_tables points to the left-most leaf. - */ - TABLE_LIST *leaf_tables; + + List of all base tables local to a subquery including all view + tables. Unlike 'next_local', this in this list views are *not* + leaves. Created in setup_tables() -> make_leaves_list(). + */ + List<TABLE_LIST> leaf_tables; + List<TABLE_LIST> leaf_tables_exec; + uint insert_tables; + const char *type; /* type of select for EXPLAIN */ SQL_LIST order_list; /* ORDER clause */ @@ -832,6 +849,28 @@ public: void clear_index_hints(void) { index_hints= NULL; } bool is_part_of_union() { return master_unit()->is_union(); } + bool handle_derived(struct st_lex *lex, uint phases); + void append_table_to_list(TABLE_LIST *TABLE_LIST::*link, TABLE_LIST *table); + bool get_free_table_map(table_map *map, uint *tablenr); + void remove_table_from_list(TABLE_LIST *table); + void remap_tables(TABLE_LIST *derived, table_map map, + uint tablenr, st_select_lex *parent_lex); + bool merge_subquery(TABLE_LIST *derived, st_select_lex *subq_lex, + uint tablenr, table_map map); + inline bool is_mergeable() + { + return (next_select() == 0 && group_list.elements == 0 && + having == 0 && with_sum_func == 0 && + table_list.elements >= 1 && !(options & SELECT_DISTINCT) && + select_limit == 0); + } + void mark_as_belong_to_derived(TABLE_LIST *derived); + void increase_derived_records(uint records); + void update_used_tables(); + void mark_const_derived(bool empty); + + bool save_leaf_tables(THD *thd); + private: /* current index hint kind. used in filling up index_hints */ enum index_hint_type current_index_hint_type; @@ -1556,8 +1595,6 @@ typedef struct st_lex : public Query_tab CHARSET_INFO *charset; bool text_string_is_7bit; - /* store original leaf_tables for INSERT SELECT and PS/SP */ - TABLE_LIST *leaf_tables_insert; /** SELECT of CREATE VIEW statement */ LEX_STRING create_view_select; @@ -1673,7 +1710,7 @@ typedef struct st_lex : public Query_tab DERIVED_SUBQUERY and DERIVED_VIEW). */ uint8 derived_tables; - uint8 create_view_algorithm; + uint16 create_view_algorithm; uint8 create_view_check; bool drop_if_exists, drop_temporary, local_file, one_shot_set; bool autocommit; @@ -1836,6 +1873,8 @@ typedef struct st_lex : public Query_tab switch (sql_command) { case SQLCOM_UPDATE: case SQLCOM_UPDATE_MULTI: + case SQLCOM_DELETE: + case SQLCOM_DELETE_MULTI: case SQLCOM_INSERT: case SQLCOM_INSERT_SELECT: case SQLCOM_REPLACE: === modified file 'sql/sql_list.h' --- a/sql/sql_list.h 2009-09-15 10:46:35 +0000 +++ b/sql/sql_list.h 2010-05-26 20:18:18 +0000 @@ -168,6 +168,11 @@ public: { if (!list->is_empty()) { + if (is_empty()) + { + *this= *list; + return; + } *last= list->first; last= list->last; elements+= list->elements; @@ -188,11 +193,13 @@ public: list_node *node= first; list_node *list_first= list->first; elements=0; - while (node && node != list_first) + while (node != list_first) { prev= &node->next; node= node->next; elements++; + if (node == &end_of_list) + return; } *prev= *last; last= prev; === modified file 'sql/sql_load.cc' --- a/sql/sql_load.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_load.cc 2010-05-26 20:18:18 +0000 @@ -164,12 +164,15 @@ int mysql_load(THD *thd,sql_exchange *ex if (open_and_lock_tables(thd, table_list)) DBUG_RETURN(TRUE); + if (mysql_handle_single_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT) || + mysql_handle_single_derived(thd->lex, table_list, DT_PREPARE)) + DBUG_RETURN(TRUE); if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, table_list, - &thd->lex->select_lex.leaf_tables, FALSE, + thd->lex->select_lex.leaf_tables, FALSE, INSERT_ACL | UPDATE_ACL, - INSERT_ACL | UPDATE_ACL)) + INSERT_ACL | UPDATE_ACL, FALSE)) DBUG_RETURN(-1); if (!table_list->table || // do not suport join view !table_list->updatable || // and derived tables === modified file 'sql/sql_olap.cc' --- a/sql/sql_olap.cc 2007-05-10 09:59:39 +0000 +++ b/sql/sql_olap.cc 2010-05-26 20:18:18 +0000 @@ -154,7 +154,7 @@ int handle_olaps(LEX *lex, SELECT_LEX *s if (setup_tables(lex->thd, &select_lex->context, &select_lex->top_join_list, (TABLE_LIST *)select_lex->table_list.first - &select_lex->leaf_tables, FALSE) || + FALSE, FALSE) || setup_fields(lex->thd, 0, select_lex->item_list, MARK_COLUMNS_READ, &all_fields,1) || setup_fields(lex->thd, 0, item_list_copy, MARK_COLUMNS_READ, === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_parse.cc 2010-05-26 20:18:18 +0000 @@ -458,7 +458,7 @@ static void handle_bootstrap_impl(THD *t thd->init_for_queries(); while (fgets(buff, thd->net.max_packet, file)) { - char *query; + char *query, *res; /* strlen() can't be deleted because fgets() doesn't return length */ ulong length= (ulong) strlen(buff); while (buff[length-1] != '\n' && !feof(file)) @@ -2769,6 +2769,9 @@ mysql_execute_command(THD *thd) } } } + if (mysql_handle_single_derived(thd->lex, create_table, + DT_MERGE_FOR_INSERT)) + DBUG_RETURN(1); /* select_create is currently not re-execution friendly and @@ -3300,6 +3303,10 @@ end_with_restore_list: if (!(res= open_and_lock_tables(thd, all_tables))) { + /* + Only the INSERT table should be merged. Other will be handled by + select. + */ /* Skip first table, which is the table we are inserting in */ TABLE_LIST *second_table= first_table->next_local; select_lex->table_list.first= (uchar*) second_table; @@ -5183,6 +5190,8 @@ bool check_single_table_access(THD *thd, /* Show only 1 table for check_grant */ if (!(all_tables->belong_to_view && (thd->lex->sql_command == SQLCOM_SHOW_FIELDS)) && + !(all_tables->is_view() && + all_tables->is_merged_derived()) && check_grant(thd, privilege, all_tables, 0, 1, no_errors)) goto deny; === modified file 'sql/sql_prepare.cc' --- a/sql/sql_prepare.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_prepare.cc 2010-05-26 20:18:18 +0000 @@ -1133,7 +1133,7 @@ static bool mysql_test_insert(Prepared_s If we would use locks, then we have to ensure we are not using TL_WRITE_DELAYED as having two such locks can cause table corruption. */ - if (open_normal_and_derived_tables(thd, table_list, 0)) + if (open_normal_and_derived_tables(thd, table_list, 0, DT_INIT)) goto error; if ((values= its++)) @@ -1217,7 +1217,10 @@ static int mysql_test_update(Prepared_st open_tables(thd, &table_list, &table_count, 0)) goto error; - if (table_list->multitable_view) + if (mysql_handle_derived(thd->lex, DT_INIT)) + goto error; + + if (table_list->is_multitable()) { DBUG_ASSERT(table_list->view != 0); DBUG_PRINT("info", ("Switch to multi-update")); @@ -1231,8 +1234,15 @@ static int mysql_test_update(Prepared_st thd->fill_derived_tables() is false here for sure (because it is preparation of PS, so we even do not check it). */ - if (mysql_handle_derived(thd->lex, &mysql_derived_prepare)) + if (table_list->handle_derived(thd->lex, DT_MERGE_FOR_INSERT) || + table_list->handle_derived(thd->lex, DT_PREPARE)) + goto error; + + if (!table_list->updatable) + { + my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "UPDATE"); goto error; + } #ifndef NO_EMBEDDED_ACCESS_CHECKS /* Force privilege re-checking for views after they have been opened. */ @@ -1286,12 +1296,18 @@ error: static bool mysql_test_delete(Prepared_statement *stmt, TABLE_LIST *table_list) { + uint table_count= 0; THD *thd= stmt->thd; LEX *lex= stmt->lex; DBUG_ENTER("mysql_test_delete"); if (delete_precheck(thd, table_list) || - open_normal_and_derived_tables(thd, table_list, 0)) + open_tables(thd, &table_list, &table_count, 0)) + goto error; + + if (mysql_handle_derived(thd->lex, DT_INIT) || + mysql_handle_list_of_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT) || + mysql_handle_list_of_derived(thd->lex, table_list, DT_PREPARE)) goto error; if (!table_list->table) @@ -1349,7 +1365,8 @@ static int mysql_test_select(Prepared_st goto error; } - if (open_normal_and_derived_tables(thd, tables, 0)) + if (open_normal_and_derived_tables(thd, tables, 0, + DT_PREPARE | DT_CREATE)) goto error; thd->used_tables= 0; // Updated by setup_fields @@ -1410,7 +1427,8 @@ static bool mysql_test_do_fields(Prepare if (tables && check_table_access(thd, SELECT_ACL, tables, UINT_MAX, FALSE)) DBUG_RETURN(TRUE); - if (open_normal_and_derived_tables(thd, tables, 0)) + if (open_normal_and_derived_tables(thd, tables, 0, + DT_PREPARE | DT_CREATE)) DBUG_RETURN(TRUE); DBUG_RETURN(setup_fields(thd, 0, *values, MARK_COLUMNS_NONE, 0, 0)); } @@ -1440,7 +1458,8 @@ static bool mysql_test_set_fields(Prepar if ((tables && check_table_access(thd, SELECT_ACL, tables, UINT_MAX, FALSE)) || - open_normal_and_derived_tables(thd, tables, 0)) + open_normal_and_derived_tables(thd, tables, 0, + DT_PREPARE | DT_CREATE)) goto error; while ((var= it++)) @@ -1477,7 +1496,7 @@ static bool mysql_test_call_fields(Prepa if ((tables && check_table_access(thd, SELECT_ACL, tables, UINT_MAX, FALSE)) || - open_normal_and_derived_tables(thd, tables, 0)) + open_normal_and_derived_tables(thd, tables, 0, DT_PREPARE)) goto err; while ((item= it++)) @@ -1552,6 +1571,7 @@ select_like_stmt_test_with_open(Prepared int (*specific_prepare)(THD *thd), ulong setup_tables_done_option) { + uint table_count= 0; DBUG_ENTER("select_like_stmt_test_with_open"); /* @@ -1560,7 +1580,8 @@ select_like_stmt_test_with_open(Prepared prepared EXPLAIN yet so derived tables will clean up after themself. */ - if (open_normal_and_derived_tables(stmt->thd, tables, 0)) + THD *thd= stmt->thd; + if (open_tables(thd, &tables, &table_count, 0)) DBUG_RETURN(TRUE); DBUG_RETURN(select_like_stmt_test(stmt, specific_prepare, @@ -1605,7 +1626,8 @@ static bool mysql_test_create_table(Prep create_table->skip_temporary= true; } - if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0)) + if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0, + DT_PREPARE | DT_CREATE)) DBUG_RETURN(TRUE); if (!(lex->create_info.options & HA_LEX_CREATE_TMP_TABLE)) @@ -1623,7 +1645,8 @@ static bool mysql_test_create_table(Prep we validate metadata of all CREATE TABLE statements, which keeps metadata validation code simple. */ - if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0)) + if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0, + DT_PREPARE)) DBUG_RETURN(TRUE); } @@ -1658,7 +1681,7 @@ static bool mysql_test_create_view(Prepa if (create_view_precheck(thd, tables, view, lex->create_view_mode)) goto err; - if (open_normal_and_derived_tables(thd, tables, 0)) + if (open_normal_and_derived_tables(thd, tables, 0, DT_PREPARE)) goto err; lex->view_prepare_mode= 1; @@ -2349,6 +2372,7 @@ void reinit_stmt_before_use(THD *thd, LE /* Fix ORDER list */ for (order= (ORDER *)sl->order_list.first; order; order= order->next) order->item= &order->item_ptr; + sl->handle_derived(lex, DT_REINIT); /* clear the no_error flag for INSERT/UPDATE IGNORE */ sl->no_error= FALSE; @@ -2392,9 +2416,6 @@ void reinit_stmt_before_use(THD *thd, LE } lex->current_select= &lex->select_lex; - /* restore original list used in INSERT ... SELECT */ - if (lex->leaf_tables_insert) - lex->select_lex.leaf_tables= lex->leaf_tables_insert; if (lex->result) { === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-03-29 20:09:40 +0000 +++ b/sql/sql_select.cc 2010-05-26 20:18:18 +0000 @@ -47,8 +47,8 @@ const char *join_type_str[]={ "UNKNOWN", struct st_sargable_param; static void optimize_keyuse(JOIN *join, DYNAMIC_ARRAY *keyuse_array); -static bool make_join_statistics(JOIN *join, TABLE_LIST *leaves, COND *conds, - DYNAMIC_ARRAY *keyuse); +static bool make_join_statistics(JOIN *join, List<TABLE_LIST> &leaves, + COND *conds, DYNAMIC_ARRAY *keyuse); static bool update_ref_and_keys(THD *thd, DYNAMIC_ARRAY *keyuse, JOIN_TAB *join_tab, uint tables, COND *conds, @@ -99,7 +99,8 @@ static void update_depend_map(JOIN *join static void update_depend_map(JOIN *join, ORDER *order); static ORDER *remove_const(JOIN *join,ORDER *first_order,COND *cond, bool change_list, bool *simple_order); -static int return_zero_rows(JOIN *join, select_result *res,TABLE_LIST *tables, +static int return_zero_rows(JOIN *join, select_result *res, + List<TABLE_LIST> &tables, List<Item> &fields, bool send_row, ulonglong select_options, const char *info, Item *having); @@ -210,7 +211,7 @@ static ORDER *create_distinct_group(THD List<Item> &all_fields, bool *all_order_by_fields_used); static bool test_if_subpart(ORDER *a,ORDER *b); -static TABLE *get_sort_by_table(ORDER *a,ORDER *b,TABLE_LIST *tables); +static TABLE *get_sort_by_table(ORDER *a,ORDER *b,List<TABLE_LIST> &tables); static void calc_group_buffer(JOIN *join,ORDER *group); static bool make_group_fields(JOIN *main_join, JOIN *curr_join); static bool alloc_group_fields(JOIN *join,ORDER *group); @@ -237,6 +238,7 @@ static void add_group_and_distinct_keys( void get_partial_join_cost(JOIN *join, uint idx, double *read_time_arg, double *record_count_arg); static uint make_join_orderinfo(JOIN *join); +static bool generate_derived_keys(DYNAMIC_ARRAY *keyuse_array); static int join_read_record_no_init(JOIN_TAB *tab); @@ -405,7 +407,7 @@ fix_inner_refs(THD *thd, List<Item> &all */ inline int setup_without_group(THD *thd, Item **ref_pointer_array, TABLE_LIST *tables, - TABLE_LIST *leaves, + List<TABLE_LIST> &leaves, List<Item> &fields, List<Item> &all_fields, COND **conds, @@ -483,28 +485,26 @@ JOIN::prepare(Item ***rref_pointer_array join_list= &select_lex->top_join_list; union_part= unit_arg->is_union(); + if (select_lex->handle_derived(thd->lex, DT_PREPARE)) + DBUG_RETURN(1); + thd->lex->current_select->is_item_list_lookup= 1; /* If we have already executed SELECT, then it have not sense to prevent its table from update (see unique_table()) + Affects only materialized derived tables. */ - if (thd->derived_tables_processing) - select_lex->exclude_from_table_unique_test= TRUE; - /* Check that all tables, fields, conds and order are ok */ - - if (!(select_options & OPTION_SETUP_TABLES_DONE) && - setup_tables_and_check_access(thd, &select_lex->context, join_list, - tables_list, &select_lex->leaf_tables, - FALSE, SELECT_ACL, SELECT_ACL)) + if (!(select_options & OPTION_SETUP_TABLES_DONE)) + { + if (setup_tables_and_check_access(thd, &select_lex->context, join_list, + tables_list, select_lex->leaf_tables, + FALSE, SELECT_ACL, SELECT_ACL, FALSE)) DBUG_RETURN(-1); - - TABLE_LIST *table_ptr; - for (table_ptr= select_lex->leaf_tables; - table_ptr; - table_ptr= table_ptr->next_leaf) - tables++; + } + tables= select_lex->leaf_tables.elements; + if (setup_wild(thd, tables_list, fields_list, &all_fields, wild_num) || select_lex->setup_ref_array(thd, og_num) || setup_fields(thd, (*rref_pointer_array), fields_list, MARK_COLUMNS_READ, @@ -605,10 +605,6 @@ JOIN::prepare(Item ***rref_pointer_array } } - if (setup_ftfuncs(select_lex)) /* should be after having->fix_fields */ - DBUG_RETURN(-1); - - /* Check if there are references to un-aggregated columns when computing aggregate functions with implicit grouping (there is no GROUP BY). @@ -720,13 +716,37 @@ JOIN::optimize() if (optimized) DBUG_RETURN(0); optimized= 1; - thd_proc_info(thd, "optimizing"); + + /* Run optimize phase for all derived tables/views used in this SELECT. */ + if (select_lex->handle_derived(thd->lex, DT_OPTIMIZE)) + DBUG_RETURN(1); + + if (select_lex->first_cond_optimization) + { + //Do it only for the first execution + /* Merge all mergeable derived tables/views in this SELECT. */ + if (select_lex->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + tables= select_lex->leaf_tables.elements; + select_lex->update_used_tables(); + + /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ + if (convert_join_subqueries_to_semijoins(this)) + DBUG_RETURN(1); /* purecov: inspected */ + /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ + select_lex->update_used_tables(); + + /* Save this info for the next executions */ + if (select_lex->save_leaf_tables(thd)) + DBUG_RETURN(1); + } + + tables= select_lex->leaf_tables.elements; + - /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ - if (convert_join_subqueries_to_semijoins(this)) - DBUG_RETURN(1); /* purecov: inspected */ - /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ + if (setup_ftfuncs(select_lex)) /* should be after having->fix_fields */ + DBUG_RETURN(-1); row_limit= ((select_distinct || order || group_list) ? HA_POS_ERROR : unit->select_limit_cnt); @@ -760,7 +780,8 @@ JOIN::optimize() } } #endif - SELECT_LEX *sel= thd->lex->current_select; + + SELECT_LEX *sel= select_lex; if (sel->first_cond_optimization) { /* @@ -785,7 +806,7 @@ JOIN::optimize() if (arena) thd->restore_active_arena(arena, &backup); } - + conds= optimize_cond(this, conds, join_list, &cond_value); if (thd->is_error()) { @@ -823,7 +844,8 @@ JOIN::optimize() #ifdef WITH_PARTITION_STORAGE_ENGINE { TABLE_LIST *tbl; - for (tbl= select_lex->leaf_tables; tbl; tbl= tbl->next_leaf) + List_iterator_fast<TABLE_LIST> li(select_lex->leaf_tables); + while ((tbl= li++)) { /* If tbl->embedding!=NULL that means that this table is in the inner @@ -930,6 +952,8 @@ JOIN::optimize() DBUG_RETURN(1); } + drop_unused_derived_keys(); + if (rollup.state != ROLLUP::STATE_NONE) { if (rollup_process_const_fields()) @@ -1030,6 +1054,7 @@ JOIN::optimize() { zero_result_cause= "Impossible WHERE noticed after reading const tables"; + select_lex->mark_const_derived(zero_result_cause); goto setup_subq_exit; } @@ -1348,7 +1373,7 @@ JOIN::optimize() if (select_options & SELECT_DESCRIBE) { error= 0; - DBUG_RETURN(0); + goto derived_exit; } having= 0; @@ -1497,6 +1522,9 @@ setup_subq_exit: if (setup_subquery_materialization()) DBUG_RETURN(1); error= 0; + +derived_exit: + select_lex->mark_const_derived(zero_result_cause); DBUG_RETURN(0); } @@ -1733,6 +1761,11 @@ JOIN::exec() !tables ? "No tables used" : NullS); DBUG_VOID_RETURN; } + else + { + /* it's a const select, materialize it. */ + select_lex->mark_const_derived(zero_result_cause); + } JOIN *curr_join= this; List<Item> *curr_all_fields= &all_fields; @@ -2232,6 +2265,7 @@ JOIN::destroy() } tmp_join->tmp_join= 0; tmp_table_param.cleanup(); + tmp_join->tmp_table_param.copy_field= 0; DBUG_RETURN(tmp_join->destroy()); } cond_equal= 0; @@ -2512,12 +2546,11 @@ typedef struct st_sargable_param */ static bool -make_join_statistics(JOIN *join, TABLE_LIST *tables_arg, COND *conds, - DYNAMIC_ARRAY *keyuse_array) +make_join_statistics(JOIN *join, List<TABLE_LIST> &tables_list, + COND *conds, DYNAMIC_ARRAY *keyuse_array) { - int error; + int error= 0; TABLE *table; - TABLE_LIST *tables= tables_arg; uint i,table_count,const_count,key; table_map found_const_table_map, all_table_map, found_ref, refs; key_map const_ref, eq_part; @@ -2528,6 +2561,8 @@ make_join_statistics(JOIN *join, TABLE_L table_map no_rows_const_tables= 0; SARGABLE_PARAM *sargables= 0; JOIN_TAB *stat_vector[MAX_TABLES+1]; + List_iterator<TABLE_LIST> ti(tables_list); + TABLE_LIST *tables; DBUG_ENTER("make_join_statistics"); table_count=join->tables; @@ -2543,9 +2578,7 @@ make_join_statistics(JOIN *join, TABLE_L found_const_table_map= all_table_map=0; const_count=0; - for (s= stat, i= 0; - tables; - s++, tables= tables->next_leaf, i++) + for (s= stat, i= 0; (tables= ti++); s++, i++) { TABLE_LIST *embedding= tables->embedding; stat_vector[i]=s; @@ -2555,7 +2588,7 @@ make_join_statistics(JOIN *join, TABLE_L s->needed_reg.init(); table_vector[i]=s->table=table=tables->table; table->pos_in_table_list= tables; - error= table->file->info(HA_STATUS_VARIABLE | HA_STATUS_NO_LOCK); + error= tables->fetch_number_of_rows(); if (error) { table->file->print_error(error, MYF(0)); @@ -2626,6 +2659,7 @@ make_join_statistics(JOIN *join, TABLE_L no_rows_const_tables |= table->map; } } + stat_vector[i]=0; join->outer_join=outer_join; @@ -2641,6 +2675,8 @@ make_join_statistics(JOIN *join, TABLE_L */ for (i= 0, s= stat ; i < table_count ; i++, s++) { + if (!s->dependent) + continue; for (uint j= 0 ; j < table_count ; j++) { table= stat[j].table; @@ -2874,7 +2910,7 @@ make_join_statistics(JOIN *join, TABLE_L } /* Approximate found rows and time to read them */ s->found_records=s->records=s->table->file->stats.records; - s->read_time=(ha_rows) s->table->file->scan_time(); + s->scan_time(); /* Set a max range of how many seeks we can expect when using keys @@ -2959,17 +2995,31 @@ make_join_statistics(JOIN *join, TABLE_L if (optimize_semijoin_nests(join, all_table_map)) DBUG_RETURN(TRUE); /* purecov: inspected */ - /* Find an optimal join order of the non-constant tables. */ - if (join->const_tables != join->tables) - { - if (choose_plan(join, all_table_map & ~join->const_table_map)) - goto error; - } - else - { - memcpy((uchar*) join->best_positions,(uchar*) join->positions, - sizeof(POSITION)*join->const_tables); - join->best_read=1.0; + { + ha_rows records= 1; + SELECT_LEX_UNIT *unit= join->select_lex->master_unit(); + + /* Find an optimal join order of the non-constant tables. */ + if (join->const_tables != join->tables) + { + if (choose_plan(join, all_table_map & ~join->const_table_map)) + goto error; + /* + Calculate estimated number of rows for materialized derived + table/view. + */ + for (i= 0; i < join->tables ; i++) + records*= join->best_positions[i].records_read ? + (ha_rows)join->best_positions[i].records_read : 1; + } + else + { + memcpy((uchar*) join->best_positions,(uchar*) join->positions, + sizeof(POSITION)*join->const_tables); + join->best_read=1.0; + } + if (unit->derived && unit->derived->is_materialized_derived()) + join->select_lex->increase_derived_records(records); } /* Generate an execution plan from the found optimal join order. */ DBUG_RETURN(join->thd->killed || get_best_combination(join)); @@ -2981,8 +3031,12 @@ error: may not be assigned yet by this function (which is building join_tab). Dangling TABLE::reginfo.join_tab may cause part_of_refkey to choke. */ - for (tables= tables_arg; tables; tables= tables->next_leaf) - tables->table->reginfo.join_tab= NULL; + { + TABLE_LIST *table; + List_iterator<TABLE_LIST> ti(tables_list); + while ((table= ti++)) + table->table->reginfo.join_tab= NULL; + } DBUG_RETURN (1); } @@ -3245,14 +3299,20 @@ add_key_field(KEY_FIELD **key_fields,uin Field *field, bool eq_func, Item **value, uint num_values, table_map usable_tables, SARGABLE_PARAM **sargables) { - uint exists_optimize= 0; - if (!(field->flags & PART_KEY_FLAG)) + uint optimize= 0; + if (eq_func && + field->table->pos_in_table_list->is_materialized_derived() && + !field->table->created) + { + optimize= KEY_OPTIMIZE_EQ; + } + else if (!(field->flags & PART_KEY_FLAG)) { // Don't remove column IS NULL on a LEFT JOIN table if (!eq_func || (*value)->type() != Item::NULL_ITEM || !field->table->maybe_null || field->null_ptr) return; // Not a key. Skip it - exists_optimize= KEY_OPTIMIZE_EXISTS; + optimize= KEY_OPTIMIZE_EXISTS; DBUG_ASSERT(num_values == 1); } else @@ -3272,12 +3332,12 @@ add_key_field(KEY_FIELD **key_fields,uin if (!eq_func || (*value)->type() != Item::NULL_ITEM || !field->table->maybe_null || field->null_ptr) return; // Can't use left join optimize - exists_optimize= KEY_OPTIMIZE_EXISTS; + optimize= KEY_OPTIMIZE_EXISTS; } else { JOIN_TAB *stat=field->table->reginfo.join_tab; - key_map possible_keys=field->key_start; + key_map possible_keys=field->get_possible_keys(); possible_keys.intersect(field->table->keys_in_use_for_query); stat[0].keys.merge(possible_keys); // Add possible keys @@ -3371,7 +3431,7 @@ add_key_field(KEY_FIELD **key_fields,uin (*key_fields)->eq_func= eq_func; (*key_fields)->val= *value; (*key_fields)->level= and_level; - (*key_fields)->optimize= exists_optimize; + (*key_fields)->optimize= optimize; /* If the condition has form "tbl.keypart = othertbl.field" and othertbl.field can be NULL, there will be no matches if othertbl.field @@ -3690,6 +3750,34 @@ max_part_bit(key_part_map bits) return found; } +static bool +add_keyuse(DYNAMIC_ARRAY *keyuse_array, KEY_FIELD *key_field, + uint key, uint part) +{ + KEYUSE keyuse; + Field *field= key_field->field; + + keyuse.table= field->table; + keyuse.val= key_field->val; + keyuse.key= key; + if (key != MAX_KEY) + { + keyuse.keypart=part; + keyuse.keypart_map= (key_part_map) 1 << part; + } + else + { + keyuse.keypart= field->field_index; + keyuse.keypart_map= (key_part_map) 0; + } + keyuse.used_tables= key_field->val->used_tables(); + keyuse.optimize= key_field->optimize & KEY_OPTIMIZE_REF_OR_NULL; + keyuse.null_rejecting= key_field->null_rejecting; + keyuse.cond_guard= key_field->cond_guard; + keyuse.sj_pred_no= key_field->sj_pred_no; + return (insert_dynamic(keyuse_array,(uchar*) &keyuse)); +} + /* Add all keys with uses 'field' for some keypart If field->and_level != and_level then only mark key_part as const_part @@ -3704,10 +3792,13 @@ add_key_part(DYNAMIC_ARRAY *keyuse_array { Field *field=key_field->field; TABLE *form= field->table; - KEYUSE keyuse; if (key_field->eq_func && !(key_field->optimize & KEY_OPTIMIZE_EXISTS)) { + if (key_field->eq_func && (key_field->optimize & KEY_OPTIMIZE_EQ)) + { + return add_keyuse(keyuse_array, key_field, MAX_KEY, 0); + } for (uint key=0 ; key < form->s->keys ; key++) { if (!(form->keys_in_use_for_query.is_set(key))) @@ -3720,17 +3811,7 @@ add_key_part(DYNAMIC_ARRAY *keyuse_array { if (field->eq(form->key_info[key].key_part[part].field)) { - keyuse.table= field->table; - keyuse.val = key_field->val; - keyuse.key = key; - keyuse.keypart=part; - keyuse.keypart_map= (key_part_map) 1 << part; - keyuse.used_tables=key_field->val->used_tables(); - keyuse.optimize= key_field->optimize & KEY_OPTIMIZE_REF_OR_NULL; - keyuse.null_rejecting= key_field->null_rejecting; - keyuse.cond_guard= key_field->cond_guard; - keyuse.sj_pred_no= key_field->sj_pred_no; - if (insert_dynamic(keyuse_array,(uchar*) &keyuse)) + if (add_keyuse(keyuse_array, key_field, key, part)) return TRUE; } } @@ -3815,6 +3896,9 @@ sort_keyuse(KEYUSE *a,KEYUSE *b) return (int) (a->table->tablenr - b->table->tablenr); if (a->key != b->key) return (int) (a->key - b->key); + if (a->key == MAX_KEY && b->key == MAX_KEY && + a->used_tables != b->used_tables) + return (int) ((ulong) a->used_tables - (ulong) b->used_tables); if (a->keypart != b->keypart) return (int) (a->keypart - b->keypart); // Place const values before other ones @@ -3965,19 +4049,21 @@ update_ref_and_keys(THD *thd, DYNAMIC_AR if (my_init_dynamic_array(keyuse,sizeof(KEYUSE),20,64)) return TRUE; + if (cond) { + KEY_FIELD *saved_field= field; add_key_fields(join_tab->join, &end, &and_level, cond, normal_tables, sargables); for (; field != end ; field++) { - if (add_key_part(keyuse,field)) - return TRUE; + /* Mark that we can optimize LEFT JOIN */ if (field->val->type() == Item::NULL_ITEM && !field->field->real_maybe_null()) field->field->table->reginfo.not_exists_optimize=1; } + field= saved_field; } for (i=0 ; i < tables ; i++) { @@ -4042,6 +4128,8 @@ update_ref_and_keys(THD *thd, DYNAMIC_AR if (insert_dynamic(keyuse,(uchar*) &key_end)) return TRUE; + generate_derived_keys(keyuse); + use=save_pos=dynamic_element(keyuse,0,KEYUSE*); prev= &key_end; found_eq_constant=0; @@ -4107,7 +4195,7 @@ static void optimize_keyuse(JOIN *join, ~OUTER_REF_TABLE_BIT))) { uint tablenr; - for (tablenr=0 ; ! (map & 1) ; map>>=1, tablenr++) ; + tablenr= my_count_bits(map); if (map == 1) // Only one table { TABLE *tmp_table=join->all_tables[tablenr]; @@ -4714,7 +4802,7 @@ best_access_path(JOIN *join, else { /* Estimate cost of reading table. */ - tmp= s->table->file->scan_time(); + tmp= s->scan_time(); if ((s->table->map & join->outer_join) || disable_jbuf) // Can't use join cache { /* @@ -5065,6 +5153,7 @@ optimize_straight_join(JOIN *join, table { JOIN_TAB *s; uint idx= join->const_tables; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; double record_count= 1.0; double read_time= 0.0; POSITION loose_scan_pos; @@ -5072,7 +5161,7 @@ optimize_straight_join(JOIN *join, table for (JOIN_TAB **pos= join->best_ref + idx ; (s= *pos) ; pos++) { /* Find the best access method from 's' to the current partial plan */ - best_access_path(join, s, join_tables, idx, FALSE, record_count, + best_access_path(join, s, join_tables, idx, disable_jbuf, record_count, join->positions + idx, &loose_scan_pos); /* compute the cost of the new plan extended with 's' */ @@ -5452,6 +5541,7 @@ best_extension_by_limited_search(JOIN JOIN_TAB *s; double best_record_count= DBL_MAX; double best_read_time= DBL_MAX; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; DBUG_EXECUTE("opt", print_plan(join, idx, record_count, read_time, read_time, "part_plan");); @@ -5473,8 +5563,8 @@ best_extension_by_limited_search(JOIN /* Find the best access method from 's' to the current partial plan */ POSITION loose_scan_pos; - best_access_path(join, s, remaining_tables, idx, FALSE, record_count, - join->positions + idx, &loose_scan_pos); + best_access_path(join, s, remaining_tables, idx, disable_jbuf, + record_count, join->positions + idx, &loose_scan_pos); /* Compute the cost of extending the plan with 's' */ @@ -5618,6 +5708,7 @@ find_best(JOIN *join,table_map rest_tabl JOIN_TAB *s; double best_record_count=DBL_MAX,best_read_time=DBL_MAX; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; for (JOIN_TAB **pos=join->best_ref+idx ; (s=*pos) ; pos++) { table_map real_table_bit=s->table->map; @@ -5626,7 +5717,7 @@ find_best(JOIN *join,table_map rest_tabl { double records, best; POSITION loose_scan_pos; - best_access_path(join, s, rest_tables, idx, FALSE, record_count, + best_access_path(join, s, rest_tables, idx, disable_jbuf, record_count, join->positions + idx, &loose_scan_pos); records= join->positions[idx].records_read; best= join->positions[idx].read_time; @@ -5999,8 +6090,7 @@ static bool create_ref_for_key(JOIN *joi if (keyuse->null_rejecting) j->ref.null_rejecting |= 1 << i; keyuse_uses_no_tables= keyuse_uses_no_tables && !keyuse->used_tables; - if (!keyuse->used_tables && - !(join->select_options & SELECT_DESCRIBE)) + if (!keyuse->used_tables && !thd->lex->describe) { // Compare against constant store_key_item tmp(thd, keyinfo->key_part[i].field, key_buff + maybe_null, @@ -6411,7 +6501,7 @@ make_outerjoin_info(JOIN *join) for ( ; embedding ; embedding= embedding->embedding) { /* Ignore sj-nests: */ - if (!embedding->on_expr) + if (!(embedding->on_expr && embedding->outer_join)) continue; NESTED_JOIN *nested_join= embedding->nested_join; if (!nested_join->counter) @@ -6902,6 +6992,123 @@ make_join_select(JOIN *join,SQL_SELECT * } +static +uint get_next_field_for_derived_key(uchar *arg) +{ + KEYUSE *keyuse= *(KEYUSE **) arg; + if (!keyuse) + return (uint) (-1); + uint key= keyuse->key; + uint fldno= keyuse->keypart; + uint keypart= keyuse->keypart_map == (key_part_map) 1 ? + 0 : (keyuse-1)->keypart+1; + for ( ; keyuse->key == key && keyuse->keypart == fldno; keyuse++) + keyuse->keypart= keypart; + if (keyuse->key != key) + keyuse= 0; + return fldno; +} + + +static +bool generate_derived_keys_for_table(KEYUSE *keyuse, uint count, uint keys) +{ + TABLE *table= keyuse->table; + if (table->alloc_keys(keys)) + return TRUE; + uint keyno= 0; + KEYUSE *first_keyuse= keyuse; + uint prev_part= (uint) (-1); + uint parts= 0; + uint i= 0; + do + { + keyuse->key= keyno; + keyuse->keypart_map= (key_part_map) (1 << parts); + keyuse++; + if (++i == count || keyuse->used_tables != first_keyuse->used_tables) + { + if (table->add_tmp_key(keyno, ++parts, + get_next_field_for_derived_key, + (uchar *) &first_keyuse)) + return TRUE; + first_keyuse= keyuse; + keyno++; + parts= 0; + } + else if (keyuse->keypart != prev_part) + { + parts++; + prev_part= keyuse->keypart; + } + } while (keyno < keys); + return FALSE; +} + + +static +bool generate_derived_keys(DYNAMIC_ARRAY *keyuse_array) +{ + KEYUSE *keyuse= dynamic_element(keyuse_array, 0, KEYUSE*); + uint elements= keyuse_array->elements; + TABLE *prev_table= 0; + for (uint i= 0; i < elements; i++, keyuse++) + { + KEYUSE *first_table_keyuse; + table_map last_used_tables; + uint count; + uint keys; + while (keyuse->key == MAX_KEY) + { + if (keyuse->table != prev_table) + { + prev_table= keyuse->table; + first_table_keyuse= keyuse; + last_used_tables= keyuse->used_tables; + count= 0; + keys= 0; + } + else if (keyuse->used_tables != last_used_tables) + { + keys++; + last_used_tables= keyuse->used_tables; + } + count++; + keyuse++; + if (keyuse->table != prev_table && + generate_derived_keys_for_table(first_table_keyuse, count, ++keys)) + return TRUE; + } + } + return FALSE; +} + + +/* + @brief + Drops unused keys for each materialized derived table/view + + @details + For materialized derived tables only ref access can be used, it employs + only one index, thus we don't need the rest. For each materialized derived + table/view call TABLE::use_index to save one index chosen by the optimizer + and free others. No key is chosen then all keys will be dropped. +*/ + +void JOIN::drop_unused_derived_keys() +{ + for (uint i= const_tables ; i < tables ; i++) + { + JOIN_TAB *tab=join_tab+i; + TABLE *table=tab->table; + if (!table->pos_in_table_list->is_materialized_derived() || + table->max_keys <= 1) + continue; + table->use_index(tab->ref.key); + tab->ref.key= 0; + } +} + /* Determine {after which table we'll produce ordered set} @@ -7695,6 +7902,30 @@ void JOIN_TAB::cleanup() /** + Initialize the join_tab before reading. + Currently only derived table/view materialization is done here. +*/ + +bool JOIN_TAB::preread_init() +{ + TABLE_LIST *derived= table->pos_in_table_list; + if (!derived || !derived->is_materialized_derived()) + { + preread_init_done= TRUE; + return FALSE; + } + + /* Materialize derived table/view. */ + if (!derived->get_unit()->executed && + mysql_handle_single_derived(join->thd->lex, + derived, DT_CREATE | DT_FILL)) + return TRUE; + preread_init_done= TRUE; + return FALSE; +} + + +/** Partially cleanup JOIN after it has executed: close index or rnd read (table cursors), free quick selects. @@ -8118,7 +8349,7 @@ remove_const(JOIN *join,ORDER *first_ord static int -return_zero_rows(JOIN *join, select_result *result,TABLE_LIST *tables, +return_zero_rows(JOIN *join, select_result *result, List<TABLE_LIST> &tables, List<Item> &fields, bool send_row, ulonglong select_options, const char *info, Item *having) { @@ -8134,7 +8365,9 @@ return_zero_rows(JOIN *join, select_resu if (send_row) { - for (TABLE_LIST *table= tables; table; table= table->next_leaf) + List_iterator<TABLE_LIST> ti(tables); + TABLE_LIST *table; + while ((table= ti++)) mark_as_null_row(table->table); // All fields are NULL if (having && having->val_int() == 0) send_row=0; @@ -9764,12 +9997,14 @@ simplify_joins(JOIN *join, List<TABLE_LI { TABLE_LIST *tbl; List_iterator<TABLE_LIST> it(nested_join->join_list); + List<TABLE_LIST> repl_list; while ((tbl= it++)) { tbl->embedding= table->embedding; tbl->join_list= table->join_list; + repl_list.push_back(tbl); } - li.replace(nested_join->join_list); + li.replace(repl_list); /* Need to update the name resolution table chain when flattening joins */ fix_name_res= TRUE; table= *li.ref(); @@ -10707,13 +10942,29 @@ Field *create_tmp_field(THD *thd, TABLE If item have to be able to store NULLs but underlaid field can't do it, create_tmp_field_from_field() can't be used for tmp field creation. */ - if (field->maybe_null && !field->field->maybe_null()) + if ((field->maybe_null || + (orig_item && orig_item->maybe_null)) && /* for outer joined views/dt*/ + !field->field->maybe_null()) { + bool save_maybe_null; + /* + The item the ref points to may have maybe_null flag set while + the ref doesn't have it. This may happen for outer fields + when the outer query decided at some point after name resolution phase + that this field might be null. Take this into account here. + */ + if (orig_item) + { + save_maybe_null= item->maybe_null; + item->maybe_null= orig_item->maybe_null; + } result= create_tmp_field_from_item(thd, item, table, NULL, modify_item, convert_blob_length); *from_field= field->field; if (result && modify_item) field->result_field= result; + if (orig_item) + item->maybe_null= save_maybe_null; } else if (table_cant_handle_bit_fields && field->field->type() == MYSQL_TYPE_BIT) @@ -10858,7 +11109,7 @@ TABLE * create_tmp_table(THD *thd,TMP_TABLE_PARAM *param,List<Item> &fields, ORDER *group, bool distinct, bool save_sum_fields, ulonglong select_options, ha_rows rows_limit, - char *table_alias) + char *table_alias, bool do_not_open) { MEM_ROOT *mem_root_save, own_root; TABLE *table; @@ -11397,7 +11648,7 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA share->uniques= test(using_unique_constraint); table->key_info= table->s->key_info= keyinfo; keyinfo->key_part=key_part_info; - keyinfo->flags=HA_NOSAME; + keyinfo->flags=HA_NOSAME | HA_BINARY_PACK_KEY | HA_PACK_KEY; keyinfo->usable_key_parts=keyinfo->key_parts= param->group_parts; keyinfo->key_length=0; keyinfo->rec_per_key=0; @@ -11483,7 +11734,7 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA bzero((void*) key_part_info, keyinfo->key_parts * sizeof(KEY_PART_INFO)); table->key_info= table->s->key_info= keyinfo; keyinfo->key_part=key_part_info; - keyinfo->flags=HA_NOSAME | HA_NULL_ARE_EQUAL; + keyinfo->flags=HA_NOSAME | HA_NULL_ARE_EQUAL | HA_BINARY_PACK_KEY | HA_PACK_KEY; keyinfo->key_length= 0; // Will compute the sum of the parts below. keyinfo->name= (char*) "distinct_key"; keyinfo->algorithm= HA_KEY_ALG_UNDEF; @@ -11551,15 +11802,17 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA if (thd->is_fatal_error) // If end of memory goto err; /* purecov: inspected */ share->db_record_offset= 1; - if (share->db_type() == TMP_ENGINE_HTON) + if (!do_not_open) { - if (create_internal_tmp_table(table, param->keyinfo, param->start_recinfo, - &param->recinfo, select_options)) + if (share->db_type() == TMP_ENGINE_HTON) + { + if (create_internal_tmp_table(table, param->keyinfo, param->start_recinfo, + &param->recinfo, select_options)) + goto err; + } + if (open_tmp_table(table)) goto err; } - if (open_tmp_table(table)) - goto err; - thd->mem_root= mem_root_save; DBUG_RETURN(table); @@ -11714,6 +11967,7 @@ bool open_tmp_table(TABLE *table) return(1); } (void) table->file->extra(HA_EXTRA_QUICK); /* Faster */ + table->created= TRUE; return(0); } @@ -12022,6 +12276,7 @@ static bool create_internal_tmp_table(TA } status_var_increment(table->in_use->status_var.created_tmp_disk_tables); share->db_record_offset= 1; + table->created= TRUE; DBUG_RETURN(0); err: DBUG_RETURN(1); @@ -12177,7 +12432,7 @@ free_tmp_table(THD *thd, TABLE *entry) save_proc_info=thd->proc_info; thd_proc_info(thd, "removing tmp table"); - if (entry->file) + if (entry->file && entry->created) { if (entry->db_stat) entry->file->ha_drop_table(entry->s->table_name.str); @@ -12777,6 +13032,9 @@ sub_select(JOIN *join,JOIN_TAB *join_tab do_sj_reset(join_tab->flush_weedout_table); } + if (!join_tab->preread_init_done && join_tab->preread_init()) + DBUG_RETURN(NESTED_LOOP_ERROR); + if (join->resume_nested_loop) { /* If not the last table, plunge down the nested loop */ @@ -13135,13 +13393,21 @@ static int join_read_const_table(JOIN_TAB *tab, POSITION *pos) { int error; + TABLE_LIST *tbl; DBUG_ENTER("join_read_const_table"); TABLE *table=tab->table; table->const_table=1; table->null_row=0; table->status=STATUS_NO_RECORD; - if (tab->type == JT_SYSTEM) + if (tab->table->pos_in_table_list->is_materialized_derived() && + !tab->table->pos_in_table_list->fill_me) + { + //TODO: don't get here at all + /* Skip materialized derived tables/views. */ + DBUG_RETURN(0); + } + else if (tab->type == JT_SYSTEM) { if ((error=join_read_system(tab))) { // Info for DESCRIBE @@ -13203,26 +13469,27 @@ join_read_const_table(JOIN_TAB *tab, POS if (!table->null_row) table->maybe_null=0; - /* Check appearance of new constant items in Item_equal objects */ - JOIN *join= tab->join; - if (join->conds) - update_const_equal_items(join->conds, tab); - TABLE_LIST *tbl; - for (tbl= join->select_lex->leaf_tables; tbl; tbl= tbl->next_leaf) { - TABLE_LIST *embedded; - TABLE_LIST *embedding= tbl; - do + JOIN *join= tab->join; + List_iterator<TABLE_LIST> ti(join->select_lex->leaf_tables); + /* Check appearance of new constant items in Item_equal objects */ + if (join->conds) + update_const_equal_items(join->conds, tab); + while ((tbl= ti++)) { - embedded= embedding; - if (embedded->on_expr) - update_const_equal_items(embedded->on_expr, tab); - embedding= embedded->embedding; + TABLE_LIST *embedded; + TABLE_LIST *embedding= tbl; + do + { + embedded= embedding; + if (embedded->on_expr) + update_const_equal_items(embedded->on_expr, tab); + embedding= embedded->embedding; + } + while (embedding && + embedding->nested_join->join_list.head() == embedded); } - while (embedding && - embedding->nested_join->join_list.head() == embedded); } - DBUG_RETURN(0); } @@ -13576,6 +13843,9 @@ int join_init_read_record(JOIN_TAB *tab) { if (tab->select && tab->select->quick && tab->select->quick->reset()) return 1; + if (!tab->preread_init_done && tab->preread_init()) + return 1; + init_read_record(&tab->read_record, tab->join->thd, tab->table, tab->select,1,1, FALSE); return (*tab->read_record.read_record)(&tab->read_record); @@ -15500,6 +15770,8 @@ create_sort_index(THD *thd, JOIN *join, get_schema_tables_result(join, PROCESSED_BY_CREATE_SORT_INDEX)) goto err; + if (!tab->preread_init_done && tab->preread_init()) + goto err; if (table->s->tmp_table) table->file->info(HA_STATUS_VARIABLE); // Get record count table->sort.found_records=filesort(thd, table,join->sortorder, length, @@ -16053,7 +16325,7 @@ find_order_in_list(THD *thd, Item **ref_ order->in_field_list= 1; order->counter= count; order->counter_used= 1; - return FALSE; + return FALSE; } /* Lookup the current GROUP/ORDER field in the SELECT clause. */ select_item= find_item_in_list(order_item, fields, &counter, @@ -16494,8 +16766,10 @@ test_if_subpart(ORDER *a,ORDER *b) */ static TABLE * -get_sort_by_table(ORDER *a,ORDER *b,TABLE_LIST *tables) +get_sort_by_table(ORDER *a,ORDER *b, List<TABLE_LIST> &tables) { + TABLE_LIST *table; + List_iterator<TABLE_LIST> ti(tables); table_map map= (table_map) 0; DBUG_ENTER("get_sort_by_table"); @@ -16513,11 +16787,11 @@ get_sort_by_table(ORDER *a,ORDER *b,TABL if (!map || (map & (RAND_TABLE_BIT | OUTER_REF_TABLE_BIT))) DBUG_RETURN(0); - for (; !(map & tables->table->map); tables= tables->next_leaf) ; - if (map != tables->table->map) + while ((table= ti++) && !(map & table->table->map)); + if (map != table->table->map) DBUG_RETURN(0); // More than one table - DBUG_PRINT("exit",("sort by table: %d",tables->table->tablenr)); - DBUG_RETURN(tables->table); + DBUG_PRINT("exit",("sort by table: %d",table->table->tablenr)); + DBUG_RETURN(table->table); } @@ -17886,7 +18160,8 @@ static void select_describe(JOIN *join, if (result->send_data(item_list)) join->error= 1; } - else + else if (!join->select_lex->master_unit()->derived || + join->select_lex->master_unit()->derived->is_materialized_derived()) { table_map used_tables=0; @@ -18194,6 +18469,7 @@ static void select_describe(JOIN *join, if (examined_rows) f= (float) (100.0 * join->best_positions[i].records_read / examined_rows); + set_if_smaller(f, 100.0); item_list.push_back(new Item_float(f, 2)); } } @@ -18456,11 +18732,32 @@ bool mysql_explain_union(THD *thd, SELEC sl; sl= sl->next_select()) { + bool is_primary= FALSE; + if (sl->next_select()) + is_primary= TRUE; + + if (!is_primary && sl->first_inner_unit()) + { + /* + If there is at least one materialized derived|view then it's a PRIMARY select. + Otherwise, all derived tables/views were merged and this select is a SIMPLE one. + */ + for (SELECT_LEX_UNIT *un= sl->first_inner_unit(); + un; + un= un->next_unit()) + { + if ((!un->derived || + un->derived->is_materialized_derived())) + { + is_primary= TRUE; + break; + } + } + } // drop UNCACHEABLE_EXPLAIN, because it is for internal usage only uint8 uncacheable= (sl->uncacheable & ~UNCACHEABLE_EXPLAIN); sl->type= (((&thd->lex->select_lex)==sl)? - (sl->first_inner_unit() || sl->next_select() ? - "PRIMARY" : "SIMPLE"): + (is_primary ? "PRIMARY" : "SIMPLE"): ((sl == first)? ((sl->linkage == DERIVED_TABLE_TYPE) ? "DERIVED": === modified file 'sql/sql_select.h' --- a/sql/sql_select.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_select.h 2010-05-26 20:18:18 +0000 @@ -37,6 +37,7 @@ /* Values in optimize */ #define KEY_OPTIMIZE_EXISTS 1 #define KEY_OPTIMIZE_REF_OR_NULL 2 +#define KEY_OPTIMIZE_EQ 4 typedef struct keyuse_t { TABLE *table; @@ -293,6 +294,8 @@ typedef struct st_join_table { */ uint sj_strategy; + bool preread_init_done; + void cleanup(); inline bool is_using_loose_index_scan() { @@ -364,6 +367,22 @@ typedef struct st_join_table { select->cond= new_cond; return tmp_select_cond; } + double scan_time() + { + double res; + if (table->created) + { + res= table->file->scan_time(); + read_time=(ha_rows) res; + } + else + { + read_time= found_records ? found_records: 10;// TODO:fix this stub + res= (double)read_time; + } + return res; + } + bool preread_init(); } JOIN_TAB; @@ -1551,6 +1570,7 @@ public: bool union_part; ///< this subselect is part of union bool optimized; ///< flag to avoid double optimization in EXPLAIN + Array<Item_in_subselect> sj_subselects; /* Temporary tables used to weed-out semi-join duplicates */ @@ -1700,6 +1720,7 @@ public: { return (table_map(1) << tables) - 1; } + void drop_unused_derived_keys(); /* Return the table for which an index scan can be used to satisfy the sort order needed by the ORDER BY/(implicit) GROUP BY clause @@ -1744,7 +1765,7 @@ Field* create_tmp_field_from_field(THD * /* functions from opt_sum.cc */ bool simple_pred(Item_func *func_item, Item **args, bool *inv_order); -int opt_sum_query(TABLE_LIST *tables, List<Item> &all_fields,COND *conds); +int opt_sum_query(List<TABLE_LIST> &tables, List<Item> &all_fields,COND *conds); /* from sql_delete.cc, used by opt_range.cc */ extern "C" int refpos_order_cmp(void* arg, const void *a,const void *b); @@ -1964,7 +1985,7 @@ void push_index_cond(JOIN_TAB *tab, uint TABLE *create_tmp_table(THD *thd,TMP_TABLE_PARAM *param,List<Item> &fields, ORDER *group, bool distinct, bool save_sum_fields, ulonglong select_options, ha_rows rows_limit, - char* alias); + char* alias, bool do_not_open=FALSE); void free_tmp_table(THD *thd, TABLE *entry); bool create_internal_tmp_table_from_heap(THD *thd, TABLE *table, ENGINE_COLUMNDEF *start_recinfo, === modified file 'sql/sql_show.cc' --- a/sql/sql_show.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_show.cc 2010-05-26 20:18:18 +0000 @@ -719,7 +719,8 @@ mysqld_show_create(THD *thd, TABLE_LIST { Show_create_error_handler view_error_suppressor(thd, table_list); thd->push_internal_handler(&view_error_suppressor); - bool error= open_normal_and_derived_tables(thd, table_list, 0); + bool error= open_normal_and_derived_tables(thd, table_list, 0, + DT_PREPARE | DT_CREATE); thd->pop_internal_handler(); if (error && (thd->killed || thd->main_da.is_error())) DBUG_RETURN(TRUE); @@ -894,7 +895,8 @@ mysqld_list_fields(THD *thd, TABLE_LIST DBUG_ENTER("mysqld_list_fields"); DBUG_PRINT("enter",("table: %s",table_list->table_name)); - if (open_normal_and_derived_tables(thd, table_list, 0)) + if (open_normal_and_derived_tables(thd, table_list, 0, + DT_PREPARE | DT_CREATE)) DBUG_VOID_RETURN; table= table_list->table; @@ -1680,7 +1682,7 @@ view_store_options(THD *thd, TABLE_LIST static void append_algorithm(TABLE_LIST *table, String *buff) { buff->append(STRING_WITH_LEN("ALGORITHM=")); - switch ((int8)table->algorithm) { + switch ((int16)table->algorithm) { case VIEW_ALGORITHM_UNDEFINED: buff->append(STRING_WITH_LEN("UNDEFINED ")); break; @@ -3360,8 +3362,9 @@ fill_schema_show_cols_or_idxs(THD *thd, SQLCOM_SHOW_FIELDS is used because it satisfies 'only_view_structure()' */ lex->sql_command= SQLCOM_SHOW_FIELDS; - res= open_normal_and_derived_tables(thd, show_table_list, - MYSQL_LOCK_IGNORE_FLUSH); + res= (open_normal_and_derived_tables(thd, show_table_list, + MYSQL_LOCK_IGNORE_FLUSH, + DT_PREPARE | DT_CREATE)); lex->sql_command= save_sql_command; /* get_all_tables() returns 1 on failure and 0 on success thus @@ -3792,8 +3795,9 @@ int get_all_tables(THD *thd, TABLE_LIST lex->sql_command= SQLCOM_SHOW_FIELDS; show_table_list->i_s_requested_object= schema_table->i_s_requested_object; - res= open_normal_and_derived_tables(thd, show_table_list, - MYSQL_LOCK_IGNORE_FLUSH); + res= (open_normal_and_derived_tables(thd, show_table_list, + MYSQL_LOCK_IGNORE_FLUSH, + DT_PREPARE | DT_CREATE)); lex->sql_command= save_sql_command; /* XXX: show_table_list has a flag i_is_requested, === modified file 'sql/sql_table.cc' --- a/sql/sql_table.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sql_table.cc 2010-05-26 20:18:18 +0000 @@ -4623,8 +4623,13 @@ static bool mysql_admin_table(THD* thd, thd->no_warnings_for_error= no_warnings_for_error; if (view_operator_func == NULL) table->required_type=FRMTYPE_TABLE; - + if (lex->sql_command == SQLCOM_CHECK || + lex->sql_command == SQLCOM_REPAIR || + lex->sql_command == SQLCOM_ANALYZE || + lex->sql_command == SQLCOM_OPTIMIZE) + thd->prepare_derived_at_open= TRUE; open_and_lock_tables(thd, table); + thd->prepare_derived_at_open= FALSE; thd->no_warnings_for_error= 0; table->next_global= save_next_global; table->next_local= save_next_local; @@ -4722,7 +4727,7 @@ static bool mysql_admin_table(THD* thd, else /* Default failure code is corrupt table */ result_code= HA_ADMIN_CORRUPT; - goto send_result; + goto send_result; } if (table->view) === modified file 'sql/sql_union.cc' --- a/sql/sql_union.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_union.cc 2010-05-26 20:18:18 +0000 @@ -105,6 +105,7 @@ bool select_union::flush() options create options table_alias name of the temporary table bit_fields_as_long convert bit fields to ulonglong + create_table whether to physically create result table DESCRIPTION Create a temporary table that is used to store the result of a UNION, @@ -119,7 +120,7 @@ bool select_union::create_result_table(THD *thd_arg, List<Item> *column_types, bool is_union_distinct, ulonglong options, const char *alias, - bool bit_fields_as_long) + bool bit_fields_as_long, bool create_table) { DBUG_ASSERT(table == 0); tmp_table_param.init(); @@ -128,10 +129,14 @@ select_union::create_result_table(THD *t if (! (table= create_tmp_table(thd_arg, &tmp_table_param, *column_types, (ORDER*) 0, is_union_distinct, 1, - options, HA_POS_ERROR, (char*) alias))) + options, HA_POS_ERROR, (char*) alias, + !create_table))) return TRUE; - table->file->extra(HA_EXTRA_WRITE_CACHE); - table->file->extra(HA_EXTRA_IGNORE_DUP_KEY); + if (create_table) + { + table->file->extra(HA_EXTRA_WRITE_CACHE); + table->file->extra(HA_EXTRA_IGNORE_DUP_KEY); + } return FALSE; } @@ -269,6 +274,7 @@ bool st_select_lex_unit::prepare(THD *th (is_union_select ? (ORDER*) 0 : (ORDER*) thd_arg->lex->proc_list.first), sl, this); + /* There are no * in the statement anymore (for PS) */ sl->with_wild= 0; last_procedure= join->procedure; @@ -331,6 +337,8 @@ bool st_select_lex_unit::prepare(THD *th List_iterator_fast<Item> tp(types); Item *type; ulonglong create_options; + uint save_tablenr; + table_map save_map; while ((type= tp++)) { @@ -383,12 +391,22 @@ bool st_select_lex_unit::prepare(THD *th create_options= create_options | TMP_TABLE_FORCE_MYISAM; if (union_result->create_result_table(thd, &types, test(union_distinct), - create_options, "", FALSE)) + create_options, "", FALSE, TRUE)) goto err; + if (fake_select_lex && !fake_select_lex->first_cond_optimization) + { + save_tablenr= result_table_list.tablenr_exec; + save_map= result_table_list.map_exec; + } bzero((char*) &result_table_list, sizeof(result_table_list)); result_table_list.db= (char*) ""; result_table_list.table_name= result_table_list.alias= (char*) "union"; result_table_list.table= table= union_result->table; + if (fake_select_lex && !fake_select_lex->first_cond_optimization) + { + result_table_list.tablenr_exec= save_tablenr; + result_table_list.map_exec= save_map; + } thd_arg->lex->current_select= lex_select_save; if (!item_list.elements) @@ -453,18 +471,21 @@ err: } -bool st_select_lex_unit::exec() +/** + Run optimization phase. + + @return FALSE unit successfully passed optimization phase. + @return TRUE an error occur. +*/ +bool st_select_lex_unit::optimize() { SELECT_LEX *lex_select_save= thd->lex->current_select; SELECT_LEX *select_cursor=first_select(); - ulonglong add_rows=0; - ha_rows examined_rows= 0; - DBUG_ENTER("st_select_lex_unit::exec"); + DBUG_ENTER("st_select_lex_unit::optimize"); - if (executed && !uncacheable && !describe) + if (optimized && !uncacheable && !describe) DBUG_RETURN(FALSE); - executed= 1; - + if (uncacheable || !item || !item->assigned() || describe) { if (item) @@ -485,7 +506,6 @@ bool st_select_lex_unit::exec() } for (SELECT_LEX *sl= select_cursor; sl; sl= sl->next_select()) { - ha_rows records_at_start= 0; thd->lex->current_select= sl; if (optimized) @@ -512,6 +532,66 @@ bool st_select_lex_unit::exec() sl->join->select_options= (select_limit_cnt == HA_POS_ERROR || sl->braces) ? sl->options & ~OPTION_FOUND_ROWS : sl->options | found_rows_for_union; + + saved_error= sl->join->optimize(); + } + + if (saved_error) + { + thd->lex->current_select= lex_select_save; + DBUG_RETURN(saved_error); + } + } + } + optimized= 1; + + thd->lex->current_select= lex_select_save; + DBUG_RETURN(saved_error); +} + + +bool st_select_lex_unit::exec() +{ + SELECT_LEX *lex_select_save= thd->lex->current_select; + SELECT_LEX *select_cursor=first_select(); + ulonglong add_rows=0; + ha_rows examined_rows= 0; + DBUG_ENTER("st_select_lex_unit::exec"); + + if (executed && !uncacheable && !describe) + DBUG_RETURN(FALSE); + executed= 1; + + saved_error= optimize(); + + if (uncacheable || !item || !item->assigned() || describe) + { + for (SELECT_LEX *sl= select_cursor; sl; sl= sl->next_select()) + { + ha_rows records_at_start= 0; + thd->lex->current_select= sl; + + { + set_limit(sl); + if (sl == global_parameters || describe) + { + offset_limit_cnt= 0; + /* + We can't use LIMIT at this stage if we are using ORDER BY for the + whole query + */ + if (sl->order_list.first || describe) + select_limit_cnt= HA_POS_ERROR; + } + + /* + When using braces, SQL_CALC_FOUND_ROWS affects the whole query: + we don't calculate found_rows() per union part. + Otherwise, SQL_CALC_FOUND_ROWS should be done on all sub parts. + */ + sl->join->select_options= + (select_limit_cnt == HA_POS_ERROR || sl->braces) ? + sl->options & ~OPTION_FOUND_ROWS : sl->options | found_rows_for_union; saved_error= sl->join->optimize(); } if (!saved_error) @@ -564,7 +644,6 @@ bool st_select_lex_unit::exec() } } } - optimized= 1; /* Send result to 'result' */ saved_error= TRUE; === modified file 'sql/sql_update.cc' --- a/sql/sql_update.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_update.cc 2010-05-26 20:18:18 +0000 @@ -212,7 +212,11 @@ int mysql_update(THD *thd, if (open_tables(thd, &table_list, &table_count, 0)) DBUG_RETURN(1); - if (table_list->multitable_view) + //Prepare views so they are handled correctly. + if (mysql_handle_derived(thd->lex, DT_INIT)) + DBUG_RETURN(1); + + if (table_list->is_multitable()) { DBUG_ASSERT(table_list->view != 0); DBUG_PRINT("info", ("Switch to multi-update")); @@ -227,15 +231,19 @@ int mysql_update(THD *thd, DBUG_RETURN(1); close_tables_for_reopen(thd, &table_list); } - - if (mysql_handle_derived(thd->lex, &mysql_derived_prepare) || - (thd->fill_derived_tables() && - mysql_handle_derived(thd->lex, &mysql_derived_filling))) + if (table_list->handle_derived(thd->lex, DT_MERGE_FOR_INSERT)) + DBUG_RETURN(1); + if (table_list->handle_derived(thd->lex, DT_PREPARE)) DBUG_RETURN(1); thd_proc_info(thd, "init"); table= table_list->table; + if (!table_list->updatable) + { + my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "UPDATE"); + DBUG_RETURN(1); + } /* Calculate "table->covering_keys" based on the WHERE */ table->covering_keys= table->s->keys_in_use; table->quick_keys.clear_all(); @@ -254,13 +262,17 @@ int mysql_update(THD *thd, table_list->grant.want_privilege= table->grant.want_privilege= want_privilege; table_list->register_want_access(want_privilege); #endif + /* 'Unfix' fields to allow correct marking by the setup_fields function. */ + if (table_list->is_view()) + unfix_fields(fields); + if (setup_fields_with_no_wrap(thd, 0, fields, MARK_COLUMNS_WRITE, 0, 0)) DBUG_RETURN(1); /* purecov: inspected */ if (table_list->view && check_fields(thd, fields)) { DBUG_RETURN(1); } - if (!table_list->updatable || check_key_in_view(thd, table_list)) + if (check_key_in_view(thd, table_list)) { my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "UPDATE"); DBUG_RETURN(1); @@ -839,6 +851,11 @@ int mysql_update(THD *thd, } thd->count_cuted_fields= CHECK_FIELD_IGNORE; /* calc cuted fields */ thd->abort_on_warning= 0; + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } DBUG_RETURN((error >= 0 || thd->is_error()) ? 1 : 0); err: @@ -903,8 +920,8 @@ bool mysql_prepare_update(THD *thd, TABL if (setup_tables_and_check_access(thd, &select_lex->context, &select_lex->top_join_list, table_list, - &select_lex->leaf_tables, - FALSE, UPDATE_ACL, SELECT_ACL) || + select_lex->leaf_tables, + FALSE, UPDATE_ACL, SELECT_ACL, TRUE) || setup_conds(thd, table_list, select_lex->leaf_tables, conds) || select_lex->setup_ref_array(thd, order_num) || setup_order(thd, select_lex->ref_pointer_array, @@ -941,8 +958,8 @@ static table_map get_table_map(List<Item Item_field *item; table_map map= 0; - while ((item= (Item_field *) item_it++)) - map|= item->used_tables(); + while ((item= (Item_field *) item_it++)) + map|= item->all_used_tables(); DBUG_PRINT("info", ("table_map: 0x%08lx", (long) map)); return map; } @@ -964,7 +981,7 @@ int mysql_multi_update_prepare(THD *thd) { LEX *lex= thd->lex; TABLE_LIST *table_list= lex->query_tables; - TABLE_LIST *tl, *leaves; + TABLE_LIST *tl; List<Item> *fields= &lex->select_lex.item_list; table_map tables_for_update; bool update_view= 0; @@ -987,19 +1004,24 @@ reopen_tables: /* open tables and create derived ones, but do not lock and fill them */ if (((original_multiupdate || need_reopen) && open_tables(thd, &table_list, &table_count, 0)) || - mysql_handle_derived(lex, &mysql_derived_prepare)) + mysql_handle_derived(lex, DT_INIT)) DBUG_RETURN(TRUE); /* setup_tables() need for VIEWs. JOIN::prepare() will call setup_tables() second time, but this call will do nothing (there are check for second call in setup_tables()). */ + //We need to merge for insert prior to prepare. + if (mysql_handle_list_of_derived(lex, table_list, DT_MERGE_FOR_INSERT)) + DBUG_RETURN(1); + if (mysql_handle_list_of_derived(lex, table_list, DT_PREPARE)) + DBUG_RETURN(1); if (setup_tables_and_check_access(thd, &lex->select_lex.context, &lex->select_lex.top_join_list, table_list, - &lex->select_lex.leaf_tables, FALSE, - UPDATE_ACL, SELECT_ACL)) + lex->select_lex.leaf_tables, FALSE, + UPDATE_ACL, SELECT_ACL, TRUE)) DBUG_RETURN(TRUE); if (setup_fields_with_no_wrap(thd, 0, *fields, MARK_COLUMNS_WRITE, 0, 0)) @@ -1024,8 +1046,8 @@ reopen_tables: /* Setup timestamp handling and locking mode */ - leaves= lex->select_lex.leaf_tables; - for (tl= leaves; tl; tl= tl->next_leaf) + List_iterator<TABLE_LIST> ti(lex->select_lex.leaf_tables); + while ((tl= ti++)) { TABLE *table= tl->table; /* Only set timestamp column if this is not modified */ @@ -1067,7 +1089,7 @@ reopen_tables: for (tl= table_list; tl; tl= tl->next_local) { /* Check access privileges for table */ - if (!tl->derived) + if (!tl->is_derived()) { uint want_privilege= tl->updating ? UPDATE_ACL : SELECT_ACL; if (check_access(thd, want_privilege, @@ -1081,7 +1103,7 @@ reopen_tables: /* check single table update for view compound from several tables */ for (tl= table_list; tl; tl= tl->next_local) { - if (tl->effective_algorithm == VIEW_ALGORITHM_MERGE) + if (tl->is_merged_derived()) { TABLE_LIST *for_update= 0; if (tl->check_single_table(&for_update, tables_for_update, tl)) @@ -1136,6 +1158,8 @@ reopen_tables: */ unit->unclean(); } + // Reset 'prepared' flags for all derived tables/views + mysql_handle_list_of_derived(thd->lex, table_list, DT_REINIT); /* Also we need to cleanup Natural_join_column::table_field items. @@ -1158,7 +1182,8 @@ reopen_tables: */ lex->select_lex.exclude_from_table_unique_test= TRUE; /* We only need SELECT privilege for columns in the values list */ - for (tl= leaves; tl; tl= tl->next_leaf) + ti.rewind(); + while ((tl= ti++)) { TABLE *table= tl->table; TABLE_LIST *tlist; @@ -1187,10 +1212,6 @@ reopen_tables: */ lex->select_lex.exclude_from_table_unique_test= FALSE; - if (thd->fill_derived_tables() && - mysql_handle_derived(lex, &mysql_derived_filling)) - DBUG_RETURN(TRUE); - DBUG_RETURN (FALSE); } @@ -1213,7 +1234,7 @@ bool mysql_multi_update(THD *thd, DBUG_ENTER("mysql_multi_update"); if (!(result= new multi_update(table_list, - thd->lex->select_lex.leaf_tables, + &thd->lex->select_lex.leaf_tables, fields, values, handle_duplicates, ignore))) DBUG_RETURN(TRUE); @@ -1247,7 +1268,7 @@ bool mysql_multi_update(THD *thd, multi_update::multi_update(TABLE_LIST *table_list, - TABLE_LIST *leaves_list, + List<TABLE_LIST> *leaves_list, List<Item> *field_list, List<Item> *value_list, enum enum_duplicates handle_duplicates_arg, bool ignore_arg) @@ -1265,6 +1286,7 @@ multi_update::multi_update(TABLE_LIST *t int multi_update::prepare(List<Item> &not_used_values, SELECT_LEX_UNIT *lex_unit) + { TABLE_LIST *table_ref; SQL_LIST update; @@ -1274,12 +1296,20 @@ int multi_update::prepare(List<Item> &no List_iterator_fast<Item> value_it(*values); uint i, max_fields; uint leaf_table_count= 0; + List_iterator<TABLE_LIST> ti(*leaves); DBUG_ENTER("multi_update::prepare"); thd->count_cuted_fields= CHECK_FIELD_WARN; thd->cuted_fields=0L; thd_proc_info(thd, "updating main table"); + SELECT_LEX *select_lex= lex_unit->first_select(); + if (select_lex->first_cond_optimization) + { + if (select_lex->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + } + tables_to_update= get_table_map(fields); if (!tables_to_update) @@ -1293,7 +1323,7 @@ int multi_update::prepare(List<Item> &no TABLE::tmp_set by pointing TABLE::read_set to it and then restore it after setup_fields(). */ - for (table_ref= leaves; table_ref; table_ref= table_ref->next_leaf) + while ((table_ref= ti++)) { TABLE *table= table_ref->table; if (tables_to_update & table->map) @@ -1311,7 +1341,8 @@ int multi_update::prepare(List<Item> &no int error= setup_fields(thd, 0, *values, MARK_COLUMNS_READ, 0, 0); - for (table_ref= leaves; table_ref; table_ref= table_ref->next_leaf) + ti.rewind(); + while ((table_ref= ti++)) { TABLE *table= table_ref->table; if (tables_to_update & table->map) @@ -1331,7 +1362,8 @@ int multi_update::prepare(List<Item> &no */ update.empty(); - for (table_ref= leaves; table_ref; table_ref= table_ref->next_leaf) + ti.rewind(); + while ((table_ref= ti++)) { /* TODO: add support of view of join support */ TABLE *table=table_ref->table; @@ -1557,9 +1589,9 @@ loop_end: { table_map unupdated_tables= table_ref->check_option->used_tables() & ~first_table_for_update->map; - for (TABLE_LIST *tbl_ref =leaves; - unupdated_tables && tbl_ref; - tbl_ref= tbl_ref->next_leaf) + List_iterator<TABLE_LIST> ti(*leaves); + TABLE_LIST *tbl_ref; + while ((tbl_ref= ti++) && unupdated_tables) { if (unupdated_tables & tbl_ref->table->map) unupdated_tables&= ~tbl_ref->table->map; === modified file 'sql/sql_view.cc' --- a/sql/sql_view.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_view.cc 2010-05-26 20:18:18 +0000 @@ -219,7 +219,7 @@ fill_defined_view_parts (THD *thd, TABLE view->definer.user= decoy.definer.user; lex->definer= &view->definer; } - if (lex->create_view_algorithm == VIEW_ALGORITHM_UNDEFINED) + if (lex->create_view_algorithm == DTYPE_ALGORITHM_UNDEFINED) lex->create_view_algorithm= (uint8) decoy.algorithm; if (lex->create_view_suid == VIEW_SUID_DEFAULT) lex->create_view_suid= decoy.view_suid ? @@ -814,7 +814,7 @@ static int mysql_register_view(THD *thd, ulong sql_mode= thd->variables.sql_mode & MODE_ANSI_QUOTES; thd->variables.sql_mode&= ~MODE_ANSI_QUOTES; - lex->unit.print(&view_query, QT_ORDINARY); + lex->unit.print(&view_query, QT_VIEW_INTERNAL); lex->unit.print(&is_query, QT_IS); thd->variables.sql_mode|= sql_mode; @@ -847,7 +847,7 @@ static int mysql_register_view(THD *thd, { push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_VIEW_MERGE, ER(ER_WARN_VIEW_MERGE)); - lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; + lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; } view->algorithm= lex->create_view_algorithm; view->definer.user= lex->definer->user; @@ -1415,7 +1415,7 @@ bool mysql_make_view(THD *thd, File_pars List_iterator_fast<TABLE_LIST> ti(view_select->top_join_list); - table->effective_algorithm= VIEW_ALGORITHM_MERGE; + table->derived_type= VIEW_ALGORITHM_MERGE; DBUG_PRINT("info", ("algorithm: MERGE")); table->updatable= (table->updatable_view != 0); table->effective_with_check= @@ -1429,67 +1429,10 @@ bool mysql_make_view(THD *thd, File_pars /* prepare view context */ lex->select_lex.context.resolve_in_table_list_only(view_main_select_tables); lex->select_lex.context.outer_context= 0; - lex->select_lex.context.select_lex= table->select_lex; lex->select_lex.select_n_having_items+= table->select_lex->select_n_having_items; - /* - Tables of the main select of the view should be marked as belonging - to the same select as original view (again we can use LEX::select_lex - for this purprose because we don't support MERGE algorithm for views - with unions). - */ - for (tbl= lex->select_lex.get_table_list(); tbl; tbl= tbl->next_local) - tbl->select_lex= table->select_lex; - - { - if (view_main_select_tables->next_local) - { - table->multitable_view= TRUE; - if (table->belong_to_view) - table->belong_to_view->multitable_view= TRUE; - } - /* make nested join structure for view tables */ - NESTED_JOIN *nested_join; - if (!(nested_join= table->nested_join= - (NESTED_JOIN *) thd->calloc(sizeof(NESTED_JOIN)))) - goto err; - nested_join->join_list= view_select->top_join_list; - - /* re-nest tables of VIEW */ - ti.rewind(); - while ((tbl= ti++)) - { - tbl->join_list= &nested_join->join_list; - tbl->embedding= table; - } - } - - /* Store WHERE clause for post-processing in setup_underlying */ table->where= view_select->where; - /* - Add subqueries units to SELECT into which we merging current view. - unit(->next)* chain starts with subqueries that are used by this - view and continues with subqueries that are used by other views. - We must not add any subquery twice (otherwise we'll form a loop), - to do this we remember in end_unit the first subquery that has - been already added. - - NOTE: we do not support UNION here, so we take only one select - */ - SELECT_LEX_NODE *end_unit= table->select_lex->slave; - SELECT_LEX_UNIT *next_unit; - for (SELECT_LEX_UNIT *unit= lex->select_lex.first_inner_unit(); - unit; - unit= next_unit) - { - if (unit == end_unit) - break; - SELECT_LEX_NODE *save_slave= unit->slave; - next_unit= unit->next_unit(); - unit->include_down(table->select_lex); - unit->slave= save_slave; // fix include_down initialisation - } /* We can safely ignore the VIEW's ORDER BY if we merge into union @@ -1506,23 +1449,22 @@ bool mysql_make_view(THD *thd, File_pars goto ok; } - table->effective_algorithm= VIEW_ALGORITHM_TMPTABLE; + table->derived_type= VIEW_ALGORITHM_TMPTABLE; DBUG_PRINT("info", ("algorithm: TEMPORARY TABLE")); view_select->linkage= DERIVED_TABLE_TYPE; table->updatable= 0; table->effective_with_check= VIEW_CHECK_NONE; old_lex->subqueries= TRUE; - /* SELECT tree link */ - lex->unit.include_down(table->select_lex); - lex->unit.slave= view_select; // fix include_down initialisation - table->derived= &lex->unit; } else goto err; ok: + /* SELECT tree link */ + lex->unit.include_down(table->select_lex); + lex->unit.slave= view_select; // fix include_down initialisation /* global SELECT list linking */ end= view_select; // primary SELECT_LEX is always last end->link_next= old_lex->all_selects_list; === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2010-03-15 11:51:23 +0000 +++ b/sql/sql_yacc.yy 2010-05-26 20:18:18 +0000 @@ -1920,7 +1920,7 @@ create: | CREATE { Lex->create_view_mode= VIEW_CREATE_NEW; - Lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; + Lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; Lex->create_view_suid= TRUE; } view_or_trigger_or_sp_or_event @@ -5858,7 +5858,7 @@ alter: my_error(ER_SP_BADSTATEMENT, MYF(0), "ALTER VIEW"); MYSQL_YYABORT; } - lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; + lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; lex->create_view_mode= VIEW_ALTER; } view_tail @@ -13369,7 +13369,7 @@ view_replace: view_algorithm: ALGORITHM_SYM EQ UNDEFINED_SYM - { Lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; } + { Lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; } | ALGORITHM_SYM EQ MERGE_SYM { Lex->create_view_algorithm= VIEW_ALGORITHM_MERGE; } | ALGORITHM_SYM EQ TEMPTABLE_SYM === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-20 12:01:47 +0000 +++ b/sql/table.cc 2010-05-26 20:18:18 +0000 @@ -20,7 +20,7 @@ #include "sql_trigger.h" #include <m_ctype.h> #include "my_md5.h" - +#include "my_bit.h" /* INFORMATION_SCHEMA name */ LEX_STRING INFORMATION_SCHEMA_NAME= {C_STRING_WITH_LEN("information_schema")}; @@ -3442,129 +3442,118 @@ void TABLE_LIST::calc_md5(char *buffer) /** - @brief Set underlying table for table place holder of view. - - @details - - Replace all views that only use one table with the table itself. This - allows us to treat the view as a simple table and even update it (it is a - kind of optimization). + @brief + Create field translation for mergeable derived table/view. - @note + @param thd Thread handle - This optimization is potentially dangerous as it makes views - masquerade as base tables: Views don't have the pointer TABLE_LIST::table - set to non-@c NULL. + @details + Create field translation for mergeable derived table/view. - We may have the case where a view accesses tables not normally accessible - in the current Security_context (only in the definer's - Security_context). According to the table's GRANT_INFO (TABLE::grant), - access is fulfilled, but this is implicitly meant in the definer's security - context. Hence we must never look at only a TABLE's GRANT_INFO without - looking at the one of the referring TABLE_LIST. + @return FALSE ok. + @return TRUE an error occur. */ -void TABLE_LIST::set_underlying_merge() +bool TABLE_LIST::create_field_translation(THD *thd) { - TABLE_LIST *tbl; + Item *item; + Field_translator *transl; + SELECT_LEX *select= get_single_select(); + List_iterator_fast<Item> it(select->item_list); + uint field_count= 0; + Query_arena *arena= thd->stmt_arena, backup; + bool res= FALSE; + + used_items.empty(); - if ((tbl= merge_underlying_list)) + if (field_translation) { - /* This is a view. Process all tables of view */ - DBUG_ASSERT(view && effective_algorithm == VIEW_ALGORITHM_MERGE); - do + /* + Update items in the field translation aftet view have been prepared. + It's needed because some items in the select list, like IN subselects, + might be substituted for optimized ones. + */ + if (is_view() && get_unit()->prepared && !field_translation_updated) { - if (tbl->merge_underlying_list) // This is a view + while ((item= it++)) { - DBUG_ASSERT(tbl->view && - tbl->effective_algorithm == VIEW_ALGORITHM_MERGE); - /* - This is the only case where set_ancestor is called on an object - that may not be a view (in which case ancestor is 0) - */ - tbl->merge_underlying_list->set_underlying_merge(); + field_translation[field_count++].item= item; } - } while ((tbl= tbl->next_local)); - - if (!multitable_view) - { - table= merge_underlying_list->table; - schema_table= merge_underlying_list->schema_table; + field_translation_updated= TRUE; } + + return FALSE; } + + if (arena->is_conventional()) + arena= 0; // For easier test + else + thd->set_n_backup_active_arena(arena, &backup); + + /* Create view fields translation table */ + + if (!(transl= + (Field_translator*)(thd->stmt_arena-> + alloc(select->item_list.elements * + sizeof(Field_translator))))) + { + res= TRUE; + goto exit; + } + + while ((item= it++)) + { + transl[field_count].name= item->name; + transl[field_count++].item= item; + } + field_translation= transl; + field_translation_end= transl + field_count; + +exit: + if (arena) + thd->restore_active_arena(arena, &backup); + + return res; } -/* - setup fields of placeholder of merged VIEW +/** + @brief + Create field translation for mergeable derived table/view. - SYNOPSIS - TABLE_LIST::setup_underlying() - thd - thread handler + @param thd Thread handle - DESCRIPTION - It is: - - preparing translation table for view columns - If there are underlying view(s) procedure first will be called for them. + @details + Create field translation for mergeable derived table/view. - RETURN - FALSE - OK - TRUE - error + @return FALSE ok. + @return TRUE an error occur. */ bool TABLE_LIST::setup_underlying(THD *thd) { DBUG_ENTER("TABLE_LIST::setup_underlying"); - if (!field_translation && merge_underlying_list) + if (!view || (!field_translation && merge_underlying_list)) { - Field_translator *transl; - SELECT_LEX *select= &view->select_lex; - Item *item; - TABLE_LIST *tbl; + SELECT_LEX *select= get_single_select(); List_iterator_fast<Item> it(select->item_list); - uint field_count= 0; + TABLE_LIST *tbl; - if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*) &field_count)) + if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*) &tbl)) { DBUG_RETURN(TRUE); } - - for (tbl= merge_underlying_list; tbl; tbl= tbl->next_local) - { - if (tbl->merge_underlying_list && - tbl->setup_underlying(thd)) - { - DBUG_RETURN(TRUE); - } - } - - /* Create view fields translation table */ - - if (!(transl= - (Field_translator*)(thd->stmt_arena-> - alloc(select->item_list.elements * - sizeof(Field_translator))))) - { + if (create_field_translation(thd)) DBUG_RETURN(TRUE); - } - - while ((item= it++)) - { - transl[field_count].name= item->name; - transl[field_count++].item= item; - } - field_translation= transl; - field_translation_end= transl + field_count; - /* TODO: use hash for big number of fields */ /* full text function moving to current select */ - if (view->select_lex.ftfunc_list->elements) + if (select->ftfunc_list->elements) { Item_func_match *ifm; SELECT_LEX *current_select= thd->lex->current_select; List_iterator_fast<Item_func_match> - li(*(view->select_lex.ftfunc_list)); + li(*(select_lex->ftfunc_list)); while ((ifm= li++)) current_select->ftfunc_list->push_front(ifm); } @@ -3574,7 +3563,7 @@ bool TABLE_LIST::setup_underlying(THD *t /* - Prepare where expression of view + Prepare where expression of derived table/view SYNOPSIS TABLE_LIST::prep_where() @@ -3598,7 +3587,8 @@ bool TABLE_LIST::prep_where(THD *thd, It for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) { - if (tbl->view && tbl->prep_where(thd, conds, no_where_clause)) + if (tbl->is_view_or_derived() && + tbl->prep_where(thd, conds, no_where_clause)) { DBUG_RETURN(TRUE); } @@ -3606,6 +3596,8 @@ bool TABLE_LIST::prep_where(THD *thd, It if (where) { + if (where->fixed) + where->update_used_tables(); if (!where->fixed && where->fix_fields(thd, &where)) { DBUG_RETURN(TRUE); @@ -3638,7 +3630,13 @@ bool TABLE_LIST::prep_where(THD *thd, It } } if (tbl == 0) + { + if (*conds && !(*conds)->fixed) + (*conds)->fix_fields(thd, conds); *conds= and_conds(*conds, where->copy_andor_structure(thd)); + if (*conds && !(*conds)->fixed) + (*conds)->fix_fields(thd, conds); + } if (arena) thd->restore_active_arena(arena, &backup); where_processed= TRUE; @@ -3677,10 +3675,11 @@ merge_on_conds(THD *thd, TABLE_LIST *tab DBUG_PRINT("info", ("alias: %s", table->alias)); if (table->on_expr) cond= table->on_expr->copy_andor_structure(thd); - if (!table->nested_join) + if (!table->view) DBUG_RETURN(cond); - List_iterator<TABLE_LIST> li(table->nested_join->join_list); - while (TABLE_LIST *tbl= li++) + for (TABLE_LIST *tbl= (TABLE_LIST*)table->view->select_lex.table_list.first; + tbl; + tbl= tbl->next_local) { if (tbl->view && !is_cascaded) continue; @@ -3720,7 +3719,7 @@ bool TABLE_LIST::prep_check_option(THD * { DBUG_ENTER("TABLE_LIST::prep_check_option"); bool is_cascaded= check_opt_type == VIEW_CHECK_CASCADED; - + TABLE_LIST *merge_underlying_list= view->select_lex.get_table_list(); for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) { /* see comment of check_opt_type parameter */ @@ -3833,10 +3832,14 @@ void TABLE_LIST::hide_view_error(THD *th TABLE_LIST *TABLE_LIST::find_underlying_table(TABLE *table_to_find) { /* is this real table and table which we are looking for? */ - if (table == table_to_find && merge_underlying_list == 0) + if (table == table_to_find && view == 0) return this; + if (!view) + return 0; - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + for (TABLE_LIST *tbl= view->select_lex.get_table_list(); + tbl; + tbl= tbl->next_local) { TABLE_LIST *result; if ((result= tbl->find_underlying_table(table_to_find))) @@ -3918,7 +3921,12 @@ bool TABLE_LIST::check_single_table(TABL table_map map, TABLE_LIST *view_arg) { - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + if (!select_lex) + return FALSE; + DBUG_ASSERT(is_merged_derived()); + for (TABLE_LIST *tbl= get_single_select()->get_table_list(); + tbl; + tbl= tbl->next_local) { if (tbl->table) { @@ -3960,8 +3968,10 @@ bool TABLE_LIST::set_insert_values(MEM_R } else { - DBUG_ASSERT(view && merge_underlying_list); - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + DBUG_ASSERT(is_view_or_derived() && is_merged_derived()); + for (TABLE_LIST *tbl= (TABLE_LIST*)view->select_lex.table_list.first; + tbl; + tbl= tbl->next_local) if (tbl->set_insert_values(mem_root)) return TRUE; } @@ -3987,7 +3997,7 @@ bool TABLE_LIST::set_insert_values(MEM_R */ bool TABLE_LIST::is_leaf_for_name_resolution() { - return (view || is_natural_join || is_join_columns_complete || + return (is_merged_derived() || is_natural_join || is_join_columns_complete || !nested_join); } @@ -4125,7 +4135,11 @@ void TABLE_LIST::register_want_access(ul if (table) table->grant.want_privilege= want_access; } - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + if (!view) + return; + for (TABLE_LIST *tbl= view->select_lex.get_table_list(); + tbl; + tbl= tbl->next_local) tbl->register_want_access(want_access); } @@ -4358,14 +4372,23 @@ const char *Natural_join_column::db_name DBUG_ASSERT(!strcmp(table_ref->db, table_ref->table->s->db.str) || (table_ref->schema_table && - table_ref->table->s->db.str[0] == 0)); + table_ref->table->s->db.str[0] == 0) || + table_ref->is_materialized_derived()); return table_ref->db; } GRANT_INFO *Natural_join_column::grant() { - if (view_field) +/* if (view_field) + return &(table_ref->grant); + return &(table_ref->table->grant);*/ + /* + Have to check algorithm because merged derived also has + field_translation. + */ +//if (table_ref->effective_algorithm == DTYPE_ALGORITHM_MERGE) + if (table_ref->is_merged_derived()) return &(table_ref->grant); return &(table_ref->table->grant); } @@ -4448,7 +4471,15 @@ Item *create_view_field(THD *thd, TABLE_ } Item *item= new Item_direct_view_ref(&view->view->select_lex.context, field_ref, view->alias, - name); + name, view); + /* + Force creation of nullable item for the result tmp table for outer joined + views/derived tables. + */ + if (view->outer_join) + item->maybe_null= TRUE; + /* Save item in case we will need to fall back to materialization. */ + view->used_items.push_back(item); DBUG_RETURN(item); } @@ -4502,8 +4533,7 @@ void Field_iterator_table_ref::set_field /* This is a merge view, so use field_translation. */ else if (table_ref->field_translation) { - DBUG_ASSERT(table_ref->view && - table_ref->effective_algorithm == VIEW_ALGORITHM_MERGE); + DBUG_ASSERT(table_ref->is_merged_derived()); field_it= &view_field_it; DBUG_PRINT("info", ("field_it for '%s' is Field_iterator_view", table_ref->alias)); @@ -5096,6 +5126,142 @@ void st_table::mark_virtual_columns_for_ file->column_bitmaps_signal(); } + +/** + @brief + Allocate space for keys + + @param key_count number of keys to allocate. + + @details + Allocate space enough to fit 'key_count' keys for this table. + + @return FALSE space was successfully allocated. + @return TRUE an error occur. +*/ + +bool TABLE::alloc_keys(uint key_count) +{ + DBUG_ASSERT(!s->keys); + key_info= s->key_info= (KEY*) alloc_root(&mem_root, sizeof(KEY)*key_count); + max_keys= key_count; + return !(key_info); +} + +/** + @brief Adds one key to a temporary table. + + @param key_parts bitmap of fields that take a part in the key. + @param key_name name of the key + + @details + Creates a key for this table from fields which corresponds the bits set to 1 + in the 'key_parts' bitmap. The 'key_name' name is given to the newly created + key. + + @return <0 an error occur. + @return >=0 number of newly added key. +*/ + +bool TABLE::add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg) +{ + DBUG_ASSERT(!created && key < max_keys); + + char buf[NAME_CHAR_LEN]; + KEY* keyinfo; + Field **reg_field; + uint i; + bool key_start= TRUE; + KEY_PART_INFO* key_part_info= + (KEY_PART_INFO*) alloc_root(&mem_root, sizeof(KEY_PART_INFO)*key_parts); + if (!key_part_info) + return TRUE; + keyinfo= key_info + key; + keyinfo->key_part= key_part_info; + keyinfo->usable_key_parts= keyinfo->key_parts = key_parts; + keyinfo->key_length=0; + keyinfo->algorithm= HA_KEY_ALG_UNDEF; + keyinfo->flags= HA_GENERATED_KEY; + sprintf(buf, "key%i", key); + if (!(keyinfo->name= strdup_root(&mem_root, buf))) + return TRUE; + keyinfo->rec_per_key= (ulong*) alloc_root(&mem_root, + sizeof(ulong)*key_parts); + if (!keyinfo->rec_per_key) + return TRUE; + bzero(keyinfo->rec_per_key, sizeof(ulong)*key_parts); + for (i= 0; i < key_parts; i++) + { + reg_field= field + next_field_no(arg); + if (key_start) + (*reg_field)->key_start.set_bit(key); + key_start= FALSE; + (*reg_field)->part_of_key.set_bit(key); + (*reg_field)->flags|= PART_KEY_FLAG; + key_part_info->null_bit= (*reg_field)->null_bit; + key_part_info->null_offset= (uint) ((*reg_field)->null_ptr - + (uchar*) record[0]); + key_part_info->field= *reg_field; + key_part_info->offset= (*reg_field)->offset(record[0]); + key_part_info->length= (uint16) (*reg_field)->pack_length(); + keyinfo->key_length+= key_part_info->length; + /* TODO: + The below method of computing the key format length of the + key part is a copy/paste from opt_range.cc, and table.cc. + This should be factored out, e.g. as a method of Field. + In addition it is not clear if any of the Field::*_length + methods is supposed to compute the same length. If so, it + might be reused. + */ + key_part_info->store_length= key_part_info->length; + + if ((*reg_field)->real_maybe_null()) + key_part_info->store_length+= HA_KEY_NULL_LENGTH; + if ((*reg_field)->type() == MYSQL_TYPE_BLOB || + (*reg_field)->real_type() == MYSQL_TYPE_VARCHAR) + key_part_info->store_length+= HA_KEY_BLOB_LENGTH; + + key_part_info->type= (uint8) (*reg_field)->key_type(); + key_part_info->key_type = + ((ha_base_keytype) key_part_info->type == HA_KEYTYPE_TEXT || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT1 || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT2) ? + 0 : FIELDFLAG_BINARY; + key_part_info++; + } + set_if_bigger(s->max_key_length, keyinfo->key_length); + s->keys++; + return FALSE; +} + +/* + @brief + Drop all indexes except specified one. + + @param key_to_save the key to save + + @details + Drop all indexes on this table except 'key_to_save'. The saved key becomes + key #0. Memory occupied by key parts of dropped keys are freed. + If the 'key_to_save' is negative then all keys are freed. +*/ + +void TABLE::use_index(int key_to_save) +{ + uint i= 1; + DBUG_ASSERT(!created && key_to_save < (int)s->keys); + if (key_to_save >= 0) + /* Save the given key. */ + memcpy(key_info, key_info + key_to_save, sizeof(KEY)); + else + /* Drop all keys; */ + i= 0; + + s->keys= (key_to_save < 0) ? 0 : 1; +} + + /** @brief Check if this is part of a MERGE table with attached children. @@ -5144,6 +5310,7 @@ void TABLE_LIST::reinit_before_use(THD * parent_embedding->nested_join->join_list.head() == embedded); } + /* Return subselect that contains the FROM list this table is taken from @@ -5412,6 +5579,296 @@ int update_virtual_fields(TABLE *table, DBUG_RETURN(0); } +/* + @brief Reset const_table flag + + @detail + Reset const_table flag for this table. If this table is a merged derived + table/view the flag is recursively reseted for all tables of the underlying + select. +*/ + +void TABLE_LIST::reset_const_table() +{ + table->const_table= 0; + if (is_merged_derived()) + { + SELECT_LEX *select_lex= get_unit()->first_select(); + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(select_lex->leaf_tables); + while ((tl= ti++)) + tl->reset_const_table(); + } +} + + +/* + @brief Run derived tables/view handling phases on underlying select_lex. + + @param lex LEX for this thread + @param phases derived tables/views handling phases to run + (set of DT_XXX constants) + @details + This function runs this derived table through specified 'phases'. + Underlying tables of this select are handled prior to this derived. + 'lex' is passed as an argument to called functions. + + @return TRUE on error + @return FALSE ok +*/ + +bool TABLE_LIST::handle_derived(struct st_lex *lex, uint phases) +{ + SELECT_LEX_UNIT *unit= get_unit(); + if (unit) + { + for (SELECT_LEX *sl= unit->first_select(); sl; sl= sl->next_select()) + if (sl->handle_derived(lex, phases)) + return TRUE; + return mysql_handle_single_derived(lex, this, phases); + } + return FALSE; +} + + +/** + @brief + Return unit of this derived table/view + + @return reference to a unit if it's a derived table/view. + @return 0 when it's not a derived table/view. +*/ + +st_select_lex_unit *TABLE_LIST::get_unit() +{ + return (view ? &view->unit : derived); +} + + +/** + @brief + Return select_lex of this derived table/view + + @return select_lex of this derived table/view. + @return 0 when it's not a derived table. +*/ + +st_select_lex *TABLE_LIST::get_single_select() +{ + SELECT_LEX_UNIT *unit= get_unit(); + return (unit ? unit->first_select() : 0); +} + + +/** + @brief + Attach a join table list as a nested join to this TABLE_LIST. + + @param join_list join table list to attach + + @details + This function wraps 'join_list' into a nested_join of this table, thus + turning it to a nested join leaf. +*/ + +void TABLE_LIST::wrap_into_nested_join(List<TABLE_LIST> &join_list) +{ + TABLE_LIST *tl; + /* + Walk through derived table top list and set 'embedding' to point to + the nesting table. + */ + nested_join->join_list.empty(); + List_iterator_fast<TABLE_LIST> li(join_list); + nested_join->join_list= join_list; + while ((tl= li++)) + { + tl->embedding= this; + tl->join_list= &nested_join->join_list; + } +} + + +/** + @brief + Initialize this derived table/view + + @param thd Thread handle + + @details + This function makes initial preparations of this derived table/view for + further processing: + if it's a derived table this function marks it either as mergeable or + materializable + creates temporary table for name resolution purposes + creates field translation for mergeable derived table/view + + @return TRUE an error occur + @return FALSE ok +*/ + +bool TABLE_LIST::init_derived(THD *thd, bool init_view) +{ + SELECT_LEX *first_select= get_single_select(); + SELECT_LEX_UNIT *unit= get_unit(); + + if (!unit) + return FALSE; + /* + Check whether we can merge this derived table into main select. + Depending on the result field translation will or will not + be created. + */ + TABLE_LIST *first_table= (TABLE_LIST *) first_select->table_list.first; + if (first_select->table_list.elements > 1 || + first_table && first_table->is_multitable()) + set_multitable(); + + unit->derived= this; + if (init_view && !view) + { + /* This is all what we can do for a derived table for now. */ + set_derived(); + } + + if (!is_view()) + { + /* A subquery might be forced to be materialized due to a side-effect. */ + if (!is_materialized_derived() && first_select->is_mergeable()) + set_merged_derived(); + else + set_materialized_derived(); + } + /* + Derived tables/view are materialized prior to UPDATE, thus we can skip + them from table uniqueness check + */ + if (is_materialized_derived()) + { + SELECT_LEX *sl; + for (sl= first_select ;sl ; sl= sl->next_select()) + sl->exclude_from_table_unique_test= TRUE; + } + /* + Create field translation for mergeable derived tables/views. + For derived tables field translation can be created only after + unit is prepared so all '*' are get unrolled. + */ + if (is_merged_derived()) + { + if (is_view() || unit->prepared) + create_field_translation(thd); + } + + return FALSE; +} + + +/** + @brief + Retrieve number of rows in the table + + @details + Retrieve number of rows in the table referred by this TABLE_LIST and + store it in the table's stats.records variable. If this TABLE_LIST refers + to a materialized derived table/view then the estimated number of rows of + the derived table/view is used instead. + + @return 0 ok + @return non zero error +*/ + +int TABLE_LIST::fetch_number_of_rows() +{ + int error= 0; + if (is_materialized_derived() && !fill_me) + + { + table->file->stats.records= ((select_union*)derived->result)->records; + set_if_bigger(table->file->stats.records, 2); + } + else + error= table->file->info(HA_STATUS_VARIABLE | HA_STATUS_NO_LOCK); + return error; +} + +/* + Procedure of keys generation for result tables of materialized derived + tables/views. + + A key is generated for each equi-join pair derived table-another table. + Each generated key consists of fields of derived table used in equi-join. + Example: + + SELECT * FROM (SELECT * FROM t1 GROUP BY 1) tt JOIN + t1 ON tt.f1=t1.f3 and tt.f2.=t1.f4; + In this case for the derived table tt one key will be generated. It will + consist of two parts f1 and f2. + Example: + + SELECT * FROM (SELECT * FROM t1 GROUP BY 1) tt JOIN + t1 ON tt.f1=t1.f3 JOIN + t2 ON tt.f2=t2.f4; + In this case for the derived table tt two keys will be generated. + One key over f1 field, and another key over f2 field. + Currently optimizer may choose to use only one such key, thus the second + one will be dropped after range optimizer is finished. + See also JOIN::drop_unused_derived_keys function. + Example: + + SELECT * FROM (SELECT * FROM t1 GROUP BY 1) tt JOIN + t1 ON tt.f1=a_function(t1.f3); + In this case for the derived table tt one key will be generated. It will + consist of one field - f1. +*/ + + + +/* + @brief + Change references to underlying items of a merged derived table/view + for fields in derived table's result table. + + @return FALSE ok + @return TRUE Out of memory +*/ +bool TABLE_LIST::change_refs_to_fields() +{ + List_iterator<Item> li(used_items); + Item_direct_ref *ref; + Field_iterator_view field_it; + THD *thd= table->in_use; + DBUG_ASSERT(is_merged_derived()); + + if (!used_items.elements) + return FALSE; + + materialized_items= (Item**)thd->calloc(sizeof(void*) * table->s->fields); + + while ((ref= (Item_direct_ref*)li++)) + { + uint idx; + Item *orig_item= *ref->ref; + field_it.set(this); + for (idx= 0; !field_it.end_of_fields(); field_it.next(), idx++) + { + if (field_it.item() == orig_item) + break; + } + DBUG_ASSERT(!field_it.end_of_fields()); + if (!materialized_items[idx]) + { + materialized_items[idx]= new Item_field(table->field[idx]); + if (!materialized_items[idx]) + return TRUE; + } + ref->ref= materialized_items + idx; + } + + return FALSE; +} + + /***************************************************************************** ** Instansiate templates *****************************************************************************/ === modified file 'sql/table.h' --- a/sql/table.h 2010-03-20 12:01:47 +0000 +++ b/sql/table.h 2010-05-26 20:18:18 +0000 @@ -858,6 +858,7 @@ struct st_table { my_bool insert_or_update; /* Can be used by the handler */ my_bool alias_name_used; /* true if table_name is alias */ my_bool get_fields_in_item_tree; /* Signal to fix_field */ + my_bool created; /* For tmp tables. TRUE <=> tmp table was actually created.*/ /* If MERGE children attached to parent. See top comment in ha_myisammrg.cc */ my_bool children_attached; @@ -870,6 +871,7 @@ struct st_table { bool no_partitions_used; /* If true, all partitions have been pruned away */ #endif + uint max_keys; /* Size of allocated key_info array. */ bool fill_item_list(List<Item> *item_list) const; void reset_item_list(List<Item> *item_list) const; void clear_column_bitmaps(void); @@ -913,6 +915,15 @@ struct st_table { */ inline bool needs_reopen_or_name_lock() { return s->version != refresh_version; } + bool alloc_keys(uint key_count); + bool add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg); + void use_index(int key_to_save); + void set_table_map(table_map map_arg, uint tablenr_arg) + { + map= map_arg; + tablenr= tablenr_arg; + } bool is_children_attached(void); }; @@ -1045,13 +1056,52 @@ typedef struct st_schema_table } ST_SCHEMA_TABLE; +/* + Types of derived tables. The ending part is a bitmap of phases that are + applicable to a derived table of the type. + * / +#define VIEW_ALGORITHM_UNDEFINED 0 +#define VIEW_ALGORITHM_MERGE 1 + DT_COMMON + DT_MERGE +#define DERIVED_ALGORITHM_MERGE 2 + DT_COMMON + DT_MERGE +#define VIEW_ALGORITHM_TMPTABLE 3 + DT_COMMON + DT_MATERIALIZE +#define DERIVED_ALGORITHM_MATERIALIZE 4 + DT_COMMON + DT_MATERIALIZE +*/ +#define DTYPE_ALGORITHM_UNDEFINED 0 +#define DTYPE_VIEW 1 +#define DTYPE_TABLE 2 +#define DTYPE_MERGE 4 +#define DTYPE_MATERIALIZE 8 +#define DTYPE_MULTITABLE 16 +#define DTYPE_MASK 19 + +/* + Phases of derived tables/views handling, see sql_derived.cc + Values are used as parts of a bitmap attached to derived table types. +*/ +#define DT_INIT 1 +#define DT_PREPARE 2 +#define DT_OPTIMIZE 4 +#define DT_MERGE 8 +#define DT_MERGE_FOR_INSERT 16 +#define DT_CREATE 32 +#define DT_FILL 64 +#define DT_REINIT 128 +#define DT_PHASES 8 +/* Phases that are applicable to all derived tables. */ +#define DT_COMMON (DT_INIT + DT_PREPARE + DT_REINIT + DT_OPTIMIZE) +/* Phases that are applicable only to materialized derived tables. */ +#define DT_MATERIALIZE (DT_CREATE + DT_FILL) + +#define DT_PHASES_MERGE (DT_COMMON | DT_MERGE | DT_MERGE_FOR_INSERT) +#define DT_PHASES_MATERIALIZE (DT_COMMON | DT_MATERIALIZE) + +#define VIEW_ALGORITHM_UNDEFINED 0 +#define VIEW_ALGORITHM_MERGE (DTYPE_VIEW | DTYPE_MERGE) +#define VIEW_ALGORITHM_TMPTABLE (DTYPE_VIEW + DTYPE_MATERIALIZE ) + #define JOIN_TYPE_LEFT 1 #define JOIN_TYPE_RIGHT 2 -#define VIEW_ALGORITHM_UNDEFINED 0 -#define VIEW_ALGORITHM_TMPTABLE 1 -#define VIEW_ALGORITHM_MERGE 2 - #define VIEW_SUID_INVOKER 0 #define VIEW_SUID_DEFINER 1 #define VIEW_SUID_DEFAULT 2 @@ -1141,6 +1191,7 @@ class Item_in_subselect; also (TABLE_LIST::field_translation != NULL) - tmptable (TABLE_LIST::effective_algorithm == VIEW_ALGORITHM_TMPTABLE) also (TABLE_LIST::field_translation == NULL) + 2.5) TODO: Add derived tables description here 3) nested table reference (TABLE_LIST::nested_join != NULL) - table sequence - e.g. (t1, t2, t3) TODO: how to distinguish from a JOIN? @@ -1153,6 +1204,7 @@ class Item_in_subselect; */ class Index_hint; +struct st_lex; struct TABLE_LIST { TABLE_LIST() {} /* Remove gcc warning */ @@ -1246,6 +1298,8 @@ struct TABLE_LIST filling procedure */ select_union *derived_result; + /* Stub used for materialized derived tables. */ + table_map map; /* ID bit of table (1,2,4,8,16...) */ /* Reference from aux_tables to local list entry of main select of multi-delete statement: @@ -1290,6 +1344,7 @@ struct TABLE_LIST Field_translator *field_translation; /* array of VIEW fields */ /* pointer to element after last one in translation table above */ Field_translator *field_translation_end; + bool field_translation_updated; /* List (based on next_local) of underlying tables of this view. I.e. it does not include the tables of subqueries used in the view. Is set only @@ -1304,11 +1359,18 @@ struct TABLE_LIST List<TABLE_LIST> *view_tables; /* most upper view this table belongs to */ TABLE_LIST *belong_to_view; + /* A derived table this table belongs to */ + TABLE_LIST *belong_to_derived; /* The view directly referencing this table (non-zero only for merged underlying tables of a view). */ TABLE_LIST *referencing_view; + + table_map view_used_tables; + table_map map_exec; + uint tablenr_exec; + /* Ptr to parent MERGE table list item. See top comment in ha_myisammrg.cc */ TABLE_LIST *parent_l; /* @@ -1321,13 +1383,7 @@ struct TABLE_LIST SQL SECURITY DEFINER) */ Security_context *view_sctx; - /* - List of all base tables local to a subquery including all view - tables. Unlike 'next_local', this in this list views are *not* - leaves. Created in setup_tables() -> make_leaves_list(). - */ bool allowed_show; - TABLE_LIST *next_leaf; Item *where; /* VIEW WHERE clause condition */ Item *check_option; /* WITH CHECK OPTION condition */ LEX_STRING select_stmt; /* text of (CREATE/SELECT) statement */ @@ -1363,7 +1419,7 @@ struct TABLE_LIST - VIEW_ALGORITHM_MERGE @to do Replace with an enum */ - uint8 effective_algorithm; + uint8 derived_type; GRANT_INFO grant; /* data need by some engines in query cache*/ ulonglong engine_data; @@ -1390,7 +1446,6 @@ struct TABLE_LIST bool skip_temporary; /* this table shouldn't be temporary */ /* TRUE if this merged view contain auto_increment field */ bool contain_auto_increment; - bool multitable_view; /* TRUE iff this is multitable view */ bool compact_view_format; /* Use compact format for SHOW CREATE VIEW */ /* view where processed */ bool where_processed; @@ -1414,6 +1469,17 @@ struct TABLE_LIST bool internal_tmp_table; bool deleting; /* going to delete this table */ + /* TRUE <=> derived table should be filled right after optimization. */ + bool fill_me; + /* TRUE <=> view/DT is merged. */ + bool merged; + bool merged_for_insert; + /* TRUE <=> don't prepare this derived table/view as it should be merged.*/ + bool skip_prepare_derived; + + List<Item> used_items; + Item **materialized_items; + /* View creation context. */ View_creation_ctx *view_creation_ctx; @@ -1451,9 +1517,10 @@ struct TABLE_LIST bool has_table_lookup_value; uint table_open_method; enum enum_schema_table_state schema_table_state; + void calc_md5(char *buffer); - void set_underlying_merge(); int view_check_option(THD *thd, bool ignore_failure); + bool create_field_translation(THD *thd); bool setup_underlying(THD *thd); void cleanup_items(); bool placeholder() @@ -1483,7 +1550,7 @@ struct TABLE_LIST inline bool prepare_where(THD *thd, Item **conds, bool no_where_clause) { - if (effective_algorithm == VIEW_ALGORITHM_MERGE) + if (!view || is_merged_derived()) return prep_where(thd, conds, no_where_clause); return FALSE; } @@ -1549,6 +1616,60 @@ struct TABLE_LIST m_table_ref_version= s->get_table_ref_version(); } + /* Set of functions returning/setting state of a derived table/view. */ + inline bool is_non_derived() + { + return (!derived_type); + } + inline bool is_view_or_derived() + { + return (derived_type); + } + inline bool is_view() + { + return (derived_type & DTYPE_VIEW); + } + inline bool is_derived() + { + return (derived_type & DTYPE_TABLE); + } + inline void set_view() + { + derived_type= DTYPE_VIEW; + } + inline void set_derived() + { + derived_type= DTYPE_TABLE; + } + inline bool is_merged_derived() + { + return (derived_type & DTYPE_MERGE); + } + inline void set_merged_derived() + { + derived_type= ((derived_type & DTYPE_MASK) | + DTYPE_TABLE | DTYPE_MERGE); + } + inline bool is_materialized_derived() + { + return (derived_type & DTYPE_MATERIALIZE); + } + inline void set_materialized_derived() + { + derived_type= ((derived_type & DTYPE_MASK) | + DTYPE_TABLE | DTYPE_MATERIALIZE); + } + inline bool is_multitable() + { + return (derived_type & DTYPE_MULTITABLE); + } + inline void set_multitable() + { + derived_type|= DTYPE_MULTITABLE; + } + void reset_const_table(); + bool handle_derived(struct st_lex *lex, uint phases); + /** @brief True if this TABLE_LIST represents an anonymous derived table, i.e. the result of a subquery. @@ -1568,6 +1689,12 @@ struct TABLE_LIST respectively. */ char *get_table_name() { return view != NULL ? view_name.str : table_name; } + st_select_lex_unit *get_unit(); + st_select_lex *get_single_select(); + void wrap_into_nested_join(List<TABLE_LIST> &join_list); + bool init_derived(THD *thd, bool init_view); + int fetch_number_of_rows(); + bool change_refs_to_fields(); private: bool prep_check_option(THD *thd, uint8 check_opt_type);

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (igor:2792)
by Igor Babaev 26 May '10

26 May '10

#At lp:maria/5.2 based on revid:igor@askmonty.org-20100518174632-e2xaeunykfmtyafm 2792 Igor Babaev 2010-05-26 MWL#106: creation of keys for materialized derived tables/views. Also fixed several bugs in the backported code. modified: mysql-test/r/derived_view.result sql/item_cmpfunc.cc sql/item_subselect.cc sql/sql_base.cc sql/sql_delete.cc sql/sql_insert.cc sql/sql_join_cache.cc sql/sql_lex.h sql/sql_parse.cc sql/sql_prepare.cc sql/sql_select.cc sql/sql_select.h sql/table.cc sql/table.h === modified file 'mysql-test/r/derived_view.result' --- a/mysql-test/r/derived_view.result 2010-05-18 17:46:32 +0000 +++ b/mysql-test/r/derived_view.result 2010-05-26 20:04:58 +0000 @@ -556,7 +556,7 @@ test two keys explain select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY t1 ALL NULL NULL NULL NULL 11 -1 PRIMARY <derived2> ALL key0 NULL NULL NULL 11 Using where; Using join buffer +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 Using where 1 PRIMARY xx ALL NULL NULL NULL NULL 11 Using where; Using join buffer 2 DERIVED t2 ALL NULL NULL NULL NULL 11 Using temporary; Using filesort select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-05-18 17:46:32 +0000 +++ b/sql/item_cmpfunc.cc 2010-05-26 20:04:58 +0000 @@ -4232,10 +4232,6 @@ Item_cond::fix_fields(THD *thd, Item **r (item= *li.ref())->check_cols(1)) return TRUE; /* purecov: inspected */ used_tables_cache|= item->used_tables(); -#if 0 - if (!item->const_item()) - const_item_cache= FALSE; -#else if (item->const_item()) and_tables_cache= (table_map) 0; else @@ -4245,7 +4241,6 @@ Item_cond::fix_fields(THD *thd, Item **r and_tables_cache&= tmp_table_map; const_item_cache= FALSE; } -#endif with_sum_func= with_sum_func || item->with_sum_func; with_subselect|= item->with_subselect; === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-04-29 21:10:39 +0000 +++ b/sql/item_subselect.cc 2010-05-26 20:04:58 +0000 @@ -2894,6 +2894,9 @@ int subselect_uniquesubquery_engine::exe DBUG_RETURN(0); } + if (!tab->preread_init_done && tab->preread_init()) + DBUG_RETURN(1); + if (null_keypart) DBUG_RETURN(scan_table()); @@ -3026,7 +3029,7 @@ subselect_uniquesubquery_engine::~subsel int subselect_indexsubquery_engine::exec() { - DBUG_ENTER("subselect_indexsubquery_engine::exec"); + DBUG_ENTER("subselect_indexsubquery_engine"); int error; bool null_finding= 0; TABLE *table= tab->table; @@ -3057,6 +3060,9 @@ int subselect_indexsubquery_engine::exec DBUG_RETURN(0); } + if (!tab->preread_init_done && tab->preread_init()) + DBUG_RETURN(1); + if (null_keypart) DBUG_RETURN(scan_table()); === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-05-12 04:09:58 +0000 +++ b/sql/sql_base.cc 2010-05-26 20:04:58 +0000 @@ -6288,7 +6288,9 @@ find_field_in_tables(THD *thd, Item_iden find_field_in_table even in the case of information schema tables when table_ref->field_translation != NULL. */ - if (table_ref->table && !table_ref->is_merged_derived()) + if (table_ref->table && + (!table_ref->is_merged_derived() || + (!table_ref->is_multitable() && table_ref->merged_for_insert))) found= find_field_in_table(thd, table_ref->table, name, length, TRUE, &(item->cached_field_index)); else === modified file 'sql/sql_delete.cc' --- a/sql/sql_delete.cc 2010-05-12 04:09:58 +0000 +++ b/sql/sql_delete.cc 2010-05-26 20:04:58 +0000 @@ -559,6 +559,11 @@ int mysql_multi_delete_prepare(THD *thd) TABLE_LIST *target_tbl; DBUG_ENTER("mysql_multi_delete_prepare"); + TABLE_LIST *tables= lex->query_tables; + if (mysql_handle_derived(lex, DT_INIT) || + mysql_handle_list_of_derived(lex, tables, DT_MERGE_FOR_INSERT) || + mysql_handle_list_of_derived(lex, tables, DT_PREPARE)) + DBUG_RETURN(TRUE); /* setup_tables() need for VIEWs. JOIN::prepare() will not do it second time. === modified file 'sql/sql_insert.cc' --- a/sql/sql_insert.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_insert.cc 2010-05-26 20:04:58 +0000 @@ -1184,8 +1184,8 @@ static bool mysql_prepare_insert_check_t if (insert_into_view && !fields.elements) { thd->lex->empty_field_list_on_rset= 1; - if (table_list->is_multitable() && !table_list->table || - !table_list->table->created) + if (!thd->lex->select_lex.leaf_tables.head()->table || + table_list->is_multitable()) { my_error(ER_VIEW_NO_INSERT_FIELD_LIST, MYF(0), table_list->view_db.str, table_list->view_name.str); @@ -1276,8 +1276,10 @@ bool mysql_prepare_insert(THD *thd, TABL /* INSERT should have a SELECT or VALUES clause */ DBUG_ASSERT (!select_insert || !values); + if (mysql_handle_derived(thd->lex, DT_INIT)) + DBUG_RETURN(TRUE); if (table_list->handle_derived(thd->lex, DT_MERGE_FOR_INSERT)) - DBUG_RETURN(TRUE); + DBUG_RETURN(TRUE); if (mysql_handle_list_of_derived(thd->lex, table_list, DT_PREPARE)) DBUG_RETURN(TRUE); /* === modified file 'sql/sql_join_cache.cc' --- a/sql/sql_join_cache.cc 2010-03-07 15:41:45 +0000 +++ b/sql/sql_join_cache.cc 2010-05-26 20:04:58 +0000 @@ -2370,6 +2370,8 @@ JOIN_CACHE_BKA::init_join_matching_recor init_mrr_buff(); + if (!join_tab->preread_init_done && join_tab->preread_init()) + return NESTED_LOOP_ERROR; /* Prepare to iterate over keys from the join buffer and to get matching candidates obtained with MMR handler functions. === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-04-29 21:10:39 +0000 +++ b/sql/sql_lex.h 2010-05-26 20:04:58 +0000 @@ -1873,6 +1873,8 @@ typedef struct st_lex : public Query_tab switch (sql_command) { case SQLCOM_UPDATE: case SQLCOM_UPDATE_MULTI: + case SQLCOM_DELETE: + case SQLCOM_DELETE_MULTI: case SQLCOM_INSERT: case SQLCOM_INSERT_SELECT: case SQLCOM_REPLACE: === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_parse.cc 2010-05-26 20:04:58 +0000 @@ -3425,9 +3425,6 @@ end_with_restore_list: thd_proc_info(thd, "init"); if ((res= open_and_lock_tables(thd, all_tables))) break; - if (mysql_handle_list_of_derived(lex, all_tables, DT_MERGE_FOR_INSERT) || - mysql_handle_list_of_derived(lex, all_tables, DT_PREPARE)) - DBUG_RETURN(1); if ((res= mysql_multi_delete_prepare(thd))) goto error; === modified file 'sql/sql_prepare.cc' --- a/sql/sql_prepare.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_prepare.cc 2010-05-26 20:04:58 +0000 @@ -1133,9 +1133,7 @@ static bool mysql_test_insert(Prepared_s If we would use locks, then we have to ensure we are not using TL_WRITE_DELAYED as having two such locks can cause table corruption. */ - if (open_normal_and_derived_tables(thd, table_list, 0, - DT_INIT | DT_PREPARE | DT_CREATE) || - mysql_handle_single_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT)) + if (open_normal_and_derived_tables(thd, table_list, 0, DT_INIT)) goto error; if ((values= its++)) @@ -1236,9 +1234,16 @@ static int mysql_test_update(Prepared_st thd->fill_derived_tables() is false here for sure (because it is preparation of PS, so we even do not check it). */ - if (mysql_handle_derived(thd->lex, DT_PREPARE)) + if (table_list->handle_derived(thd->lex, DT_MERGE_FOR_INSERT) || + table_list->handle_derived(thd->lex, DT_PREPARE)) goto error; + if (!table_list->updatable) + { + my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "UPDATE"); + goto error; + } + #ifndef NO_EMBEDDED_ACCESS_CHECKS /* Force privilege re-checking for views after they have been opened. */ want_privilege= (table_list->view ? UPDATE_ACL : @@ -1291,13 +1296,18 @@ error: static bool mysql_test_delete(Prepared_statement *stmt, TABLE_LIST *table_list) { + uint table_count= 0; THD *thd= stmt->thd; LEX *lex= stmt->lex; DBUG_ENTER("mysql_test_delete"); if (delete_precheck(thd, table_list) || - open_normal_and_derived_tables(thd, table_list, 0, - DT_PREPARE | DT_CREATE)) + open_tables(thd, &table_list, &table_count, 0)) + goto error; + + if (mysql_handle_derived(thd->lex, DT_INIT) || + mysql_handle_list_of_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT) || + mysql_handle_list_of_derived(thd->lex, table_list, DT_PREPARE)) goto error; if (!table_list->table) @@ -1561,6 +1571,7 @@ select_like_stmt_test_with_open(Prepared int (*specific_prepare)(THD *thd), ulong setup_tables_done_option) { + uint table_count= 0; DBUG_ENTER("select_like_stmt_test_with_open"); /* @@ -1569,8 +1580,8 @@ select_like_stmt_test_with_open(Prepared prepared EXPLAIN yet so derived tables will clean up after themself. */ - if (open_normal_and_derived_tables(stmt->thd, tables, 0, - DT_PREPARE | DT_CREATE)) + THD *thd= stmt->thd; + if (open_tables(thd, &tables, &table_count, 0)) DBUG_RETURN(TRUE); DBUG_RETURN(select_like_stmt_test(stmt, specific_prepare, === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-18 17:46:32 +0000 +++ b/sql/sql_select.cc 2010-05-26 20:04:58 +0000 @@ -238,7 +238,7 @@ static void add_group_and_distinct_keys( void get_partial_join_cost(JOIN *join, uint idx, double *read_time_arg, double *record_count_arg); static uint make_join_orderinfo(JOIN *join); -static bool generate_derived_keys(List<TABLE_LIST> &tables); +static bool generate_derived_keys(DYNAMIC_ARRAY *keyuse_array); static int join_read_record_no_init(JOIN_TAB *tab); @@ -744,18 +744,6 @@ JOIN::optimize() tables= select_lex->leaf_tables.elements; -#if 0 - if (thd->lex->describe) - { - /* - Force join->join_tmp creation, because we will use this JOIN - twice for EXPLAIN and we have to have unchanged join for EXPLAINing - */ - select_lex->uncacheable|= UNCACHEABLE_EXPLAIN; - select_lex->master_unit()->uncacheable|= UNCACHEABLE_EXPLAIN; - } -#else -#endif if (setup_ftfuncs(select_lex)) /* should be after having->fix_fields */ DBUG_RETURN(-1); @@ -3311,18 +3299,20 @@ add_key_field(KEY_FIELD **key_fields,uin Field *field, bool eq_func, Item **value, uint num_values, table_map usable_tables, SARGABLE_PARAM **sargables) { - uint exists_optimize= 0; - if (field->table->pos_in_table_list->is_materialized_derived() && + uint optimize= 0; + if (eq_func && + field->table->pos_in_table_list->is_materialized_derived() && !field->table->created) - field->table->pos_in_table_list->update_derived_keys(field, value, - num_values); - if (!(field->flags & PART_KEY_FLAG)) + { + optimize= KEY_OPTIMIZE_EQ; + } + else if (!(field->flags & PART_KEY_FLAG)) { // Don't remove column IS NULL on a LEFT JOIN table if (!eq_func || (*value)->type() != Item::NULL_ITEM || !field->table->maybe_null || field->null_ptr) return; // Not a key. Skip it - exists_optimize= KEY_OPTIMIZE_EXISTS; + optimize= KEY_OPTIMIZE_EXISTS; DBUG_ASSERT(num_values == 1); } else @@ -3342,7 +3332,7 @@ add_key_field(KEY_FIELD **key_fields,uin if (!eq_func || (*value)->type() != Item::NULL_ITEM || !field->table->maybe_null || field->null_ptr) return; // Can't use left join optimize - exists_optimize= KEY_OPTIMIZE_EXISTS; + optimize= KEY_OPTIMIZE_EXISTS; } else { @@ -3441,7 +3431,7 @@ add_key_field(KEY_FIELD **key_fields,uin (*key_fields)->eq_func= eq_func; (*key_fields)->val= *value; (*key_fields)->level= and_level; - (*key_fields)->optimize= exists_optimize; + (*key_fields)->optimize= optimize; /* If the condition has form "tbl.keypart = othertbl.field" and othertbl.field can be NULL, there will be no matches if othertbl.field @@ -3760,6 +3750,34 @@ max_part_bit(key_part_map bits) return found; } +static bool +add_keyuse(DYNAMIC_ARRAY *keyuse_array, KEY_FIELD *key_field, + uint key, uint part) +{ + KEYUSE keyuse; + Field *field= key_field->field; + + keyuse.table= field->table; + keyuse.val= key_field->val; + keyuse.key= key; + if (key != MAX_KEY) + { + keyuse.keypart=part; + keyuse.keypart_map= (key_part_map) 1 << part; + } + else + { + keyuse.keypart= field->field_index; + keyuse.keypart_map= (key_part_map) 0; + } + keyuse.used_tables= key_field->val->used_tables(); + keyuse.optimize= key_field->optimize & KEY_OPTIMIZE_REF_OR_NULL; + keyuse.null_rejecting= key_field->null_rejecting; + keyuse.cond_guard= key_field->cond_guard; + keyuse.sj_pred_no= key_field->sj_pred_no; + return (insert_dynamic(keyuse_array,(uchar*) &keyuse)); +} + /* Add all keys with uses 'field' for some keypart If field->and_level != and_level then only mark key_part as const_part @@ -3774,10 +3792,13 @@ add_key_part(DYNAMIC_ARRAY *keyuse_array { Field *field=key_field->field; TABLE *form= field->table; - KEYUSE keyuse; if (key_field->eq_func && !(key_field->optimize & KEY_OPTIMIZE_EXISTS)) { + if (key_field->eq_func && (key_field->optimize & KEY_OPTIMIZE_EQ)) + { + return add_keyuse(keyuse_array, key_field, MAX_KEY, 0); + } for (uint key=0 ; key < form->s->keys ; key++) { if (!(form->keys_in_use_for_query.is_set(key))) @@ -3790,17 +3811,7 @@ add_key_part(DYNAMIC_ARRAY *keyuse_array { if (field->eq(form->key_info[key].key_part[part].field)) { - keyuse.table= field->table; - keyuse.val = key_field->val; - keyuse.key = key; - keyuse.keypart=part; - keyuse.keypart_map= (key_part_map) 1 << part; - keyuse.used_tables=key_field->val->used_tables(); - keyuse.optimize= key_field->optimize & KEY_OPTIMIZE_REF_OR_NULL; - keyuse.null_rejecting= key_field->null_rejecting; - keyuse.cond_guard= key_field->cond_guard; - keyuse.sj_pred_no= key_field->sj_pred_no; - if (insert_dynamic(keyuse_array,(uchar*) &keyuse)) + if (add_keyuse(keyuse_array, key_field, key, part)) return TRUE; } } @@ -3885,6 +3896,9 @@ sort_keyuse(KEYUSE *a,KEYUSE *b) return (int) (a->table->tablenr - b->table->tablenr); if (a->key != b->key) return (int) (a->key - b->key); + if (a->key == MAX_KEY && b->key == MAX_KEY && + a->used_tables != b->used_tables) + return (int) ((ulong) a->used_tables - (ulong) b->used_tables); if (a->keypart != b->keypart) return (int) (a->keypart - b->keypart); // Place const values before other ones @@ -4080,9 +4094,6 @@ update_ref_and_keys(THD *thd, DYNAMIC_AR } } - /* Generate keys descriptions for derived tables */ - generate_derived_keys(select_lex->leaf_tables); - /* fill keyuse with found key parts */ for ( ; field != end ; field++) { @@ -4117,6 +4128,8 @@ update_ref_and_keys(THD *thd, DYNAMIC_AR if (insert_dynamic(keyuse,(uchar*) &key_end)) return TRUE; + generate_derived_keys(keyuse); + use=save_pos=dynamic_element(keyuse,0,KEYUSE*); prev= &key_end; found_eq_constant=0; @@ -6979,33 +6992,93 @@ make_join_select(JOIN *join,SQL_SELECT * } -/** - @brief - Add keys to derived tables'/views' result tables in a list - - @param tables list of tables to generate keys for - - @details - This function generates keys for all derived tables/views in the 'tables' - list with help of the TABLE_LIST:generate_keys function. +static +uint get_next_field_for_derived_key(uchar *arg) +{ + KEYUSE *keyuse= *(KEYUSE **) arg; + if (!keyuse) + return (uint) (-1); + uint key= keyuse->key; + uint fldno= keyuse->keypart; + uint keypart= keyuse->keypart_map == (key_part_map) 1 ? + 0 : (keyuse-1)->keypart+1; + for ( ; keyuse->key == key && keyuse->keypart == fldno; keyuse++) + keyuse->keypart= keypart; + if (keyuse->key != key) + keyuse= 0; + return fldno; +} - @note currently this function can't fail because errors from the - TABLE_LIST:generate_keys function is ignored as they aren't critical to the - query execution. - @return FALSE all keys were successfully added. -*/ +static +bool generate_derived_keys_for_table(KEYUSE *keyuse, uint count, uint keys) +{ + TABLE *table= keyuse->table; + if (table->alloc_keys(keys)) + return TRUE; + uint keyno= 0; + KEYUSE *first_keyuse= keyuse; + uint prev_part= (uint) (-1); + uint parts= 0; + uint i= 0; + do + { + keyuse->key= keyno; + keyuse->keypart_map= (key_part_map) (1 << parts); + keyuse++; + if (++i == count || keyuse->used_tables != first_keyuse->used_tables) + { + if (table->add_tmp_key(keyno, ++parts, + get_next_field_for_derived_key, + (uchar *) &first_keyuse)) + return TRUE; + first_keyuse= keyuse; + keyno++; + parts= 0; + } + else if (keyuse->keypart != prev_part) + { + parts++; + prev_part= keyuse->keypart; + } + } while (keyno < keys); + return FALSE; +} + static -bool generate_derived_keys(List<TABLE_LIST> &tables) +bool generate_derived_keys(DYNAMIC_ARRAY *keyuse_array) { - TABLE_LIST *table; - List_iterator<TABLE_LIST> ti(tables); - while ((table= ti++)) + KEYUSE *keyuse= dynamic_element(keyuse_array, 0, KEYUSE*); + uint elements= keyuse_array->elements; + TABLE *prev_table= 0; + for (uint i= 0; i < elements; i++, keyuse++) { - /* Process tables that aren't materialized yet. */ - if (table->is_materialized_derived() && !table->table->created) - table->generate_keys(); + KEYUSE *first_table_keyuse; + table_map last_used_tables; + uint count; + uint keys; + while (keyuse->key == MAX_KEY) + { + if (keyuse->table != prev_table) + { + prev_table= keyuse->table; + first_table_keyuse= keyuse; + last_used_tables= keyuse->used_tables; + count= 0; + keys= 0; + } + else if (keyuse->used_tables != last_used_tables) + { + keys++; + last_used_tables= keyuse->used_tables; + } + count++; + keyuse++; + if (keyuse->table != prev_table && + generate_derived_keys_for_table(first_table_keyuse, count, ++keys)) + return TRUE; + } } return FALSE; } === modified file 'sql/sql_select.h' --- a/sql/sql_select.h 2010-05-18 17:46:32 +0000 +++ b/sql/sql_select.h 2010-05-26 20:04:58 +0000 @@ -37,6 +37,7 @@ /* Values in optimize */ #define KEY_OPTIMIZE_EXISTS 1 #define KEY_OPTIMIZE_REF_OR_NULL 2 +#define KEY_OPTIMIZE_EQ 4 typedef struct keyuse_t { TABLE *table; === modified file 'sql/table.cc' --- a/sql/table.cc 2010-05-12 04:09:58 +0000 +++ b/sql/table.cc 2010-05-26 20:04:58 +0000 @@ -5143,12 +5143,11 @@ void st_table::mark_virtual_columns_for_ bool TABLE::alloc_keys(uint key_count) { DBUG_ASSERT(!s->keys); - key_info= s->key_info= (KEY*) my_malloc(sizeof(KEY)*key_count, MYF(0)); + key_info= s->key_info= (KEY*) alloc_root(&mem_root, sizeof(KEY)*key_count); max_keys= key_count; return !(key_info); } - /** @brief Adds one key to a temporary table. @@ -5164,40 +5163,41 @@ bool TABLE::alloc_keys(uint key_count) @return >=0 number of newly added key. */ -int TABLE::add_tmp_key(ulonglong key_parts, char *key_name) +bool TABLE::add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg) { - DBUG_ASSERT(!created && s->keys< max_keys); + DBUG_ASSERT(!created && key < max_keys); + char buf[NAME_CHAR_LEN]; KEY* keyinfo; Field **reg_field; uint i; bool key_start= TRUE; - uint key_part_count= my_count_bits(key_parts); KEY_PART_INFO* key_part_info= - (KEY_PART_INFO*) my_malloc(sizeof(KEY_PART_INFO)* key_part_count, MYF(0)); + (KEY_PART_INFO*) alloc_root(&mem_root, sizeof(KEY_PART_INFO)*key_parts); if (!key_part_info) - return -1; - keyinfo= key_info + s->keys; - keyinfo->key_part=key_part_info; - keyinfo->usable_key_parts=keyinfo->key_parts= key_part_count; + return TRUE; + keyinfo= key_info + key; + keyinfo->key_part= key_part_info; + keyinfo->usable_key_parts= keyinfo->key_parts = key_parts; keyinfo->key_length=0; keyinfo->algorithm= HA_KEY_ALG_UNDEF; - keyinfo->name= key_name; keyinfo->flags= HA_GENERATED_KEY; - keyinfo->rec_per_key= (ulong*)my_malloc(sizeof(ulong)*key_part_count, MYF(0)); + sprintf(buf, "key%i", key); + if (!(keyinfo->name= strdup_root(&mem_root, buf))) + return TRUE; + keyinfo->rec_per_key= (ulong*) alloc_root(&mem_root, + sizeof(ulong)*key_parts); if (!keyinfo->rec_per_key) - return -1; - bzero(keyinfo->rec_per_key, sizeof(ulong)*key_part_count); - for (i= 0, reg_field=field ; - *reg_field; - i++, reg_field++) + return TRUE; + bzero(keyinfo->rec_per_key, sizeof(ulong)*key_parts); + for (i= 0; i < key_parts; i++) { - if (!(key_parts & (1 << i))) - continue; + reg_field= field + next_field_no(arg); if (key_start) - (*reg_field)->key_start.set_bit(s->keys); + (*reg_field)->key_start.set_bit(key); key_start= FALSE; - (*reg_field)->part_of_key.set_bit(s->keys); + (*reg_field)->part_of_key.set_bit(key); (*reg_field)->flags|= PART_KEY_FLAG; key_part_info->null_bit= (*reg_field)->null_bit; key_part_info->null_offset= (uint) ((*reg_field)->null_ptr - @@ -5231,10 +5231,10 @@ int TABLE::add_tmp_key(ulonglong key_par key_part_info++; } set_if_bigger(s->max_key_length, keyinfo->key_length); - return ++s->keys - 1; + s->keys++; + return FALSE; } - /* @brief Drop all indexes except specified one. @@ -5310,6 +5310,7 @@ void TABLE_LIST::reinit_before_use(THD * parent_embedding->nested_join->join_list.head() == embedded); } + /* Return subselect that contains the FROM list this table is taken from @@ -5819,143 +5820,8 @@ int TABLE_LIST::fetch_number_of_rows() t1 ON tt.f1=a_function(t1.f3); In this case for the derived table tt one key will be generated. It will consist of one field - f1. - - Implementation is split in two steps: - gathering information on all used fields of derived tables/view and - store it in lists of possible keys, one per a derived table/view. - add keys to result tables of derived tables/view using info from above - lists. - - The above procedure is implemented in 4 functions: - TABLE_LIST::update_derived_keys - Create/extend list of possible keys for one derived - table/view based on given field/used tables info. - (Step one) - generate_derived_keys This function is called at the moment when all - possible info on keys is gathered and it's safe to - add keys. Walk over list of derived tables/views and - calls to TABLE_LIST::generate_keys to actually - generate keys. (Step two) - TABLE_LIST::generate_keys - Walks over list of possible keys for this derived - table/view to add keys to the result table. - Calls to TABLE::add_tmp_index to actually add - keys. (Step two) - TABLE::add_tmp_index Creates one index description according to given - bitmap of used fields. (Step two) - There is also the fifth function called TABLE::use_index. It saves used - key and frees others. It is called when the optimizer has chosen which key - it will use, thus we don't need other keys anymore. -*/ - - -/* - @brief - Update derived table's list of possible keys - - @param field derived table's field to take part in a key - @param values array of values - @param num_values number of elements in the array values - - @details - This function creates/extends a list of possible keys for this derived - table/view. For each table used by a value from the 'values' array the - corresponding possible key is extended to include the 'field'. - If there is no such possible key then it is created. field's - key_start/part_of_key bitmaps are updated accordingly. - - @return TRUE new possible key can't be allocated. - @return FALSE list of possible keys successfully updated. -*/ - -bool TABLE_LIST::update_derived_keys(Field *field, Item **values, - uint num_values) -{ - DERIVED_KEY_MAP *entry= 0; - List_iterator<DERIVED_KEY_MAP> ki(derived_keymap_list); - uint i; - - /* Allow all keys to be used. */ - if (!derived_keymap_list.elements) - { - table->keys_in_use_for_query.set_all(); - table->s->uniques= 0; - derived_keymap_list.empty(); - } - - for (i= 0; i < num_values; i++) - { - uint tbl; - table_map tables= values[i]->used_tables() & ~OUTER_REF_TABLE_BIT; - for (tbl= 1; tables >= tbl; tbl<<= 1) - { - uint key= 0; - if (! (tables & tbl)) - continue; - ki.rewind(); - while ((entry= ki++)) - { - key++; - if (entry->referenced_by & tbl) - break; - } - if (!entry) - { - key++; - entry= (DERIVED_KEY_MAP*)my_malloc(sizeof(DERIVED_KEY_MAP), MYF(0)); - if (!entry) - return TRUE; - entry->referenced_by|= tbl; - entry->used_fields.clear_all(); - derived_keymap_list.push_back(entry); - field->key_start.set_bit(key); - table->max_keys++; - } - field->part_of_key.set_bit(key - 1); - field->flags|= PART_KEY_FLAG; - entry->used_fields.set_bit(field->field_index); - } - } - return FALSE; -} - - -/** - @brief - Generate keys for a materialized derived table/view - - @details - This function adds keys to the result table by walking over the list of - possible keys for this derived table/view and calling to the - TABLE::add_tmp_index to actually add keys. A name "key" with a sequential - number is given to each key to ease debugging. - - @return TRUE an error occur. - @return FALSE all keys were successfully added. */ -bool TABLE_LIST::generate_keys() -{ - List_iterator<DERIVED_KEY_MAP> it(derived_keymap_list); - DERIVED_KEY_MAP *entry; - uint key= 0; - char buf[NAME_CHAR_LEN]; - DBUG_ASSERT(is_materialized_derived()); - - if (!derived_keymap_list.elements) - return FALSE; - - table->alloc_keys(table->max_keys); - while ((entry= it++)) - { - table->s->key_parts+= entry->used_fields.bits_set(); - sprintf(buf, "key%i", key++); - if (table->add_tmp_key(entry->used_fields.to_ulonglong(), - table->in_use->strdup(buf)) < 0) - return TRUE; - } - return FALSE; -} /* === modified file 'sql/table.h' --- a/sql/table.h 2010-04-29 21:10:39 +0000 +++ b/sql/table.h 2010-05-26 20:04:58 +0000 @@ -916,7 +916,8 @@ struct st_table { inline bool needs_reopen_or_name_lock() { return s->version != refresh_version; } bool alloc_keys(uint key_count); - int add_tmp_key(ulonglong key_parts, char *key_name); + bool add_tmp_key(uint key, uint key_parts, + uint (*next_field_no) (uchar *), uchar *arg); void use_index(int key_to_save); void set_table_map(table_map map_arg, uint tablenr_arg) { @@ -1202,20 +1203,6 @@ class Item_in_subselect; (TABLE_LIST::join_using_fields != NULL) */ -/* - This structure is used to keep info about possible key for the result table - of a derived table/view. - The 'referenced_by' is the table map of the table to which this possible - key corresponds. - The 'used_field' is a map of fields of which this key consists of. - See also the comment for the TABLE_LIST::update_derived_keys function. -*/ -struct st_derived_table_key_map { - table_map referenced_by; - key_map used_fields; -}; -typedef st_derived_table_key_map DERIVED_KEY_MAP; - class Index_hint; struct st_lex; struct TABLE_LIST @@ -1531,8 +1518,6 @@ struct TABLE_LIST uint table_open_method; enum enum_schema_table_state schema_table_state; - List<DERIVED_KEY_MAP> derived_keymap_list; - void calc_md5(char *buffer); int view_check_option(THD *thd, bool ignore_failure); bool create_field_translation(THD *thd); @@ -1709,8 +1694,6 @@ struct TABLE_LIST void wrap_into_nested_join(List<TABLE_LIST> &join_list); bool init_derived(THD *thd, bool init_view); int fetch_number_of_rows(); - bool update_derived_keys(Field *field, Item **values, uint num_values); - bool generate_keys(); bool change_refs_to_fields(); private:

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1]
by Sergei 26 May '10

26 May '10

Status: Development => Mature -- lp:maria/5.1 https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria/5.1. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] Bzr upgrade needed for buildbot slaves
by Kristian Nielsen 26 May '10

26 May '10

Hi Adam, We're upgrading the bzr repositories on launchpad to a new repository format 2a. Because of this, you need to do some changes on your buildbot hosts. These are the slaves mariadb-brs and adutko-ultrasparc3. (The slave adutko-centos5-amd64 does not need any changes, as it is not using bzr for the build). You have to do two things: 1. make sure you are using at least bzr 1.16 (or better 2.x). If the bzr on either slave is older, then you need to upgrade it. 2. Upgrade the shared repository on the slave to format 2a. These are the following directories on the slaves: /export/home/buildbot/maria-slave/adutko-ultrasparc3/.bzr /var/lib/buildbot/maria-slave/.bzr You need to move away/delete those .bzr directories and replace them with a new format 2a repository .bzr directory. You can obtain the new .bzr directory in two ways: - Download http://hasky.askmonty.org/download/mariadb-shared-repo.tgz , there is a suitable .bzr included (300Mb). - Delete the old .bzr and run bzr init-repo --format=2a /export/home/buildbot/maria-slave/adutko-ultrasparc3 respectively bzr init-repo --format=2a /var/lib/buildbot/maria-slave In the latter case, buildbot will need to download (once) all revisions from launchpad again, so the first way is best on slow connections (around 200Mb of download). Afterwards you can verify that things work by running /export/home/buildbot/maria-slave/adutko-ultrasparc3 /var/lib/buildbot/maria-slave It should say something like Shared repository with trees (format: 2a) Just let me know if you need any help, and sorry for the inconvenience, - Kristian.

2 2

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2859: Fix short version number
by noreply＠launchpad.net 26 May '10

26 May '10

------------------------------------------------------------ revno: 2859 committer: Bo Thorsen <bo(a)askmonty.org> branch nick: trunk-work timestamp: Wed 2010-05-26 10:40:01 +0200 message: Fix short version number modified: win/make_mariadb_win_dist -- lp:maria/5.1 https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria/5.1. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2849)
by knielsen＠knielsen-hq.org 26 May '10

26 May '10

#At lp:maria 2849 knielsen(a)knielsen-hq.org 2010-05-26 Preliminary commit of group commit patch. added: mysql-test/suite/binlog/r/binlog_ioerr.result mysql-test/suite/binlog/t/binlog_ioerr.test modified: sql/handler.cc sql/handler.h sql/log.cc sql/log.h sql/log_event.h sql/sql_class.cc sql/sql_class.h sql/sql_load.cc sql/table.cc sql/table.h storage/xtradb/handler/ha_innodb.cc === added file 'mysql-test/suite/binlog/r/binlog_ioerr.result' --- a/mysql-test/suite/binlog/r/binlog_ioerr.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/binlog/r/binlog_ioerr.result 2010-05-26 08:16:18 +0000 @@ -0,0 +1,28 @@ +CALL mtr.add_suppression("Error writing file 'master-bin'"); +RESET MASTER; +CREATE TABLE t1 (a INT PRIMARY KEY) ENGINE=innodb; +INSERT INTO t1 VALUES(0); +SET SESSION debug='+d,fail_binlog_write_1'; +INSERT INTO t1 VALUES(1); +ERROR HY000: Error writing file 'master-bin' (errno: 22) +INSERT INTO t1 VALUES(2); +ERROR HY000: Error writing file 'master-bin' (errno: 22) +SET SESSION debug=''; +INSERT INTO t1 VALUES(3); +SELECT * FROM t1; +a +0 +3 +SHOW BINLOG EVENTS; +Log_name Pos Event_type Server_id End_log_pos Info +BINLOG POS Format_desc 1 ENDPOS Server ver: #, Binlog ver: # +BINLOG POS Query 1 ENDPOS use `test`; CREATE TABLE t1 (a INT PRIMARY KEY) ENGINE=innodb +BINLOG POS Query 1 ENDPOS BEGIN +BINLOG POS Query 1 ENDPOS use `test`; INSERT INTO t1 VALUES(0) +BINLOG POS Xid 1 ENDPOS COMMIT /* XID */ +BINLOG POS Query 1 ENDPOS BEGIN +BINLOG POS Query 1 ENDPOS BEGIN +BINLOG POS Query 1 ENDPOS BEGIN +BINLOG POS Query 1 ENDPOS use `test`; INSERT INTO t1 VALUES(3) +BINLOG POS Xid 1 ENDPOS COMMIT /* XID */ +DROP TABLE t1; === added file 'mysql-test/suite/binlog/t/binlog_ioerr.test' --- a/mysql-test/suite/binlog/t/binlog_ioerr.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/binlog/t/binlog_ioerr.test 2010-05-26 08:16:18 +0000 @@ -0,0 +1,29 @@ +source include/have_debug.inc; +source include/have_innodb.inc; +source include/have_log_bin.inc; +source include/have_binlog_format_mixed_or_statement.inc; + +CALL mtr.add_suppression("Error writing file 'master-bin'"); + +RESET MASTER; + +CREATE TABLE t1 (a INT PRIMARY KEY) ENGINE=innodb; +INSERT INTO t1 VALUES(0); +SET SESSION debug='+d,fail_binlog_write_1'; +--error ER_ERROR_ON_WRITE +INSERT INTO t1 VALUES(1); +--error ER_ERROR_ON_WRITE +INSERT INTO t1 VALUES(2); +SET SESSION debug=''; +INSERT INTO t1 VALUES(3); +SELECT * FROM t1; + +# Actually the output from this currently shows a bug. +# The injected IO error leaves partially written transactions in the binlog in +# the form of stray "BEGIN" events. +# These should disappear from the output if binlog error handling is improved. +--replace_regex /\/\* xid=.* \*\//\/* XID *\// /Server ver: .*, Binlog ver: .*/Server ver: #, Binlog ver: #/ /table_id: [0-9]+/table_id: #/ +--replace_column 1 BINLOG 2 POS 5 ENDPOS +SHOW BINLOG EVENTS; + +DROP TABLE t1; === modified file 'sql/handler.cc' --- a/sql/handler.cc 2010-04-06 22:47:08 +0000 +++ b/sql/handler.cc 2010-05-26 08:16:18 +0000 @@ -76,6 +76,7 @@ TYPELIB tx_isolation_typelib= {array_ele static TYPELIB known_extensions= {0,"known_exts", NULL, NULL}; uint known_extensions_id= 0; +static int commit_one_phase_2(THD *thd, bool all, bool do_commit_ordered); static plugin_ref ha_default_plugin(THD *thd) @@ -544,6 +545,26 @@ err: DBUG_RETURN(1); } +/* + This is a queue of THDs waiting for being group committed with + tc_log->group_log_xid(). +*/ +static THD *group_commit_queue; +/* + This mutex protects the group_commit_queue on platforms without native + atomic operations. + */ +static pthread_mutex_t LOCK_group_commit_queue; +/* This mutex is used to serialize calls to handler prepare_ordered methods. */ +static pthread_mutex_t LOCK_prepare_ordered; +/* This mutex is used to serialize calls to handler commit_ordered methods. */ +static pthread_mutex_t LOCK_commit_ordered; +/* This mutex is used to serialize calls to group_log_xid(). */ +static pthread_mutex_t LOCK_group_commit; +static pthread_cond_t COND_group_commit; + +static bool mutexes_inited= FALSE; + int ha_init() { int error= 0; @@ -557,6 +578,19 @@ int ha_init() */ opt_using_transactions= total_ha>(ulong)opt_bin_log; savepoint_alloc_size+= sizeof(SAVEPOINT); + + group_commit_queue= NULL; + my_pthread_mutex_init(&LOCK_group_commit_queue, MY_MUTEX_INIT_FAST, + "LOCK_group_commit_queue", MYF(0)); + my_pthread_mutex_init(&LOCK_prepare_ordered, MY_MUTEX_INIT_SLOW, + "LOCK_prepare_ordered", MYF(0)); + my_pthread_mutex_init(&LOCK_commit_ordered, MY_MUTEX_INIT_SLOW, + "LOCK_commit_ordered", MYF(0)); + my_pthread_mutex_init(&LOCK_group_commit, MY_MUTEX_INIT_SLOW, + "LOCK_group_commit", MYF(0)); + pthread_cond_init(&COND_group_commit, 0); + mutexes_inited= TRUE; + DBUG_RETURN(error); } @@ -574,6 +608,15 @@ int ha_end() if (ha_finish_errors()) error= 1; + if (mutexes_inited) + { + pthread_mutex_destroy(&LOCK_group_commit_queue); + pthread_mutex_destroy(&LOCK_prepare_ordered); + pthread_mutex_destroy(&LOCK_commit_ordered); + pthread_mutex_destroy(&LOCK_group_commit); + mutexes_inited= FALSE; + } + DBUG_RETURN(error); } @@ -1053,6 +1096,108 @@ ha_check_and_coalesce_trx_read_only(THD return rw_ha_count; } +/* + Atomically enqueue a THD at the head of the queue of threads waiting to + group commit, and return the previous head of the queue. +*/ +static THD * +enqueue_atomic(THD *thd) +{ + my_atomic_rwlock_wrlock(&LOCK_group_commit_queue); + thd->next_commit_ordered= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&thd->next_commit_ordered), + thd)) + ; + my_atomic_rwlock_wrunlock(&LOCK_group_commit_queue); + return thd->next_commit_ordered; +} + +static THD * +atomic_grab_reverse_queue() +{ + my_atomic_rwlock_wrlock(&LOCK_group_commit_queue); + THD *queue= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&queue), + NULL)) + ; + my_atomic_rwlock_wrunlock(&LOCK_group_commit_queue); + + /* + Since we enqueue at the head, the queue is actually in reverse order. + So reverse it back into correct commit order before returning. + */ + THD *prev= NULL; + while (queue) + { + THD *next= queue->next_commit_ordered; + queue->next_commit_ordered= prev; + prev= queue; + queue= next; + } + + return prev; +} + +static void +call_commit_ordered(Ha_trx_info *ha_info, THD *thd, bool all) +{ + for (; ha_info; ha_info= ha_info->next()) + { + handlerton *ht= ha_info->ht(); + if (!ht->commit_ordered) + continue; + ht->commit_ordered(ht, thd, all); + } +} + +static void +group_commit_wait_for_wakeup(THD *thd) +{ + pthread_mutex_lock(&thd->LOCK_commit_ordered); + while (!thd->group_commit_ready) + pthread_cond_wait(&thd->COND_commit_ordered, + &thd->LOCK_commit_ordered); + pthread_mutex_unlock(&thd->LOCK_commit_ordered); +} + +static void +group_commit_wakeup_other(THD *other_thd) +{ + pthread_mutex_lock(&other_thd->LOCK_commit_ordered); + other_thd->group_commit_ready= TRUE; + pthread_cond_signal(&other_thd->COND_commit_ordered); + pthread_mutex_unlock(&other_thd->LOCK_commit_ordered); +} + +static bool group_commit_queue_busy= 0; + +static void +group_commit_mark_queue_idle() +{ + pthread_mutex_lock(&LOCK_group_commit); + group_commit_queue_busy= FALSE; + pthread_cond_signal(&COND_group_commit); + pthread_mutex_unlock(&LOCK_group_commit); +} + +static void +group_commit_mark_queue_busy() +{ + safe_mutex_assert_owner(&LOCK_group_commit); + group_commit_queue_busy= TRUE; +} + +static void +group_commit_wait_queue_idle() +{ + /* Wait for any existing queue run to finish. */ + safe_mutex_assert_owner(&LOCK_group_commit); + while (group_commit_queue_busy) + pthread_cond_wait(&COND_group_commit, &LOCK_group_commit); +} + /** @retval @@ -1070,7 +1215,7 @@ ha_check_and_coalesce_trx_read_only(THD */ int ha_commit_trans(THD *thd, bool all) { - int error= 0, cookie= 0; + int error= 0; /* 'all' means that this is either an explicit commit issued by user, or an implicit commit issued by a DDL. @@ -1085,7 +1230,10 @@ int ha_commit_trans(THD *thd, bool all) */ bool is_real_trans= all || thd->transaction.all.ha_list == 0; Ha_trx_info *ha_info= trans->ha_list; - my_xid xid= thd->transaction.xid_state.xid.get_my_xid(); + bool need_prepare_ordered, need_commit_ordered; + bool need_enqueue; + bool is_group_commit_leader; + my_xid xid; DBUG_ENTER("ha_commit_trans"); /* @@ -1118,85 +1266,277 @@ int ha_commit_trans(THD *thd, bool all) DBUG_RETURN(2); } #ifdef USING_TRANSACTIONS - if (ha_info) + if (!ha_info) { - uint rw_ha_count; - bool rw_trans; + /* Free resources and perform other cleanup even for 'empty' transactions. */ + if (is_real_trans) + thd->transaction.cleanup(); + DBUG_RETURN(0); + } - DBUG_EXECUTE_IF("crash_commit_before", abort();); + DBUG_EXECUTE_IF("crash_commit_before", abort();); - /* Close all cursors that can not survive COMMIT */ - if (is_real_trans) /* not a statement commit */ - thd->stmt_map.close_transient_cursors(); + /* Close all cursors that can not survive COMMIT */ + if (is_real_trans) /* not a statement commit */ + thd->stmt_map.close_transient_cursors(); - rw_ha_count= ha_check_and_coalesce_trx_read_only(thd, ha_info, all); - /* rw_trans is TRUE when we in a transaction changing data */ - rw_trans= is_real_trans && (rw_ha_count > 0); + uint rw_ha_count= ha_check_and_coalesce_trx_read_only(thd, ha_info, all); + /* rw_trans is TRUE when we in a transaction changing data */ + bool rw_trans= is_real_trans && (rw_ha_count > 0); - if (rw_trans && - wait_if_global_read_lock(thd, 0, 0)) - { - ha_rollback_trans(thd, all); - DBUG_RETURN(1); - } + if (rw_trans && + wait_if_global_read_lock(thd, 0, 0)) + { + ha_rollback_trans(thd, all); + DBUG_RETURN(1); + } + + if (rw_trans && + opt_readonly && + !(thd->security_ctx->master_access & SUPER_ACL) && + !thd->slave_thread) + { + my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--read-only"); + goto err; + } - if (rw_trans && - opt_readonly && - !(thd->security_ctx->master_access & SUPER_ACL) && - !thd->slave_thread) + if (trans->no_2pc || (rw_ha_count <= 1)) + { + error= ha_commit_one_phase(thd, all); + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + } + + need_prepare_ordered= FALSE; + need_commit_ordered= FALSE; + xid= thd->transaction.xid_state.xid.get_my_xid(); + + for (Ha_trx_info *hi= ha_info; hi; hi= hi->next()) + { + int err; + handlerton *ht= hi->ht(); + /* + Do not call two-phase commit if this particular + transaction is read-only. This allows for simpler + implementation in engines that are always read-only. + */ + if (! hi->is_trx_read_write()) + continue; + /* + Sic: we know that prepare() is not NULL since otherwise + trans->no_2pc would have been set. + */ + if ((err= ht->prepare(ht, thd, all))) + my_error(ER_ERROR_DURING_COMMIT, MYF(0), err); + status_var_increment(thd->status_var.ha_prepare_count); + + if (err) + goto err; + + if (ht->prepare_ordered) + need_prepare_ordered= TRUE; + if (ht->commit_ordered) + need_commit_ordered= TRUE; + } + DBUG_EXECUTE_IF("crash_commit_after_prepare", DBUG_ABORT();); + + if (!is_real_trans) + { + error= commit_one_phase_2(thd, all, FALSE); + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + } + + /* + We can optimise away some of the thread synchronisation that may not be + needed. + + If need_prepare_ordered, then we need to take LOCK_prepare_ordered. + + If (xid && use_group_log_xid), then we need to enqueue (and this must + be done under LOCK_prepare_ordered if we take that lock). + + Similarly, if (need_prepare_ordered && need_commit_ordered), then we + need to enqueue under the LOCK_prepare_ordered. + + If (xid && use_group_log_xid), then we need to take LOCK_group_commit. + + If need_commit_ordered, then we need to take LOCK_commit_ordered. + + Cases not covered by above can be skipped to optimise things a bit. + */ + need_enqueue= (xid && tc_log->use_group_log_xid) || + (need_prepare_ordered && need_commit_ordered); + + thd->group_commit_ready= FALSE; + thd->group_commit_all= all; + if (need_prepare_ordered) + { + pthread_mutex_lock(&LOCK_prepare_ordered); + + for (Ha_trx_info *hi= ha_info; hi; hi= hi->next()) { - my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--read-only"); - ha_rollback_trans(thd, all); - error= 1; - goto end; + int err; + handlerton *ht= hi->ht(); + if (! hi->is_trx_read_write()) + continue; + if (ht->prepare_ordered && (err= ht->prepare_ordered(ht, thd, all))) + { + my_error(ER_ERROR_DURING_COMMIT, MYF(0), err); + pthread_mutex_unlock(&LOCK_prepare_ordered); + goto err; + } } + } + if (need_enqueue) + { + THD *previous_queue= enqueue_atomic(thd); + is_group_commit_leader= (previous_queue == NULL); + } + if (need_prepare_ordered) + pthread_mutex_unlock(&LOCK_prepare_ordered); - if (!trans->no_2pc && (rw_ha_count > 1)) + int cookie; + if (tc_log->use_group_log_xid) + { + if (is_group_commit_leader) { - for (; ha_info && !error; ha_info= ha_info->next()) + pthread_mutex_lock(&LOCK_group_commit); + group_commit_wait_queue_idle(); + + THD *queue= atomic_grab_reverse_queue(); + /* The first in the queue is the leader. */ + DBUG_ASSERT(queue == thd); + + /* + This will set individual error codes in each thd->xid_error, as + well as set thd->xid_cookie for later unlog() call. + */ + tc_log->group_log_xid(queue); + + /* + Call commit_ordered methods for all transactions in the queue + (that did not get an error in group_log_xid()). + + We do this under an additional global LOCK_commit_ordered; this is + so that transactions that do not need 2-phase commit do not have + to wait for the potentially long duration of LOCK_group_commit. + */ + if (need_commit_ordered) { - int err; - handlerton *ht= ha_info->ht(); - /* - Do not call two-phase commit if this particular - transaction is read-only. This allows for simpler - implementation in engines that are always read-only. - */ - if (! ha_info->is_trx_read_write()) - continue; - /* - Sic: we know that prepare() is not NULL since otherwise - trans->no_2pc would have been set. - */ - if ((err= ht->prepare(ht, thd, all))) + pthread_mutex_lock(&LOCK_commit_ordered); + for (THD *thd2= queue; thd2 != NULL; thd2= thd2->next_commit_ordered) { - my_error(ER_ERROR_DURING_COMMIT, MYF(0), err); - error= 1; + if (!queue->xid_error) + call_commit_ordered(ha_info, thd2, thd2->group_commit_all); } - status_var_increment(thd->status_var.ha_prepare_count); + pthread_mutex_unlock(&LOCK_commit_ordered); } - DBUG_EXECUTE_IF("crash_commit_after_prepare", DBUG_ABORT();); - if (error || (is_real_trans && xid && - (error= !(cookie= tc_log->log_xid(thd, xid))))) - { - ha_rollback_trans(thd, all); - error= 1; - goto end; - } - DBUG_EXECUTE_IF("crash_commit_after_log", DBUG_ABORT();); + pthread_mutex_unlock(&LOCK_group_commit); + + /* Wake up everyone except ourself. */ + while ((queue= queue->next_commit_ordered) != NULL) + group_commit_wakeup_other(queue); + } + else + { + /* If not leader, just wait until leader wakes us up. */ + group_commit_wait_for_wakeup(thd); + } + + /* + Now that we're back in our own thread context, do any delayed error + reporting. + */ + if (thd->xid_error) + { + tc_log->xid_delayed_error(thd); + goto err; + } + cookie= thd->xid_cookie; + /* The cookie must be non-zero in the non-error case. */ + DBUG_ASSERT(cookie); + } + else + { + if (xid) + cookie= tc_log->log_xid(thd, xid); + + if (!need_enqueue) + { + error= commit_one_phase_2(thd, all, TRUE); + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + } + + /* + We only get here to do correctly sequenced prepare_ordered and + commit_ordered() calls. + + In this case, we need to wait for the previous in queue to finish + commit_ordered before us to get the correct sequence. + */ + DBUG_ASSERT(need_prepare_ordered && need_commit_ordered); + + if (is_group_commit_leader) + { + pthread_mutex_lock(&LOCK_group_commit); + group_commit_wait_queue_idle(); + THD *queue= atomic_grab_reverse_queue(); + /* + Mark the queue busy while we bounce it from one thread to the + next. + */ + group_commit_mark_queue_busy(); + pthread_mutex_unlock(&LOCK_group_commit); + + /* The first in the queue is the leader. */ + DBUG_ASSERT(queue == thd); } - error=ha_commit_one_phase(thd, all) ? (cookie ? 2 : 1) : 0; - DBUG_EXECUTE_IF("crash_commit_before_unlog", DBUG_ABORT();); + else + { + /* If not leader, just wait until previous thread wakes us up. */ + group_commit_wait_for_wakeup(thd); + } + + /* Only run commit_ordered() if log_xid was successful. */ if (cookie) - tc_log->unlog(cookie, xid); - DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); -end: - if (rw_trans) - start_waiting_global_read_lock(thd); + { + pthread_mutex_lock(&LOCK_commit_ordered); + call_commit_ordered(ha_info, thd, all); + pthread_mutex_unlock(&LOCK_commit_ordered); + } + + THD *next= thd->next_commit_ordered; + if (next) + group_commit_wakeup_other(next); + else + group_commit_mark_queue_idle(); + + if (!cookie) + goto err; } - /* Free resources and perform other cleanup even for 'empty' transactions. */ - else if (is_real_trans) - thd->transaction.cleanup(); + + DBUG_EXECUTE_IF("crash_commit_after_log", DBUG_ABORT();); + + error= commit_one_phase_2(thd, all, FALSE) ? 2 : 0; + + DBUG_EXECUTE_IF("crash_commit_before_unlog", DBUG_ABORT();); + DBUG_ASSERT(cookie); + tc_log->unlog(cookie, xid); + + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + + /* Come here if error and we need to rollback. */ +err: + if (!error) + error= 1; + ha_rollback_trans(thd, all); + +end: + if (rw_trans) + start_waiting_global_read_lock(thd); #endif /* USING_TRANSACTIONS */ DBUG_RETURN(error); } @@ -1207,6 +1547,17 @@ end: */ int ha_commit_one_phase(THD *thd, bool all) { + /* + When we come here, we did not call handler commit_ordered() methods in + ha_commit_trans() 2-phase commit, so we pass TRUE to do it in + commit_one_phase_2(). + */ + return commit_one_phase_2(thd, all, TRUE); +} + +static int +commit_one_phase_2(THD *thd, bool all, bool do_commit_ordered) +{ int error=0; THD_TRANS *trans=all ? &thd->transaction.all : &thd->transaction.stmt; /* @@ -1218,10 +1569,40 @@ int ha_commit_one_phase(THD *thd, bool a */ bool is_real_trans=all || thd->transaction.all.ha_list == 0; Ha_trx_info *ha_info= trans->ha_list, *ha_info_next; - DBUG_ENTER("ha_commit_one_phase"); + DBUG_ENTER("commit_one_phase_2"); #ifdef USING_TRANSACTIONS if (ha_info) { + if (is_real_trans && do_commit_ordered) + { + /* + If we did not do it already, call any commit_ordered() method. + + Even though we do not need to keep any ordering with other threads + (as there is no prepare or log_xid for this commit), we still need to + do this under the LOCK_commit_ordered mutex to not run in parallel + with other commit_ordered calls. + */ + + bool locked= FALSE; + + for (Ha_trx_info *hi= ha_info; hi; hi= hi->next()) + { + handlerton *ht= hi->ht(); + if (ht->commit_ordered) + { + if (!locked) + { + pthread_mutex_lock(&LOCK_commit_ordered); + locked= 1; + } + ht->commit_ordered(ht, thd, all); + } + } + if (locked) + pthread_mutex_unlock(&LOCK_commit_ordered); + } + for (; ha_info; ha_info= ha_info_next) { int err; === modified file 'sql/handler.h' --- a/sql/handler.h 2010-01-14 16:51:00 +0000 +++ b/sql/handler.h 2010-05-26 08:16:18 +0000 @@ -656,9 +656,81 @@ struct handlerton NOTE 'all' is also false in auto-commit mode where 'end of statement' and 'real commit' mean the same event. */ - int (*commit)(handlerton *hton, THD *thd, bool all); + int (*commit)(handlerton *hton, THD *thd, bool all); + /* + The commit_ordered() method is called prior to the commit() method, after + the transaction manager has decided to commit (not rollback) the + transaction. + + The calls to commit_ordered() in multiple parallel transactions is + guaranteed to happen in the same order in every participating + handler. This can be used to ensure the same commit order among multiple + handlers (eg. in table handler and binlog). So if transaction T1 calls + into commit_ordered() of handler A before T2, then T1 will also call + commit_ordered() of handler B before T2. + + The intension is that commit_ordered() should do the minimal amount of + work that needs to happen in consistent commit order among handlers. To + preserve ordering, calls need to be serialised on a global mutex, so + doing any time-consuming or blocking operations in commit_ordered() will + limit scalability. + + Handlers can rely on commit_ordered() calls being serialised (no two + calls can run in parallel, so no extra locking on the handler part is + required to ensure this). + + Note that commit_ordered() can be called from a different thread than the + one handling the transaction! So it can not do anything that depends on + thread local storage, in particular it can not call my_error() and + friends (instead it can store the error code and delay the call to + my_error() to the commit() method). + + Similarly, since commit_ordered() returns void, any return error code + must be saved and returned from the commit() method instead. + + commit_ordered() is called only when actually committing a transaction + (autocommit or not), not when ending a statement in the middle of a + transaction. + + The commit_ordered method is optional, and can be left unset if not + needed in a particular handler. + */ + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); int (*rollback)(handlerton *hton, THD *thd, bool all); int (*prepare)(handlerton *hton, THD *thd, bool all); + /* + The prepare_ordered method is optional. If set, it will be called after + successful prepare() in all handlers participating in 2-phase commit. + + The calls to prepare_ordered() among multiple parallel transactions are + ordered consistently with calls to commit_ordered(). This means that + calls to prepare_ordered() effectively define the commit order, and that + each handler will see the same sequence of transactions calling into + prepare_ordered() and commit_ordered(). + + Thus, prepare_ordered() can be used to define commit order for handlers + that need to do this in the prepare step (like binlog). It can also be + used to release transactions locks early in an order consistent with the + order transactions will be eventually committed. + + Like commit_ordered(), prepare_ordered() calls are serialised to maintain + ordering, so the intension is that they should execute fast, with only + the minimal amount of work needed to define commit order. Handlers can + rely on this serialisation, and do not need to do any extra locking to + avoid two prepare_ordered() calls running in parallel. + + Unlike commit_ordered(), prepare_ordered() _is_ guaranteed to be called + in the context of the thread handling the rest of the transaction. + + Note that for user-level XA SQL commands, no consistent ordering among + prepare_ordered() and commit_ordered() is guaranteed (as that would + require blocking all other commits for an indefinite time). + + prepare_ordered() is called only when actually committing a transaction + (autocommit or not), not when ending a statement in the middle of a + transaction. + */ + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); int (*recover)(handlerton *hton, XID *xid_list, uint len); int (*commit_by_xid)(handlerton *hton, XID *xid); int (*rollback_by_xid)(handlerton *hton, XID *xid); === modified file 'sql/log.cc' --- a/sql/log.cc 2010-04-06 22:47:08 +0000 +++ b/sql/log.cc 2010-05-26 08:16:18 +0000 @@ -154,9 +154,12 @@ class binlog_trx_data { public: binlog_trx_data() : at_least_one_stmt_committed(0), incident(FALSE), m_pending(0), - before_stmt_pos(MY_OFF_T_UNDEF) + before_stmt_pos(MY_OFF_T_UNDEF), using_xa(0) { trans_log.end_of_file= max_binlog_cache_size; + (void) my_pthread_mutex_init(&LOCK_group_commit, MY_MUTEX_INIT_SLOW, + "LOCK_group_commit", MYF(0)); + (void) pthread_cond_init(&COND_group_commit, 0); } ~binlog_trx_data() @@ -208,11 +211,12 @@ public: completely. */ void reset() { - if (!empty()) + if (trans_log.type != WRITE_CACHE || !empty()) truncate(0); before_stmt_pos= MY_OFF_T_UNDEF; incident= FALSE; trans_log.end_of_file= max_binlog_cache_size; + using_xa= FALSE; DBUG_ASSERT(empty()); } @@ -257,6 +261,41 @@ public: Binlog position before the start of the current statement. */ my_off_t before_stmt_pos; + + /* 0 or error when writing to binlog; set during group commit. */ + int error; + /* If error != 0, value of errno (for my_error() reporting). */ + int commit_errno; + /* Link for queueing transactions up for group commit to binlog. */ + binlog_trx_data *next; + /* + Flag set true when group commit for this transaction is finished; used + with pthread_cond_wait() to wait until commit is done. + This flag is protected by LOCK_group_commit. + */ + bool done; + /* + Flag set if this transaction is the group commit leader that will handle + the actual writing to the binlog. + This flag is protected by LOCK_group_commit. + */ + bool group_commit_leader; + /* + Flag set true if this transaction is committed with log_xid() as part of + XA, false if not. + */ + bool using_xa; + /* + Extra events (BEGIN, COMMIT/ROLLBACK/XID, and possibly INCIDENT) to be + written during group commit. The incident_event is only valid if + has_incident() is true. + */ + Log_event *begin_event; + Log_event *end_event; + Log_event *incident_event; + /* Mutex and condition for wakeup after group commit. */ + pthread_mutex_t LOCK_group_commit; + pthread_cond_t COND_group_commit; }; handlerton *binlog_hton; @@ -1391,117 +1430,188 @@ static int binlog_close_connection(handl return 0; } +/* Helper functions for binlog_flush_trx_cache(). */ +static int +binlog_flush_trx_cache_prepare(THD *thd) +{ + if (thd->binlog_flush_pending_rows_event(TRUE)) + return 1; + return 0; +} + +static void +binlog_flush_trx_cache_finish(THD *thd, binlog_trx_data *trx_data) +{ + IO_CACHE *trans_log= &trx_data->trans_log; + + trx_data->reset(); + + statistic_increment(binlog_cache_use, &LOCK_status); + if (trans_log->disk_writes != 0) + { + statistic_increment(binlog_cache_disk_use, &LOCK_status); + trans_log->disk_writes= 0; + } +} + /* - End a transaction. + End a transaction, writing events to the binary log. SYNOPSIS - binlog_end_trans() + binlog_flush_trx_cache() thd The thread whose transaction should be ended trx_data Pointer to the transaction data to use - end_ev The end event to use, or NULL - all True if the entire transaction should be ended, false if - only the statement transaction should be ended. + end_ev The end event to use (COMMIT, ROLLBACK, or commit XID) DESCRIPTION End the currently open transaction. The transaction can be either - a real transaction (if 'all' is true) or a statement transaction - (if 'all' is false). + a real transaction or a statement transaction. - If 'end_ev' is NULL, the transaction is a rollback of only - transactional tables, so the transaction cache will be truncated - to either just before the last opened statement transaction (if - 'all' is false), or reset completely (if 'all' is true). + This can be to commit a transaction, with a COMMIT query event or an XA + commit XID event. But it can also be to rollback a transaction with a + ROLLBACK query event, used for rolling back transactions which also + contain updates to non-transactional tables. */ static int -binlog_end_trans(THD *thd, binlog_trx_data *trx_data, - Log_event *end_ev, bool all) +binlog_flush_trx_cache(THD *thd, binlog_trx_data *trx_data, + Log_event *end_ev) { - DBUG_ENTER("binlog_end_trans"); - int error=0; - IO_CACHE *trans_log= &trx_data->trans_log; - DBUG_PRINT("enter", ("transaction: %s end_ev: 0x%lx", - all ? "all" : "stmt", (long) end_ev)); + DBUG_ENTER("binlog_flush_trx_cache"); DBUG_PRINT("info", ("thd->options={ %s%s}", FLAGSTR(thd->options, OPTION_NOT_AUTOCOMMIT), FLAGSTR(thd->options, OPTION_BEGIN))); + if (binlog_flush_trx_cache_prepare(thd)) + DBUG_RETURN(1); + /* - NULL denotes ROLLBACK with nothing to replicate: i.e., rollback of - only transactional tables. If the transaction contain changes to - any non-transactiona tables, we need write the transaction and log - a ROLLBACK last. - */ - if (end_ev != NULL) - { - if (thd->binlog_flush_pending_rows_event(TRUE)) - DBUG_RETURN(1); - /* - Doing a commit or a rollback including non-transactional tables, - i.e., ending a transaction where we might write the transaction - cache to the binary log. - - We can always end the statement when ending a transaction since - transactions are not allowed inside stored functions. If they - were, we would have to ensure that we're not ending a statement - inside a stored function. - */ - error= mysql_bin_log.write(thd, &trx_data->trans_log, end_ev, - trx_data->has_incident()); - trx_data->reset(); + Doing a commit or a rollback including non-transactional tables, + i.e., ending a transaction where we might write the transaction + cache to the binary log. + + We can always end the statement when ending a transaction since + transactions are not allowed inside stored functions. If they + were, we would have to ensure that we're not ending a statement + inside a stored function. + */ + int error= mysql_bin_log.write_transaction_to_binlog(thd, trx_data, end_ev); - /* - We need to step the table map version after writing the - transaction cache to disk. - */ - mysql_bin_log.update_table_map_version(); - statistic_increment(binlog_cache_use, &LOCK_status); - if (trans_log->disk_writes != 0) - { - statistic_increment(binlog_cache_disk_use, &LOCK_status); - trans_log->disk_writes= 0; - } - } - else - { - /* - If rolling back an entire transaction or a single statement not - inside a transaction, we reset the transaction cache. + binlog_flush_trx_cache_finish(thd, trx_data); - If rolling back a statement in a transaction, we truncate the - transaction cache to remove the statement. - */ - thd->binlog_remove_pending_rows_event(TRUE); - if (all || !(thd->options & (OPTION_BEGIN | OPTION_NOT_AUTOCOMMIT))) - { - if (trx_data->has_incident()) - error= mysql_bin_log.write_incident(thd, TRUE); - trx_data->reset(); - } - else // ...statement - trx_data->truncate(trx_data->before_stmt_pos); + DBUG_ASSERT(thd->binlog_get_pending_rows_event() == NULL); + DBUG_RETURN(error); +} - /* - We need to step the table map version on a rollback to ensure - that a new table map event is generated instead of the one that - was written to the thrown-away transaction cache. - */ - mysql_bin_log.update_table_map_version(); +/* + Discard a transaction, ie. ROLLBACK with only transactional table updates. + + SYNOPSIS + binlog_truncate_trx_cache() + + thd The thread whose transaction should be ended + trx_data Pointer to the transaction data to use + all True if the entire transaction should be ended, false if + only the statement transaction should be ended. + + DESCRIPTION + + Rollback (and end) a transaction that only modifies transactional + tables. The transaction can be either a real transaction (if 'all' is + true) or a statement transaction (if 'all' is false). + + The transaction cache will be truncated to either just before the last + opened statement transaction (if 'all' is false), or reset completely (if + 'all' is true). + */ +static int +binlog_truncate_trx_cache(THD *thd, binlog_trx_data *trx_data, bool all) +{ + DBUG_ENTER("binlog_truncate_trx_cache"); + int error= 0; + DBUG_PRINT("enter", ("transaction: %s", all ? "all" : "stmt")); + DBUG_PRINT("info", ("thd->options={ %s%s}", + FLAGSTR(thd->options, OPTION_NOT_AUTOCOMMIT), + FLAGSTR(thd->options, OPTION_BEGIN))); + + /* + ROLLBACK with nothing to replicate: i.e., rollback of only transactional + tables. + */ + + /* + If rolling back an entire transaction or a single statement not + inside a transaction, we reset the transaction cache. + + If rolling back a statement in a transaction, we truncate the + transaction cache to remove the statement. + */ + thd->binlog_remove_pending_rows_event(TRUE); + if (all || !(thd->options & (OPTION_BEGIN | OPTION_NOT_AUTOCOMMIT))) + { + if (trx_data->has_incident()) + error= mysql_bin_log.write_incident(thd); + trx_data->reset(); } + else // ...statement + trx_data->truncate(trx_data->before_stmt_pos); DBUG_ASSERT(thd->binlog_get_pending_rows_event() == NULL); DBUG_RETURN(error); } +static LEX_STRING const write_error_msg= + { C_STRING_WITH_LEN("error writing to the binary log") }; + static int binlog_prepare(handlerton *hton, THD *thd, bool all) { /* - do nothing. - just pretend we can do 2pc, so that MySQL won't - switch to 1pc. - real work will be done in MYSQL_BIN_LOG::log_xid() + If this prepare is for a single statement in the middle of a transactions, + not the actual transaction commit, then we do nothing. The real work is + only done later, in the prepare for making persistent changes. */ + if (!all && (thd->options & (OPTION_BEGIN | OPTION_NOT_AUTOCOMMIT))) + return 0; + + binlog_trx_data *trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + + trx_data->using_xa= TRUE; + + if (binlog_flush_trx_cache_prepare(thd)) + return 1; + + my_xid xid= thd->transaction.xid_state.xid.get_my_xid(); + if (!xid) + { + /* Skip logging this transaction, marked by setting end_event to NULL. */ + trx_data->end_event= NULL; + return 0; + } + + /* + Allocate the extra events that will be logged to the binlog in binlog group + commit. Use placement new to allocate them on the THD memroot, as they need + to remain live until log_xid() returns. + */ + size_t needed_size= sizeof(Query_log_event) + sizeof(Xid_log_event); + if (trx_data->has_incident()) + needed_size+= sizeof(Incident_log_event); + uchar *mem= (uchar *)thd->alloc(needed_size); + if (!mem) + return 1; + + trx_data->begin_event= new ((void *)mem) + Query_log_event(thd, STRING_WITH_LEN("BEGIN"), TRUE, TRUE, 0); + mem+= sizeof(Query_log_event); + + trx_data->end_event= new ((void *)mem) Xid_log_event(thd, xid); + + if (trx_data->has_incident()) + trx_data->incident_event= new ((void *)(mem + sizeof(Xid_log_event))) + Incident_log_event(thd, INCIDENT_LOST_EVENTS, write_error_msg); + return 0; } @@ -1525,11 +1635,11 @@ static int binlog_commit(handlerton *hto binlog_trx_data *const trx_data= (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); - if (trx_data->empty()) + if (trx_data->using_xa) { // we're here because trans_log was flushed in MYSQL_BIN_LOG::log_xid() - trx_data->reset(); - DBUG_RETURN(0); + binlog_flush_trx_cache_finish(thd, trx_data); + DBUG_RETURN(error); } /* @@ -1556,8 +1666,8 @@ static int binlog_commit(handlerton *hto !stmt_has_updated_trans_table(thd) && thd->transaction.stmt.modified_non_trans_table)) { - Query_log_event qev(thd, STRING_WITH_LEN("COMMIT"), TRUE, TRUE, 0); - error= binlog_end_trans(thd, trx_data, &qev, all); + Query_log_event end_ev(thd, STRING_WITH_LEN("COMMIT"), TRUE, TRUE, 0); + error= binlog_flush_trx_cache(thd, trx_data, &end_ev); } trx_data->at_least_one_stmt_committed = my_b_tell(&trx_data->trans_log) > 0; @@ -1621,7 +1731,7 @@ static int binlog_rollback(handlerton *h (thd->options & OPTION_KEEP_LOG)) && mysql_bin_log.check_write_error(thd)) trx_data->set_incident(); - error= binlog_end_trans(thd, trx_data, 0, all); + error= binlog_truncate_trx_cache(thd, trx_data, all); } else { @@ -1641,8 +1751,8 @@ static int binlog_rollback(handlerton *h thd->current_stmt_binlog_row_based) || ((thd->options & OPTION_KEEP_LOG))) { - Query_log_event qev(thd, STRING_WITH_LEN("ROLLBACK"), TRUE, TRUE, 0); - error= binlog_end_trans(thd, trx_data, &qev, all); + Query_log_event end_ev(thd, STRING_WITH_LEN("ROLLBACK"), TRUE, TRUE, 0); + error= binlog_flush_trx_cache(thd, trx_data, &end_ev); } /* Otherwise, we simply truncate the cache as there is no change on @@ -1650,7 +1760,7 @@ static int binlog_rollback(handlerton *h */ else if ((all && !thd->transaction.all.modified_non_trans_table) || (!all && !thd->transaction.stmt.modified_non_trans_table)) - error= binlog_end_trans(thd, trx_data, 0, all); + error= binlog_truncate_trx_cache(thd, trx_data, all); } if (!all) trx_data->before_stmt_pos = MY_OFF_T_UNDEF; // part of the stmt rollback @@ -2464,7 +2574,7 @@ const char *MYSQL_LOG::generate_name(con MYSQL_BIN_LOG::MYSQL_BIN_LOG() :bytes_written(0), prepared_xids(0), file_id(1), open_count(1), - need_start_event(TRUE), m_table_map_version(0), + need_start_event(TRUE), is_relay_log(0), description_event_for_exec(0), description_event_for_queue(0) { @@ -2477,6 +2587,7 @@ MYSQL_BIN_LOG::MYSQL_BIN_LOG() index_file_name[0] = 0; bzero((char*) &index_file, sizeof(index_file)); bzero((char*) &purge_index_file, sizeof(purge_index_file)); + use_group_log_xid= TRUE; } /* this is called only once */ @@ -2492,6 +2603,7 @@ void MYSQL_BIN_LOG::cleanup() delete description_event_for_exec; (void) pthread_mutex_destroy(&LOCK_log); (void) pthread_mutex_destroy(&LOCK_index); + (void) pthread_mutex_destroy(&LOCK_queue); (void) pthread_cond_destroy(&update_cond); } DBUG_VOID_RETURN; @@ -2520,6 +2632,8 @@ void MYSQL_BIN_LOG::init_pthread_objects */ (void) my_pthread_mutex_init(&LOCK_index, MY_MUTEX_INIT_SLOW, "LOCK_index", MYF_NO_DEADLOCK_DETECTION); + (void) my_pthread_mutex_init(&LOCK_queue, MY_MUTEX_INIT_FAST, "LOCK_queue", + MYF(0)); (void) pthread_cond_init(&update_cond, 0); } @@ -4113,7 +4227,6 @@ int THD::binlog_write_table_map(TABLE *t DBUG_RETURN(error); binlog_table_maps++; - table->s->table_map_version= mysql_bin_log.table_map_version(); DBUG_RETURN(0); } @@ -4194,64 +4307,41 @@ MYSQL_BIN_LOG::flush_and_set_pending_row if (Rows_log_event* pending= trx_data->pending()) { - IO_CACHE *file= &log_file; - /* Decide if we should write to the log file directly or to the transaction log. */ if (pending->get_cache_stmt() || my_b_tell(&trx_data->trans_log)) - file= &trx_data->trans_log; - - /* - If we are writing to the log file directly, we could avoid - locking the log. This does not work since we need to step the - m_table_map_version below, and that change has to be protected - by the LOCK_log mutex. - */ - pthread_mutex_lock(&LOCK_log); - - /* - Write pending event to log file or transaction cache - */ - if (pending->write(file)) { - pthread_mutex_unlock(&LOCK_log); - set_write_error(thd); - DBUG_RETURN(1); + /* Write to transaction log/cache. */ + if (pending->write(&trx_data->trans_log)) + { + set_write_error(thd); + DBUG_RETURN(1); + } } - - /* - We step the table map version if we are writing an event - representing the end of a statement. We do this regardless of - wheather we write to the transaction cache or to directly to the - file. - - In an ideal world, we could avoid stepping the table map version - if we were writing to a transaction cache, since we could then - reuse the table map that was written earlier in the transaction - cache. This does not work since STMT_END_F implies closing all - table mappings on the slave side. - - TODO: Find a solution so that table maps does not have to be - written several times within a transaction. - */ - if (pending->get_flags(Rows_log_event::STMT_END_F)) - ++m_table_map_version; - - delete pending; - - if (file == &log_file) + else { + /* Write directly to log file. */ + pthread_mutex_lock(&LOCK_log); + if (pending->write(&log_file)) + { + pthread_mutex_unlock(&LOCK_log); + set_write_error(thd); + DBUG_RETURN(1); + } + error= flush_and_sync(); if (!error) { signal_update(); rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); } + + pthread_mutex_unlock(&LOCK_log); } - pthread_mutex_unlock(&LOCK_log); + delete pending; } thd->binlog_set_pending_rows_event(event); @@ -4450,9 +4540,6 @@ err: set_write_error(thd); } - if (event_info->flags & LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F) - ++m_table_map_version; - pthread_mutex_unlock(&LOCK_log); DBUG_RETURN(error); } @@ -4575,18 +4662,14 @@ uint MYSQL_BIN_LOG::next_file_id() SYNOPSIS write_cache() cache Cache to write to the binary log - lock_log True if the LOCK_log mutex should be aquired, false otherwise - sync_log True if the log should be flushed and sync:ed DESCRIPTION Write the contents of the cache to the binary log. The cache will be reset as a READ_CACHE to be able to read the contents from it. */ -int MYSQL_BIN_LOG::write_cache(IO_CACHE *cache, bool lock_log, bool sync_log) +int MYSQL_BIN_LOG::write_cache(IO_CACHE *cache) { - Mutex_sentry sentry(lock_log ? &LOCK_log : NULL); - if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0)) return ER_ERROR_ON_WRITE; uint length= my_b_bytes_in_cache(cache), group, carry, hdr_offs; @@ -4697,6 +4780,7 @@ int MYSQL_BIN_LOG::write_cache(IO_CACHE } /* Write data to the binary log file */ + DBUG_EXECUTE_IF("fail_binlog_write_1", return ER_ERROR_ON_WRITE;); if (my_b_write(&log_file, cache->read_pos, length)) return ER_ERROR_ON_WRITE; cache->read_pos=cache->read_end; // Mark buffer used up @@ -4704,9 +4788,6 @@ int MYSQL_BIN_LOG::write_cache(IO_CACHE DBUG_ASSERT(carry == 0); - if (sync_log) - flush_and_sync(); - return 0; // All OK } @@ -4739,26 +4820,22 @@ int query_error_code(THD *thd, bool not_ return error; } -bool MYSQL_BIN_LOG::write_incident(THD *thd, bool lock) +bool MYSQL_BIN_LOG::write_incident(THD *thd) { uint error= 0; DBUG_ENTER("MYSQL_BIN_LOG::write_incident"); - LEX_STRING const write_error_msg= - { C_STRING_WITH_LEN("error writing to the binary log") }; Incident incident= INCIDENT_LOST_EVENTS; Incident_log_event ev(thd, incident, write_error_msg); - if (lock) - pthread_mutex_lock(&LOCK_log); + + pthread_mutex_lock(&LOCK_log); error= ev.write(&log_file); - if (lock) + if (!error && !(error= flush_and_sync())) { - if (!error && !(error= flush_and_sync())) - { - signal_update(); - rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); - } - pthread_mutex_unlock(&LOCK_log); + signal_update(); + rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); } + pthread_mutex_unlock(&LOCK_log); + DBUG_RETURN(error); } @@ -4786,103 +4863,364 @@ bool MYSQL_BIN_LOG::write_incident(THD * 'cache' needs to be reinitialized after this functions returns. */ -bool MYSQL_BIN_LOG::write(THD *thd, IO_CACHE *cache, Log_event *commit_event, - bool incident) +bool +MYSQL_BIN_LOG::write_transaction_to_binlog(THD *thd, binlog_trx_data *trx_data, + Log_event *end_ev) { - DBUG_ENTER("MYSQL_BIN_LOG::write(THD *, IO_CACHE *, Log_event *)"); + DBUG_ENTER("MYSQL_BIN_LOG::write_transaction_to_binlog"); + + /* + Create the necessary events here, where we have the correct THD (and + thread context). + + Due to group commit the actual writing to binlog may happen in a different + thread. + */ + Query_log_event qinfo(thd, STRING_WITH_LEN("BEGIN"), TRUE, TRUE, 0); + trx_data->begin_event= &qinfo; + trx_data->end_event= end_ev; + if (trx_data->has_incident()) + { + Incident_log_event inc_ev(thd, INCIDENT_LOST_EVENTS, write_error_msg); + trx_data->incident_event= &inc_ev; + DBUG_RETURN(write_transaction_to_binlog_events(trx_data)); + } + else + { + trx_data->incident_event= NULL; + DBUG_RETURN(write_transaction_to_binlog_events(trx_data)); + } +} + +bool +MYSQL_BIN_LOG::write_transaction_to_binlog_events(binlog_trx_data *trx_data) +{ + /* + To facilitate group commit for the binlog, we first queue up ourselves in + the group commit queue. Then the first thread to enter the queue waits for + the LOCK_log mutex, and commits for everyone in the queue once it gets the + lock. Any other threads in the queue just wait for the first one to finish + the commit and wake them up. + */ + + pthread_mutex_lock(&trx_data->LOCK_group_commit); + const binlog_trx_data *orig_queue= atomic_enqueue_trx(trx_data); + + if (orig_queue != NULL) + { + trx_data->group_commit_leader= FALSE; + trx_data->done= FALSE; + trx_group_commit_participant(trx_data); + } + else + { + trx_data->group_commit_leader= TRUE; + pthread_mutex_unlock(&trx_data->LOCK_group_commit); + trx_group_commit_leader(NULL); + } + + return trx_group_commit_finish(trx_data); +} + +/* + Participate as secondary transaction in group commit. + + Another thread is already waiting to obtain the LOCK_log, and should include + this thread in the group commit once the log is obtained. So here we put + ourself in the queue and wait to be signalled that the group commit is done. + + Note that this function must be called with the trs_data->LOCK_group_commit + locked; the mutex will be released before return. +*/ +void +MYSQL_BIN_LOG::trx_group_commit_participant(binlog_trx_data *trx_data) +{ + safe_mutex_assert_owner(&trx_data->LOCK_group_commit); + + /* Wait until trx_data.done == true and woken up by the leader. */ + while (!trx_data->done) + pthread_cond_wait(&trx_data->COND_group_commit, + &trx_data->LOCK_group_commit); + pthread_mutex_unlock(&trx_data->LOCK_group_commit); +} + +bool +MYSQL_BIN_LOG::trx_group_commit_finish(binlog_trx_data *trx_data) +{ + if (trx_data->error) + { + switch (trx_data->error) + { + case ER_ERROR_ON_WRITE: + my_error(ER_ERROR_ON_WRITE, MYF(ME_NOREFRESH), name, trx_data->commit_errno); + break; + case ER_ERROR_ON_READ: + my_error(ER_ERROR_ON_READ, MYF(ME_NOREFRESH), + trx_data->trans_log.file_name, trx_data->commit_errno); + break; + default: + /* + There are not (and should not be) any errors thrown not covered above. + But just in case one is added later without updating the above switch + statement, include a catch-all. + */ + my_printf_error(trx_data->error, + "Error writing transaction to binary log: %d", + MYF(ME_NOREFRESH), trx_data->error); + } + + /* + Since we return error, this transaction XID will not be committed, so + we need to mark it as not needed for recovery (unlog() is not called + for a transaction if log_xid() fails). + */ + if (trx_data->end_event->get_type_code() == XID_EVENT) + mark_xid_done(); + + return 1; + } + + return 0; +} + +/* + Do binlog group commit as the lead thread. + + This must be called when this thread/transaction is queued at the start of + the group_commit_queue. It will wait to obtain the LOCK_log mutex, then group + commit all the transactions in the queue (more may have entered while waiting + for LOCK_log). After commit is done, all other threads in the queue will be + signalled. + + */ +void +MYSQL_BIN_LOG::trx_group_commit_leader(THD *first_thd) +{ + uint xid_count= 0; + uint write_count= 0; + + /* First, put anything from group_log_xid into the queue. */ + binlog_trx_data *full_queue= NULL; + binlog_trx_data **next_ptr= &full_queue; + for (THD *thd= first_thd; thd; thd= thd->next_commit_ordered) + { + binlog_trx_data *const trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + + /* Skip log_xid for transactions without xid, marked by NULL end_event. */ + if (!trx_data->end_event) + continue; + + trx_data->error= 0; + *next_ptr= trx_data; + next_ptr= &(trx_data->next); + } + + /* + Next, lock the LOCK_log(), and once we get it, add any additional writes + that queued up while we were waiting. + + Note that if some writer not going through log_xid() comes in and gets the + LOCK_log before us, they will not be able to include us in their group + commit (and they are not able to handle ensuring same commit order between + us and participating transactional storage engines anyway). + + On the other hand, when we get the LOCK_log, we will be able to include + any non-trasactional writes that queued up in our group commit. This + should hopefully not be too big of a problem, as group commit is most + important for the transactional case anyway when durability (fsync) is + enabled. + */ VOID(pthread_mutex_lock(&LOCK_log)); - /* NULL would represent nothing to replicate after ROLLBACK */ - DBUG_ASSERT(commit_event != NULL); + /* + As the queue is in reverse order of entering, reverse the queue as we add + it to the existing one. Note that there is no ordering defined between + transactional and non-transactional commits. + */ + binlog_trx_data *current= atomic_grab_trx_queue(); + binlog_trx_data *xtra_queue= NULL; + while (current) + { + current->error= 0; + binlog_trx_data *next= current->next; + current->next= xtra_queue; + xtra_queue= current; + current= next; + } + *next_ptr= xtra_queue; + /* + Now we have in full_queue the list of transactions to be committed in + order. + */ DBUG_ASSERT(is_open()); if (likely(is_open())) // Should always be true { /* - We only bother to write to the binary log if there is anything - to write. - */ - if (my_b_tell(cache) > 0) + Commit every transaction in the queue. + + Note that we are doing this in a different thread than the one running + the transaction! So we are limited in the operations we can do. In + particular, we cannot call my_error() on behalf of a transaction, as + that obtains the THD from thread local storage. Instead, we must set + current->error and let the thread do the error reporting itself once + we wake it up. + */ + for (current= full_queue; current != NULL; current= current->next) { - /* - Log "BEGIN" at the beginning of every transaction. Here, a - transaction is either a BEGIN..COMMIT block or a single - statement in autocommit mode. - */ - Query_log_event qinfo(thd, STRING_WITH_LEN("BEGIN"), TRUE, TRUE, 0); + IO_CACHE *cache= &current->trans_log; /* - Now this Query_log_event has artificial log_pos 0. It must be - adjusted to reflect the real position in the log. Not doing it - would confuse the slave: it would prevent this one from - knowing where he is in the master's binlog, which would result - in wrong positions being shown to the user, MASTER_POS_WAIT - undue waiting etc. + We only bother to write to the binary log if there is anything + to write. */ - if (qinfo.write(&log_file)) - goto err; - - DBUG_EXECUTE_IF("crash_before_writing_xid", - { - if ((write_error= write_cache(cache, false, true))) - DBUG_PRINT("info", ("error writing binlog cache: %d", - write_error)); - DBUG_PRINT("info", ("crashing before writing xid")); - abort(); - }); - - if ((write_error= write_cache(cache, false, false))) - goto err; + if (my_b_tell(cache) > 0) + { + current->error= write_transaction(current); + if (current->error) + current->commit_errno= errno; - if (commit_event && commit_event->write(&log_file)) - goto err; + write_count++; + } - if (incident && write_incident(thd, FALSE)) - goto err; + if (current->end_event->get_type_code() == XID_EVENT) + xid_count++; + } + if (write_count > 0) + { if (flush_and_sync()) - goto err; - DBUG_EXECUTE_IF("half_binlogged_transaction", DBUG_ABORT();); - if (cache->error) // Error on read { - sql_print_error(ER(ER_ERROR_ON_READ), cache->file_name, errno); - write_error=1; // Don't give more errors - goto err; + for (current= full_queue; current != NULL; current= current->next) + { + if (!current->error) + { + current->error= ER_ERROR_ON_WRITE; + current->commit_errno= errno; + } + } + } + else + { + signal_update(); } - signal_update(); } /* - if commit_event is Xid_log_event, increase the number of + if any commit_events are Xid_log_event, increase the number of prepared_xids (it's decreasd in ::unlog()). Binlog cannot be rotated if there're prepared xids in it - see the comment in new_file() for an explanation. - If the commit_event is not Xid_log_event (then it's a Query_log_event) - rotate binlog, if necessary. + If no Xid_log_events (then it's all Query_log_event) rotate binlog, + if necessary. */ - if (commit_event && commit_event->get_type_code() == XID_EVENT) + if (xid_count > 0) { - pthread_mutex_lock(&LOCK_prep_xids); - prepared_xids++; - pthread_mutex_unlock(&LOCK_prep_xids); + mark_xids_active(xid_count); } else rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); } + VOID(pthread_mutex_unlock(&LOCK_log)); - DBUG_RETURN(0); + /* + Signal those that are not part of group_log_xid, and are not group leaders + running the queue. -err: - if (!write_error) + Since a group leader runs the queue itself if a group_log_xid does not get + to do it forst, such leader threads do not need wait or wakeup. + */ + for (current= xtra_queue; current != NULL; current= current->next) { - write_error= 1; - sql_print_error(ER(ER_ERROR_ON_WRITE), name, errno); + /* + Note that we need to take LOCK_group_commit even in the case of a leader! + + Otherwise there is a race between setting and testing the + group_commit_leader flag. + */ + pthread_mutex_lock(&current->LOCK_group_commit); + if (!current->group_commit_leader) + { + current->done= true; + pthread_cond_signal(&current->COND_group_commit); + } + pthread_mutex_unlock(&current->LOCK_group_commit); } - VOID(pthread_mutex_unlock(&LOCK_log)); - DBUG_RETURN(1); } +int +MYSQL_BIN_LOG::write_transaction(binlog_trx_data *trx_data) +{ + IO_CACHE *cache= &trx_data->trans_log; + /* + Log "BEGIN" at the beginning of every transaction. Here, a transaction is + either a BEGIN..COMMIT block or a single statement in autocommit mode. The + event was constructed in write_transaction_to_binlog(), in the thread + running the transaction. + + Now this Query_log_event has artificial log_pos 0. It must be + adjusted to reflect the real position in the log. Not doing it + would confuse the slave: it would prevent this one from + knowing where he is in the master's binlog, which would result + in wrong positions being shown to the user, MASTER_POS_WAIT + undue waiting etc. + */ + if (trx_data->begin_event->write(&log_file)) + return ER_ERROR_ON_WRITE; + + DBUG_EXECUTE_IF("crash_before_writing_xid", + { + if ((write_cache(cache))) + DBUG_PRINT("info", ("error writing binlog cache")); + else + flush_and_sync(); + + DBUG_PRINT("info", ("crashing before writing xid")); + abort(); + }); + + if (write_cache(cache)) + return ER_ERROR_ON_WRITE; + + if (trx_data->end_event->write(&log_file)) + return ER_ERROR_ON_WRITE; + + if (trx_data->has_incident() && trx_data->incident_event->write(&log_file)) + return ER_ERROR_ON_WRITE; + + if (cache->error) // Error on read + return ER_ERROR_ON_READ; + + return 0; +} + +binlog_trx_data * +MYSQL_BIN_LOG::atomic_enqueue_trx(binlog_trx_data *trx_data) +{ + my_atomic_rwlock_wrlock(&LOCK_queue); + trx_data->next= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&trx_data->next), + trx_data)) + ; + my_atomic_rwlock_wrunlock(&LOCK_queue); + return trx_data->next; +} + +binlog_trx_data * +MYSQL_BIN_LOG::atomic_grab_trx_queue() +{ + my_atomic_rwlock_wrlock(&LOCK_queue); + binlog_trx_data *queue= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&queue), + NULL)) + ; + my_atomic_rwlock_wrunlock(&LOCK_queue); + return queue; +} /** Wait until we get a signal that the binary log has been updated. @@ -5879,9 +6217,6 @@ void TC_LOG_BINLOG::close() } /** - @todo - group commit - @retval 0 error @retval @@ -5889,19 +6224,83 @@ void TC_LOG_BINLOG::close() */ int TC_LOG_BINLOG::log_xid(THD *thd, my_xid xid) { - DBUG_ENTER("TC_LOG_BINLOG::log"); - Xid_log_event xle(thd, xid); - binlog_trx_data *trx_data= - (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + int error; + DBUG_ENTER("TC_LOG_BINLOG::log_xid"); + + thd->next_commit_ordered= 0; + group_log_xid(thd); + if (thd->xid_error) + error= xid_delayed_error(thd); + else + error= 0; + /* - We always commit the entire transaction when writing an XID. Also - note that the return value is inverted. - */ - DBUG_RETURN(!binlog_end_trans(thd, trx_data, &xle, TRUE)); + Note that the return value is inverted: zero on failure, private non-zero + 'cookie' on success. + */ + DBUG_RETURN(!error); } -void TC_LOG_BINLOG::unlog(ulong cookie, my_xid xid) +/* + Do a binlog log_xid() for a group of transactions, linked through + thd->next_commit_ordered. +*/ +void +TC_LOG_BINLOG::group_log_xid(THD *first_thd) +{ + DBUG_ENTER("TC_LOG_BINLOG::group_log_xid"); + trx_group_commit_leader(first_thd); + for (THD *thd= first_thd; thd; thd= thd->next_commit_ordered) + { + binlog_trx_data *const trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + thd->xid_error= trx_data->error; + thd->xid_cookie= !trx_data->error; + } + DBUG_VOID_RETURN; +} + +int +TC_LOG_BINLOG::xid_delayed_error(THD *thd) { + binlog_trx_data *const trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + return trx_group_commit_finish(trx_data); +} + +/* + After an XID is logged, we need to hold on to the current binlog file until + it is fully committed in the storage engine. The reason is that crash + recovery only looks at the latest binlog, so we must make sure there are no + outstanding prepared (but not committed) transactions before rotating the + binlog. + + To handle this, we keep a count of outstanding XIDs. This function is used + to increase this count when committing one or more transactions to the + binary log. +*/ +void +TC_LOG_BINLOG::mark_xids_active(uint xid_count) +{ + DBUG_ENTER("TC_LOG_BINLOG::mark_xids_active"); + DBUG_PRINT("info", ("xid_count=%u", xid_count)); + pthread_mutex_lock(&LOCK_prep_xids); + prepared_xids+= xid_count; + pthread_mutex_unlock(&LOCK_prep_xids); + DBUG_VOID_RETURN; +} + +/* + Once an XID is committed, it is safe to rotate the binary log, as it can no + longer be needed during crash recovery. + + This function is called to mark an XID this way. It needs to decrease the + count of pending XIDs, and signal the log rotator thread when it reaches zero. +*/ +void +TC_LOG_BINLOG::mark_xid_done() +{ + DBUG_ENTER("TC_LOG_BINLOG::mark_xid_done"); pthread_mutex_lock(&LOCK_prep_xids); DBUG_ASSERT(prepared_xids > 0); if (--prepared_xids == 0) { @@ -5909,7 +6308,16 @@ void TC_LOG_BINLOG::unlog(ulong cookie, pthread_cond_signal(&COND_prep_xids); } pthread_mutex_unlock(&LOCK_prep_xids); - rotate_and_purge(0); // as ::write() did not rotate + DBUG_VOID_RETURN; +} + +void TC_LOG_BINLOG::unlog(ulong cookie, my_xid xid) +{ + DBUG_ENTER("TC_LOG_BINLOG::unlog"); + if (xid) + mark_xid_done(); + rotate_and_purge(0); // as ::write_transaction_to_binlog() did not rotate + DBUG_VOID_RETURN; } int TC_LOG_BINLOG::recover(IO_CACHE *log, Format_description_log_event *fdle) === modified file 'sql/log.h' --- a/sql/log.h 2009-12-04 14:40:42 +0000 +++ b/sql/log.h 2010-05-26 08:16:18 +0000 @@ -28,13 +28,49 @@ class TC_LOG { public: int using_heuristic_recover(); - TC_LOG() {} + /* True if we should use group_log_xid(), false to use log_xid(). */ + bool use_group_log_xid; + + TC_LOG() : use_group_log_xid(0) {} virtual ~TC_LOG() {} virtual int open(const char *opt_name)=0; virtual void close()=0; virtual int log_xid(THD *thd, my_xid xid)=0; virtual void unlog(ulong cookie, my_xid xid)=0; + /* + If use_group_log_xid is true, then this method is used instead of + log_xid() to do logging of a group of transactions all at once. + + The transactions will be linked through THD::next_commit_ordered. + + Additionally, when this method is used instead of log_xid(), the order in + which handler->prepare_ordered() and handler->commit_ordered() are called + is guaranteed to be the same as the order of calls and THD list elements + for group_log_xid(). + + This can be used to efficiently implement group commit that at the same + time preserves the order of commits among handlers and TC (eg. to get same + commit order in InnoDB and binary log). + + For TCs that do not need this, it can be preferable to use plain log_xid() + instead, as it allows threads to run log_xid() in parallel with each + other. In contrast, group_log_xid() runs under a global mutex, so it is + guaranteed that only once call into it will be active at once. + + Since this call handles multiple threads/THDs at once, my_error() (and + other code that relies on thread local storage) cannot be used in this + method. Instead, in case of error, thd->xid_error should be set to the + error code, and xid_delayed_error() will be called later in the correct + thread context to actually report the error. + + In the success case, this method must set thd->xid_cookie for each thread + to the cookie that is normally returned from log_xid() (which must be + non-zero in the non-error case). + */ + virtual void group_log_xid(THD *first_thd) { DBUG_ASSERT(FALSE); } + /* Error reporting for group_log_xid(). */ + virtual int xid_delayed_error(THD *thd) { DBUG_ASSERT(FALSE); return 0; } }; class TC_LOG_DUMMY: public TC_LOG // use it to disable the logging @@ -227,12 +263,19 @@ private: time_t last_time; }; +class binlog_trx_data; class MYSQL_BIN_LOG: public TC_LOG, private MYSQL_LOG { private: /* LOCK_log and LOCK_index are inited by init_pthread_objects() */ pthread_mutex_t LOCK_index; pthread_mutex_t LOCK_prep_xids; + /* + Mutex to protect the queue of transactions waiting to participate in group + commit. (Only used on platforms without native atomic operations). + */ + pthread_mutex_t LOCK_queue; + pthread_cond_t COND_prep_xids; pthread_cond_t update_cond; ulonglong bytes_written; @@ -271,8 +314,8 @@ class MYSQL_BIN_LOG: public TC_LOG, priv In 5.0 it's 0 for relay logs too! */ bool no_auto_events; - - ulonglong m_table_map_version; + /* Queue of transactions queued up to participate in group commit. */ + binlog_trx_data *group_commit_queue; int write_to_file(IO_CACHE *cache); /* @@ -282,6 +325,14 @@ class MYSQL_BIN_LOG: public TC_LOG, priv */ void new_file_without_locking(); void new_file_impl(bool need_lock); + int write_transaction(binlog_trx_data *trx_data); + bool write_transaction_to_binlog_events(binlog_trx_data *trx_data); + void trx_group_commit_participant(binlog_trx_data *trx_data); + void trx_group_commit_leader(THD *first_thd); + binlog_trx_data *atomic_enqueue_trx(binlog_trx_data *trx_data); + binlog_trx_data *atomic_grab_trx_queue(); + void mark_xid_done(); + void mark_xids_active(uint xid_count); public: MYSQL_LOG::generate_name; @@ -311,17 +362,11 @@ public: int open(const char *opt_name); void close(); int log_xid(THD *thd, my_xid xid); + int xid_delayed_error(THD *thd); + void group_log_xid(THD *first_thd); void unlog(ulong cookie, my_xid xid); int recover(IO_CACHE *log, Format_description_log_event *fdle); #if !defined(MYSQL_CLIENT) - bool is_table_mapped(TABLE *table) const - { - return table->s->table_map_version == table_map_version(); - } - - ulonglong table_map_version() const { return m_table_map_version; } - void update_table_map_version() { ++m_table_map_version; } - int flush_and_set_pending_rows_event(THD *thd, Rows_log_event* event); int remove_pending_rows_event(THD *thd); @@ -362,10 +407,12 @@ public: void new_file(); bool write(Log_event* event_info); // binary log write - bool write(THD *thd, IO_CACHE *cache, Log_event *commit_event, bool incident); - bool write_incident(THD *thd, bool lock); + bool write_transaction_to_binlog(THD *thd, binlog_trx_data *trx_data, + Log_event *end_ev); + bool trx_group_commit_finish(binlog_trx_data *trx_data); + bool write_incident(THD *thd); - int write_cache(IO_CACHE *cache, bool lock_log, bool flush_and_sync); + int write_cache(IO_CACHE *cache); void set_write_error(THD *thd); bool check_write_error(THD *thd); === modified file 'sql/log_event.h' --- a/sql/log_event.h 2010-03-04 08:03:07 +0000 +++ b/sql/log_event.h 2010-05-26 08:16:18 +0000 @@ -463,10 +463,9 @@ struct sql_ex_info #define LOG_EVENT_SUPPRESS_USE_F 0x8 /* - The table map version internal to the log should be increased after - the event has been written to the binary log. + This used to be LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F, but is now unused. */ -#define LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F 0x10 +#define LOG_EVENT_UNUSED1_F 0x10 /** @def LOG_EVENT_ARTIFICIAL_F === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-01-15 15:27:55 +0000 +++ b/sql/sql_class.cc 2010-05-26 08:16:18 +0000 @@ -673,6 +673,8 @@ THD::THD() active_vio = 0; #endif pthread_mutex_init(&LOCK_thd_data, MY_MUTEX_INIT_FAST); + pthread_mutex_init(&LOCK_commit_ordered, MY_MUTEX_INIT_FAST); + pthread_cond_init(&COND_commit_ordered, 0); /* Variables with default values */ proc_info="login"; @@ -3773,7 +3775,6 @@ int THD::binlog_flush_pending_rows_event if (stmt_end) { pending->set_flags(Rows_log_event::STMT_END_F); - pending->flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; binlog_table_maps= 0; } @@ -3901,7 +3902,6 @@ int THD::binlog_query(THD::enum_binlog_q { Query_log_event qinfo(this, query_arg, query_len, is_trans, suppress_use, errcode); - qinfo.flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; /* Binlog table maps will be irrelevant after a Query_log_event (they are just removed on the slave side) so after the query === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-03-30 12:36:49 +0000 +++ b/sql/sql_class.h 2010-05-26 08:16:18 +0000 @@ -1438,6 +1438,21 @@ public: /* container for handler's private per-connection data */ Ha_data ha_data[MAX_HA]; + /* Mutex and condition for waking up threads after group commit. */ + pthread_mutex_t LOCK_commit_ordered; + pthread_cond_t COND_commit_ordered; + bool group_commit_ready; + /* Pointer for linking THDs into queue waiting for group commit. */ + THD *next_commit_ordered; + /* + The "all" parameter of commit(), to communicate it to the thread that + calls commit_ordered(). + */ + bool group_commit_all; + /* Set by TC_LOG::group_log_xid(), to return per-thd error and cookie. */ + int xid_error; + int xid_cookie; + #ifndef MYSQL_CLIENT int binlog_setup_trx_data(); === modified file 'sql/sql_load.cc' --- a/sql/sql_load.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_load.cc 2010-05-26 08:16:18 +0000 @@ -516,7 +516,6 @@ int mysql_load(THD *thd,sql_exchange *ex else { Delete_file_log_event d(thd, db, transactional_table); - d.flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; (void) mysql_bin_log.write(&d); } } @@ -698,7 +697,6 @@ static bool write_execute_load_query_log (duplicates == DUP_REPLACE) ? LOAD_DUP_REPLACE : (ignore ? LOAD_DUP_IGNORE : LOAD_DUP_ERROR), transactional_table, FALSE, errcode); - e.flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; return mysql_bin_log.write(&e); } === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-10 10:32:14 +0000 +++ b/sql/table.cc 2010-05-26 08:16:18 +0000 @@ -297,13 +297,6 @@ TABLE_SHARE *alloc_table_share(TABLE_LIS share->version= refresh_version; /* - This constant is used to mark that no table map version has been - assigned. No arithmetic is done on the value: it will be - overwritten with a value taken from MYSQL_BIN_LOG. - */ - share->table_map_version= ~(ulonglong)0; - - /* Since alloc_table_share() can be called without any locking (for example, ha_create_table... functions), we do not assign a table map id here. Instead we assign a value that is not used @@ -367,10 +360,9 @@ void init_tmp_table_share(THD *thd, TABL share->frm_version= FRM_VER_TRUE_VARCHAR; /* - Temporary tables are not replicated, but we set up these fields + Temporary tables are not replicated, but we set up this fields anyway to be able to catch errors. */ - share->table_map_version= ~(ulonglong)0; share->cached_row_logging_check= -1; /* === modified file 'sql/table.h' --- a/sql/table.h 2010-02-10 19:06:24 +0000 +++ b/sql/table.h 2010-05-26 08:16:18 +0000 @@ -433,7 +433,6 @@ typedef struct st_table_share bool waiting_on_cond; /* Protection against free */ bool deleting; /* going to delete this table */ ulong table_map_id; /* for row-based replication */ - ulonglong table_map_version; /* Cache for row-based replication table share checks that does not === modified file 'storage/xtradb/handler/ha_innodb.cc' --- a/storage/xtradb/handler/ha_innodb.cc 2010-01-15 21:12:30 +0000 +++ b/storage/xtradb/handler/ha_innodb.cc 2010-05-26 08:16:18 +0000 @@ -138,8 +138,6 @@ bool check_global_access(THD *thd, ulong /** to protect innobase_open_files */ static pthread_mutex_t innobase_share_mutex; -/** to force correct commit order in binlog */ -static pthread_mutex_t prepare_commit_mutex; static ulong commit_threads = 0; static pthread_mutex_t commit_threads_m; static pthread_cond_t commit_cond; @@ -239,6 +237,7 @@ static const char* innobase_change_buffe static INNOBASE_SHARE *get_share(const char *table_name); static void free_share(INNOBASE_SHARE *share); static int innobase_close_connection(handlerton *hton, THD* thd); +static void innobase_commit_ordered(handlerton *hton, THD* thd, bool all); static int innobase_commit(handlerton *hton, THD* thd, bool all); static int innobase_rollback(handlerton *hton, THD* thd, bool all); static int innobase_rollback_to_savepoint(handlerton *hton, THD* thd, @@ -1356,7 +1355,6 @@ innobase_trx_init( trx_t* trx) /*!< in/out: InnoDB transaction handle */ { DBUG_ENTER("innobase_trx_init"); - DBUG_ASSERT(EQ_CURRENT_THD(thd)); DBUG_ASSERT(thd == trx->mysql_thd); trx->check_foreigns = !thd_test_options( @@ -1416,8 +1414,6 @@ check_trx_exists( { trx_t*& trx = thd_to_trx(thd); - ut_ad(EQ_CURRENT_THD(thd)); - if (trx == NULL) { trx = innobase_trx_allocate(thd); } else if (UNIV_UNLIKELY(trx->magic_n != TRX_MAGIC_N)) { @@ -2024,6 +2020,7 @@ innobase_init( innobase_hton->savepoint_set=innobase_savepoint; innobase_hton->savepoint_rollback=innobase_rollback_to_savepoint; innobase_hton->savepoint_release=innobase_release_savepoint; + innobase_hton->commit_ordered=innobase_commit_ordered; innobase_hton->commit=innobase_commit; innobase_hton->rollback=innobase_rollback; innobase_hton->prepare=innobase_xa_prepare; @@ -2492,7 +2489,6 @@ skip_overwrite: innobase_open_tables = hash_create(200); pthread_mutex_init(&innobase_share_mutex, MY_MUTEX_INIT_FAST); - pthread_mutex_init(&prepare_commit_mutex, MY_MUTEX_INIT_FAST); pthread_mutex_init(&commit_threads_m, MY_MUTEX_INIT_FAST); pthread_mutex_init(&commit_cond_m, MY_MUTEX_INIT_FAST); pthread_mutex_init(&analyze_mutex, MY_MUTEX_INIT_FAST); @@ -2547,7 +2543,6 @@ innobase_end( my_free(internal_innobase_data_file_path, MYF(MY_ALLOW_ZERO_PTR)); pthread_mutex_destroy(&innobase_share_mutex); - pthread_mutex_destroy(&prepare_commit_mutex); pthread_mutex_destroy(&commit_threads_m); pthread_mutex_destroy(&commit_cond_m); pthread_mutex_destroy(&analyze_mutex); @@ -2681,6 +2676,101 @@ innobase_start_trx_and_assign_read_view( } /*****************************************************************//** +Perform the first, fast part of InnoDB commit. + +Doing it in this call ensures that we get the same commit order here +as in binlog and any other participating transactional storage engines. + +Note that we want to do as little as really needed here, as we run +under a global mutex. The expensive fsync() is done later, in +innobase_commit(), without a lock so group commit can take place. + +Note also that this method can be called from a different thread than +the one handling the rest of the transaction. */ +static +void +innobase_commit_ordered( +/*============*/ + handlerton *hton, /*!< in: Innodb handlerton */ + THD* thd, /*!< in: MySQL thread handle of the user for whom + the transaction should be committed */ + bool all) /*!< in: TRUE - commit transaction + FALSE - the current SQL statement ended */ +{ + trx_t* trx; + DBUG_ENTER("innobase_commit_ordered"); + DBUG_ASSERT(hton == innodb_hton_ptr); + + trx = check_trx_exists(thd); + + if (trx->active_trans == 0 + && trx->conc_state != TRX_NOT_STARTED) { + /* We throw an error here; instead we will catch this error + again in innobase_commit() and report it from there. */ + DBUG_VOID_RETURN; + } + /* Since we will reserve the kernel mutex, we have to release + the search system latch first to obey the latching order. */ + + if (trx->has_search_latch) { + trx_search_latch_release_if_reserved(trx); + } + + /* commit_ordered is only called when committing the whole transaction + (or an SQL statement when autocommit is on). */ + DBUG_ASSERT(all || (!thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN))); + + /* We need current binlog position for ibbackup to work. + Note, the position is current because commit_ordered is guaranteed + to be called in same sequenece as writing to binlog. */ + +retry: + if (innobase_commit_concurrency > 0) { + pthread_mutex_lock(&commit_cond_m); + commit_threads++; + + if (commit_threads > innobase_commit_concurrency) { + commit_threads--; + pthread_cond_wait(&commit_cond, + &commit_cond_m); + pthread_mutex_unlock(&commit_cond_m); + goto retry; + } + else { + pthread_mutex_unlock(&commit_cond_m); + } + } + + /* The following calls to read the MySQL binary log + file name and the position return consistent results: + 1) We use commit_ordered() to get same commit order + in InnoDB as in binary log. + 2) A MySQL log file rotation cannot happen because + MySQL protects against this by having a counter of + transactions in prepared state and it only allows + a rotation when the counter drops to zero. See + LOCK_prep_xids and COND_prep_xids in log.cc. */ + trx->mysql_log_file_name = mysql_bin_log_file_name(); + trx->mysql_log_offset = (ib_int64_t) mysql_bin_log_file_pos(); + + /* Don't do write + flush right now. For group commit + to work we want to do the flush in the innobase_commit() + method, which runs without holding any locks. */ + trx->flush_log_later = TRUE; + innobase_commit_low(trx); + trx->flush_log_later = FALSE; + + if (innobase_commit_concurrency > 0) { + pthread_mutex_lock(&commit_cond_m); + commit_threads--; + pthread_cond_signal(&commit_cond); + pthread_mutex_unlock(&commit_cond_m); + } + + DBUG_VOID_RETURN; +} + +/*****************************************************************//** Commits a transaction in an InnoDB database or marks an SQL statement ended. @return 0 */ @@ -2702,13 +2792,6 @@ innobase_commit( trx = check_trx_exists(thd); - /* Since we will reserve the kernel mutex, we have to release - the search system latch first to obey the latching order. */ - - if (trx->has_search_latch) { - trx_search_latch_release_if_reserved(trx); - } - /* The flag trx->active_trans is set to 1 in 1. ::external_lock(), @@ -2736,62 +2819,8 @@ innobase_commit( /* We were instructed to commit the whole transaction, or this is an SQL statement end and autocommit is on */ - /* We need current binlog position for ibbackup to work. - Note, the position is current because of - prepare_commit_mutex */ -retry: - if (innobase_commit_concurrency > 0) { - pthread_mutex_lock(&commit_cond_m); - commit_threads++; - - if (commit_threads > innobase_commit_concurrency) { - commit_threads--; - pthread_cond_wait(&commit_cond, - &commit_cond_m); - pthread_mutex_unlock(&commit_cond_m); - goto retry; - } - else { - pthread_mutex_unlock(&commit_cond_m); - } - } - - /* The following calls to read the MySQL binary log - file name and the position return consistent results: - 1) Other InnoDB transactions cannot intervene between - these calls as we are holding prepare_commit_mutex. - 2) Binary logging of other engines is not relevant - to InnoDB as all InnoDB requires is that committing - InnoDB transactions appear in the same order in the - MySQL binary log as they appear in InnoDB logs. - 3) A MySQL log file rotation cannot happen because - MySQL protects against this by having a counter of - transactions in prepared state and it only allows - a rotation when the counter drops to zero. See - LOCK_prep_xids and COND_prep_xids in log.cc. */ - trx->mysql_log_file_name = mysql_bin_log_file_name(); - trx->mysql_log_offset = (ib_int64_t) mysql_bin_log_file_pos(); - - /* Don't do write + flush right now. For group commit - to work we want to do the flush after releasing the - prepare_commit_mutex. */ - trx->flush_log_later = TRUE; - innobase_commit_low(trx); - trx->flush_log_later = FALSE; - - if (innobase_commit_concurrency > 0) { - pthread_mutex_lock(&commit_cond_m); - commit_threads--; - pthread_cond_signal(&commit_cond); - pthread_mutex_unlock(&commit_cond_m); - } - - if (trx->active_trans == 2) { - - pthread_mutex_unlock(&prepare_commit_mutex); - } - - /* Now do a write + flush of logs. */ + /* We did the first part already in innobase_commit_ordered(), + Now finish by doing a write + flush of logs. */ trx_commit_complete_for_mysql(trx); trx->active_trans = 0; @@ -4621,6 +4650,7 @@ no_commit: no need to re-acquire locks on it. */ /* Altering to InnoDB format */ + innobase_commit_ordered(ht, user_thd, 1); innobase_commit(ht, user_thd, 1); /* Note that this transaction is still active. */ prebuilt->trx->active_trans = 1; @@ -4637,6 +4667,7 @@ no_commit: /* Commit the transaction. This will release the table locks, so they have to be acquired again. */ + innobase_commit_ordered(ht, user_thd, 1); innobase_commit(ht, user_thd, 1); /* Note that this transaction is still active. */ prebuilt->trx->active_trans = 1; @@ -8339,6 +8370,7 @@ ha_innobase::external_lock( if (!thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN)) { if (trx->active_trans != 0) { + innobase_commit_ordered(ht, thd, TRUE); innobase_commit(ht, thd, TRUE); } } else { @@ -9448,36 +9480,6 @@ innobase_xa_prepare( srv_active_wake_master_thread(); - if (thd_sql_command(thd) != SQLCOM_XA_PREPARE && - (all || !thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN))) - { - if (srv_enable_unsafe_group_commit && !THDVAR(thd, support_xa)) { - /* choose group commit rather than binlog order */ - return(error); - } - - /* For ibbackup to work the order of transactions in binlog - and InnoDB must be the same. Consider the situation - - thread1> prepare; write to binlog; ... - <context switch> - thread2> prepare; write to binlog; commit - thread1> ... commit - - To ensure this will not happen we're taking the mutex on - prepare, and releasing it on commit. - - Note: only do it for normal commits, done via ha_commit_trans. - If 2pc protocol is executed by external transaction - coordinator, it will be just a regular MySQL client - executing XA PREPARE and XA COMMIT commands. - In this case we cannot know how many minutes or hours - will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. */ - pthread_mutex_lock(&prepare_commit_mutex); - trx->active_trans = 2; - } - return(error); } @@ -10669,11 +10671,6 @@ static MYSQL_SYSVAR_ENUM(adaptive_checkp "Enable/Disable flushing along modified age. (none, reflex, [estimate])", NULL, innodb_adaptive_checkpoint_update, 2, &adaptive_checkpoint_typelib); -static MYSQL_SYSVAR_ULONG(enable_unsafe_group_commit, srv_enable_unsafe_group_commit, - PLUGIN_VAR_RQCMDARG, - "Enable/Disable unsafe group commit when support_xa=OFF and use with binlog or other XA storage engine.", - NULL, NULL, 0, 0, 1, 0); - static MYSQL_SYSVAR_ULONG(expand_import, srv_expand_import, PLUGIN_VAR_RQCMDARG, "Enable/Disable converting automatically *.ibd files when import tablespace.", @@ -10763,7 +10760,6 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(flush_neighbor_pages), MYSQL_SYSVAR(read_ahead), MYSQL_SYSVAR(adaptive_checkpoint), - MYSQL_SYSVAR(enable_unsafe_group_commit), MYSQL_SYSVAR(expand_import), MYSQL_SYSVAR(extra_rsegments), MYSQL_SYSVAR(dict_size_limit),

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2849)
by knielsen＠knielsen-hq.org 26 May '10

26 May '10

#At lp:maria 2849 knielsen(a)knielsen-hq.org 2010-05-26 Preliminary commit of group commit proof-of-concept. modified: sql/handler.cc sql/handler.h sql/log.cc sql/log.h sql/log_event.h sql/sql_class.cc sql/sql_class.h sql/sql_load.cc sql/table.cc sql/table.h storage/xtradb/handler/ha_innodb.cc === modified file 'sql/handler.cc' --- a/sql/handler.cc 2010-04-06 22:47:08 +0000 +++ b/sql/handler.cc 2010-05-26 08:13:32 +0000 @@ -76,6 +76,7 @@ TYPELIB tx_isolation_typelib= {array_ele static TYPELIB known_extensions= {0,"known_exts", NULL, NULL}; uint known_extensions_id= 0; +static int commit_one_phase_2(THD *thd, bool all, bool do_commit_ordered); static plugin_ref ha_default_plugin(THD *thd) @@ -544,6 +545,26 @@ err: DBUG_RETURN(1); } +/* + This is a queue of THDs waiting for being group committed with + tc_log->group_log_xid(). +*/ +static THD *group_commit_queue; +/* + This mutex protects the group_commit_queue on platforms without native + atomic operations. + */ +static pthread_mutex_t LOCK_group_commit_queue; +/* This mutex is used to serialize calls to handler prepare_ordered methods. */ +static pthread_mutex_t LOCK_prepare_ordered; +/* This mutex is used to serialize calls to handler commit_ordered methods. */ +static pthread_mutex_t LOCK_commit_ordered; +/* This mutex is used to serialize calls to group_log_xid(). */ +static pthread_mutex_t LOCK_group_commit; +static pthread_cond_t COND_group_commit; + +static bool mutexes_inited= FALSE; + int ha_init() { int error= 0; @@ -557,6 +578,19 @@ int ha_init() */ opt_using_transactions= total_ha>(ulong)opt_bin_log; savepoint_alloc_size+= sizeof(SAVEPOINT); + + group_commit_queue= NULL; + my_pthread_mutex_init(&LOCK_group_commit_queue, MY_MUTEX_INIT_FAST, + "LOCK_group_commit_queue", MYF(0)); + my_pthread_mutex_init(&LOCK_prepare_ordered, MY_MUTEX_INIT_SLOW, + "LOCK_prepare_ordered", MYF(0)); + my_pthread_mutex_init(&LOCK_commit_ordered, MY_MUTEX_INIT_SLOW, + "LOCK_commit_ordered", MYF(0)); + my_pthread_mutex_init(&LOCK_group_commit, MY_MUTEX_INIT_SLOW, + "LOCK_group_commit", MYF(0)); + pthread_cond_init(&COND_group_commit, 0); + mutexes_inited= TRUE; + DBUG_RETURN(error); } @@ -574,6 +608,15 @@ int ha_end() if (ha_finish_errors()) error= 1; + if (mutexes_inited) + { + pthread_mutex_destroy(&LOCK_group_commit_queue); + pthread_mutex_destroy(&LOCK_prepare_ordered); + pthread_mutex_destroy(&LOCK_commit_ordered); + pthread_mutex_destroy(&LOCK_group_commit); + mutexes_inited= FALSE; + } + DBUG_RETURN(error); } @@ -1053,6 +1096,108 @@ ha_check_and_coalesce_trx_read_only(THD return rw_ha_count; } +/* + Atomically enqueue a THD at the head of the queue of threads waiting to + group commit, and return the previous head of the queue. +*/ +static THD * +enqueue_atomic(THD *thd) +{ + my_atomic_rwlock_wrlock(&LOCK_group_commit_queue); + thd->next_commit_ordered= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&thd->next_commit_ordered), + thd)) + ; + my_atomic_rwlock_wrunlock(&LOCK_group_commit_queue); + return thd->next_commit_ordered; +} + +static THD * +atomic_grab_reverse_queue() +{ + my_atomic_rwlock_wrlock(&LOCK_group_commit_queue); + THD *queue= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&queue), + NULL)) + ; + my_atomic_rwlock_wrunlock(&LOCK_group_commit_queue); + + /* + Since we enqueue at the head, the queue is actually in reverse order. + So reverse it back into correct commit order before returning. + */ + THD *prev= NULL; + while (queue) + { + THD *next= queue->next_commit_ordered; + queue->next_commit_ordered= prev; + prev= queue; + queue= next; + } + + return prev; +} + +static void +call_commit_ordered(Ha_trx_info *ha_info, THD *thd, bool all) +{ + for (; ha_info; ha_info= ha_info->next()) + { + handlerton *ht= ha_info->ht(); + if (!ht->commit_ordered) + continue; + ht->commit_ordered(ht, thd, all); + } +} + +static void +group_commit_wait_for_wakeup(THD *thd) +{ + pthread_mutex_lock(&thd->LOCK_commit_ordered); + while (!thd->group_commit_ready) + pthread_cond_wait(&thd->COND_commit_ordered, + &thd->LOCK_commit_ordered); + pthread_mutex_unlock(&thd->LOCK_commit_ordered); +} + +static void +group_commit_wakeup_other(THD *other_thd) +{ + pthread_mutex_lock(&other_thd->LOCK_commit_ordered); + other_thd->group_commit_ready= TRUE; + pthread_cond_signal(&other_thd->COND_commit_ordered); + pthread_mutex_unlock(&other_thd->LOCK_commit_ordered); +} + +static bool group_commit_queue_busy= 0; + +static void +group_commit_mark_queue_idle() +{ + pthread_mutex_lock(&LOCK_group_commit); + group_commit_queue_busy= FALSE; + pthread_cond_signal(&COND_group_commit); + pthread_mutex_unlock(&LOCK_group_commit); +} + +static void +group_commit_mark_queue_busy() +{ + safe_mutex_assert_owner(&LOCK_group_commit); + group_commit_queue_busy= TRUE; +} + +static void +group_commit_wait_queue_idle() +{ + /* Wait for any existing queue run to finish. */ + safe_mutex_assert_owner(&LOCK_group_commit); + while (group_commit_queue_busy) + pthread_cond_wait(&COND_group_commit, &LOCK_group_commit); +} + /** @retval @@ -1070,7 +1215,7 @@ ha_check_and_coalesce_trx_read_only(THD */ int ha_commit_trans(THD *thd, bool all) { - int error= 0, cookie= 0; + int error= 0; /* 'all' means that this is either an explicit commit issued by user, or an implicit commit issued by a DDL. @@ -1085,7 +1230,10 @@ int ha_commit_trans(THD *thd, bool all) */ bool is_real_trans= all || thd->transaction.all.ha_list == 0; Ha_trx_info *ha_info= trans->ha_list; - my_xid xid= thd->transaction.xid_state.xid.get_my_xid(); + bool need_prepare_ordered, need_commit_ordered; + bool need_enqueue; + bool is_group_commit_leader; + my_xid xid; DBUG_ENTER("ha_commit_trans"); /* @@ -1118,85 +1266,277 @@ int ha_commit_trans(THD *thd, bool all) DBUG_RETURN(2); } #ifdef USING_TRANSACTIONS - if (ha_info) + if (!ha_info) { - uint rw_ha_count; - bool rw_trans; + /* Free resources and perform other cleanup even for 'empty' transactions. */ + if (is_real_trans) + thd->transaction.cleanup(); + DBUG_RETURN(0); + } - DBUG_EXECUTE_IF("crash_commit_before", abort();); + DBUG_EXECUTE_IF("crash_commit_before", abort();); - /* Close all cursors that can not survive COMMIT */ - if (is_real_trans) /* not a statement commit */ - thd->stmt_map.close_transient_cursors(); + /* Close all cursors that can not survive COMMIT */ + if (is_real_trans) /* not a statement commit */ + thd->stmt_map.close_transient_cursors(); - rw_ha_count= ha_check_and_coalesce_trx_read_only(thd, ha_info, all); - /* rw_trans is TRUE when we in a transaction changing data */ - rw_trans= is_real_trans && (rw_ha_count > 0); + uint rw_ha_count= ha_check_and_coalesce_trx_read_only(thd, ha_info, all); + /* rw_trans is TRUE when we in a transaction changing data */ + bool rw_trans= is_real_trans && (rw_ha_count > 0); - if (rw_trans && - wait_if_global_read_lock(thd, 0, 0)) - { - ha_rollback_trans(thd, all); - DBUG_RETURN(1); - } + if (rw_trans && + wait_if_global_read_lock(thd, 0, 0)) + { + ha_rollback_trans(thd, all); + DBUG_RETURN(1); + } + + if (rw_trans && + opt_readonly && + !(thd->security_ctx->master_access & SUPER_ACL) && + !thd->slave_thread) + { + my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--read-only"); + goto err; + } - if (rw_trans && - opt_readonly && - !(thd->security_ctx->master_access & SUPER_ACL) && - !thd->slave_thread) + if (trans->no_2pc || (rw_ha_count <= 1)) + { + error= ha_commit_one_phase(thd, all); + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + } + + need_prepare_ordered= FALSE; + need_commit_ordered= FALSE; + xid= thd->transaction.xid_state.xid.get_my_xid(); + + for (Ha_trx_info *hi= ha_info; hi; hi= hi->next()) + { + int err; + handlerton *ht= hi->ht(); + /* + Do not call two-phase commit if this particular + transaction is read-only. This allows for simpler + implementation in engines that are always read-only. + */ + if (! hi->is_trx_read_write()) + continue; + /* + Sic: we know that prepare() is not NULL since otherwise + trans->no_2pc would have been set. + */ + if ((err= ht->prepare(ht, thd, all))) + my_error(ER_ERROR_DURING_COMMIT, MYF(0), err); + status_var_increment(thd->status_var.ha_prepare_count); + + if (err) + goto err; + + if (ht->prepare_ordered) + need_prepare_ordered= TRUE; + if (ht->commit_ordered) + need_commit_ordered= TRUE; + } + DBUG_EXECUTE_IF("crash_commit_after_prepare", DBUG_ABORT();); + + if (!is_real_trans) + { + error= commit_one_phase_2(thd, all, FALSE); + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + } + + /* + We can optimise away some of the thread synchronisation that may not be + needed. + + If need_prepare_ordered, then we need to take LOCK_prepare_ordered. + + If (xid && use_group_log_xid), then we need to enqueue (and this must + be done under LOCK_prepare_ordered if we take that lock). + + Similarly, if (need_prepare_ordered && need_commit_ordered), then we + need to enqueue under the LOCK_prepare_ordered. + + If (xid && use_group_log_xid), then we need to take LOCK_group_commit. + + If need_commit_ordered, then we need to take LOCK_commit_ordered. + + Cases not covered by above can be skipped to optimise things a bit. + */ + need_enqueue= (xid && tc_log->use_group_log_xid) || + (need_prepare_ordered && need_commit_ordered); + + thd->group_commit_ready= FALSE; + thd->group_commit_all= all; + if (need_prepare_ordered) + { + pthread_mutex_lock(&LOCK_prepare_ordered); + + for (Ha_trx_info *hi= ha_info; hi; hi= hi->next()) { - my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--read-only"); - ha_rollback_trans(thd, all); - error= 1; - goto end; + int err; + handlerton *ht= hi->ht(); + if (! hi->is_trx_read_write()) + continue; + if (ht->prepare_ordered && (err= ht->prepare_ordered(ht, thd, all))) + { + my_error(ER_ERROR_DURING_COMMIT, MYF(0), err); + pthread_mutex_unlock(&LOCK_prepare_ordered); + goto err; + } } + } + if (need_enqueue) + { + THD *previous_queue= enqueue_atomic(thd); + is_group_commit_leader= (previous_queue == NULL); + } + if (need_prepare_ordered) + pthread_mutex_unlock(&LOCK_prepare_ordered); - if (!trans->no_2pc && (rw_ha_count > 1)) + int cookie; + if (tc_log->use_group_log_xid) + { + if (is_group_commit_leader) { - for (; ha_info && !error; ha_info= ha_info->next()) + pthread_mutex_lock(&LOCK_group_commit); + group_commit_wait_queue_idle(); + + THD *queue= atomic_grab_reverse_queue(); + /* The first in the queue is the leader. */ + DBUG_ASSERT(queue == thd); + + /* + This will set individual error codes in each thd->xid_error, as + well as set thd->xid_cookie for later unlog() call. + */ + tc_log->group_log_xid(queue); + + /* + Call commit_ordered methods for all transactions in the queue + (that did not get an error in group_log_xid()). + + We do this under an additional global LOCK_commit_ordered; this is + so that transactions that do not need 2-phase commit do not have + to wait for the potentially long duration of LOCK_group_commit. + */ + if (need_commit_ordered) { - int err; - handlerton *ht= ha_info->ht(); - /* - Do not call two-phase commit if this particular - transaction is read-only. This allows for simpler - implementation in engines that are always read-only. - */ - if (! ha_info->is_trx_read_write()) - continue; - /* - Sic: we know that prepare() is not NULL since otherwise - trans->no_2pc would have been set. - */ - if ((err= ht->prepare(ht, thd, all))) + pthread_mutex_lock(&LOCK_commit_ordered); + for (THD *thd2= queue; thd2 != NULL; thd2= thd2->next_commit_ordered) { - my_error(ER_ERROR_DURING_COMMIT, MYF(0), err); - error= 1; + if (!queue->xid_error) + call_commit_ordered(ha_info, thd2, thd2->group_commit_all); } - status_var_increment(thd->status_var.ha_prepare_count); + pthread_mutex_unlock(&LOCK_commit_ordered); } - DBUG_EXECUTE_IF("crash_commit_after_prepare", DBUG_ABORT();); - if (error || (is_real_trans && xid && - (error= !(cookie= tc_log->log_xid(thd, xid))))) - { - ha_rollback_trans(thd, all); - error= 1; - goto end; - } - DBUG_EXECUTE_IF("crash_commit_after_log", DBUG_ABORT();); + pthread_mutex_unlock(&LOCK_group_commit); + + /* Wake up everyone except ourself. */ + while ((queue= queue->next_commit_ordered) != NULL) + group_commit_wakeup_other(queue); + } + else + { + /* If not leader, just wait until leader wakes us up. */ + group_commit_wait_for_wakeup(thd); + } + + /* + Now that we're back in our own thread context, do any delayed error + reporting. + */ + if (thd->xid_error) + { + tc_log->xid_delayed_error(thd); + goto err; + } + cookie= thd->xid_cookie; + /* The cookie must be non-zero in the non-error case. */ + DBUG_ASSERT(cookie); + } + else + { + if (xid) + cookie= tc_log->log_xid(thd, xid); + + if (!need_enqueue) + { + error= commit_one_phase_2(thd, all, TRUE); + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + } + + /* + We only get here to do correctly sequenced prepare_ordered and + commit_ordered() calls. + + In this case, we need to wait for the previous in queue to finish + commit_ordered before us to get the correct sequence. + */ + DBUG_ASSERT(need_prepare_ordered && need_commit_ordered); + + if (is_group_commit_leader) + { + pthread_mutex_lock(&LOCK_group_commit); + group_commit_wait_queue_idle(); + THD *queue= atomic_grab_reverse_queue(); + /* + Mark the queue busy while we bounce it from one thread to the + next. + */ + group_commit_mark_queue_busy(); + pthread_mutex_unlock(&LOCK_group_commit); + + /* The first in the queue is the leader. */ + DBUG_ASSERT(queue == thd); } - error=ha_commit_one_phase(thd, all) ? (cookie ? 2 : 1) : 0; - DBUG_EXECUTE_IF("crash_commit_before_unlog", DBUG_ABORT();); + else + { + /* If not leader, just wait until previous thread wakes us up. */ + group_commit_wait_for_wakeup(thd); + } + + /* Only run commit_ordered() if log_xid was successful. */ if (cookie) - tc_log->unlog(cookie, xid); - DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); -end: - if (rw_trans) - start_waiting_global_read_lock(thd); + { + pthread_mutex_lock(&LOCK_commit_ordered); + call_commit_ordered(ha_info, thd, all); + pthread_mutex_unlock(&LOCK_commit_ordered); + } + + THD *next= thd->next_commit_ordered; + if (next) + group_commit_wakeup_other(next); + else + group_commit_mark_queue_idle(); + + if (!cookie) + goto err; } - /* Free resources and perform other cleanup even for 'empty' transactions. */ - else if (is_real_trans) - thd->transaction.cleanup(); + + DBUG_EXECUTE_IF("crash_commit_after_log", DBUG_ABORT();); + + error= commit_one_phase_2(thd, all, FALSE) ? 2 : 0; + + DBUG_EXECUTE_IF("crash_commit_before_unlog", DBUG_ABORT();); + DBUG_ASSERT(cookie); + tc_log->unlog(cookie, xid); + + DBUG_EXECUTE_IF("crash_commit_after", DBUG_ABORT();); + goto end; + + /* Come here if error and we need to rollback. */ +err: + if (!error) + error= 1; + ha_rollback_trans(thd, all); + +end: + if (rw_trans) + start_waiting_global_read_lock(thd); #endif /* USING_TRANSACTIONS */ DBUG_RETURN(error); } @@ -1207,6 +1547,17 @@ end: */ int ha_commit_one_phase(THD *thd, bool all) { + /* + When we come here, we did not call handler commit_ordered() methods in + ha_commit_trans() 2-phase commit, so we pass TRUE to do it in + commit_one_phase_2(). + */ + return commit_one_phase_2(thd, all, TRUE); +} + +static int +commit_one_phase_2(THD *thd, bool all, bool do_commit_ordered) +{ int error=0; THD_TRANS *trans=all ? &thd->transaction.all : &thd->transaction.stmt; /* @@ -1218,10 +1569,40 @@ int ha_commit_one_phase(THD *thd, bool a */ bool is_real_trans=all || thd->transaction.all.ha_list == 0; Ha_trx_info *ha_info= trans->ha_list, *ha_info_next; - DBUG_ENTER("ha_commit_one_phase"); + DBUG_ENTER("commit_one_phase_2"); #ifdef USING_TRANSACTIONS if (ha_info) { + if (is_real_trans && do_commit_ordered) + { + /* + If we did not do it already, call any commit_ordered() method. + + Even though we do not need to keep any ordering with other threads + (as there is no prepare or log_xid for this commit), we still need to + do this under the LOCK_commit_ordered mutex to not run in parallel + with other commit_ordered calls. + */ + + bool locked= FALSE; + + for (Ha_trx_info *hi= ha_info; hi; hi= hi->next()) + { + handlerton *ht= hi->ht(); + if (ht->commit_ordered) + { + if (!locked) + { + pthread_mutex_lock(&LOCK_commit_ordered); + locked= 1; + } + ht->commit_ordered(ht, thd, all); + } + } + if (locked) + pthread_mutex_unlock(&LOCK_commit_ordered); + } + for (; ha_info; ha_info= ha_info_next) { int err; === modified file 'sql/handler.h' --- a/sql/handler.h 2010-01-14 16:51:00 +0000 +++ b/sql/handler.h 2010-05-26 08:13:32 +0000 @@ -656,9 +656,81 @@ struct handlerton NOTE 'all' is also false in auto-commit mode where 'end of statement' and 'real commit' mean the same event. */ - int (*commit)(handlerton *hton, THD *thd, bool all); + int (*commit)(handlerton *hton, THD *thd, bool all); + /* + The commit_ordered() method is called prior to the commit() method, after + the transaction manager has decided to commit (not rollback) the + transaction. + + The calls to commit_ordered() in multiple parallel transactions is + guaranteed to happen in the same order in every participating + handler. This can be used to ensure the same commit order among multiple + handlers (eg. in table handler and binlog). So if transaction T1 calls + into commit_ordered() of handler A before T2, then T1 will also call + commit_ordered() of handler B before T2. + + The intension is that commit_ordered() should do the minimal amount of + work that needs to happen in consistent commit order among handlers. To + preserve ordering, calls need to be serialised on a global mutex, so + doing any time-consuming or blocking operations in commit_ordered() will + limit scalability. + + Handlers can rely on commit_ordered() calls being serialised (no two + calls can run in parallel, so no extra locking on the handler part is + required to ensure this). + + Note that commit_ordered() can be called from a different thread than the + one handling the transaction! So it can not do anything that depends on + thread local storage, in particular it can not call my_error() and + friends (instead it can store the error code and delay the call to + my_error() to the commit() method). + + Similarly, since commit_ordered() returns void, any return error code + must be saved and returned from the commit() method instead. + + commit_ordered() is called only when actually committing a transaction + (autocommit or not), not when ending a statement in the middle of a + transaction. + + The commit_ordered method is optional, and can be left unset if not + needed in a particular handler. + */ + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); int (*rollback)(handlerton *hton, THD *thd, bool all); int (*prepare)(handlerton *hton, THD *thd, bool all); + /* + The prepare_ordered method is optional. If set, it will be called after + successful prepare() in all handlers participating in 2-phase commit. + + The calls to prepare_ordered() among multiple parallel transactions are + ordered consistently with calls to commit_ordered(). This means that + calls to prepare_ordered() effectively define the commit order, and that + each handler will see the same sequence of transactions calling into + prepare_ordered() and commit_ordered(). + + Thus, prepare_ordered() can be used to define commit order for handlers + that need to do this in the prepare step (like binlog). It can also be + used to release transactions locks early in an order consistent with the + order transactions will be eventually committed. + + Like commit_ordered(), prepare_ordered() calls are serialised to maintain + ordering, so the intension is that they should execute fast, with only + the minimal amount of work needed to define commit order. Handlers can + rely on this serialisation, and do not need to do any extra locking to + avoid two prepare_ordered() calls running in parallel. + + Unlike commit_ordered(), prepare_ordered() _is_ guaranteed to be called + in the context of the thread handling the rest of the transaction. + + Note that for user-level XA SQL commands, no consistent ordering among + prepare_ordered() and commit_ordered() is guaranteed (as that would + require blocking all other commits for an indefinite time). + + prepare_ordered() is called only when actually committing a transaction + (autocommit or not), not when ending a statement in the middle of a + transaction. + */ + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); int (*recover)(handlerton *hton, XID *xid_list, uint len); int (*commit_by_xid)(handlerton *hton, XID *xid); int (*rollback_by_xid)(handlerton *hton, XID *xid); === modified file 'sql/log.cc' --- a/sql/log.cc 2010-04-06 22:47:08 +0000 +++ b/sql/log.cc 2010-05-26 08:13:32 +0000 @@ -154,9 +154,12 @@ class binlog_trx_data { public: binlog_trx_data() : at_least_one_stmt_committed(0), incident(FALSE), m_pending(0), - before_stmt_pos(MY_OFF_T_UNDEF) + before_stmt_pos(MY_OFF_T_UNDEF), using_xa(0) { trans_log.end_of_file= max_binlog_cache_size; + (void) my_pthread_mutex_init(&LOCK_group_commit, MY_MUTEX_INIT_SLOW, + "LOCK_group_commit", MYF(0)); + (void) pthread_cond_init(&COND_group_commit, 0); } ~binlog_trx_data() @@ -208,11 +211,12 @@ public: completely. */ void reset() { - if (!empty()) + if (trans_log.type != WRITE_CACHE || !empty()) truncate(0); before_stmt_pos= MY_OFF_T_UNDEF; incident= FALSE; trans_log.end_of_file= max_binlog_cache_size; + using_xa= FALSE; DBUG_ASSERT(empty()); } @@ -257,6 +261,41 @@ public: Binlog position before the start of the current statement. */ my_off_t before_stmt_pos; + + /* 0 or error when writing to binlog; set during group commit. */ + int error; + /* If error != 0, value of errno (for my_error() reporting). */ + int commit_errno; + /* Link for queueing transactions up for group commit to binlog. */ + binlog_trx_data *next; + /* + Flag set true when group commit for this transaction is finished; used + with pthread_cond_wait() to wait until commit is done. + This flag is protected by LOCK_group_commit. + */ + bool done; + /* + Flag set if this transaction is the group commit leader that will handle + the actual writing to the binlog. + This flag is protected by LOCK_group_commit. + */ + bool group_commit_leader; + /* + Flag set true if this transaction is committed with log_xid() as part of + XA, false if not. + */ + bool using_xa; + /* + Extra events (BEGIN, COMMIT/ROLLBACK/XID, and possibly INCIDENT) to be + written during group commit. The incident_event is only valid if + has_incident() is true. + */ + Log_event *begin_event; + Log_event *end_event; + Log_event *incident_event; + /* Mutex and condition for wakeup after group commit. */ + pthread_mutex_t LOCK_group_commit; + pthread_cond_t COND_group_commit; }; handlerton *binlog_hton; @@ -1391,117 +1430,188 @@ static int binlog_close_connection(handl return 0; } +/* Helper functions for binlog_flush_trx_cache(). */ +static int +binlog_flush_trx_cache_prepare(THD *thd) +{ + if (thd->binlog_flush_pending_rows_event(TRUE)) + return 1; + return 0; +} + +static void +binlog_flush_trx_cache_finish(THD *thd, binlog_trx_data *trx_data) +{ + IO_CACHE *trans_log= &trx_data->trans_log; + + trx_data->reset(); + + statistic_increment(binlog_cache_use, &LOCK_status); + if (trans_log->disk_writes != 0) + { + statistic_increment(binlog_cache_disk_use, &LOCK_status); + trans_log->disk_writes= 0; + } +} + /* - End a transaction. + End a transaction, writing events to the binary log. SYNOPSIS - binlog_end_trans() + binlog_flush_trx_cache() thd The thread whose transaction should be ended trx_data Pointer to the transaction data to use - end_ev The end event to use, or NULL - all True if the entire transaction should be ended, false if - only the statement transaction should be ended. + end_ev The end event to use (COMMIT, ROLLBACK, or commit XID) DESCRIPTION End the currently open transaction. The transaction can be either - a real transaction (if 'all' is true) or a statement transaction - (if 'all' is false). + a real transaction or a statement transaction. - If 'end_ev' is NULL, the transaction is a rollback of only - transactional tables, so the transaction cache will be truncated - to either just before the last opened statement transaction (if - 'all' is false), or reset completely (if 'all' is true). + This can be to commit a transaction, with a COMMIT query event or an XA + commit XID event. But it can also be to rollback a transaction with a + ROLLBACK query event, used for rolling back transactions which also + contain updates to non-transactional tables. */ static int -binlog_end_trans(THD *thd, binlog_trx_data *trx_data, - Log_event *end_ev, bool all) +binlog_flush_trx_cache(THD *thd, binlog_trx_data *trx_data, + Log_event *end_ev) { - DBUG_ENTER("binlog_end_trans"); - int error=0; - IO_CACHE *trans_log= &trx_data->trans_log; - DBUG_PRINT("enter", ("transaction: %s end_ev: 0x%lx", - all ? "all" : "stmt", (long) end_ev)); + DBUG_ENTER("binlog_flush_trx_cache"); DBUG_PRINT("info", ("thd->options={ %s%s}", FLAGSTR(thd->options, OPTION_NOT_AUTOCOMMIT), FLAGSTR(thd->options, OPTION_BEGIN))); + if (binlog_flush_trx_cache_prepare(thd)) + DBUG_RETURN(1); + /* - NULL denotes ROLLBACK with nothing to replicate: i.e., rollback of - only transactional tables. If the transaction contain changes to - any non-transactiona tables, we need write the transaction and log - a ROLLBACK last. - */ - if (end_ev != NULL) - { - if (thd->binlog_flush_pending_rows_event(TRUE)) - DBUG_RETURN(1); - /* - Doing a commit or a rollback including non-transactional tables, - i.e., ending a transaction where we might write the transaction - cache to the binary log. - - We can always end the statement when ending a transaction since - transactions are not allowed inside stored functions. If they - were, we would have to ensure that we're not ending a statement - inside a stored function. - */ - error= mysql_bin_log.write(thd, &trx_data->trans_log, end_ev, - trx_data->has_incident()); - trx_data->reset(); + Doing a commit or a rollback including non-transactional tables, + i.e., ending a transaction where we might write the transaction + cache to the binary log. + + We can always end the statement when ending a transaction since + transactions are not allowed inside stored functions. If they + were, we would have to ensure that we're not ending a statement + inside a stored function. + */ + int error= mysql_bin_log.write_transaction_to_binlog(thd, trx_data, end_ev); - /* - We need to step the table map version after writing the - transaction cache to disk. - */ - mysql_bin_log.update_table_map_version(); - statistic_increment(binlog_cache_use, &LOCK_status); - if (trans_log->disk_writes != 0) - { - statistic_increment(binlog_cache_disk_use, &LOCK_status); - trans_log->disk_writes= 0; - } - } - else - { - /* - If rolling back an entire transaction or a single statement not - inside a transaction, we reset the transaction cache. + binlog_flush_trx_cache_finish(thd, trx_data); - If rolling back a statement in a transaction, we truncate the - transaction cache to remove the statement. - */ - thd->binlog_remove_pending_rows_event(TRUE); - if (all || !(thd->options & (OPTION_BEGIN | OPTION_NOT_AUTOCOMMIT))) - { - if (trx_data->has_incident()) - error= mysql_bin_log.write_incident(thd, TRUE); - trx_data->reset(); - } - else // ...statement - trx_data->truncate(trx_data->before_stmt_pos); + DBUG_ASSERT(thd->binlog_get_pending_rows_event() == NULL); + DBUG_RETURN(error); +} - /* - We need to step the table map version on a rollback to ensure - that a new table map event is generated instead of the one that - was written to the thrown-away transaction cache. - */ - mysql_bin_log.update_table_map_version(); +/* + Discard a transaction, ie. ROLLBACK with only transactional table updates. + + SYNOPSIS + binlog_truncate_trx_cache() + + thd The thread whose transaction should be ended + trx_data Pointer to the transaction data to use + all True if the entire transaction should be ended, false if + only the statement transaction should be ended. + + DESCRIPTION + + Rollback (and end) a transaction that only modifies transactional + tables. The transaction can be either a real transaction (if 'all' is + true) or a statement transaction (if 'all' is false). + + The transaction cache will be truncated to either just before the last + opened statement transaction (if 'all' is false), or reset completely (if + 'all' is true). + */ +static int +binlog_truncate_trx_cache(THD *thd, binlog_trx_data *trx_data, bool all) +{ + DBUG_ENTER("binlog_truncate_trx_cache"); + int error= 0; + DBUG_PRINT("enter", ("transaction: %s", all ? "all" : "stmt")); + DBUG_PRINT("info", ("thd->options={ %s%s}", + FLAGSTR(thd->options, OPTION_NOT_AUTOCOMMIT), + FLAGSTR(thd->options, OPTION_BEGIN))); + + /* + ROLLBACK with nothing to replicate: i.e., rollback of only transactional + tables. + */ + + /* + If rolling back an entire transaction or a single statement not + inside a transaction, we reset the transaction cache. + + If rolling back a statement in a transaction, we truncate the + transaction cache to remove the statement. + */ + thd->binlog_remove_pending_rows_event(TRUE); + if (all || !(thd->options & (OPTION_BEGIN | OPTION_NOT_AUTOCOMMIT))) + { + if (trx_data->has_incident()) + error= mysql_bin_log.write_incident(thd); + trx_data->reset(); } + else // ...statement + trx_data->truncate(trx_data->before_stmt_pos); DBUG_ASSERT(thd->binlog_get_pending_rows_event() == NULL); DBUG_RETURN(error); } +static LEX_STRING const write_error_msg= + { C_STRING_WITH_LEN("error writing to the binary log") }; + static int binlog_prepare(handlerton *hton, THD *thd, bool all) { /* - do nothing. - just pretend we can do 2pc, so that MySQL won't - switch to 1pc. - real work will be done in MYSQL_BIN_LOG::log_xid() + If this prepare is for a single statement in the middle of a transactions, + not the actual transaction commit, then we do nothing. The real work is + only done later, in the prepare for making persistent changes. */ + if (!all && (thd->options & (OPTION_BEGIN | OPTION_NOT_AUTOCOMMIT))) + return 0; + + binlog_trx_data *trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + + trx_data->using_xa= TRUE; + + if (binlog_flush_trx_cache_prepare(thd)) + return 1; + + my_xid xid= thd->transaction.xid_state.xid.get_my_xid(); + if (!xid) + { + /* Skip logging this transaction, marked by setting end_event to NULL. */ + trx_data->end_event= NULL; + return 0; + } + + /* + Allocate the extra events that will be logged to the binlog in binlog group + commit. Use placement new to allocate them on the THD memroot, as they need + to remain live until log_xid() returns. + */ + size_t needed_size= sizeof(Query_log_event) + sizeof(Xid_log_event); + if (trx_data->has_incident()) + needed_size+= sizeof(Incident_log_event); + uchar *mem= (uchar *)thd->alloc(needed_size); + if (!mem) + return 1; + + trx_data->begin_event= new ((void *)mem) + Query_log_event(thd, STRING_WITH_LEN("BEGIN"), TRUE, TRUE, 0); + mem+= sizeof(Query_log_event); + + trx_data->end_event= new ((void *)mem) Xid_log_event(thd, xid); + + if (trx_data->has_incident()) + trx_data->incident_event= new ((void *)(mem + sizeof(Xid_log_event))) + Incident_log_event(thd, INCIDENT_LOST_EVENTS, write_error_msg); + return 0; } @@ -1525,11 +1635,11 @@ static int binlog_commit(handlerton *hto binlog_trx_data *const trx_data= (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); - if (trx_data->empty()) + if (trx_data->using_xa) { // we're here because trans_log was flushed in MYSQL_BIN_LOG::log_xid() - trx_data->reset(); - DBUG_RETURN(0); + binlog_flush_trx_cache_finish(thd, trx_data); + DBUG_RETURN(error); } /* @@ -1556,8 +1666,8 @@ static int binlog_commit(handlerton *hto !stmt_has_updated_trans_table(thd) && thd->transaction.stmt.modified_non_trans_table)) { - Query_log_event qev(thd, STRING_WITH_LEN("COMMIT"), TRUE, TRUE, 0); - error= binlog_end_trans(thd, trx_data, &qev, all); + Query_log_event end_ev(thd, STRING_WITH_LEN("COMMIT"), TRUE, TRUE, 0); + error= binlog_flush_trx_cache(thd, trx_data, &end_ev); } trx_data->at_least_one_stmt_committed = my_b_tell(&trx_data->trans_log) > 0; @@ -1621,7 +1731,7 @@ static int binlog_rollback(handlerton *h (thd->options & OPTION_KEEP_LOG)) && mysql_bin_log.check_write_error(thd)) trx_data->set_incident(); - error= binlog_end_trans(thd, trx_data, 0, all); + error= binlog_truncate_trx_cache(thd, trx_data, all); } else { @@ -1641,8 +1751,8 @@ static int binlog_rollback(handlerton *h thd->current_stmt_binlog_row_based) || ((thd->options & OPTION_KEEP_LOG))) { - Query_log_event qev(thd, STRING_WITH_LEN("ROLLBACK"), TRUE, TRUE, 0); - error= binlog_end_trans(thd, trx_data, &qev, all); + Query_log_event end_ev(thd, STRING_WITH_LEN("ROLLBACK"), TRUE, TRUE, 0); + error= binlog_flush_trx_cache(thd, trx_data, &end_ev); } /* Otherwise, we simply truncate the cache as there is no change on @@ -1650,7 +1760,7 @@ static int binlog_rollback(handlerton *h */ else if ((all && !thd->transaction.all.modified_non_trans_table) || (!all && !thd->transaction.stmt.modified_non_trans_table)) - error= binlog_end_trans(thd, trx_data, 0, all); + error= binlog_truncate_trx_cache(thd, trx_data, all); } if (!all) trx_data->before_stmt_pos = MY_OFF_T_UNDEF; // part of the stmt rollback @@ -2464,7 +2574,7 @@ const char *MYSQL_LOG::generate_name(con MYSQL_BIN_LOG::MYSQL_BIN_LOG() :bytes_written(0), prepared_xids(0), file_id(1), open_count(1), - need_start_event(TRUE), m_table_map_version(0), + need_start_event(TRUE), is_relay_log(0), description_event_for_exec(0), description_event_for_queue(0) { @@ -2477,6 +2587,7 @@ MYSQL_BIN_LOG::MYSQL_BIN_LOG() index_file_name[0] = 0; bzero((char*) &index_file, sizeof(index_file)); bzero((char*) &purge_index_file, sizeof(purge_index_file)); + use_group_log_xid= TRUE; } /* this is called only once */ @@ -2492,6 +2603,7 @@ void MYSQL_BIN_LOG::cleanup() delete description_event_for_exec; (void) pthread_mutex_destroy(&LOCK_log); (void) pthread_mutex_destroy(&LOCK_index); + (void) pthread_mutex_destroy(&LOCK_queue); (void) pthread_cond_destroy(&update_cond); } DBUG_VOID_RETURN; @@ -2520,6 +2632,8 @@ void MYSQL_BIN_LOG::init_pthread_objects */ (void) my_pthread_mutex_init(&LOCK_index, MY_MUTEX_INIT_SLOW, "LOCK_index", MYF_NO_DEADLOCK_DETECTION); + (void) my_pthread_mutex_init(&LOCK_queue, MY_MUTEX_INIT_FAST, "LOCK_queue", + MYF(0)); (void) pthread_cond_init(&update_cond, 0); } @@ -4113,7 +4227,6 @@ int THD::binlog_write_table_map(TABLE *t DBUG_RETURN(error); binlog_table_maps++; - table->s->table_map_version= mysql_bin_log.table_map_version(); DBUG_RETURN(0); } @@ -4194,64 +4307,41 @@ MYSQL_BIN_LOG::flush_and_set_pending_row if (Rows_log_event* pending= trx_data->pending()) { - IO_CACHE *file= &log_file; - /* Decide if we should write to the log file directly or to the transaction log. */ if (pending->get_cache_stmt() || my_b_tell(&trx_data->trans_log)) - file= &trx_data->trans_log; - - /* - If we are writing to the log file directly, we could avoid - locking the log. This does not work since we need to step the - m_table_map_version below, and that change has to be protected - by the LOCK_log mutex. - */ - pthread_mutex_lock(&LOCK_log); - - /* - Write pending event to log file or transaction cache - */ - if (pending->write(file)) { - pthread_mutex_unlock(&LOCK_log); - set_write_error(thd); - DBUG_RETURN(1); + /* Write to transaction log/cache. */ + if (pending->write(&trx_data->trans_log)) + { + set_write_error(thd); + DBUG_RETURN(1); + } } - - /* - We step the table map version if we are writing an event - representing the end of a statement. We do this regardless of - wheather we write to the transaction cache or to directly to the - file. - - In an ideal world, we could avoid stepping the table map version - if we were writing to a transaction cache, since we could then - reuse the table map that was written earlier in the transaction - cache. This does not work since STMT_END_F implies closing all - table mappings on the slave side. - - TODO: Find a solution so that table maps does not have to be - written several times within a transaction. - */ - if (pending->get_flags(Rows_log_event::STMT_END_F)) - ++m_table_map_version; - - delete pending; - - if (file == &log_file) + else { + /* Write directly to log file. */ + pthread_mutex_lock(&LOCK_log); + if (pending->write(&log_file)) + { + pthread_mutex_unlock(&LOCK_log); + set_write_error(thd); + DBUG_RETURN(1); + } + error= flush_and_sync(); if (!error) { signal_update(); rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); } + + pthread_mutex_unlock(&LOCK_log); } - pthread_mutex_unlock(&LOCK_log); + delete pending; } thd->binlog_set_pending_rows_event(event); @@ -4450,9 +4540,6 @@ err: set_write_error(thd); } - if (event_info->flags & LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F) - ++m_table_map_version; - pthread_mutex_unlock(&LOCK_log); DBUG_RETURN(error); } @@ -4575,18 +4662,14 @@ uint MYSQL_BIN_LOG::next_file_id() SYNOPSIS write_cache() cache Cache to write to the binary log - lock_log True if the LOCK_log mutex should be aquired, false otherwise - sync_log True if the log should be flushed and sync:ed DESCRIPTION Write the contents of the cache to the binary log. The cache will be reset as a READ_CACHE to be able to read the contents from it. */ -int MYSQL_BIN_LOG::write_cache(IO_CACHE *cache, bool lock_log, bool sync_log) +int MYSQL_BIN_LOG::write_cache(IO_CACHE *cache) { - Mutex_sentry sentry(lock_log ? &LOCK_log : NULL); - if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0)) return ER_ERROR_ON_WRITE; uint length= my_b_bytes_in_cache(cache), group, carry, hdr_offs; @@ -4697,6 +4780,7 @@ int MYSQL_BIN_LOG::write_cache(IO_CACHE } /* Write data to the binary log file */ + DBUG_EXECUTE_IF("fail_binlog_write_1", return ER_ERROR_ON_WRITE;); if (my_b_write(&log_file, cache->read_pos, length)) return ER_ERROR_ON_WRITE; cache->read_pos=cache->read_end; // Mark buffer used up @@ -4704,9 +4788,6 @@ int MYSQL_BIN_LOG::write_cache(IO_CACHE DBUG_ASSERT(carry == 0); - if (sync_log) - flush_and_sync(); - return 0; // All OK } @@ -4739,26 +4820,22 @@ int query_error_code(THD *thd, bool not_ return error; } -bool MYSQL_BIN_LOG::write_incident(THD *thd, bool lock) +bool MYSQL_BIN_LOG::write_incident(THD *thd) { uint error= 0; DBUG_ENTER("MYSQL_BIN_LOG::write_incident"); - LEX_STRING const write_error_msg= - { C_STRING_WITH_LEN("error writing to the binary log") }; Incident incident= INCIDENT_LOST_EVENTS; Incident_log_event ev(thd, incident, write_error_msg); - if (lock) - pthread_mutex_lock(&LOCK_log); + + pthread_mutex_lock(&LOCK_log); error= ev.write(&log_file); - if (lock) + if (!error && !(error= flush_and_sync())) { - if (!error && !(error= flush_and_sync())) - { - signal_update(); - rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); - } - pthread_mutex_unlock(&LOCK_log); + signal_update(); + rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); } + pthread_mutex_unlock(&LOCK_log); + DBUG_RETURN(error); } @@ -4786,103 +4863,364 @@ bool MYSQL_BIN_LOG::write_incident(THD * 'cache' needs to be reinitialized after this functions returns. */ -bool MYSQL_BIN_LOG::write(THD *thd, IO_CACHE *cache, Log_event *commit_event, - bool incident) +bool +MYSQL_BIN_LOG::write_transaction_to_binlog(THD *thd, binlog_trx_data *trx_data, + Log_event *end_ev) { - DBUG_ENTER("MYSQL_BIN_LOG::write(THD *, IO_CACHE *, Log_event *)"); + DBUG_ENTER("MYSQL_BIN_LOG::write_transaction_to_binlog"); + + /* + Create the necessary events here, where we have the correct THD (and + thread context). + + Due to group commit the actual writing to binlog may happen in a different + thread. + */ + Query_log_event qinfo(thd, STRING_WITH_LEN("BEGIN"), TRUE, TRUE, 0); + trx_data->begin_event= &qinfo; + trx_data->end_event= end_ev; + if (trx_data->has_incident()) + { + Incident_log_event inc_ev(thd, INCIDENT_LOST_EVENTS, write_error_msg); + trx_data->incident_event= &inc_ev; + DBUG_RETURN(write_transaction_to_binlog_events(trx_data)); + } + else + { + trx_data->incident_event= NULL; + DBUG_RETURN(write_transaction_to_binlog_events(trx_data)); + } +} + +bool +MYSQL_BIN_LOG::write_transaction_to_binlog_events(binlog_trx_data *trx_data) +{ + /* + To facilitate group commit for the binlog, we first queue up ourselves in + the group commit queue. Then the first thread to enter the queue waits for + the LOCK_log mutex, and commits for everyone in the queue once it gets the + lock. Any other threads in the queue just wait for the first one to finish + the commit and wake them up. + */ + + pthread_mutex_lock(&trx_data->LOCK_group_commit); + const binlog_trx_data *orig_queue= atomic_enqueue_trx(trx_data); + + if (orig_queue != NULL) + { + trx_data->group_commit_leader= FALSE; + trx_data->done= FALSE; + trx_group_commit_participant(trx_data); + } + else + { + trx_data->group_commit_leader= TRUE; + pthread_mutex_unlock(&trx_data->LOCK_group_commit); + trx_group_commit_leader(NULL); + } + + return trx_group_commit_finish(trx_data); +} + +/* + Participate as secondary transaction in group commit. + + Another thread is already waiting to obtain the LOCK_log, and should include + this thread in the group commit once the log is obtained. So here we put + ourself in the queue and wait to be signalled that the group commit is done. + + Note that this function must be called with the trs_data->LOCK_group_commit + locked; the mutex will be released before return. +*/ +void +MYSQL_BIN_LOG::trx_group_commit_participant(binlog_trx_data *trx_data) +{ + safe_mutex_assert_owner(&trx_data->LOCK_group_commit); + + /* Wait until trx_data.done == true and woken up by the leader. */ + while (!trx_data->done) + pthread_cond_wait(&trx_data->COND_group_commit, + &trx_data->LOCK_group_commit); + pthread_mutex_unlock(&trx_data->LOCK_group_commit); +} + +bool +MYSQL_BIN_LOG::trx_group_commit_finish(binlog_trx_data *trx_data) +{ + if (trx_data->error) + { + switch (trx_data->error) + { + case ER_ERROR_ON_WRITE: + my_error(ER_ERROR_ON_WRITE, MYF(ME_NOREFRESH), name, trx_data->commit_errno); + break; + case ER_ERROR_ON_READ: + my_error(ER_ERROR_ON_READ, MYF(ME_NOREFRESH), + trx_data->trans_log.file_name, trx_data->commit_errno); + break; + default: + /* + There are not (and should not be) any errors thrown not covered above. + But just in case one is added later without updating the above switch + statement, include a catch-all. + */ + my_printf_error(trx_data->error, + "Error writing transaction to binary log: %d", + MYF(ME_NOREFRESH), trx_data->error); + } + + /* + Since we return error, this transaction XID will not be committed, so + we need to mark it as not needed for recovery (unlog() is not called + for a transaction if log_xid() fails). + */ + if (trx_data->end_event->get_type_code() == XID_EVENT) + mark_xid_done(); + + return 1; + } + + return 0; +} + +/* + Do binlog group commit as the lead thread. + + This must be called when this thread/transaction is queued at the start of + the group_commit_queue. It will wait to obtain the LOCK_log mutex, then group + commit all the transactions in the queue (more may have entered while waiting + for LOCK_log). After commit is done, all other threads in the queue will be + signalled. + + */ +void +MYSQL_BIN_LOG::trx_group_commit_leader(THD *first_thd) +{ + uint xid_count= 0; + uint write_count= 0; + + /* First, put anything from group_log_xid into the queue. */ + binlog_trx_data *full_queue= NULL; + binlog_trx_data **next_ptr= &full_queue; + for (THD *thd= first_thd; thd; thd= thd->next_commit_ordered) + { + binlog_trx_data *const trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + + /* Skip log_xid for transactions without xid, marked by NULL end_event. */ + if (!trx_data->end_event) + continue; + + trx_data->error= 0; + *next_ptr= trx_data; + next_ptr= &(trx_data->next); + } + + /* + Next, lock the LOCK_log(), and once we get it, add any additional writes + that queued up while we were waiting. + + Note that if some writer not going through log_xid() comes in and gets the + LOCK_log before us, they will not be able to include us in their group + commit (and they are not able to handle ensuring same commit order between + us and participating transactional storage engines anyway). + + On the other hand, when we get the LOCK_log, we will be able to include + any non-trasactional writes that queued up in our group commit. This + should hopefully not be too big of a problem, as group commit is most + important for the transactional case anyway when durability (fsync) is + enabled. + */ VOID(pthread_mutex_lock(&LOCK_log)); - /* NULL would represent nothing to replicate after ROLLBACK */ - DBUG_ASSERT(commit_event != NULL); + /* + As the queue is in reverse order of entering, reverse the queue as we add + it to the existing one. Note that there is no ordering defined between + transactional and non-transactional commits. + */ + binlog_trx_data *current= atomic_grab_trx_queue(); + binlog_trx_data *xtra_queue= NULL; + while (current) + { + current->error= 0; + binlog_trx_data *next= current->next; + current->next= xtra_queue; + xtra_queue= current; + current= next; + } + *next_ptr= xtra_queue; + /* + Now we have in full_queue the list of transactions to be committed in + order. + */ DBUG_ASSERT(is_open()); if (likely(is_open())) // Should always be true { /* - We only bother to write to the binary log if there is anything - to write. - */ - if (my_b_tell(cache) > 0) + Commit every transaction in the queue. + + Note that we are doing this in a different thread than the one running + the transaction! So we are limited in the operations we can do. In + particular, we cannot call my_error() on behalf of a transaction, as + that obtains the THD from thread local storage. Instead, we must set + current->error and let the thread do the error reporting itself once + we wake it up. + */ + for (current= full_queue; current != NULL; current= current->next) { - /* - Log "BEGIN" at the beginning of every transaction. Here, a - transaction is either a BEGIN..COMMIT block or a single - statement in autocommit mode. - */ - Query_log_event qinfo(thd, STRING_WITH_LEN("BEGIN"), TRUE, TRUE, 0); + IO_CACHE *cache= &current->trans_log; /* - Now this Query_log_event has artificial log_pos 0. It must be - adjusted to reflect the real position in the log. Not doing it - would confuse the slave: it would prevent this one from - knowing where he is in the master's binlog, which would result - in wrong positions being shown to the user, MASTER_POS_WAIT - undue waiting etc. + We only bother to write to the binary log if there is anything + to write. */ - if (qinfo.write(&log_file)) - goto err; - - DBUG_EXECUTE_IF("crash_before_writing_xid", - { - if ((write_error= write_cache(cache, false, true))) - DBUG_PRINT("info", ("error writing binlog cache: %d", - write_error)); - DBUG_PRINT("info", ("crashing before writing xid")); - abort(); - }); - - if ((write_error= write_cache(cache, false, false))) - goto err; + if (my_b_tell(cache) > 0) + { + current->error= write_transaction(current); + if (current->error) + current->commit_errno= errno; - if (commit_event && commit_event->write(&log_file)) - goto err; + write_count++; + } - if (incident && write_incident(thd, FALSE)) - goto err; + if (current->end_event->get_type_code() == XID_EVENT) + xid_count++; + } + if (write_count > 0) + { if (flush_and_sync()) - goto err; - DBUG_EXECUTE_IF("half_binlogged_transaction", DBUG_ABORT();); - if (cache->error) // Error on read { - sql_print_error(ER(ER_ERROR_ON_READ), cache->file_name, errno); - write_error=1; // Don't give more errors - goto err; + for (current= full_queue; current != NULL; current= current->next) + { + if (!current->error) + { + current->error= ER_ERROR_ON_WRITE; + current->commit_errno= errno; + } + } + } + else + { + signal_update(); } - signal_update(); } /* - if commit_event is Xid_log_event, increase the number of + if any commit_events are Xid_log_event, increase the number of prepared_xids (it's decreasd in ::unlog()). Binlog cannot be rotated if there're prepared xids in it - see the comment in new_file() for an explanation. - If the commit_event is not Xid_log_event (then it's a Query_log_event) - rotate binlog, if necessary. + If no Xid_log_events (then it's all Query_log_event) rotate binlog, + if necessary. */ - if (commit_event && commit_event->get_type_code() == XID_EVENT) + if (xid_count > 0) { - pthread_mutex_lock(&LOCK_prep_xids); - prepared_xids++; - pthread_mutex_unlock(&LOCK_prep_xids); + mark_xids_active(xid_count); } else rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED); } + VOID(pthread_mutex_unlock(&LOCK_log)); - DBUG_RETURN(0); + /* + Signal those that are not part of group_log_xid, and are not group leaders + running the queue. -err: - if (!write_error) + Since a group leader runs the queue itself if a group_log_xid does not get + to do it forst, such leader threads do not need wait or wakeup. + */ + for (current= xtra_queue; current != NULL; current= current->next) { - write_error= 1; - sql_print_error(ER(ER_ERROR_ON_WRITE), name, errno); + /* + Note that we need to take LOCK_group_commit even in the case of a leader! + + Otherwise there is a race between setting and testing the + group_commit_leader flag. + */ + pthread_mutex_lock(&current->LOCK_group_commit); + if (!current->group_commit_leader) + { + current->done= true; + pthread_cond_signal(&current->COND_group_commit); + } + pthread_mutex_unlock(&current->LOCK_group_commit); } - VOID(pthread_mutex_unlock(&LOCK_log)); - DBUG_RETURN(1); } +int +MYSQL_BIN_LOG::write_transaction(binlog_trx_data *trx_data) +{ + IO_CACHE *cache= &trx_data->trans_log; + /* + Log "BEGIN" at the beginning of every transaction. Here, a transaction is + either a BEGIN..COMMIT block or a single statement in autocommit mode. The + event was constructed in write_transaction_to_binlog(), in the thread + running the transaction. + + Now this Query_log_event has artificial log_pos 0. It must be + adjusted to reflect the real position in the log. Not doing it + would confuse the slave: it would prevent this one from + knowing where he is in the master's binlog, which would result + in wrong positions being shown to the user, MASTER_POS_WAIT + undue waiting etc. + */ + if (trx_data->begin_event->write(&log_file)) + return ER_ERROR_ON_WRITE; + + DBUG_EXECUTE_IF("crash_before_writing_xid", + { + if ((write_cache(cache))) + DBUG_PRINT("info", ("error writing binlog cache")); + else + flush_and_sync(); + + DBUG_PRINT("info", ("crashing before writing xid")); + abort(); + }); + + if (write_cache(cache)) + return ER_ERROR_ON_WRITE; + + if (trx_data->end_event->write(&log_file)) + return ER_ERROR_ON_WRITE; + + if (trx_data->has_incident() && trx_data->incident_event->write(&log_file)) + return ER_ERROR_ON_WRITE; + + if (cache->error) // Error on read + return ER_ERROR_ON_READ; + + return 0; +} + +binlog_trx_data * +MYSQL_BIN_LOG::atomic_enqueue_trx(binlog_trx_data *trx_data) +{ + my_atomic_rwlock_wrlock(&LOCK_queue); + trx_data->next= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&trx_data->next), + trx_data)) + ; + my_atomic_rwlock_wrunlock(&LOCK_queue); + return trx_data->next; +} + +binlog_trx_data * +MYSQL_BIN_LOG::atomic_grab_trx_queue() +{ + my_atomic_rwlock_wrlock(&LOCK_queue); + binlog_trx_data *queue= group_commit_queue; + while (!my_atomic_casptr((void **)(&group_commit_queue), + (void **)(&queue), + NULL)) + ; + my_atomic_rwlock_wrunlock(&LOCK_queue); + return queue; +} /** Wait until we get a signal that the binary log has been updated. @@ -5879,9 +6217,6 @@ void TC_LOG_BINLOG::close() } /** - @todo - group commit - @retval 0 error @retval @@ -5889,19 +6224,83 @@ void TC_LOG_BINLOG::close() */ int TC_LOG_BINLOG::log_xid(THD *thd, my_xid xid) { - DBUG_ENTER("TC_LOG_BINLOG::log"); - Xid_log_event xle(thd, xid); - binlog_trx_data *trx_data= - (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + int error; + DBUG_ENTER("TC_LOG_BINLOG::log_xid"); + + thd->next_commit_ordered= 0; + group_log_xid(thd); + if (thd->xid_error) + error= xid_delayed_error(thd); + else + error= 0; + /* - We always commit the entire transaction when writing an XID. Also - note that the return value is inverted. - */ - DBUG_RETURN(!binlog_end_trans(thd, trx_data, &xle, TRUE)); + Note that the return value is inverted: zero on failure, private non-zero + 'cookie' on success. + */ + DBUG_RETURN(!error); } -void TC_LOG_BINLOG::unlog(ulong cookie, my_xid xid) +/* + Do a binlog log_xid() for a group of transactions, linked through + thd->next_commit_ordered. +*/ +void +TC_LOG_BINLOG::group_log_xid(THD *first_thd) +{ + DBUG_ENTER("TC_LOG_BINLOG::group_log_xid"); + trx_group_commit_leader(first_thd); + for (THD *thd= first_thd; thd; thd= thd->next_commit_ordered) + { + binlog_trx_data *const trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + thd->xid_error= trx_data->error; + thd->xid_cookie= !trx_data->error; + } + DBUG_VOID_RETURN; +} + +int +TC_LOG_BINLOG::xid_delayed_error(THD *thd) { + binlog_trx_data *const trx_data= + (binlog_trx_data*) thd_get_ha_data(thd, binlog_hton); + return trx_group_commit_finish(trx_data); +} + +/* + After an XID is logged, we need to hold on to the current binlog file until + it is fully committed in the storage engine. The reason is that crash + recovery only looks at the latest binlog, so we must make sure there are no + outstanding prepared (but not committed) transactions before rotating the + binlog. + + To handle this, we keep a count of outstanding XIDs. This function is used + to increase this count when committing one or more transactions to the + binary log. +*/ +void +TC_LOG_BINLOG::mark_xids_active(uint xid_count) +{ + DBUG_ENTER("TC_LOG_BINLOG::mark_xids_active"); + DBUG_PRINT("info", ("xid_count=%u", xid_count)); + pthread_mutex_lock(&LOCK_prep_xids); + prepared_xids+= xid_count; + pthread_mutex_unlock(&LOCK_prep_xids); + DBUG_VOID_RETURN; +} + +/* + Once an XID is committed, it is safe to rotate the binary log, as it can no + longer be needed during crash recovery. + + This function is called to mark an XID this way. It needs to decrease the + count of pending XIDs, and signal the log rotator thread when it reaches zero. +*/ +void +TC_LOG_BINLOG::mark_xid_done() +{ + DBUG_ENTER("TC_LOG_BINLOG::mark_xid_done"); pthread_mutex_lock(&LOCK_prep_xids); DBUG_ASSERT(prepared_xids > 0); if (--prepared_xids == 0) { @@ -5909,7 +6308,16 @@ void TC_LOG_BINLOG::unlog(ulong cookie, pthread_cond_signal(&COND_prep_xids); } pthread_mutex_unlock(&LOCK_prep_xids); - rotate_and_purge(0); // as ::write() did not rotate + DBUG_VOID_RETURN; +} + +void TC_LOG_BINLOG::unlog(ulong cookie, my_xid xid) +{ + DBUG_ENTER("TC_LOG_BINLOG::unlog"); + if (xid) + mark_xid_done(); + rotate_and_purge(0); // as ::write_transaction_to_binlog() did not rotate + DBUG_VOID_RETURN; } int TC_LOG_BINLOG::recover(IO_CACHE *log, Format_description_log_event *fdle) === modified file 'sql/log.h' --- a/sql/log.h 2009-12-04 14:40:42 +0000 +++ b/sql/log.h 2010-05-26 08:13:32 +0000 @@ -28,13 +28,49 @@ class TC_LOG { public: int using_heuristic_recover(); - TC_LOG() {} + /* True if we should use group_log_xid(), false to use log_xid(). */ + bool use_group_log_xid; + + TC_LOG() : use_group_log_xid(0) {} virtual ~TC_LOG() {} virtual int open(const char *opt_name)=0; virtual void close()=0; virtual int log_xid(THD *thd, my_xid xid)=0; virtual void unlog(ulong cookie, my_xid xid)=0; + /* + If use_group_log_xid is true, then this method is used instead of + log_xid() to do logging of a group of transactions all at once. + + The transactions will be linked through THD::next_commit_ordered. + + Additionally, when this method is used instead of log_xid(), the order in + which handler->prepare_ordered() and handler->commit_ordered() are called + is guaranteed to be the same as the order of calls and THD list elements + for group_log_xid(). + + This can be used to efficiently implement group commit that at the same + time preserves the order of commits among handlers and TC (eg. to get same + commit order in InnoDB and binary log). + + For TCs that do not need this, it can be preferable to use plain log_xid() + instead, as it allows threads to run log_xid() in parallel with each + other. In contrast, group_log_xid() runs under a global mutex, so it is + guaranteed that only once call into it will be active at once. + + Since this call handles multiple threads/THDs at once, my_error() (and + other code that relies on thread local storage) cannot be used in this + method. Instead, in case of error, thd->xid_error should be set to the + error code, and xid_delayed_error() will be called later in the correct + thread context to actually report the error. + + In the success case, this method must set thd->xid_cookie for each thread + to the cookie that is normally returned from log_xid() (which must be + non-zero in the non-error case). + */ + virtual void group_log_xid(THD *first_thd) { DBUG_ASSERT(FALSE); } + /* Error reporting for group_log_xid(). */ + virtual int xid_delayed_error(THD *thd) { DBUG_ASSERT(FALSE); return 0; } }; class TC_LOG_DUMMY: public TC_LOG // use it to disable the logging @@ -227,12 +263,19 @@ private: time_t last_time; }; +class binlog_trx_data; class MYSQL_BIN_LOG: public TC_LOG, private MYSQL_LOG { private: /* LOCK_log and LOCK_index are inited by init_pthread_objects() */ pthread_mutex_t LOCK_index; pthread_mutex_t LOCK_prep_xids; + /* + Mutex to protect the queue of transactions waiting to participate in group + commit. (Only used on platforms without native atomic operations). + */ + pthread_mutex_t LOCK_queue; + pthread_cond_t COND_prep_xids; pthread_cond_t update_cond; ulonglong bytes_written; @@ -271,8 +314,8 @@ class MYSQL_BIN_LOG: public TC_LOG, priv In 5.0 it's 0 for relay logs too! */ bool no_auto_events; - - ulonglong m_table_map_version; + /* Queue of transactions queued up to participate in group commit. */ + binlog_trx_data *group_commit_queue; int write_to_file(IO_CACHE *cache); /* @@ -282,6 +325,14 @@ class MYSQL_BIN_LOG: public TC_LOG, priv */ void new_file_without_locking(); void new_file_impl(bool need_lock); + int write_transaction(binlog_trx_data *trx_data); + bool write_transaction_to_binlog_events(binlog_trx_data *trx_data); + void trx_group_commit_participant(binlog_trx_data *trx_data); + void trx_group_commit_leader(THD *first_thd); + binlog_trx_data *atomic_enqueue_trx(binlog_trx_data *trx_data); + binlog_trx_data *atomic_grab_trx_queue(); + void mark_xid_done(); + void mark_xids_active(uint xid_count); public: MYSQL_LOG::generate_name; @@ -311,17 +362,11 @@ public: int open(const char *opt_name); void close(); int log_xid(THD *thd, my_xid xid); + int xid_delayed_error(THD *thd); + void group_log_xid(THD *first_thd); void unlog(ulong cookie, my_xid xid); int recover(IO_CACHE *log, Format_description_log_event *fdle); #if !defined(MYSQL_CLIENT) - bool is_table_mapped(TABLE *table) const - { - return table->s->table_map_version == table_map_version(); - } - - ulonglong table_map_version() const { return m_table_map_version; } - void update_table_map_version() { ++m_table_map_version; } - int flush_and_set_pending_rows_event(THD *thd, Rows_log_event* event); int remove_pending_rows_event(THD *thd); @@ -362,10 +407,12 @@ public: void new_file(); bool write(Log_event* event_info); // binary log write - bool write(THD *thd, IO_CACHE *cache, Log_event *commit_event, bool incident); - bool write_incident(THD *thd, bool lock); + bool write_transaction_to_binlog(THD *thd, binlog_trx_data *trx_data, + Log_event *end_ev); + bool trx_group_commit_finish(binlog_trx_data *trx_data); + bool write_incident(THD *thd); - int write_cache(IO_CACHE *cache, bool lock_log, bool flush_and_sync); + int write_cache(IO_CACHE *cache); void set_write_error(THD *thd); bool check_write_error(THD *thd); === modified file 'sql/log_event.h' --- a/sql/log_event.h 2010-03-04 08:03:07 +0000 +++ b/sql/log_event.h 2010-05-26 08:13:32 +0000 @@ -463,10 +463,9 @@ struct sql_ex_info #define LOG_EVENT_SUPPRESS_USE_F 0x8 /* - The table map version internal to the log should be increased after - the event has been written to the binary log. + This used to be LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F, but is now unused. */ -#define LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F 0x10 +#define LOG_EVENT_UNUSED1_F 0x10 /** @def LOG_EVENT_ARTIFICIAL_F === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-01-15 15:27:55 +0000 +++ b/sql/sql_class.cc 2010-05-26 08:13:32 +0000 @@ -673,6 +673,8 @@ THD::THD() active_vio = 0; #endif pthread_mutex_init(&LOCK_thd_data, MY_MUTEX_INIT_FAST); + pthread_mutex_init(&LOCK_commit_ordered, MY_MUTEX_INIT_FAST); + pthread_cond_init(&COND_commit_ordered, 0); /* Variables with default values */ proc_info="login"; @@ -3773,7 +3775,6 @@ int THD::binlog_flush_pending_rows_event if (stmt_end) { pending->set_flags(Rows_log_event::STMT_END_F); - pending->flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; binlog_table_maps= 0; } @@ -3901,7 +3902,6 @@ int THD::binlog_query(THD::enum_binlog_q { Query_log_event qinfo(this, query_arg, query_len, is_trans, suppress_use, errcode); - qinfo.flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; /* Binlog table maps will be irrelevant after a Query_log_event (they are just removed on the slave side) so after the query === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-03-30 12:36:49 +0000 +++ b/sql/sql_class.h 2010-05-26 08:13:32 +0000 @@ -1438,6 +1438,21 @@ public: /* container for handler's private per-connection data */ Ha_data ha_data[MAX_HA]; + /* Mutex and condition for waking up threads after group commit. */ + pthread_mutex_t LOCK_commit_ordered; + pthread_cond_t COND_commit_ordered; + bool group_commit_ready; + /* Pointer for linking THDs into queue waiting for group commit. */ + THD *next_commit_ordered; + /* + The "all" parameter of commit(), to communicate it to the thread that + calls commit_ordered(). + */ + bool group_commit_all; + /* Set by TC_LOG::group_log_xid(), to return per-thd error and cookie. */ + int xid_error; + int xid_cookie; + #ifndef MYSQL_CLIENT int binlog_setup_trx_data(); === modified file 'sql/sql_load.cc' --- a/sql/sql_load.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_load.cc 2010-05-26 08:13:32 +0000 @@ -516,7 +516,6 @@ int mysql_load(THD *thd,sql_exchange *ex else { Delete_file_log_event d(thd, db, transactional_table); - d.flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; (void) mysql_bin_log.write(&d); } } @@ -698,7 +697,6 @@ static bool write_execute_load_query_log (duplicates == DUP_REPLACE) ? LOAD_DUP_REPLACE : (ignore ? LOAD_DUP_IGNORE : LOAD_DUP_ERROR), transactional_table, FALSE, errcode); - e.flags|= LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F; return mysql_bin_log.write(&e); } === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-10 10:32:14 +0000 +++ b/sql/table.cc 2010-05-26 08:13:32 +0000 @@ -297,13 +297,6 @@ TABLE_SHARE *alloc_table_share(TABLE_LIS share->version= refresh_version; /* - This constant is used to mark that no table map version has been - assigned. No arithmetic is done on the value: it will be - overwritten with a value taken from MYSQL_BIN_LOG. - */ - share->table_map_version= ~(ulonglong)0; - - /* Since alloc_table_share() can be called without any locking (for example, ha_create_table... functions), we do not assign a table map id here. Instead we assign a value that is not used @@ -367,10 +360,9 @@ void init_tmp_table_share(THD *thd, TABL share->frm_version= FRM_VER_TRUE_VARCHAR; /* - Temporary tables are not replicated, but we set up these fields + Temporary tables are not replicated, but we set up this fields anyway to be able to catch errors. */ - share->table_map_version= ~(ulonglong)0; share->cached_row_logging_check= -1; /* === modified file 'sql/table.h' --- a/sql/table.h 2010-02-10 19:06:24 +0000 +++ b/sql/table.h 2010-05-26 08:13:32 +0000 @@ -433,7 +433,6 @@ typedef struct st_table_share bool waiting_on_cond; /* Protection against free */ bool deleting; /* going to delete this table */ ulong table_map_id; /* for row-based replication */ - ulonglong table_map_version; /* Cache for row-based replication table share checks that does not === modified file 'storage/xtradb/handler/ha_innodb.cc' --- a/storage/xtradb/handler/ha_innodb.cc 2010-01-15 21:12:30 +0000 +++ b/storage/xtradb/handler/ha_innodb.cc 2010-05-26 08:13:32 +0000 @@ -138,8 +138,6 @@ bool check_global_access(THD *thd, ulong /** to protect innobase_open_files */ static pthread_mutex_t innobase_share_mutex; -/** to force correct commit order in binlog */ -static pthread_mutex_t prepare_commit_mutex; static ulong commit_threads = 0; static pthread_mutex_t commit_threads_m; static pthread_cond_t commit_cond; @@ -239,6 +237,7 @@ static const char* innobase_change_buffe static INNOBASE_SHARE *get_share(const char *table_name); static void free_share(INNOBASE_SHARE *share); static int innobase_close_connection(handlerton *hton, THD* thd); +static void innobase_commit_ordered(handlerton *hton, THD* thd, bool all); static int innobase_commit(handlerton *hton, THD* thd, bool all); static int innobase_rollback(handlerton *hton, THD* thd, bool all); static int innobase_rollback_to_savepoint(handlerton *hton, THD* thd, @@ -1356,7 +1355,6 @@ innobase_trx_init( trx_t* trx) /*!< in/out: InnoDB transaction handle */ { DBUG_ENTER("innobase_trx_init"); - DBUG_ASSERT(EQ_CURRENT_THD(thd)); DBUG_ASSERT(thd == trx->mysql_thd); trx->check_foreigns = !thd_test_options( @@ -1416,8 +1414,6 @@ check_trx_exists( { trx_t*& trx = thd_to_trx(thd); - ut_ad(EQ_CURRENT_THD(thd)); - if (trx == NULL) { trx = innobase_trx_allocate(thd); } else if (UNIV_UNLIKELY(trx->magic_n != TRX_MAGIC_N)) { @@ -2024,6 +2020,7 @@ innobase_init( innobase_hton->savepoint_set=innobase_savepoint; innobase_hton->savepoint_rollback=innobase_rollback_to_savepoint; innobase_hton->savepoint_release=innobase_release_savepoint; + innobase_hton->commit_ordered=innobase_commit_ordered; innobase_hton->commit=innobase_commit; innobase_hton->rollback=innobase_rollback; innobase_hton->prepare=innobase_xa_prepare; @@ -2492,7 +2489,6 @@ skip_overwrite: innobase_open_tables = hash_create(200); pthread_mutex_init(&innobase_share_mutex, MY_MUTEX_INIT_FAST); - pthread_mutex_init(&prepare_commit_mutex, MY_MUTEX_INIT_FAST); pthread_mutex_init(&commit_threads_m, MY_MUTEX_INIT_FAST); pthread_mutex_init(&commit_cond_m, MY_MUTEX_INIT_FAST); pthread_mutex_init(&analyze_mutex, MY_MUTEX_INIT_FAST); @@ -2547,7 +2543,6 @@ innobase_end( my_free(internal_innobase_data_file_path, MYF(MY_ALLOW_ZERO_PTR)); pthread_mutex_destroy(&innobase_share_mutex); - pthread_mutex_destroy(&prepare_commit_mutex); pthread_mutex_destroy(&commit_threads_m); pthread_mutex_destroy(&commit_cond_m); pthread_mutex_destroy(&analyze_mutex); @@ -2681,6 +2676,101 @@ innobase_start_trx_and_assign_read_view( } /*****************************************************************//** +Perform the first, fast part of InnoDB commit. + +Doing it in this call ensures that we get the same commit order here +as in binlog and any other participating transactional storage engines. + +Note that we want to do as little as really needed here, as we run +under a global mutex. The expensive fsync() is done later, in +innobase_commit(), without a lock so group commit can take place. + +Note also that this method can be called from a different thread than +the one handling the rest of the transaction. */ +static +void +innobase_commit_ordered( +/*============*/ + handlerton *hton, /*!< in: Innodb handlerton */ + THD* thd, /*!< in: MySQL thread handle of the user for whom + the transaction should be committed */ + bool all) /*!< in: TRUE - commit transaction + FALSE - the current SQL statement ended */ +{ + trx_t* trx; + DBUG_ENTER("innobase_commit_ordered"); + DBUG_ASSERT(hton == innodb_hton_ptr); + + trx = check_trx_exists(thd); + + if (trx->active_trans == 0 + && trx->conc_state != TRX_NOT_STARTED) { + /* We throw an error here; instead we will catch this error + again in innobase_commit() and report it from there. */ + DBUG_VOID_RETURN; + } + /* Since we will reserve the kernel mutex, we have to release + the search system latch first to obey the latching order. */ + + if (trx->has_search_latch) { + trx_search_latch_release_if_reserved(trx); + } + + /* commit_ordered is only called when committing the whole transaction + (or an SQL statement when autocommit is on). */ + DBUG_ASSERT(all || (!thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN))); + + /* We need current binlog position for ibbackup to work. + Note, the position is current because commit_ordered is guaranteed + to be called in same sequenece as writing to binlog. */ + +retry: + if (innobase_commit_concurrency > 0) { + pthread_mutex_lock(&commit_cond_m); + commit_threads++; + + if (commit_threads > innobase_commit_concurrency) { + commit_threads--; + pthread_cond_wait(&commit_cond, + &commit_cond_m); + pthread_mutex_unlock(&commit_cond_m); + goto retry; + } + else { + pthread_mutex_unlock(&commit_cond_m); + } + } + + /* The following calls to read the MySQL binary log + file name and the position return consistent results: + 1) We use commit_ordered() to get same commit order + in InnoDB as in binary log. + 2) A MySQL log file rotation cannot happen because + MySQL protects against this by having a counter of + transactions in prepared state and it only allows + a rotation when the counter drops to zero. See + LOCK_prep_xids and COND_prep_xids in log.cc. */ + trx->mysql_log_file_name = mysql_bin_log_file_name(); + trx->mysql_log_offset = (ib_int64_t) mysql_bin_log_file_pos(); + + /* Don't do write + flush right now. For group commit + to work we want to do the flush in the innobase_commit() + method, which runs without holding any locks. */ + trx->flush_log_later = TRUE; + innobase_commit_low(trx); + trx->flush_log_later = FALSE; + + if (innobase_commit_concurrency > 0) { + pthread_mutex_lock(&commit_cond_m); + commit_threads--; + pthread_cond_signal(&commit_cond); + pthread_mutex_unlock(&commit_cond_m); + } + + DBUG_VOID_RETURN; +} + +/*****************************************************************//** Commits a transaction in an InnoDB database or marks an SQL statement ended. @return 0 */ @@ -2702,13 +2792,6 @@ innobase_commit( trx = check_trx_exists(thd); - /* Since we will reserve the kernel mutex, we have to release - the search system latch first to obey the latching order. */ - - if (trx->has_search_latch) { - trx_search_latch_release_if_reserved(trx); - } - /* The flag trx->active_trans is set to 1 in 1. ::external_lock(), @@ -2736,62 +2819,8 @@ innobase_commit( /* We were instructed to commit the whole transaction, or this is an SQL statement end and autocommit is on */ - /* We need current binlog position for ibbackup to work. - Note, the position is current because of - prepare_commit_mutex */ -retry: - if (innobase_commit_concurrency > 0) { - pthread_mutex_lock(&commit_cond_m); - commit_threads++; - - if (commit_threads > innobase_commit_concurrency) { - commit_threads--; - pthread_cond_wait(&commit_cond, - &commit_cond_m); - pthread_mutex_unlock(&commit_cond_m); - goto retry; - } - else { - pthread_mutex_unlock(&commit_cond_m); - } - } - - /* The following calls to read the MySQL binary log - file name and the position return consistent results: - 1) Other InnoDB transactions cannot intervene between - these calls as we are holding prepare_commit_mutex. - 2) Binary logging of other engines is not relevant - to InnoDB as all InnoDB requires is that committing - InnoDB transactions appear in the same order in the - MySQL binary log as they appear in InnoDB logs. - 3) A MySQL log file rotation cannot happen because - MySQL protects against this by having a counter of - transactions in prepared state and it only allows - a rotation when the counter drops to zero. See - LOCK_prep_xids and COND_prep_xids in log.cc. */ - trx->mysql_log_file_name = mysql_bin_log_file_name(); - trx->mysql_log_offset = (ib_int64_t) mysql_bin_log_file_pos(); - - /* Don't do write + flush right now. For group commit - to work we want to do the flush after releasing the - prepare_commit_mutex. */ - trx->flush_log_later = TRUE; - innobase_commit_low(trx); - trx->flush_log_later = FALSE; - - if (innobase_commit_concurrency > 0) { - pthread_mutex_lock(&commit_cond_m); - commit_threads--; - pthread_cond_signal(&commit_cond); - pthread_mutex_unlock(&commit_cond_m); - } - - if (trx->active_trans == 2) { - - pthread_mutex_unlock(&prepare_commit_mutex); - } - - /* Now do a write + flush of logs. */ + /* We did the first part already in innobase_commit_ordered(), + Now finish by doing a write + flush of logs. */ trx_commit_complete_for_mysql(trx); trx->active_trans = 0; @@ -4621,6 +4650,7 @@ no_commit: no need to re-acquire locks on it. */ /* Altering to InnoDB format */ + innobase_commit_ordered(ht, user_thd, 1); innobase_commit(ht, user_thd, 1); /* Note that this transaction is still active. */ prebuilt->trx->active_trans = 1; @@ -4637,6 +4667,7 @@ no_commit: /* Commit the transaction. This will release the table locks, so they have to be acquired again. */ + innobase_commit_ordered(ht, user_thd, 1); innobase_commit(ht, user_thd, 1); /* Note that this transaction is still active. */ prebuilt->trx->active_trans = 1; @@ -8339,6 +8370,7 @@ ha_innobase::external_lock( if (!thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN)) { if (trx->active_trans != 0) { + innobase_commit_ordered(ht, thd, TRUE); innobase_commit(ht, thd, TRUE); } } else { @@ -9448,36 +9480,6 @@ innobase_xa_prepare( srv_active_wake_master_thread(); - if (thd_sql_command(thd) != SQLCOM_XA_PREPARE && - (all || !thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN))) - { - if (srv_enable_unsafe_group_commit && !THDVAR(thd, support_xa)) { - /* choose group commit rather than binlog order */ - return(error); - } - - /* For ibbackup to work the order of transactions in binlog - and InnoDB must be the same. Consider the situation - - thread1> prepare; write to binlog; ... - <context switch> - thread2> prepare; write to binlog; commit - thread1> ... commit - - To ensure this will not happen we're taking the mutex on - prepare, and releasing it on commit. - - Note: only do it for normal commits, done via ha_commit_trans. - If 2pc protocol is executed by external transaction - coordinator, it will be just a regular MySQL client - executing XA PREPARE and XA COMMIT commands. - In this case we cannot know how many minutes or hours - will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. */ - pthread_mutex_lock(&prepare_commit_mutex); - trx->active_trans = 2; - } - return(error); } @@ -10669,11 +10671,6 @@ static MYSQL_SYSVAR_ENUM(adaptive_checkp "Enable/Disable flushing along modified age. (none, reflex, [estimate])", NULL, innodb_adaptive_checkpoint_update, 2, &adaptive_checkpoint_typelib); -static MYSQL_SYSVAR_ULONG(enable_unsafe_group_commit, srv_enable_unsafe_group_commit, - PLUGIN_VAR_RQCMDARG, - "Enable/Disable unsafe group commit when support_xa=OFF and use with binlog or other XA storage engine.", - NULL, NULL, 0, 0, 1, 0); - static MYSQL_SYSVAR_ULONG(expand_import, srv_expand_import, PLUGIN_VAR_RQCMDARG, "Enable/Disable converting automatically *.ibd files when import tablespace.", @@ -10763,7 +10760,6 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(flush_neighbor_pages), MYSQL_SYSVAR(read_ahead), MYSQL_SYSVAR(adaptive_checkpoint), - MYSQL_SYSVAR(enable_unsafe_group_commit), MYSQL_SYSVAR(expand_import), MYSQL_SYSVAR(extra_rsegments), MYSQL_SYSVAR(dict_size_limit),

1 0

[Maria-developers] Rev 2792: fixed problem with subselect_debug.test in file:///home/bell/maria/bzr/work-maria-5.3-scache/
by sanja＠askmonty.org 25 May '10

25 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache/ ------------------------------------------------------------ revno: 2792 revision-id: sanja(a)askmonty.org-20100525182914-z3zeviggq9026x1n parent: sanja(a)askmonty.org-20100525125457-5rwbiihh0vtghdrj committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache timestamp: Tue 2010-05-25 21:29:14 +0300 message: fixed problem with subselect_debug.test === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-05-25 12:54:57 +0000 +++ b/sql/item_subselect.cc 2010-05-25 18:29:14 +0000 @@ -1201,7 +1201,8 @@ if (exec()) { reset(); - DBUG_RETURN(NULL); + str->set((ulonglong)0,&my_charset_bin); + DBUG_RETURN(str); } if (scache) @@ -1244,7 +1245,8 @@ if (exec()) { reset(); - DBUG_RETURN(0); + int2my_decimal(E_DEC_FATAL_ERROR, 0, 0, decimal_value); + DBUG_RETURN(decimal_value); } if (scache)

1 0

[Maria-developers] String and decimal results in case of error.
by Oleksandr Byelkin 25 May '10

25 May '10

Hi! There was that in case of errors Item::val_str() and Item::val_decimal() can return NULL (0) but I found some parts of the code do not allow it, for example Item_sum_sum::add(). Should we revise val_* methods so that always retun something (for example subquery EXISTS was changed (and even we have subselect_debug.test about it), but single row subquery was not) or change places where val_str() and val_decimal() that it will allow NULL (0).

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2858: Fix a couple of problems in the pack script, and disable a check feature that doesn't work right now
by noreply＠launchpad.net 25 May '10

25 May '10

------------------------------------------------------------ revno: 2858 committer: Bo Thorsen <bo(a)askmonty.org> branch nick: trunk-work timestamp: Tue 2010-05-25 16:56:35 +0200 message: Fix a couple of problems in the pack script, and disable a check feature that doesn't work right now modified: win/make_mariadb_win_dist -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] Updated (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 13:19)=-=- Low Level Design modified. --- /tmp/wklog.116.old.14255 2010-05-25 13:19:00.000000000 +0000 +++ /tmp/wklog.116.new.14255 2010-05-25 13:19:00.000000000 +0000 @@ -1 +1,363 @@ +1. Changes for ha_commit_trans() + +The gut of the code for commit is in the function ha_commit_trans() (and in +commit_one_phase() which is called from it). This must be extended to use the +new prepare_ordered(), group_log_xid(), and commit_ordered() calls. + +1.1 Atomic queue of committing transactions + +To keep the right commit order among participants, we put transactions into a +queue. The operations on the queue are non-locking: + + - Insert THD at the head of the queue, and return old queue. + + THD *enqueue_atomic(THD *thd) + + - Fetch (and delete) the whole queue. + + THD *atomic_grab_reverse_queue() + +These are simple to implement with atomic compare-and-set. Note that there is +no ABA problem [2], as we do not delete individual elements from the queue, we +grab the whole queue and replace it with NULL. + +A transaction enters the queue when it does prepare_ordered(). This way, the +scheduling order for prepare_ordered() calls is what determines the sequence +in the queue and effectively the commit order. + +The queue is grabbed by the code doing group_log_xid() and commit_ordered() +calls. The queue is passed directly to group_log_xid(), and afterwards +iterated to do individual commit_ordered() calls. + +Using a lock-free queue allows prepare_ordered() (for one transaction) to run +in parallel with commit_ordered (in another transaction), increasing potential +parallelism. + +The queue is simply a linked list of THD objects, linked through a +THD::next_commit_ordered field. Since we add at the head of the queue, the +list is actually in reverse order, so must be reversed when we grab and delete +it. + +The reason that enqueue_atomic() returns the old queue is so that we can check +if an insert goes to the head of the queue. The thread at the head of the +queue will do the sequential part of group commit for everyone. + + +1.2 Locks + +1.2.1 Global LOCK_prepare_ordered + +This lock is taken to serialise calls to prepare_ordered(). Note that +effectively, the commit order is decided by the order in which threads obtain +this lock. + + +1.2.2 Global LOCK_group_commit and COND_group_commit + +This lock is used to protect the serial part of group commit. It is taken +around the code where we grab the queue, call group_log_xid() on the queue, +and call commit_ordered() on each element of the queue, to make sure they +happen serialised and in consistent order. It also protects the variable +group_commit_queue_busy, which is used when not using group_log_xid() to delay +running over a new queue until the first queue is completely done. + + +1.2.3 Global LOCK_commit_ordered + +This lock is taken around calls to commit_ordered(), to ensure they happen +serialised. + + +1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered + +This lock protects the thd->group_commit_ready variable, as well as the +condition variable used to wake up threads after log_xid() and +commit_ordered() finishes. + + +1.2.5 Global LOCK_group_commit_queue + +This is only used on platforms with no native compare-and-set operations, to +make the queue operations atomic. + + +1.3 Commit algorithm. + +This is the basic algorithm, simplified by + + - omitting some error handling + + - omitting looping over all handlers when invoking handler methods + + - omitting some possible optimisations when not all calls needed (see next + section). + + - Omitting the case where no group_log_xid() is used, see below. + +---- BEGIN ALGORITHM ---- + ht->prepare() + + // Call prepare_ordered() and enqueue in correct commit order + lock(LOCK_prepare_ordered) + ht->prepare_ordered() + old_queue= enqueue_atomic(thd) + thd->group_commit_ready= FALSE + is_group_commit_leader= (old_queue == NULL) + unlock(LOCK_prepare_ordered) + + if (is_group_commit_leader) + + // The first in queue handles group commit for everyone + + lock(LOCK_group_commit) + // Wait while queue is busy, see below for when this occurs + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + + // Grab and reverse the queue to get correct order of transactions + queue= atomic_grab_reverse_queue() + + // This call will set individual error codes in thd->xid_error + // It also sets the cookie for unlog() in thd->xid_cookie + group_log_xid(queue) + + lock(LOCK_commit_ordered) + for (other IN queue) + if (!other->xid_error) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + unlock(LOCK_group_commit) + + // Now we are done, so wake up all the others. + for (other IN TAIL(queue)) + lock(other->LOCK_commit_ordered) + other->group_commit_ready= TRUE + cond_signal(other->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + else + // If not the leader, just wait until leader did the work for us. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + // Finally do any error reporting now that we're back in own thread. + if (thd->xid_error) + xid_delayed_error(thd) + else + ht->commit(thd) + unlog(thd->xid_cookie, thd->xid) +---- END ALGORITHM ---- + +If the transaction coordinator does not support group_log_xid(), we have to do +things differently. In this case after the serialisation point at +prepare_ordered(), we have to parallelise again when running log_xid() +(otherwise we would loose group commit). But then when log_xid() is done, we +have to serialise again to check for any error and call commit_ordered() in +correct sequence for any transaction where log_xid() did not return error. + +The central part of the algorithm in this case (when using log_xid()) is: + +---- BEGIN ALGORITHM ---- + cookie= log_xid(thd) + error= (cookie == 0) + + if (is_group_commit_leader) + + // The first to enqueue grabs the queue and runs first. + // But we must wait until a previous queue run is fully done. + + lock(LOCK_group_commit) + while (group_commit_queue_busy) + cond_wait(COND_group_commit) + queue= atomic_grab_reverse_queue() + // The queue will be busy until last thread in it is done. + group_commit_queue_busy= TRUE + unlock(LOCK_group_commit) + else + // Not first in queue -> wait for previous one to wake us up. + lock(thd->LOCK_commit_ordered) + while (!thd->group_commit_ready) + cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) + unlock(other->LOCK_commit_ordered) + + if (!error) // Only if log_xid() was successful + lock(LOCK_commit_ordered) + ht->commit_ordered() + unlock(LOCK_commit_ordered) + + // Wake up the next thread, and release queue in last. + next= thd->next_commit_ordered + + if (next) + lock(next->LOCK_commit_ordered) + next->group_commit_ready= TRUE + cond_signal(next->COND_commit_ordered) + unlock(next->LOCK_commit_ordered) + else + lock(LOCK_group_commit) + group_commit_queue_busy= FALSE + unlock(LOCK_group_commit) +---- END ALGORITHM ---- + +There are a number of locks taken in the algorithm, but in the group_log_xid() +case most of them should be uncontended most of the time. The +LOCK_group_commit of course will be contended, as new threads queue up waiting +for the previous group commit (and binlog fsync()) to finish so they can do +the next group commit. This is the whole point of implementing group commit. + +The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much +contended as long as handlers follow the intension of having the corresponding +handler calls execute quickly. + +The per-thread LOCK_commit_ordered mutexes should not be contended; they are +only used to wake up a sleeping thread. + + +1.4 Optimisations when not using all three new calls + + +The prepare_ordered(), group_log_xid(), and commit_ordered() methods are +optional, and if not implemented by a particular handler/transaction +coordinator, we can optimise the algorithm to take advantage of not having to +keep ordering for the missing parts. + +If there is no prepare_ordered(), then we need not take the +LOCK_prepare_ordered mutex. + +If there is no commit_ordered(), then we need not take the LOCK_commit_ordered +mutex. + +If there is no group_log_xid(), then we only need the queue to ensure same +ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, +if either of these (or both) are also not present, we do not need to use the +queue at all. + + +2. Binlog code changes (log.cc) + + +The bulk of the work needed for the binary log is to extend the code to allow +group commit to the log. Unlike InnoDB/XtraDB, there is no existing support +inside the binlog code for group commit. + +The existing code runs most of the write + fsync to the binary lock under the +global LOCK_log mutex, preventing any group commit. + +To enable group commit, this code must be split into two parts: + + - one part that runs per transaction, re-writing the embedded event positions + for the correct offset, and writing this into the in-memory log cache. + + - another part that writes a set of transactions to the disk, and runs + fsync(). + +Then in group_log_xid(), we can run the first part in a loop over all the +transactions in the passed-in queue, and run the second part only once. + +The binlog code also has other code paths that write into the binlog, +eg. non-transactional statements. These have to be adapted also to work with +the new code. + +In order to get some group commit facility for these also, we change that part +of the code in a similar way to ha_commit_trans. We keep another, +binlog-internal queue of such non-transactional binlog writes, and such writes +queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the +LOCK_log, it loops over the queue for the fast part, and does the slow part +once, then finally wakes up the others in the queue. + +In the transactional case in group_log_xid(), before we run the passed-in +queue, we add any members found in the binlog-internal queue. This allows +these non-transactional writes to share the group commit. + +However, in the case where it is a non-transactional write that gets the +LOCK_log, the transactional transactions from the ha_commit_trans() queue will +not be able to take part (they will have to wait for their turn to do another +fsync). It seems difficult to cleanly let the binlog code grab the queue from +out of the ha_commit_trans() algorithm. I think the group commit is mostly +useful in transactional workloads anyway (non-transactional engines will loose +data anyway in case of crash, so why fsync() after each transaction?) + + +3. XtraDB changes (ha_innodb.cc) + +The changes needed in XtraDB are comparatively simple, as XtraDB already +implements group commit, it just needs to be enabled with the new +commit_ordered() call. + +The existing commit() method already is logically in two parts. The first part +runs under the prepare_commit_mutex() and must be run in same order as binlog +commit. This part needs to be moved to commit_ordered(). The second part runs +after releasing prepare_commit_mutex and does transaction log write+fsync; it +can remain. + +Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit +XtraDB option to disable it). + +There are two asserts that check that the thread running the first part of +XtraDB commit is the same as the thread running the other operations for the +transaction. These have to be removed (as commit_ordered() can run in a +different thread). Also an error reporting with sql_print_error() has to be +delayed until commit() time. + + +4. Proof-of-concept implementation + +There is a proof-of-concept implementation of this architecture, in the form +of a quilt patch series [3]. + +A quick benchmark was done, with sync_binlog=1 and +innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row +transactions against one table. + +Without the patch, we get only 25 queries per second. + +With the patch, we get 650 queries per second. + + +5. Open issues/tasks + +5.1 XA / other prepare() and commit() call sites. + +Check that user-level XA is handled correctly and working. And covered +sufficiently with tests. Also check that any other calls of ha->prepare() and +ha->commit() outside of ha_commit_trans() are handled correctly. + +5.2 Testing + +This worklog needs additions to the test suite, including error inserts to +check error handling, and synchronisation points to check thread parallelism +correctness. + + +6. Alternative implementations + + - The binlog code maintains its own extra atomic transaction queue to handle + non-transactional commits in a good way together with transactional (with + respect to group commit). Alternatively, we could ignore this issue and + just give up on group commit for non-transactional statements, for some + code simplifications. + + - The binlog code has two ways to prepare end_event and similar, one that + uses stack-allocation, and another for when stack allocation is not + possible that uses thd->mem_root. Probably the overhead of thd->mem_root is + so small that it would make sense to use the same code for both cases. + + - Instead of adding extra fields to THD, we could allocate a separate + structure on the thd->mem_root() with the required extra fields (including + the THD pointer). Would seem to require initialising mutexes at every + commit though. + + - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() + (should not be hard). + + +----------------------------------------------------------------------- + +References: + +[2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem + +[3] https://knielsen-hq.org/maria/patches.mwl116/ -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. LOW-LEVEL DESIGN: 1. Changes for ha_commit_trans() The gut of the code for commit is in the function ha_commit_trans() (and in commit_one_phase() which is called from it). This must be extended to use the new prepare_ordered(), group_log_xid(), and commit_ordered() calls. 1.1 Atomic queue of committing transactions To keep the right commit order among participants, we put transactions into a queue. The operations on the queue are non-locking: - Insert THD at the head of the queue, and return old queue. THD *enqueue_atomic(THD *thd) - Fetch (and delete) the whole queue. THD *atomic_grab_reverse_queue() These are simple to implement with atomic compare-and-set. Note that there is no ABA problem [2], as we do not delete individual elements from the queue, we grab the whole queue and replace it with NULL. A transaction enters the queue when it does prepare_ordered(). This way, the scheduling order for prepare_ordered() calls is what determines the sequence in the queue and effectively the commit order. The queue is grabbed by the code doing group_log_xid() and commit_ordered() calls. The queue is passed directly to group_log_xid(), and afterwards iterated to do individual commit_ordered() calls. Using a lock-free queue allows prepare_ordered() (for one transaction) to run in parallel with commit_ordered (in another transaction), increasing potential parallelism. The queue is simply a linked list of THD objects, linked through a THD::next_commit_ordered field. Since we add at the head of the queue, the list is actually in reverse order, so must be reversed when we grab and delete it. The reason that enqueue_atomic() returns the old queue is so that we can check if an insert goes to the head of the queue. The thread at the head of the queue will do the sequential part of group commit for everyone. 1.2 Locks 1.2.1 Global LOCK_prepare_ordered This lock is taken to serialise calls to prepare_ordered(). Note that effectively, the commit order is decided by the order in which threads obtain this lock. 1.2.2 Global LOCK_group_commit and COND_group_commit This lock is used to protect the serial part of group commit. It is taken around the code where we grab the queue, call group_log_xid() on the queue, and call commit_ordered() on each element of the queue, to make sure they happen serialised and in consistent order. It also protects the variable group_commit_queue_busy, which is used when not using group_log_xid() to delay running over a new queue until the first queue is completely done. 1.2.3 Global LOCK_commit_ordered This lock is taken around calls to commit_ordered(), to ensure they happen serialised. 1.2.4 Per-thread thd->LOCK_commit_ordered and thd->COND_commit_ordered This lock protects the thd->group_commit_ready variable, as well as the condition variable used to wake up threads after log_xid() and commit_ordered() finishes. 1.2.5 Global LOCK_group_commit_queue This is only used on platforms with no native compare-and-set operations, to make the queue operations atomic. 1.3 Commit algorithm. This is the basic algorithm, simplified by - omitting some error handling - omitting looping over all handlers when invoking handler methods - omitting some possible optimisations when not all calls needed (see next section). - Omitting the case where no group_log_xid() is used, see below. ---- BEGIN ALGORITHM ---- ht->prepare() // Call prepare_ordered() and enqueue in correct commit order lock(LOCK_prepare_ordered) ht->prepare_ordered() old_queue= enqueue_atomic(thd) thd->group_commit_ready= FALSE is_group_commit_leader= (old_queue == NULL) unlock(LOCK_prepare_ordered) if (is_group_commit_leader) // The first in queue handles group commit for everyone lock(LOCK_group_commit) // Wait while queue is busy, see below for when this occurs while (group_commit_queue_busy) cond_wait(COND_group_commit) // Grab and reverse the queue to get correct order of transactions queue= atomic_grab_reverse_queue() // This call will set individual error codes in thd->xid_error // It also sets the cookie for unlog() in thd->xid_cookie group_log_xid(queue) lock(LOCK_commit_ordered) for (other IN queue) if (!other->xid_error) ht->commit_ordered() unlock(LOCK_commit_ordered) unlock(LOCK_group_commit) // Now we are done, so wake up all the others. for (other IN TAIL(queue)) lock(other->LOCK_commit_ordered) other->group_commit_ready= TRUE cond_signal(other->COND_commit_ordered) unlock(other->LOCK_commit_ordered) else // If not the leader, just wait until leader did the work for us. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) // Finally do any error reporting now that we're back in own thread. if (thd->xid_error) xid_delayed_error(thd) else ht->commit(thd) unlog(thd->xid_cookie, thd->xid) ---- END ALGORITHM ---- If the transaction coordinator does not support group_log_xid(), we have to do things differently. In this case after the serialisation point at prepare_ordered(), we have to parallelise again when running log_xid() (otherwise we would loose group commit). But then when log_xid() is done, we have to serialise again to check for any error and call commit_ordered() in correct sequence for any transaction where log_xid() did not return error. The central part of the algorithm in this case (when using log_xid()) is: ---- BEGIN ALGORITHM ---- cookie= log_xid(thd) error= (cookie == 0) if (is_group_commit_leader) // The first to enqueue grabs the queue and runs first. // But we must wait until a previous queue run is fully done. lock(LOCK_group_commit) while (group_commit_queue_busy) cond_wait(COND_group_commit) queue= atomic_grab_reverse_queue() // The queue will be busy until last thread in it is done. group_commit_queue_busy= TRUE unlock(LOCK_group_commit) else // Not first in queue -> wait for previous one to wake us up. lock(thd->LOCK_commit_ordered) while (!thd->group_commit_ready) cond_wait(thd->LOCK_commit_ordered, thd->COND_commit_ordered) unlock(other->LOCK_commit_ordered) if (!error) // Only if log_xid() was successful lock(LOCK_commit_ordered) ht->commit_ordered() unlock(LOCK_commit_ordered) // Wake up the next thread, and release queue in last. next= thd->next_commit_ordered if (next) lock(next->LOCK_commit_ordered) next->group_commit_ready= TRUE cond_signal(next->COND_commit_ordered) unlock(next->LOCK_commit_ordered) else lock(LOCK_group_commit) group_commit_queue_busy= FALSE unlock(LOCK_group_commit) ---- END ALGORITHM ---- There are a number of locks taken in the algorithm, but in the group_log_xid() case most of them should be uncontended most of the time. The LOCK_group_commit of course will be contended, as new threads queue up waiting for the previous group commit (and binlog fsync()) to finish so they can do the next group commit. This is the whole point of implementing group commit. The LOCK_prepare_ordered and LOCK_commit_ordered mutexes should be not much contended as long as handlers follow the intension of having the corresponding handler calls execute quickly. The per-thread LOCK_commit_ordered mutexes should not be contended; they are only used to wake up a sleeping thread. 1.4 Optimisations when not using all three new calls The prepare_ordered(), group_log_xid(), and commit_ordered() methods are optional, and if not implemented by a particular handler/transaction coordinator, we can optimise the algorithm to take advantage of not having to keep ordering for the missing parts. If there is no prepare_ordered(), then we need not take the LOCK_prepare_ordered mutex. If there is no commit_ordered(), then we need not take the LOCK_commit_ordered mutex. If there is no group_log_xid(), then we only need the queue to ensure same ordering of transactions for commit_ordered() as for prepare_ordered(). Thus, if either of these (or both) are also not present, we do not need to use the queue at all. 2. Binlog code changes (log.cc) The bulk of the work needed for the binary log is to extend the code to allow group commit to the log. Unlike InnoDB/XtraDB, there is no existing support inside the binlog code for group commit. The existing code runs most of the write + fsync to the binary lock under the global LOCK_log mutex, preventing any group commit. To enable group commit, this code must be split into two parts: - one part that runs per transaction, re-writing the embedded event positions for the correct offset, and writing this into the in-memory log cache. - another part that writes a set of transactions to the disk, and runs fsync(). Then in group_log_xid(), we can run the first part in a loop over all the transactions in the passed-in queue, and run the second part only once. The binlog code also has other code paths that write into the binlog, eg. non-transactional statements. These have to be adapted also to work with the new code. In order to get some group commit facility for these also, we change that part of the code in a similar way to ha_commit_trans. We keep another, binlog-internal queue of such non-transactional binlog writes, and such writes queue up here before sleeping on the LOCK_log mutex. Once a thread obtains the LOCK_log, it loops over the queue for the fast part, and does the slow part once, then finally wakes up the others in the queue. In the transactional case in group_log_xid(), before we run the passed-in queue, we add any members found in the binlog-internal queue. This allows these non-transactional writes to share the group commit. However, in the case where it is a non-transactional write that gets the LOCK_log, the transactional transactions from the ha_commit_trans() queue will not be able to take part (they will have to wait for their turn to do another fsync). It seems difficult to cleanly let the binlog code grab the queue from out of the ha_commit_trans() algorithm. I think the group commit is mostly useful in transactional workloads anyway (non-transactional engines will loose data anyway in case of crash, so why fsync() after each transaction?) 3. XtraDB changes (ha_innodb.cc) The changes needed in XtraDB are comparatively simple, as XtraDB already implements group commit, it just needs to be enabled with the new commit_ordered() call. The existing commit() method already is logically in two parts. The first part runs under the prepare_commit_mutex() and must be run in same order as binlog commit. This part needs to be moved to commit_ordered(). The second part runs after releasing prepare_commit_mutex and does transaction log write+fsync; it can remain. Then the prepare_commit_mutex is removed (and the enable_unsafe_group_commit XtraDB option to disable it). There are two asserts that check that the thread running the first part of XtraDB commit is the same as the thread running the other operations for the transaction. These have to be removed (as commit_ordered() can run in a different thread). Also an error reporting with sql_print_error() has to be delayed until commit() time. 4. Proof-of-concept implementation There is a proof-of-concept implementation of this architecture, in the form of a quilt patch series [3]. A quick benchmark was done, with sync_binlog=1 and innodb_flush_log_at_trx_commit=1. 64 parallel threads doing single-row transactions against one table. Without the patch, we get only 25 queries per second. With the patch, we get 650 queries per second. 5. Open issues/tasks 5.1 XA / other prepare() and commit() call sites. Check that user-level XA is handled correctly and working. And covered sufficiently with tests. Also check that any other calls of ha->prepare() and ha->commit() outside of ha_commit_trans() are handled correctly. 5.2 Testing This worklog needs additions to the test suite, including error inserts to check error handling, and synchronisation points to check thread parallelism correctness. 6. Alternative implementations - The binlog code maintains its own extra atomic transaction queue to handle non-transactional commits in a good way together with transactional (with respect to group commit). Alternatively, we could ignore this issue and just give up on group commit for non-transactional statements, for some code simplifications. - The binlog code has two ways to prepare end_event and similar, one that uses stack-allocation, and another for when stack allocation is not possible that uses thd->mem_root. Probably the overhead of thd->mem_root is so small that it would make sense to use the same code for both cases. - Instead of adding extra fields to THD, we could allocate a separate structure on the thd->mem_root() with the required extra fields (including the THD pointer). Would seem to require initialising mutexes at every commit though. - It would probably be a good idea to implement TC_LOG_MMAP::group_log_xid() (should not be hard). ----------------------------------------------------------------------- References: [2] https://secure.wikimedia.org/wikipedia/en/wiki/ABA_problem [3] https://knielsen-hq.org/maria/patches.mwl116/ ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High-Level Specification modified. --- /tmp/wklog.116.old.14249 2010-05-25 13:18:34.000000000 +0000 +++ /tmp/wklog.116.new.14249 2010-05-25 13:18:34.000000000 +0000 @@ -1 +1,157 @@ +The basic idea in group commit is that multiple threads, each handling one +transaction, prepare for commit and then queue up together waiting to do an +fsync() on the transaction log. Then once the log is available, a single +thread does the fsync() + other necessary book-keeping for all of the threads +at once. After this, the single thread signals the other threads that it's +done and they can finish up and return success (or failure) from the commit +operation. + +So group commit has a parallel part, and a sequential part. So we need a +facility for engines/binlog to participate in both the parallel and the +sequential part. + +To do this, we add two new handlerton methods: + + int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); + void (*commit_ordered)(handlerton *hton, THD *thd, bool all); + +The idea is that the existing prepare() and commit() methods run in the +parallel part of group commit, and the new prepare_ordered() and +commit_ordered() run in the sequential part. + +The prepare_ordered() method is called after prepare(). The order of +tranctions that call into prepare_ordered() is guaranteed to be the same among +all storage engines and binlog, and it is serialised so no two calls can be +running inside the same engine at the same time. + +The commit_ordered() method is called before commit(), and similarly is +guaranteed to have same transaction order in all participants, and to be +serialised within one engine. + +As the prepare_ordered() and commit_ordered() calls are serialised, the idea +is that handlers should do the minimum amount of work needed in these calls, +relaying most of the work (eg. fsync() ...) to prepare() and commit(). + +As a concrete example, for InnoDB the commit_ordered() method will do the +first part of commit that fixed the commit order in the transaction log +buffer, and the commit() method will write the log to disk and fsync() +it. This split already exists inside the InnoDB code, running before +respectively after releasing the prepare_commit_mutex. + +In addition, the XA transaction coordinator (TC_LOG) is special, since it is +the one responsible for deciding whether to commit or rollback the +transaction. For this we need an extra method, since this decision can be done +only after we know that all prepare() and prepare_ordered() calls succeed, and +must be done to know whether to call commit_ordered()/commit(), or do rollback. + +The existing method for this is TC_LOG::log_xid(). To make implementing group +commit simpler to implement in a transaction coordinator and more efficient, +we introduce a new method: + + void group_log_xid(THD *first_thd); + +This method runs in the sequential part of group commit. It receives a list of +transactions to perform log_xid() on, in the correct commit order. (Note that +TC_LOG can do parallel parts of group commit in its own prepare() and commit() +methods). + +This method can make it easier to implement the group commit in TC_LOG, as it +gets directly the list of transactions in the right order. Without it, it +might need to compute such order anyway in a prepare_ordered() method, and the +server has to create this ordered list anyway to implement the order guarantee +for prepare_ordered() and commit_ordered(). + +This group_log_xid() method also is more efficient, as it avoids some +inter-thread synchronisation. Since group_log_xid() is serialised, we can run +it together with all the commit_ordered() method calls and need only a single +sequential code section. With the log_xid() methods, we would need first a +sequential part for the prepare_ordered() calls, then a parallel part with +log_xid() calls (to not loose group commit ability for log_xid()), then again +a sequential part for the commit_ordered() method calls. + +The extra synchronisation is needed, as each commit_ordered() call will have +to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() +should not be called), and also wait for commit_ordered() to finish in all +threads handling earlier commits. In effect we will need to bounce the +execution from one thread to the other among all participants in the group +commit. + +As a consequence of the group_log_xid() optimisation, handlers must be aware +that the commit_ordered() call can happen in another thread than the one +running commit() (so thread local storage is not available). This should not +be a big issue as the THD is available for storing any needed information. + +Since group_log_xid() runs for multiple transactions in a single thread, it +can not do error reporting (my_error()) as that relies on thread local +storage. Instead it sets an error code in THD::xid_error, and if there is an +error then later another method will be called (in correct thread context) to +actually report the error: + + int xid_delayed_error(THD *thd) + +The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() +are optional (as is xid_delayed_error). A storage engine or transaction +coordinator is free to not implement them if they are not needed. In this case +there will be no order guarantee for the corresponding stage of group commit +for that engine. For example, InnoDB needs no ordering of the prepare phase, +so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at +all, so does not need to implement any of them. + +Note in particular that all existing engines (/binlog implementations if they +exist) will work unmodified (and also without any change in group commit +facilities or commit order guaranteed). + +Using these new APIs, the work will be to + + - In ha_commit_trans(), implement the correct semantics for the three new + calls. + + - In XtraDB, use the new commit_ordered() call to remove the + prepare_commit_mutex (and resurrect group commit) without loosing the + consistency with binlog commit order. + + - In log.cc (binlog module), implement group_log_xid() to do group commit of + multiple transactions to the binlog with a single shared fsync() call. + +----------------------------------------------------------------------- +Some possible alternative for this worklog: + + - We could eliminate the group_log_xid() method for a simpler API, at the + cost of extra synchronisation between threads to do in-order + commit_ordered() method calls. This would also allow to call + commit_ordered() in the correct thread context. + + - Alternatively, we could eliminate log_xid() and require that all + transaction coordinators implement group_log_xid() instead, again for some + moderate simplification. + + - At the moment there is no plugin actually using prepare_ordered(), so, it + could be removed from the design. But it fits in well, is efficient to + implement, and could be useful later (eg. for the requested feature of + releasing locks early in InnoDB). + +----------------------------------------------------------------------- +Some possible follow-up projects after this is implemented: + + - Add statistics about how efficient group commit is (#fsyncs/#commits in + each engine and binlog). + + - Implement an XtraDB prepare_ordered() methods that can release row locks + early (Mark Callaghan from Facebook advocates this, but need to determine + exactly how to do this safely). + + - Implement a new crash recovery algorithm that uses the consistent commit + ordering to need only fsync() for the binlog. At crash recovery, any + missing transactions in an engine is replayed from the correct point in the + binlog (this point must be stored transactionally inside the engine, as + XtraDB already does today). + + - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a + consistent snapshow, with same set of committed and not committed + transactions in all engines, 2) returns a corresponding consistent binlog + position. This should be easy by piggybacking on the synchronisation + implemented for ha_commit_trans(). + + - Use this in XtraBackup to get consistent binlog position without having to + block all updates with FLUSH TABLES WITH READ LOCK. -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication HIGH-LEVEL SPECIFICATION: The basic idea in group commit is that multiple threads, each handling one transaction, prepare for commit and then queue up together waiting to do an fsync() on the transaction log. Then once the log is available, a single thread does the fsync() + other necessary book-keeping for all of the threads at once. After this, the single thread signals the other threads that it's done and they can finish up and return success (or failure) from the commit operation. So group commit has a parallel part, and a sequential part. So we need a facility for engines/binlog to participate in both the parallel and the sequential part. To do this, we add two new handlerton methods: int (*prepare_ordered)(handlerton *hton, THD *thd, bool all); void (*commit_ordered)(handlerton *hton, THD *thd, bool all); The idea is that the existing prepare() and commit() methods run in the parallel part of group commit, and the new prepare_ordered() and commit_ordered() run in the sequential part. The prepare_ordered() method is called after prepare(). The order of tranctions that call into prepare_ordered() is guaranteed to be the same among all storage engines and binlog, and it is serialised so no two calls can be running inside the same engine at the same time. The commit_ordered() method is called before commit(), and similarly is guaranteed to have same transaction order in all participants, and to be serialised within one engine. As the prepare_ordered() and commit_ordered() calls are serialised, the idea is that handlers should do the minimum amount of work needed in these calls, relaying most of the work (eg. fsync() ...) to prepare() and commit(). As a concrete example, for InnoDB the commit_ordered() method will do the first part of commit that fixed the commit order in the transaction log buffer, and the commit() method will write the log to disk and fsync() it. This split already exists inside the InnoDB code, running before respectively after releasing the prepare_commit_mutex. In addition, the XA transaction coordinator (TC_LOG) is special, since it is the one responsible for deciding whether to commit or rollback the transaction. For this we need an extra method, since this decision can be done only after we know that all prepare() and prepare_ordered() calls succeed, and must be done to know whether to call commit_ordered()/commit(), or do rollback. The existing method for this is TC_LOG::log_xid(). To make implementing group commit simpler to implement in a transaction coordinator and more efficient, we introduce a new method: void group_log_xid(THD *first_thd); This method runs in the sequential part of group commit. It receives a list of transactions to perform log_xid() on, in the correct commit order. (Note that TC_LOG can do parallel parts of group commit in its own prepare() and commit() methods). This method can make it easier to implement the group commit in TC_LOG, as it gets directly the list of transactions in the right order. Without it, it might need to compute such order anyway in a prepare_ordered() method, and the server has to create this ordered list anyway to implement the order guarantee for prepare_ordered() and commit_ordered(). This group_log_xid() method also is more efficient, as it avoids some inter-thread synchronisation. Since group_log_xid() is serialised, we can run it together with all the commit_ordered() method calls and need only a single sequential code section. With the log_xid() methods, we would need first a sequential part for the prepare_ordered() calls, then a parallel part with log_xid() calls (to not loose group commit ability for log_xid()), then again a sequential part for the commit_ordered() method calls. The extra synchronisation is needed, as each commit_ordered() call will have to wait for log_xid() in one thread (if log_xid() fails then commit_ordered() should not be called), and also wait for commit_ordered() to finish in all threads handling earlier commits. In effect we will need to bounce the execution from one thread to the other among all participants in the group commit. As a consequence of the group_log_xid() optimisation, handlers must be aware that the commit_ordered() call can happen in another thread than the one running commit() (so thread local storage is not available). This should not be a big issue as the THD is available for storing any needed information. Since group_log_xid() runs for multiple transactions in a single thread, it can not do error reporting (my_error()) as that relies on thread local storage. Instead it sets an error code in THD::xid_error, and if there is an error then later another method will be called (in correct thread context) to actually report the error: int xid_delayed_error(THD *thd) The three new methods prepare_ordered(), group_log_xid(), and commit_ordered() are optional (as is xid_delayed_error). A storage engine or transaction coordinator is free to not implement them if they are not needed. In this case there will be no order guarantee for the corresponding stage of group commit for that engine. For example, InnoDB needs no ordering of the prepare phase, so can omit implementing prepare_ordered(); TC_LOG_MMAP needs no ordering at all, so does not need to implement any of them. Note in particular that all existing engines (/binlog implementations if they exist) will work unmodified (and also without any change in group commit facilities or commit order guaranteed). Using these new APIs, the work will be to - In ha_commit_trans(), implement the correct semantics for the three new calls. - In XtraDB, use the new commit_ordered() call to remove the prepare_commit_mutex (and resurrect group commit) without loosing the consistency with binlog commit order. - In log.cc (binlog module), implement group_log_xid() to do group commit of multiple transactions to the binlog with a single shared fsync() call. ----------------------------------------------------------------------- Some possible alternative for this worklog: - We could eliminate the group_log_xid() method for a simpler API, at the cost of extra synchronisation between threads to do in-order commit_ordered() method calls. This would also allow to call commit_ordered() in the correct thread context. - Alternatively, we could eliminate log_xid() and require that all transaction coordinators implement group_log_xid() instead, again for some moderate simplification. - At the moment there is no plugin actually using prepare_ordered(), so, it could be removed from the design. But it fits in well, is efficient to implement, and could be useful later (eg. for the requested feature of releasing locks early in InnoDB). ----------------------------------------------------------------------- Some possible follow-up projects after this is implemented: - Add statistics about how efficient group commit is (#fsyncs/#commits in each engine and binlog). - Implement an XtraDB prepare_ordered() methods that can release row locks early (Mark Callaghan from Facebook advocates this, but need to determine exactly how to do this safely). - Implement a new crash recovery algorithm that uses the consistent commit ordering to need only fsync() for the binlog. At crash recovery, any missing transactions in an engine is replayed from the correct point in the binlog (this point must be stored transactionally inside the engine, as XtraDB already does today). - Implement that START TRANSACTION WITH CONSISTENT SNAPSHOT 1) really gets a consistent snapshow, with same set of committed and not committed transactions in all engines, 2) returns a corresponding consistent binlog position. This should be easy by piggybacking on the synchronisation implemented for ha_commit_trans(). - Use this in XtraBackup to get consistent binlog position without having to block all updates with FLUSH TABLES WITH READ LOCK. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 13:18)=-=- High Level Description modified. --- /tmp/wklog.116.old.14234 2010-05-25 13:18:07.000000000 +0000 +++ /tmp/wklog.116.new.14234 2010-05-25 13:18:07.000000000 +0000 @@ -21,3 +21,69 @@ http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html +---- + +Implementing group commit in MySQL faces some challenges from the handler +plugin architecture: + +1. Because storage engine handlers have separate transaction log from the +mysql binlog (and from each other), there are multiple fsync() calls per +commit that need the group commit optimisation (2 per participating storage +engine + 1 for binlog). + +2. The code handling commit is split in several places, in main server code +and in storage engine code. With pluggable binlog it will be split even +more. This requires a good abstract yet powerful API to be able to implement +group commit simply and efficiently in plugins without the different parts +having to rely on iternals of the others. + +3. We want the order of commits to be the same in all engines participating in +multiple transactions. This requirement is the reason that InnoDB currently +breaks group commit with the infamous prepare_commit_mutex. + +While currently there is no server guarantee to get same commit order in +engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are +several reasons why this could be desirable: + + - InnoDB hot backup needs to be able to extract a binlog position that is + consistent with the hot backup to be able to provision a new slave, and + this is impossible without imposing at least partial consistent ordering + between InnoDB and binlog. + + - Other backup methods could have similar needs, eg. XtraBackup or + `mysqldump --single-transaction`, to have consistent commit order between + binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK + or similar expensive blocking operation. (other backup methods, like LVM + snapshot, don't need consistent commit order, as they can restore + out-of-order commits during crash recovery using XA). + + - If we have consistent commit order, we can think about optimising commit to + need only one fsync (for binlog); lost commits in storage engines can then + be recovered from the binlog at crash recovery by re-playing against the + engine from a particular point in the binlog. + + - With consistent commit order, we can get better semantics for START + TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we + could even get it to return also a matching binlog position). Currently, + this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage + engines. + + - In InnoDB, the performance in the presense of hotspots can be improved if + we can release row locks early in the commit phase, but this requires that we +release them in + the same order as commits in the binlog to ensure consistency between + master and slaves. + + - There was some discussions around Galera [1] synchroneous replication and + global transaction ID that it needed consistent commit order among + participating engines. + + - I believe there could be other applications for guaranteed consistent + commit order, and that the architecture described in this worklog can + implement such guarantee with reasonable overhead. + + +References: + +[1] Galera: http://www.codership.com/products/galera_replication + -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ---- Implementing group commit in MySQL faces some challenges from the handler plugin architecture: 1. Because storage engine handlers have separate transaction log from the mysql binlog (and from each other), there are multiple fsync() calls per commit that need the group commit optimisation (2 per participating storage engine + 1 for binlog). 2. The code handling commit is split in several places, in main server code and in storage engine code. With pluggable binlog it will be split even more. This requires a good abstract yet powerful API to be able to implement group commit simply and efficiently in plugins without the different parts having to rely on iternals of the others. 3. We want the order of commits to be the same in all engines participating in multiple transactions. This requirement is the reason that InnoDB currently breaks group commit with the infamous prepare_commit_mutex. While currently there is no server guarantee to get same commit order in engines an binlog (except for the InnoDB prepare_commit_mutex hack), there are several reasons why this could be desirable: - InnoDB hot backup needs to be able to extract a binlog position that is consistent with the hot backup to be able to provision a new slave, and this is impossible without imposing at least partial consistent ordering between InnoDB and binlog. - Other backup methods could have similar needs, eg. XtraBackup or `mysqldump --single-transaction`, to have consistent commit order between binlog and storage engines without having to do FLUSH TABLES WITH READ LOCK or similar expensive blocking operation. (other backup methods, like LVM snapshot, don't need consistent commit order, as they can restore out-of-order commits during crash recovery using XA). - If we have consistent commit order, we can think about optimising commit to need only one fsync (for binlog); lost commits in storage engines can then be recovered from the binlog at crash recovery by re-playing against the engine from a particular point in the binlog. - With consistent commit order, we can get better semantics for START TRANSACTION WITH CONSISTENT SNAPSHOT with multi-engine transactions (and we could even get it to return also a matching binlog position). Currently, this "CONSISTENT SNAPSHOT" can be inconsistent among multiple storage engines. - In InnoDB, the performance in the presense of hotspots can be improved if we can release row locks early in the commit phase, but this requires that we release them in the same order as commits in the binlog to ensure consistency between master and slaves. - There was some discussions around Galera [1] synchroneous replication and global transaction ID that it needed consistent commit order among participating engines. - I believe there could be other applications for guaranteed consistent commit order, and that the architecture described in this worklog can implement such guarantee with reasonable overhead. References: [1] Galera: http://www.codership.com/products/galera_replication ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 2791: Prevent cacheing subqueries with random parameters and side effect functions. in file:///home/bell/maria/bzr/work-maria-5.3-scache/
by sanja＠askmonty.org 25 May '10

25 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache/ ------------------------------------------------------------ revno: 2791 revision-id: sanja(a)askmonty.org-20100525125457-5rwbiihh0vtghdrj parent: sanja(a)askmonty.org-20100525104536-zw06otfk8ut7fias committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache timestamp: Tue 2010-05-25 15:54:57 +0300 message: Prevent cacheing subqueries with random parameters and side effect functions. === modified file 'mysql-test/r/subquery_cache.result' --- a/mysql-test/r/subquery_cache.result 2010-05-25 10:45:36 +0000 +++ b/mysql-test/r/subquery_cache.result 2010-05-25 12:54:57 +0000 @@ -1,4 +1,5 @@ set optimizer_switch='subquery_cache=on'; +flush status; create table t1 (a int, b int); insert into t1 values (1,2),(3,4),(1,2),(3,4),(3,4),(4,5),(4,5),(5,6),(5,6),(4,5); create table t2 (c int, d int); @@ -552,4 +553,38 @@ POINT(1 1) POINT(3 3) DROP TABLE t1; +#uncacheable queries test (random and side effect) +flush status; +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (2), (4), (1), (3); +select a, a in (select a from t1) from t1 as ext; +a a in (select a from t1) +2 1 +4 1 +1 1 +3 1 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 4 +select a, a in (select a from t1 where -1 < rand()) from t1 as ext; +a a in (select a from t1 where -1 < rand()) +2 1 +4 1 +1 1 +3 1 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 4 +select a, a in (select a from t1 where -1 < benchmark(a,100)) from t1 as ext; +a a in (select a from t1 where -1 < benchmark(a,100)) +2 1 +4 1 +1 1 +3 1 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 4 set optimizer_switch='subquery_cache=default'; === modified file 'mysql-test/t/subquery_cache.test' --- a/mysql-test/t/subquery_cache.test 2010-05-25 10:45:36 +0000 +++ b/mysql-test/t/subquery_cache.test 2010-05-25 12:54:57 +0000 @@ -1,5 +1,6 @@ set optimizer_switch='subquery_cache=on'; +flush status; create table t1 (a int, b int); insert into t1 values (1,2),(3,4),(1,2),(3,4),(3,4),(4,5),(4,5),(5,6),(5,6),(4,5); @@ -188,4 +189,15 @@ DROP TABLE t1; +--echo #uncacheable queries test (random and side effect) +flush status; +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (2), (4), (1), (3); +select a, a in (select a from t1) from t1 as ext; +show status like "subquery_cache%"; +select a, a in (select a from t1 where -1 < rand()) from t1 as ext; +show status like "subquery_cache%"; +select a, a in (select a from t1 where -1 < benchmark(a,100)) from t1 as ext; +show status like "subquery_cache%"; + set optimizer_switch='subquery_cache=default'; === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-05-24 17:29:56 +0000 +++ b/sql/item_cmpfunc.cc 2010-05-25 12:54:57 +0000 @@ -1738,7 +1738,9 @@ const_item_cache&= args[1]->const_item(); DBUG_ASSERT(scache == NULL); if (args[0]->cols() ==1 && - thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE) + thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE && + !(sub->engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) { sub->depends_on.push_front((Item**)&cache); scache= new Subquery_cache_tmptable(thd, sub->depends_on, &result); === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-05-24 17:29:56 +0000 +++ b/sql/item_subselect.cc 2010-05-25 12:54:57 +0000 @@ -760,7 +760,10 @@ (uint)depends_on.elements, (uint)test(thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE))); engine->fix_length_and_dec(row= &value); - if (depends_on.elements && optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE)) + if (depends_on.elements && + optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && + !(engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) { DBUG_ASSERT(scache == NULL); scache= new Subquery_cache_tmptable(thd, depends_on, value); @@ -1100,7 +1103,9 @@ /* We need only 1 row to determine existence */ unit->global_parameters->select_limit= new Item_int((int32) 1); if (substype() == EXISTS_SUBS && depends_on.elements && - optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE)) + optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE) && + !(engine->uncacheable() & (UNCACHEABLE_RAND | + UNCACHEABLE_SIDEEFFECT))) { DBUG_ASSERT(scache == NULL); scache= new Subquery_cache_tmptable(thd, depends_on, &result);

1 0

[Maria-developers] Rev 2790: Fixed sum functions dependency for subqueries in file:///home/bell/maria/bzr/work-maria-5.3-scache/
by sanja＠askmonty.org 25 May '10

25 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache/ ------------------------------------------------------------ revno: 2790 revision-id: sanja(a)askmonty.org-20100525104536-zw06otfk8ut7fias parent: sanja(a)askmonty.org-20100524172956-7b14x01aodizr3sq committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache timestamp: Tue 2010-05-25 13:45:36 +0300 message: Fixed sum functions dependency for subqueries Forgoten files added. === added file 'mysql-test/r/subquery_cache.result' --- a/mysql-test/r/subquery_cache.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/subquery_cache.result 2010-05-25 10:45:36 +0000 @@ -0,0 +1,555 @@ +set optimizer_switch='subquery_cache=on'; +create table t1 (a int, b int); +insert into t1 values (1,2),(3,4),(1,2),(3,4),(3,4),(4,5),(4,5),(5,6),(5,6),(4,5); +create table t2 (c int, d int); +insert into t2 values (2,3),(3,4),(5,6); +#single value subquery test +select a, (select d from t2 where b=c) + 1 from t1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 6 +Subquery_cache_miss 4 +#single value subquery test (PS) +prepare stmt1 from 'select a, (select d from t2 where b=c) + 1 from t1'; +execute stmt1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 12 +Subquery_cache_miss 8 +execute stmt1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 18 +Subquery_cache_miss 12 +deallocate prepare stmt1; +#single value subquery test (SP) +CREATE PROCEDURE p1() select a, (select d from t2 where b=c) + 1 from t1; +call p1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +call p1; +a (select d from t2 where b=c) + 1 +1 4 +3 NULL +1 4 +3 NULL +3 NULL +4 7 +4 7 +5 NULL +5 NULL +4 7 +drop procedure p1; +#IN subquery test +flush status; +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 0 +Subquery_cache_miss 0 +select a, b , b in (select d from t2) as SUBS from t1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 6 +Subquery_cache_miss 4 +insert into t1 values (7,8),(9,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +7 8 0 +9 NULL NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 12 +Subquery_cache_miss 10 +insert into t2 values (8,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +7 8 NULL +9 NULL NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 18 +Subquery_cache_miss 16 +#IN subquery tesy (PS) +delete from t1 where a > 6; +delete from t2 where c > 6; +prepare stmt1 from 'select a, b , b in (select d from t2) as SUBS from t1'; +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 24 +Subquery_cache_miss 20 +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 30 +Subquery_cache_miss 24 +insert into t1 values (7,8),(9,NULL); +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 36 +Subquery_cache_miss 30 +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 42 +Subquery_cache_miss 36 +insert into t2 values (8,NULL); +execute stmt1; +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 48 +Subquery_cache_miss 42 +execute stmt1; +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 54 +Subquery_cache_miss 48 +deallocate prepare stmt1; +#IN subquery tesy (SP) +delete from t1 where a > 6; +delete from t2 where c > 6; +CREATE PROCEDURE p1() select a, b , b in (select d from t2) as SUBS from t1; +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 60 +Subquery_cache_miss 52 +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 66 +Subquery_cache_miss 56 +insert into t1 values (7,8),(9,NULL); +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 72 +Subquery_cache_miss 62 +call p1(); +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL NULL +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 78 +Subquery_cache_miss 68 +insert into t2 values (8,NULL); +call p1(); +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 84 +Subquery_cache_miss 74 +call p1(); +a b SUBS +1 2 NULL +3 4 1 +1 2 NULL +3 4 1 +3 4 1 +4 5 NULL +4 5 NULL +5 6 1 +5 6 1 +4 5 NULL +9 NULL NULL +7 8 NULL +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 90 +Subquery_cache_miss 80 +drop procedure p1; +# test of simple exists +select a, b , exists (select * from t2 where b=d) as SUBS from t1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +# test of prepared statement exists +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 96 +Subquery_cache_miss 86 +prepare stmt1 from 'select a, b , exists (select * from t2 where b=d) as SUBS from t1'; +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 102 +Subquery_cache_miss 92 +execute stmt1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +show status like "subquery_cache%"; +Variable_name Value +Subquery_cache_hit 108 +Subquery_cache_miss 98 +deallocate prepare stmt1; +# test of stored procedure exists +CREATE PROCEDURE p1() select a, b , exists (select * from t2 where b=d) as SUBS from t1; +call p1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +call p1; +a b SUBS +1 2 0 +3 4 1 +1 2 0 +3 4 1 +3 4 1 +4 5 0 +4 5 0 +5 6 1 +5 6 1 +4 5 0 +9 NULL 0 +7 8 0 +drop procedure p1; +#clean up +drop table t1,t2; +test different types +#int +CREATE TABLE t1 ( a int, b int); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +a +1 +3 +DROP TABLE t1; +#char +CREATE TABLE t1 ( a char(1), b char (1)); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#decimal +CREATE TABLE t1 ( a decimal(3,1), b decimal(3,1)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +a +1.0 +3.0 +DROP TABLE t1; +#date +CREATE TABLE t1 ( a date, b date); +INSERT INTO t1 VALUES('1000-01-01','1000-01-01'),('2000-02-01','2000-02-01'),('3000-03-03','3000-03-03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-01'); +a +1000-01-01 +3000-03-03 +DROP TABLE t1; +#datetime +CREATE TABLE t1 ( a datetime, b datetime); +INSERT INTO t1 VALUES('1000-01-01 01:01:01','1000-01-01 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('3000-03-03 03:03:03','3000-03-03 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +a +1000-01-01 01:01:01 +3000-03-03 03:03:03 +DROP TABLE t1; +#time +CREATE TABLE t1 ( a time, b time); +INSERT INTO t1 VALUES('01:01:01','01:01:01'),('02:02:02','02:02:02'),('03:03:03','03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '02:02:02'); +a +01:01:01 +03:03:03 +DROP TABLE t1; +#timestamp +CREATE TABLE t1 ( a timestamp, b timestamp); +INSERT INTO t1 VALUES('2000-02-02 01:01:01','2000-02-02 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('2000-02-02 03:03:03','2000-02-02 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +a +2000-02-02 01:01:01 +2000-02-02 03:03:03 +DROP TABLE t1; +#bit +CREATE TABLE t1 ( a bit(20), b bit(20)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a+0 FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +a+0 +1 +3 +DROP TABLE t1; +#enum +CREATE TABLE t1 ( a enum('1','2','3'), b enum('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#set +CREATE TABLE t1 ( a set('1','2','3'), b set('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#blob +CREATE TABLE t1 ( a blob, b blob); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +a +1 +3 +DROP TABLE t1; +#geometry +CREATE TABLE t1 ( a geometry, b geometry); +INSERT INTO t1 VALUES(POINT(1,1),POINT(1,1)),(POINT(2,2),POINT(2,2)),(POINT(3,3),POINT(3,3)); +SELECT astext(a) FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = POINT(2,2)); +astext(a) +POINT(1 1) +POINT(3 3) +DROP TABLE t1; +set optimizer_switch='subquery_cache=default'; === added file 'mysql-test/t/subquery_cache.test' --- a/mysql-test/t/subquery_cache.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/subquery_cache.test 2010-05-25 10:45:36 +0000 @@ -0,0 +1,191 @@ + +set optimizer_switch='subquery_cache=on'; + +create table t1 (a int, b int); +insert into t1 values (1,2),(3,4),(1,2),(3,4),(3,4),(4,5),(4,5),(5,6),(5,6),(4,5); +create table t2 (c int, d int); +insert into t2 values (2,3),(3,4),(5,6); + +--echo #single value subquery test +select a, (select d from t2 where b=c) + 1 from t1; + +show status like "subquery_cache%"; + +--echo #single value subquery test (PS) +prepare stmt1 from 'select a, (select d from t2 where b=c) + 1 from t1'; +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; +deallocate prepare stmt1; + +--echo #single value subquery test (SP) +CREATE PROCEDURE p1() select a, (select d from t2 where b=c) + 1 from t1; + +call p1; +call p1; + +drop procedure p1; + +--echo #IN subquery test +flush status; + +show status like "subquery_cache%"; +select a, b , b in (select d from t2) as SUBS from t1; +show status like "subquery_cache%"; + +insert into t1 values (7,8),(9,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +show status like "subquery_cache%"; + +insert into t2 values (8,NULL); +select a, b , b in (select d from t2) as SUBS from t1; +show status like "subquery_cache%"; + +--echo #IN subquery tesy (PS) +delete from t1 where a > 6; +delete from t2 where c > 6; + +prepare stmt1 from 'select a, b , b in (select d from t2) as SUBS from t1'; +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; + +insert into t1 values (7,8),(9,NULL); +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; + +insert into t2 values (8,NULL); +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; + +deallocate prepare stmt1; + + +--echo #IN subquery tesy (SP) +delete from t1 where a > 6; +delete from t2 where c > 6; + +CREATE PROCEDURE p1() select a, b , b in (select d from t2) as SUBS from t1; + +call p1(); +show status like "subquery_cache%"; +call p1(); +show status like "subquery_cache%"; + +insert into t1 values (7,8),(9,NULL); +call p1(); +show status like "subquery_cache%"; +call p1(); +show status like "subquery_cache%"; + +insert into t2 values (8,NULL); +call p1(); +show status like "subquery_cache%"; +call p1(); +show status like "subquery_cache%"; + +drop procedure p1; + + +--echo # test of simple exists +select a, b , exists (select * from t2 where b=d) as SUBS from t1; + +--echo # test of prepared statement exists +show status like "subquery_cache%"; +prepare stmt1 from 'select a, b , exists (select * from t2 where b=d) as SUBS from t1'; +execute stmt1; +show status like "subquery_cache%"; +execute stmt1; +show status like "subquery_cache%"; +deallocate prepare stmt1; + +--echo # test of stored procedure exists +CREATE PROCEDURE p1() select a, b , exists (select * from t2 where b=d) as SUBS from t1; +call p1; +call p1; +drop procedure p1; + +--echo #clean up +drop table t1,t2; + +--echo test different types +--echo #int +CREATE TABLE t1 ( a int, b int); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +DROP TABLE t1; + +--echo #char +CREATE TABLE t1 ( a char(1), b char (1)); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #decimal +CREATE TABLE t1 ( a decimal(3,1), b decimal(3,1)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +DROP TABLE t1; + +--echo #date +CREATE TABLE t1 ( a date, b date); +INSERT INTO t1 VALUES('1000-01-01','1000-01-01'),('2000-02-01','2000-02-01'),('3000-03-03','3000-03-03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-01'); +DROP TABLE t1; + +--echo #datetime +CREATE TABLE t1 ( a datetime, b datetime); +INSERT INTO t1 VALUES('1000-01-01 01:01:01','1000-01-01 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('3000-03-03 03:03:03','3000-03-03 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +DROP TABLE t1; + +--echo #time +CREATE TABLE t1 ( a time, b time); +INSERT INTO t1 VALUES('01:01:01','01:01:01'),('02:02:02','02:02:02'),('03:03:03','03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '02:02:02'); +DROP TABLE t1; + +--echo #timestamp +CREATE TABLE t1 ( a timestamp, b timestamp); +INSERT INTO t1 VALUES('2000-02-02 01:01:01','2000-02-02 01:01:01'),('2000-02-02 02:02:02','2000-02-02 02:02:02'),('2000-02-02 03:03:03','2000-02-02 03:03:03'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2000-02-02 02:02:02'); +DROP TABLE t1; + +--echo #bit +CREATE TABLE t1 ( a bit(20), b bit(20)); +INSERT INTO t1 VALUES(1,1),(2,2),(3,3); +SELECT a+0 FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = 2); +DROP TABLE t1; + +--echo #enum +CREATE TABLE t1 ( a enum('1','2','3'), b enum('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #set +CREATE TABLE t1 ( a set('1','2','3'), b set('1','2','3')); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #blob +CREATE TABLE t1 ( a blob, b blob); +INSERT INTO t1 VALUES('1','1'),('2','2'),('3','3'); +SELECT a FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = '2'); +DROP TABLE t1; + +--echo #geometry +CREATE TABLE t1 ( a geometry, b geometry); +INSERT INTO t1 VALUES(POINT(1,1),POINT(1,1)),(POINT(2,2),POINT(2,2)),(POINT(3,3),POINT(3,3)); +SELECT astext(a) FROM t1 WHERE NOT a IN (SELECT a FROM t1 WHERE b = POINT(2,2)); +DROP TABLE t1; + + +set optimizer_switch='subquery_cache=default'; === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_sum.cc 2010-05-25 10:45:36 +0000 @@ -319,6 +319,7 @@ if (aggr_level >= 0) { ref_by= ref; + thd->lex->current_select->register_dependency_item(aggr_sel, ref); /* Add the object to the list of registered objects assigned to aggr_sel */ if (!aggr_sel->inner_sum_func_list) next= this; === added file 'sql/sql_subquery_cache.cc' --- a/sql/sql_subquery_cache.cc 1970-01-01 00:00:00 +0000 +++ b/sql/sql_subquery_cache.cc 2010-05-25 10:45:36 +0000 @@ -0,0 +1,319 @@ + +#include "mysql_priv.h" +#include "sql_select.h" + +ulonglong subquery_cache_miss, subquery_cache_hit; + +/** + Creates structures which we need for index look up + + @retval FALSE OK + @retval TRUE Error +*/ + +static my_bool createtmp_table_search_structures(THD *thd, + TABLE *table, + List_iterator_fast<Item> &li, + TABLE_REF **ref) +{ + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + TABLE_REF *tab_ref; + KEY *tmp_key; /* The only index on the temporary table. */ + Item *item; + uint tmp_key_parts; /* Number of keyparts in tmp_key. */ + uint i; + + DBUG_ENTER("createtmp_table_search_structures"); + + tmp_key= table->key_info; + tmp_key_parts= tmp_key->key_parts; + + if (!(tab_ref= (TABLE_REF*) thd->alloc(sizeof(TABLE_REF)))) + DBUG_RETURN(TRUE); + + tab_ref->key= 0; /* The only temp table index. */ + tab_ref->key_length= tmp_key->key_length; + if (!(tab_ref->key_buff= + (uchar*) thd->calloc(ALIGN_SIZE(tmp_key->key_length) * 2)) || + !(tab_ref->key_copy= + (store_key**) thd->alloc((sizeof(store_key*) * + (tmp_key_parts + 1)))) || + !(tab_ref->items= + (Item**) thd->alloc(sizeof(Item*) * tmp_key_parts))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + + tab_ref->key_buff2=tab_ref->key_buff+ALIGN_SIZE(tmp_key->key_length); + tab_ref->key_err=1; + tab_ref->null_rejecting= 1; + tab_ref->disable_cache= FALSE; + tab_ref->has_record= 0; + + KEY_PART_INFO *cur_key_part= tmp_key->key_part; + store_key **ref_key= tab_ref->key_copy; + uchar *cur_ref_buff= tab_ref->key_buff; + + for (i= 0; i < tmp_key_parts; i++, cur_key_part++, ref_key++) + { + item= li++; + DBUG_ASSERT(item); + tab_ref->items[i]= item; + int null_count= test(cur_key_part->field->real_maybe_null()); + *ref_key= new store_key_item(thd, cur_key_part->field, + /* TODO: + the NULL byte is taken into account in + cur_key_part->store_length, so instead of + cur_ref_buff + test(maybe_null), we could + use that information instead. + */ + cur_ref_buff + null_count, + null_count ? tab_ref->key_buff : 0, + cur_key_part->length, tab_ref->items[i]); + cur_ref_buff+= cur_key_part->store_length; + } + *ref_key= NULL; /* End marker. */ + tab_ref->key_err= 1; + tab_ref->key_parts= tmp_key_parts; + *ref= tab_ref; + + DBUG_RETURN(FALSE); +} + + +Subquery_cache_tmptable::Subquery_cache_tmptable(THD *thd, + List<Item*> &dependance, + Item *value) + :cache_table(NULL), table_thd(thd), list(&dependance), val(value), + equalities(NULL), inited (0) +{ + DBUG_ENTER("Subquery_cache_tmptable::Subquery_cache_tmptable"); + DBUG_VOID_RETURN; +}; + + +/** + Creates equalities expression. + + @retval FALSE OK + @retval TRUE Error +*/ + +bool Subquery_cache_tmptable::make_equalities() +{ + List<Item> args; + List_iterator_fast<Item*> li(*list); + Item **ref; + Name_resolution_context *cn= NULL; + DBUG_ENTER("Subquery_cache_tmptable::make_equalities"); + + for (uint i= 1 /* skip result filed */; (ref= li++); i++) + { + Field *fld= cache_table->field[i]; + if (fld->type() == MYSQL_TYPE_VARCHAR || + fld->type() == MYSQL_TYPE_TINY_BLOB || + fld->type() == MYSQL_TYPE_MEDIUM_BLOB || + fld->type() == MYSQL_TYPE_LONG_BLOB || + fld->type() == MYSQL_TYPE_BLOB || + fld->type() == MYSQL_TYPE_VAR_STRING || + fld->type() == MYSQL_TYPE_STRING || + fld->type() == MYSQL_TYPE_NEWDECIMAL || + fld->type() == MYSQL_TYPE_DECIMAL) + { + if (!cn) + { + // dummy resolution context + cn= new Name_resolution_context(); + cn->init(); + } + args.push_front(new Item_func_eq(new Item_ref(cn, ref, "", "", FALSE), + new Item_field(fld))); + } + } + if (args.elements == 1) + equalities= args.head(); + else + equalities= new Item_cond_and(args); + + DBUG_RETURN(equalities->fix_fields(table_thd, &equalities)); +} + +void Subquery_cache_tmptable::init() +{ + ulonglong keymap; + List_iterator_fast<Item*> li(*list); + List_iterator_fast<Item> li_items(items); + Item **item; + DBUG_ENTER("Subquery_cache_tmptable::init"); + DBUG_ASSERT(!inited); + inited= TRUE; + + if (!(ULONGLONG_MAX >> (list->elements + 1))) + { + DBUG_PRINT("info", ("Too many dependencies")); + DBUG_VOID_RETURN; + } + + cache_table= NULL; + while ((item= li++)) + { + DBUG_ASSERT(item); + DBUG_ASSERT(*item); + DBUG_ASSERT((*item)->fixed); + items.push_back((*item)); + } + + cache_table_param.init(); + /* dependance items and result */ + cache_table_param.field_count= list->elements + 1; + /* postpone table creation to index description */ + cache_table_param.skip_create_table= 1; + + + items.push_front(val); + if (!(cache_table= create_tmp_table(table_thd, &cache_table_param, + items, (ORDER*) NULL, + FALSE, FALSE, + (table_thd->options | + TMP_TABLE_ALL_COLUMNS), + HA_POS_ERROR, + (char *)"subquery-cache-table"))) + { + DBUG_PRINT("error", ("create_tmp_table failed, caching switched off")); + DBUG_VOID_RETURN; + } + + if (cache_table->s->blob_fields) + { + DBUG_PRINT("error", ("we do not need blobs")); + goto error; + } + + /* makes all bits set for keys */ + keymap= 1 << (items.elements); /* + 1 - 1 */ + if (!keymap) + keymap= ULONGLONG_MAX; + else + keymap--; + keymap&=~1; + + li_items++; + if (cache_table->alloc_keys(1) || + (cache_table->add_tmp_key(keymap, "cache-table-key") < 0) || + createtmp_table_search_structures(table_thd, cache_table, li_items, + &tab_ref) || + !(tab= create_index_lookup_join_tab(cache_table))) + { + DBUG_PRINT("error", ("creating index failed")); + goto error; + } + cache_table->s->keys= 1; + cache_table->s->uniques= 1; + + if (open_tmp_table(cache_table)) + { + DBUG_PRINT("error", ("Opening (creating) temporary table failed")); + goto error; + } + + if (!(chached_result= new Item_field(cache_table->field[0]))) + { + DBUG_PRINT("error", ("Creating Item_field failed")); + goto error; + } + if (make_equalities()) + { + DBUG_PRINT("error", ("Creating equalities failed")); + goto error; + } + + DBUG_VOID_RETURN; + +error: + /* switch off cache */ + free_tmp_table(table_thd, cache_table); + cache_table= NULL; + DBUG_VOID_RETURN; +} + + +Subquery_cache_tmptable::~Subquery_cache_tmptable() +{ + if (cache_table) + free_tmp_table(table_thd, cache_table); +} + + +Subquery_cache::result Subquery_cache_tmptable::check_value(Item **value) +{ + int res; + DBUG_ENTER("Subquery_cache_tmptable::check_value"); + + if (!inited) + init(); + + if (cache_table) + { + DBUG_PRINT("info", ("status: %u has_record %u", + (uint)cache_table->status, (uint)tab_ref->has_record)); + if ((res= join_read_key2(table_thd, tab, cache_table, tab_ref)) == 1) + DBUG_RETURN(ERROR); + if (res || (equalities && !equalities->val_int())) + { + subquery_cache_miss++; + DBUG_RETURN(MISS); + } + + subquery_cache_hit++; + *value= chached_result; + DBUG_RETURN(Subquery_cache::HIT); + } + DBUG_RETURN(Subquery_cache::MISS); +} + + +my_bool Subquery_cache_tmptable::put_value(Item *value) +{ + int error; + DBUG_ENTER("Subquery_cache_tmptable::put_value"); + DBUG_ASSERT(inited); + + if (!cache_table) + { + DBUG_PRINT("info", ("No table so behave as we successfully put value")); + DBUG_RETURN(FALSE); + } + + *(items.head_ref())= value; + fill_record(table_thd, cache_table->field, items, 1); + if (table_thd->is_error()) + goto err;; + + if ((error= cache_table->file->ha_write_row(cache_table->record[0]))) + { + /* create_myisam_from_heap will generate error if needed */ + if (cache_table->file->is_fatal_error(error, HA_CHECK_DUP) && + create_internal_tmp_table_from_heap(table_thd, cache_table, + cache_table_param.start_recinfo, + &cache_table_param.recinfo, + error, 1)) + goto err; + } + cache_table->status= 0; /* cache_table->record contains an existed record */ + tab_ref->has_record= TRUE; /* the same as above */ + DBUG_PRINT("info", ("has_record: TRUE status: 0")); + + DBUG_RETURN(FALSE); + +err: + free_tmp_table(table_thd, cache_table); + cache_table= NULL; + DBUG_RETURN(TRUE); +} + + +void Subquery_cache_tmptable::cleanup() +{ + cache_table->file->ha_delete_all_rows(); +} === added file 'sql/sql_subquery_cache.h' --- a/sql/sql_subquery_cache.h 1970-01-01 00:00:00 +0000 +++ b/sql/sql_subquery_cache.h 2010-05-25 10:45:36 +0000 @@ -0,0 +1,79 @@ +#ifndef _SQL_SUBQUERY_CACHE_H_ +#define _SQL_SUBQUERY_CACHE_H_ + +/** + Interface for subquery cache +*/ + +extern ulonglong subquery_cache_miss, subquery_cache_hit; + +class Subquery_cache :public Sql_alloc +{ +public: + enum result {ERROR, HIT, MISS}; + + Subquery_cache(){}; + virtual ~Subquery_cache() {}; + /** + Checks presence of the key (taken from cache owner) and if found return + it via value parameter + */ + virtual result check_value(Item **value)= 0; + /** + Puts value into this cache (key should be taken from cache owner) + */ + virtual my_bool put_value(Item *value)= 0; + /** + Cleans up and reset cache before reusing + */ + virtual void cleanup()= 0; +}; + +struct st_table_ref; +struct st_join_table; +//class Item_cache; +class Item_field; + +/** + Implementation of subquery cache over temporary table +*/ + +class Subquery_cache_tmptable :public Subquery_cache +{ +public: + Subquery_cache_tmptable(THD *thd, List<Item*> &dependance, Item *value); + virtual ~Subquery_cache_tmptable(); + virtual result check_value(Item **value); + virtual my_bool put_value(Item *value); + virtual void cleanup(); + void init(); + +private: + bool make_equalities(); + + /* tmp table parameters */ + TMP_TABLE_PARAM cache_table_param; + /* temporary table to store this cache */ + TABLE *cache_table; + /* Thread handler for the temporary table */ + THD *table_thd; + /* tab_ref for index search */ + struct st_table_ref *tab_ref; + /* cache of subquery value to avoid evaluating it twice */ + //Item_cache *value_cache; + /* JOIN_TAB for index lookup */ + st_join_table *tab; + /* Chached result */ + Item_field *chached_result; + /* List of references to items */ + List<Item*> *list; + /* List of items */ + List<Item> items; + /* Value Item example */ + Item *val; + /* Expression to check after index lookup */ + Item *equalities; + /* set if structures are inited */ + bool inited; +}; +#endif

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 31 ESTIMATE.......: 4 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint ------------------------------------------------------------ -=-=(View All Progress Notes, 31 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 31 ESTIMATE.......: 4 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint ------------------------------------------------------------ -=-=(View All Progress Notes, 31 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 31 ESTIMATE.......: 4 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint ------------------------------------------------------------ -=-=(View All Progress Notes, 31 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 31 ESTIMATE.......: 4 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 08:29)=-=- Help debug strange problem in mysqlbinlog.test. Worked 1 hour and estimate 4 hours remain (original estimate unchanged). -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint ------------------------------------------------------------ -=-=(View All Progress Notes, 31 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 49 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Tue, 25 May 2010, 08:28)=-=- More thoughts on and changes to the archtecture. Got to something now that I am satisfied with and that seems to be able to handle all issues. Implement new prepare_ordered and commit_ordered handler methods and the logic in ha_commit_trans(). Implement TC_LOG::group_log_xid() method and logic in ha_commit_trans(). Implement XtraDB part, using commit_ordered() rather than prepare_commit_mutex. Fix test suite failures. Proof-of-concept patch series complete now. Do initial benchmark, getting good results. With 64 threads, see 26x improvement in queries-per-sec. Next step: write up the architecture description. Worked 21 hours and estimate 0 hours remain (original estimate increased by 21 hours). -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Push conditions down into non-mergeable VIEWs when possible (119)
by worklog-noreply＠askmonty.org 25 May '10

25 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Push conditions down into non-mergeable VIEWs when possible CREATION DATE..: Mon, 24 May 2010, 20:52 SUPERVISOR.....: Igor IMPLEMENTOR....: COPIES TO......: Psergey, Timour CATEGORY.......: Server-RawIdeaBin TASK ID........: 119 (http://askmonty.org/worklog/?tid=119) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 24 May 2010, 20:59)=-=- High-Level Specification modified. --- /tmp/wklog.119.old.25116 2010-05-24 20:59:40.000000000 +0000 +++ /tmp/wklog.119.new.25116 2010-05-24 20:59:40.000000000 +0000 @@ -1 +1,113 @@ +<contents> +HLS +1. Problems to be addressed in this WL +2. Pushdown of conditions into non-mergeable VIEWs +2.1 A note about VIEWs with ORDER BY ... LIMIT +2.2 What condition can be pushed +3. Pushdown from HAVING into WHERE +4. When to do the pushdown +5. Other things to take care of + + +</contents> + +1. Problems to be addressed in this WL +====================================== +The problem actually consists of two parts: +1. Condition on VIEW columns are not pushed down into VIEWs. +2. Even if conditions were pushed, they would have been put into VIEW's +HAVING clause, which would not give enough of speedup. In order to get a +real speedup, the optimizer must be able to move relevant part of HAVING +into WHERE (and then use it for further optimizations) in order to provide +the desired speedup. Note that HAVING->WHERE condition move is orthogonal +to VIEW processing. + +2. Pushdown of conditions into non-mergeable VIEWs +================================================== +We can push a condition into non-mergeable VIEW when VIEW's top-most operation +is selection (i.e., filtering). This is true, for example, when the VIEW is +defined as + + SELECT select_list FROM from_clause [WHERE where_cond] [HAVING having_cond] + +and not true when the VIEW is defined as + + SELECT select_list FROM from_clause [WHERE where_cond] ORDER BY expr LIMIT n + +Generalizing the above, we arrive at the following rule: + + For non-mergeable VIEWs, + - pushdown must not be done if VIEW uses ORDER BY .. LIMIT + - when pushdown is done, the pushed condition should be added to the WHERE + clause. + +Note: In scope of this WL, we will not hande VIEWs that have UNION [ALL] as +top operation. + +(TODO: what about SELECT DISTINCT?) +(TODO: pushdown down into IN subqueries?) + +2.1 A note about VIEWs with ORDER BY ... LIMIT +---------------------------------------------- +Although it is not possible to push a condition below the ORDER BY ... LIMIT +operation, there is still some benefit from checking the condition early as +that would allow to avoid writing non-matching rows into temporary table. + +We could do that if we introduced a post-ORDERBY selection operation. That +operation would also allow to support ORDER BY ... LIMIT inside subqueries +(we don't currently support those because default subquery strategy, +IN->EXISTS rewrite, also needs to push a condition into subquery). + +2.2 What condition can be pushed +-------------------------------- +Assuming simplify_joins() operation has done normalization: +* If the VIEW is in top-level join list, or inside a semi-join that's in + top-level join list, we can push parts of WHERE condition. +* If the VIEW is inside an outer join, we can push it's ON expression. + +We can reuse make_cond_for_index()/make_remainder_cond() code to extract part +of condition that can be pushed, and the remainder, respectively. + +Pushability criteria for an atomic (i.e. not AND/OR) condition is that + + the condition only uses VIEW's fields. + +(TODO: what about fields of const tables? Do we have const tables already +retrived by the time VIEW is materialized? If yes, we could push down +expressions that refer to const tables, too) + +3. Pushdown from HAVING into WHERE +================================== +The idea is: + + Parts of HAVING that refer to columns/expressions we're doing GROUP BY on + can be put into WHERE. + +(TODO: do we need to handle case of grouping over expressions?) + +(TODO: when moving expression for HAVING to WHERE, do we need +to do something with it? Replace all Item_ref objects with items that +they refer to? + - In case of referring to expression, do we get + Item_ref(where_clause_expr) or expr( Item_ref(..), .., Item_ref(..))? +) + +4. When to do the pushdown +========================== +In order to do pushdown, we must have prepare phase finished +for both parent (so that we can make sense of its WHERE condition) and +child (so that we know what it has in its select list). + +We can do pushdown before we've done join optimization (i.e. choose_plan() +call) of the parent. + +We must do pushdown before we've done JOIN::optimize() of the child +(in particular, it must be done before we do update_ref_and_keys() and +range analysis in the child). + + +5. Other things to take care of +=============================== +* Pushing down fulltext predicates (it seems one needs to "register" a + fulltext predicate when it is moved from one select from another? Ask Serg) DESCRIPTION: There are complaints (see links below) about cases with non-mergeable VIEW (because it has a GROUP BY), a query that has restrictions on the grouped column, and poor performance that is caused by VIEW processing code ignoring the restriction. This WL is about addressing this issue. links to complaints: http://code.openark.org/blog/mysql/views-better-performance-with-condition-… http://www.mysqlperformanceblog.com/2010/05/19/a-workaround-for-the-perform… The target version is MariaDB 5.3+, because it has late optimization/execution for FROM-subqueries/non mergeable VIEWs, which makes it much more feasible to inject something into VIEW before it is optimized/executed. HIGH-LEVEL SPECIFICATION: <contents> HLS 1. Problems to be addressed in this WL 2. Pushdown of conditions into non-mergeable VIEWs 2.1 A note about VIEWs with ORDER BY ... LIMIT 2.2 What condition can be pushed 3. Pushdown from HAVING into WHERE 4. When to do the pushdown 5. Other things to take care of </contents> 1. Problems to be addressed in this WL ====================================== The problem actually consists of two parts: 1. Condition on VIEW columns are not pushed down into VIEWs. 2. Even if conditions were pushed, they would have been put into VIEW's HAVING clause, which would not give enough of speedup. In order to get a real speedup, the optimizer must be able to move relevant part of HAVING into WHERE (and then use it for further optimizations) in order to provide the desired speedup. Note that HAVING->WHERE condition move is orthogonal to VIEW processing. 2. Pushdown of conditions into non-mergeable VIEWs ================================================== We can push a condition into non-mergeable VIEW when VIEW's top-most operation is selection (i.e., filtering). This is true, for example, when the VIEW is defined as SELECT select_list FROM from_clause [WHERE where_cond] [HAVING having_cond] and not true when the VIEW is defined as SELECT select_list FROM from_clause [WHERE where_cond] ORDER BY expr LIMIT n Generalizing the above, we arrive at the following rule: For non-mergeable VIEWs, - pushdown must not be done if VIEW uses ORDER BY .. LIMIT - when pushdown is done, the pushed condition should be added to the WHERE clause. Note: In scope of this WL, we will not hande VIEWs that have UNION [ALL] as top operation. (TODO: what about SELECT DISTINCT?) (TODO: pushdown down into IN subqueries?) 2.1 A note about VIEWs with ORDER BY ... LIMIT ---------------------------------------------- Although it is not possible to push a condition below the ORDER BY ... LIMIT operation, there is still some benefit from checking the condition early as that would allow to avoid writing non-matching rows into temporary table. We could do that if we introduced a post-ORDERBY selection operation. That operation would also allow to support ORDER BY ... LIMIT inside subqueries (we don't currently support those because default subquery strategy, IN->EXISTS rewrite, also needs to push a condition into subquery). 2.2 What condition can be pushed -------------------------------- Assuming simplify_joins() operation has done normalization: * If the VIEW is in top-level join list, or inside a semi-join that's in top-level join list, we can push parts of WHERE condition. * If the VIEW is inside an outer join, we can push it's ON expression. We can reuse make_cond_for_index()/make_remainder_cond() code to extract part of condition that can be pushed, and the remainder, respectively. Pushability criteria for an atomic (i.e. not AND/OR) condition is that the condition only uses VIEW's fields. (TODO: what about fields of const tables? Do we have const tables already retrived by the time VIEW is materialized? If yes, we could push down expressions that refer to const tables, too) 3. Pushdown from HAVING into WHERE ================================== The idea is: Parts of HAVING that refer to columns/expressions we're doing GROUP BY on can be put into WHERE. (TODO: do we need to handle case of grouping over expressions?) (TODO: when moving expression for HAVING to WHERE, do we need to do something with it? Replace all Item_ref objects with items that they refer to? - In case of referring to expression, do we get Item_ref(where_clause_expr) or expr( Item_ref(..), .., Item_ref(..))? ) 4. When to do the pushdown ========================== In order to do pushdown, we must have prepare phase finished for both parent (so that we can make sense of its WHERE condition) and child (so that we know what it has in its select list). We can do pushdown before we've done join optimization (i.e. choose_plan() call) of the parent. We must do pushdown before we've done JOIN::optimize() of the child (in particular, it must be done before we do update_ref_and_keys() and range analysis in the child). 5. Other things to take care of =============================== * Pushing down fulltext predicates (it seems one needs to "register" a fulltext predicate when it is moved from one select from another? Ask Serg) ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

3 2

[Maria-developers] Updated (by Psergey): Push conditions down into non-mergeable VIEWs when possible (119)
by worklog-noreply＠askmonty.org 24 May '10

24 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Push conditions down into non-mergeable VIEWs when possible CREATION DATE..: Mon, 24 May 2010, 20:52 SUPERVISOR.....: Igor IMPLEMENTOR....: COPIES TO......: Psergey, Timour CATEGORY.......: Server-RawIdeaBin TASK ID........: 119 (http://askmonty.org/worklog/?tid=119) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 24 May 2010, 20:59)=-=- High-Level Specification modified. --- /tmp/wklog.119.old.25116 2010-05-24 20:59:40.000000000 +0000 +++ /tmp/wklog.119.new.25116 2010-05-24 20:59:40.000000000 +0000 @@ -1 +1,113 @@ +<contents> +HLS +1. Problems to be addressed in this WL +2. Pushdown of conditions into non-mergeable VIEWs +2.1 A note about VIEWs with ORDER BY ... LIMIT +2.2 What condition can be pushed +3. Pushdown from HAVING into WHERE +4. When to do the pushdown +5. Other things to take care of + + +</contents> + +1. Problems to be addressed in this WL +====================================== +The problem actually consists of two parts: +1. Condition on VIEW columns are not pushed down into VIEWs. +2. Even if conditions were pushed, they would have been put into VIEW's +HAVING clause, which would not give enough of speedup. In order to get a +real speedup, the optimizer must be able to move relevant part of HAVING +into WHERE (and then use it for further optimizations) in order to provide +the desired speedup. Note that HAVING->WHERE condition move is orthogonal +to VIEW processing. + +2. Pushdown of conditions into non-mergeable VIEWs +================================================== +We can push a condition into non-mergeable VIEW when VIEW's top-most operation +is selection (i.e., filtering). This is true, for example, when the VIEW is +defined as + + SELECT select_list FROM from_clause [WHERE where_cond] [HAVING having_cond] + +and not true when the VIEW is defined as + + SELECT select_list FROM from_clause [WHERE where_cond] ORDER BY expr LIMIT n + +Generalizing the above, we arrive at the following rule: + + For non-mergeable VIEWs, + - pushdown must not be done if VIEW uses ORDER BY .. LIMIT + - when pushdown is done, the pushed condition should be added to the WHERE + clause. + +Note: In scope of this WL, we will not hande VIEWs that have UNION [ALL] as +top operation. + +(TODO: what about SELECT DISTINCT?) +(TODO: pushdown down into IN subqueries?) + +2.1 A note about VIEWs with ORDER BY ... LIMIT +---------------------------------------------- +Although it is not possible to push a condition below the ORDER BY ... LIMIT +operation, there is still some benefit from checking the condition early as +that would allow to avoid writing non-matching rows into temporary table. + +We could do that if we introduced a post-ORDERBY selection operation. That +operation would also allow to support ORDER BY ... LIMIT inside subqueries +(we don't currently support those because default subquery strategy, +IN->EXISTS rewrite, also needs to push a condition into subquery). + +2.2 What condition can be pushed +-------------------------------- +Assuming simplify_joins() operation has done normalization: +* If the VIEW is in top-level join list, or inside a semi-join that's in + top-level join list, we can push parts of WHERE condition. +* If the VIEW is inside an outer join, we can push it's ON expression. + +We can reuse make_cond_for_index()/make_remainder_cond() code to extract part +of condition that can be pushed, and the remainder, respectively. + +Pushability criteria for an atomic (i.e. not AND/OR) condition is that + + the condition only uses VIEW's fields. + +(TODO: what about fields of const tables? Do we have const tables already +retrived by the time VIEW is materialized? If yes, we could push down +expressions that refer to const tables, too) + +3. Pushdown from HAVING into WHERE +================================== +The idea is: + + Parts of HAVING that refer to columns/expressions we're doing GROUP BY on + can be put into WHERE. + +(TODO: do we need to handle case of grouping over expressions?) + +(TODO: when moving expression for HAVING to WHERE, do we need +to do something with it? Replace all Item_ref objects with items that +they refer to? + - In case of referring to expression, do we get + Item_ref(where_clause_expr) or expr( Item_ref(..), .., Item_ref(..))? +) + +4. When to do the pushdown +========================== +In order to do pushdown, we must have prepare phase finished +for both parent (so that we can make sense of its WHERE condition) and +child (so that we know what it has in its select list). + +We can do pushdown before we've done join optimization (i.e. choose_plan() +call) of the parent. + +We must do pushdown before we've done JOIN::optimize() of the child +(in particular, it must be done before we do update_ref_and_keys() and +range analysis in the child). + + +5. Other things to take care of +=============================== +* Pushing down fulltext predicates (it seems one needs to "register" a + fulltext predicate when it is moved from one select from another? Ask Serg) DESCRIPTION: There are complaints (see links below) about cases with non-mergeable VIEW (because it has a GROUP BY), a query that has restrictions on the grouped column, and poor performance that is caused by VIEW processing code ignoring the restriction. This WL is about addressing this issue. links to complaints: http://code.openark.org/blog/mysql/views-better-performance-with-condition-… http://www.mysqlperformanceblog.com/2010/05/19/a-workaround-for-the-perform… The target version is MariaDB 5.3+, because it has late optimization/execution for FROM-subqueries/non mergeable VIEWs, which makes it much more feasible to inject something into VIEW before it is optimized/executed. HIGH-LEVEL SPECIFICATION: <contents> HLS 1. Problems to be addressed in this WL 2. Pushdown of conditions into non-mergeable VIEWs 2.1 A note about VIEWs with ORDER BY ... LIMIT 2.2 What condition can be pushed 3. Pushdown from HAVING into WHERE 4. When to do the pushdown 5. Other things to take care of </contents> 1. Problems to be addressed in this WL ====================================== The problem actually consists of two parts: 1. Condition on VIEW columns are not pushed down into VIEWs. 2. Even if conditions were pushed, they would have been put into VIEW's HAVING clause, which would not give enough of speedup. In order to get a real speedup, the optimizer must be able to move relevant part of HAVING into WHERE (and then use it for further optimizations) in order to provide the desired speedup. Note that HAVING->WHERE condition move is orthogonal to VIEW processing. 2. Pushdown of conditions into non-mergeable VIEWs ================================================== We can push a condition into non-mergeable VIEW when VIEW's top-most operation is selection (i.e., filtering). This is true, for example, when the VIEW is defined as SELECT select_list FROM from_clause [WHERE where_cond] [HAVING having_cond] and not true when the VIEW is defined as SELECT select_list FROM from_clause [WHERE where_cond] ORDER BY expr LIMIT n Generalizing the above, we arrive at the following rule: For non-mergeable VIEWs, - pushdown must not be done if VIEW uses ORDER BY .. LIMIT - when pushdown is done, the pushed condition should be added to the WHERE clause. Note: In scope of this WL, we will not hande VIEWs that have UNION [ALL] as top operation. (TODO: what about SELECT DISTINCT?) (TODO: pushdown down into IN subqueries?) 2.1 A note about VIEWs with ORDER BY ... LIMIT ---------------------------------------------- Although it is not possible to push a condition below the ORDER BY ... LIMIT operation, there is still some benefit from checking the condition early as that would allow to avoid writing non-matching rows into temporary table. We could do that if we introduced a post-ORDERBY selection operation. That operation would also allow to support ORDER BY ... LIMIT inside subqueries (we don't currently support those because default subquery strategy, IN->EXISTS rewrite, also needs to push a condition into subquery). 2.2 What condition can be pushed -------------------------------- Assuming simplify_joins() operation has done normalization: * If the VIEW is in top-level join list, or inside a semi-join that's in top-level join list, we can push parts of WHERE condition. * If the VIEW is inside an outer join, we can push it's ON expression. We can reuse make_cond_for_index()/make_remainder_cond() code to extract part of condition that can be pushed, and the remainder, respectively. Pushability criteria for an atomic (i.e. not AND/OR) condition is that the condition only uses VIEW's fields. (TODO: what about fields of const tables? Do we have const tables already retrived by the time VIEW is materialized? If yes, we could push down expressions that refer to const tables, too) 3. Pushdown from HAVING into WHERE ================================== The idea is: Parts of HAVING that refer to columns/expressions we're doing GROUP BY on can be put into WHERE. (TODO: do we need to handle case of grouping over expressions?) (TODO: when moving expression for HAVING to WHERE, do we need to do something with it? Replace all Item_ref objects with items that they refer to? - In case of referring to expression, do we get Item_ref(where_clause_expr) or expr( Item_ref(..), .., Item_ref(..))? ) 4. When to do the pushdown ========================== In order to do pushdown, we must have prepare phase finished for both parent (so that we can make sense of its WHERE condition) and child (so that we know what it has in its select list). We can do pushdown before we've done join optimization (i.e. choose_plan() call) of the parent. We must do pushdown before we've done JOIN::optimize() of the child (in particular, it must be done before we do update_ref_and_keys() and range analysis in the child). 5. Other things to take care of =============================== * Pushing down fulltext predicates (it seems one needs to "register" a fulltext predicate when it is moved from one select from another? Ask Serg) ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Push conditions down into non-mergeable VIEWs when possible (119)
by worklog-noreply＠askmonty.org 24 May '10

24 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Push conditions down into non-mergeable VIEWs when possible CREATION DATE..: Mon, 24 May 2010, 20:52 SUPERVISOR.....: Igor IMPLEMENTOR....: COPIES TO......: Psergey, Timour CATEGORY.......: Server-RawIdeaBin TASK ID........: 119 (http://askmonty.org/worklog/?tid=119) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 24 May 2010, 20:59)=-=- High-Level Specification modified. --- /tmp/wklog.119.old.25116 2010-05-24 20:59:40.000000000 +0000 +++ /tmp/wklog.119.new.25116 2010-05-24 20:59:40.000000000 +0000 @@ -1 +1,113 @@ +<contents> +HLS +1. Problems to be addressed in this WL +2. Pushdown of conditions into non-mergeable VIEWs +2.1 A note about VIEWs with ORDER BY ... LIMIT +2.2 What condition can be pushed +3. Pushdown from HAVING into WHERE +4. When to do the pushdown +5. Other things to take care of + + +</contents> + +1. Problems to be addressed in this WL +====================================== +The problem actually consists of two parts: +1. Condition on VIEW columns are not pushed down into VIEWs. +2. Even if conditions were pushed, they would have been put into VIEW's +HAVING clause, which would not give enough of speedup. In order to get a +real speedup, the optimizer must be able to move relevant part of HAVING +into WHERE (and then use it for further optimizations) in order to provide +the desired speedup. Note that HAVING->WHERE condition move is orthogonal +to VIEW processing. + +2. Pushdown of conditions into non-mergeable VIEWs +================================================== +We can push a condition into non-mergeable VIEW when VIEW's top-most operation +is selection (i.e., filtering). This is true, for example, when the VIEW is +defined as + + SELECT select_list FROM from_clause [WHERE where_cond] [HAVING having_cond] + +and not true when the VIEW is defined as + + SELECT select_list FROM from_clause [WHERE where_cond] ORDER BY expr LIMIT n + +Generalizing the above, we arrive at the following rule: + + For non-mergeable VIEWs, + - pushdown must not be done if VIEW uses ORDER BY .. LIMIT + - when pushdown is done, the pushed condition should be added to the WHERE + clause. + +Note: In scope of this WL, we will not hande VIEWs that have UNION [ALL] as +top operation. + +(TODO: what about SELECT DISTINCT?) +(TODO: pushdown down into IN subqueries?) + +2.1 A note about VIEWs with ORDER BY ... LIMIT +---------------------------------------------- +Although it is not possible to push a condition below the ORDER BY ... LIMIT +operation, there is still some benefit from checking the condition early as +that would allow to avoid writing non-matching rows into temporary table. + +We could do that if we introduced a post-ORDERBY selection operation. That +operation would also allow to support ORDER BY ... LIMIT inside subqueries +(we don't currently support those because default subquery strategy, +IN->EXISTS rewrite, also needs to push a condition into subquery). + +2.2 What condition can be pushed +-------------------------------- +Assuming simplify_joins() operation has done normalization: +* If the VIEW is in top-level join list, or inside a semi-join that's in + top-level join list, we can push parts of WHERE condition. +* If the VIEW is inside an outer join, we can push it's ON expression. + +We can reuse make_cond_for_index()/make_remainder_cond() code to extract part +of condition that can be pushed, and the remainder, respectively. + +Pushability criteria for an atomic (i.e. not AND/OR) condition is that + + the condition only uses VIEW's fields. + +(TODO: what about fields of const tables? Do we have const tables already +retrived by the time VIEW is materialized? If yes, we could push down +expressions that refer to const tables, too) + +3. Pushdown from HAVING into WHERE +================================== +The idea is: + + Parts of HAVING that refer to columns/expressions we're doing GROUP BY on + can be put into WHERE. + +(TODO: do we need to handle case of grouping over expressions?) + +(TODO: when moving expression for HAVING to WHERE, do we need +to do something with it? Replace all Item_ref objects with items that +they refer to? + - In case of referring to expression, do we get + Item_ref(where_clause_expr) or expr( Item_ref(..), .., Item_ref(..))? +) + +4. When to do the pushdown +========================== +In order to do pushdown, we must have prepare phase finished +for both parent (so that we can make sense of its WHERE condition) and +child (so that we know what it has in its select list). + +We can do pushdown before we've done join optimization (i.e. choose_plan() +call) of the parent. + +We must do pushdown before we've done JOIN::optimize() of the child +(in particular, it must be done before we do update_ref_and_keys() and +range analysis in the child). + + +5. Other things to take care of +=============================== +* Pushing down fulltext predicates (it seems one needs to "register" a + fulltext predicate when it is moved from one select from another? Ask Serg) DESCRIPTION: There are complaints (see links below) about cases with non-mergeable VIEW (because it has a GROUP BY), a query that has restrictions on the grouped column, and poor performance that is caused by VIEW processing code ignoring the restriction. This WL is about addressing this issue. links to complaints: http://code.openark.org/blog/mysql/views-better-performance-with-condition-… http://www.mysqlperformanceblog.com/2010/05/19/a-workaround-for-the-perform… The target version is MariaDB 5.3+, because it has late optimization/execution for FROM-subqueries/non mergeable VIEWs, which makes it much more feasible to inject something into VIEW before it is optimized/executed. HIGH-LEVEL SPECIFICATION: <contents> HLS 1. Problems to be addressed in this WL 2. Pushdown of conditions into non-mergeable VIEWs 2.1 A note about VIEWs with ORDER BY ... LIMIT 2.2 What condition can be pushed 3. Pushdown from HAVING into WHERE 4. When to do the pushdown 5. Other things to take care of </contents> 1. Problems to be addressed in this WL ====================================== The problem actually consists of two parts: 1. Condition on VIEW columns are not pushed down into VIEWs. 2. Even if conditions were pushed, they would have been put into VIEW's HAVING clause, which would not give enough of speedup. In order to get a real speedup, the optimizer must be able to move relevant part of HAVING into WHERE (and then use it for further optimizations) in order to provide the desired speedup. Note that HAVING->WHERE condition move is orthogonal to VIEW processing. 2. Pushdown of conditions into non-mergeable VIEWs ================================================== We can push a condition into non-mergeable VIEW when VIEW's top-most operation is selection (i.e., filtering). This is true, for example, when the VIEW is defined as SELECT select_list FROM from_clause [WHERE where_cond] [HAVING having_cond] and not true when the VIEW is defined as SELECT select_list FROM from_clause [WHERE where_cond] ORDER BY expr LIMIT n Generalizing the above, we arrive at the following rule: For non-mergeable VIEWs, - pushdown must not be done if VIEW uses ORDER BY .. LIMIT - when pushdown is done, the pushed condition should be added to the WHERE clause. Note: In scope of this WL, we will not hande VIEWs that have UNION [ALL] as top operation. (TODO: what about SELECT DISTINCT?) (TODO: pushdown down into IN subqueries?) 2.1 A note about VIEWs with ORDER BY ... LIMIT ---------------------------------------------- Although it is not possible to push a condition below the ORDER BY ... LIMIT operation, there is still some benefit from checking the condition early as that would allow to avoid writing non-matching rows into temporary table. We could do that if we introduced a post-ORDERBY selection operation. That operation would also allow to support ORDER BY ... LIMIT inside subqueries (we don't currently support those because default subquery strategy, IN->EXISTS rewrite, also needs to push a condition into subquery). 2.2 What condition can be pushed -------------------------------- Assuming simplify_joins() operation has done normalization: * If the VIEW is in top-level join list, or inside a semi-join that's in top-level join list, we can push parts of WHERE condition. * If the VIEW is inside an outer join, we can push it's ON expression. We can reuse make_cond_for_index()/make_remainder_cond() code to extract part of condition that can be pushed, and the remainder, respectively. Pushability criteria for an atomic (i.e. not AND/OR) condition is that the condition only uses VIEW's fields. (TODO: what about fields of const tables? Do we have const tables already retrived by the time VIEW is materialized? If yes, we could push down expressions that refer to const tables, too) 3. Pushdown from HAVING into WHERE ================================== The idea is: Parts of HAVING that refer to columns/expressions we're doing GROUP BY on can be put into WHERE. (TODO: do we need to handle case of grouping over expressions?) (TODO: when moving expression for HAVING to WHERE, do we need to do something with it? Replace all Item_ref objects with items that they refer to? - In case of referring to expression, do we get Item_ref(where_clause_expr) or expr( Item_ref(..), .., Item_ref(..))? ) 4. When to do the pushdown ========================== In order to do pushdown, we must have prepare phase finished for both parent (so that we can make sense of its WHERE condition) and child (so that we know what it has in its select list). We can do pushdown before we've done join optimization (i.e. choose_plan() call) of the parent. We must do pushdown before we've done JOIN::optimize() of the child (in particular, it must be done before we do update_ref_and_keys() and range analysis in the child). 5. Other things to take care of =============================== * Pushing down fulltext predicates (it seems one needs to "register" a fulltext predicate when it is moved from one select from another? Ask Serg) ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Push conditions down into non-mergeable VIEWs when possible (119)
by worklog-noreply＠askmonty.org 24 May '10

24 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Push conditions down into non-mergeable VIEWs when possible CREATION DATE..: Mon, 24 May 2010, 20:52 SUPERVISOR.....: Igor IMPLEMENTOR....: COPIES TO......: Psergey, Timour CATEGORY.......: Server-RawIdeaBin TASK ID........: 119 (http://askmonty.org/worklog/?tid=119) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: There are complaints (see links below) about cases with non-mergeable VIEW (because it has a GROUP BY), a query that has restrictions on the grouped column, and poor performance that is caused by VIEW processing code ignoring the restriction. This WL is about addressing this issue. links to complaints: http://code.openark.org/blog/mysql/views-better-performance-with-condition-… http://www.mysqlperformanceblog.com/2010/05/19/a-workaround-for-the-perform… The target version is MariaDB 5.3+, because it has late optimization/execution for FROM-subqueries/non mergeable VIEWs, which makes it much more feasible to inject something into VIEW before it is optimized/executed. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Push conditions down into non-mergeable VIEWs when possible (119)
by worklog-noreply＠askmonty.org 24 May '10

24 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Push conditions down into non-mergeable VIEWs when possible CREATION DATE..: Mon, 24 May 2010, 20:52 SUPERVISOR.....: Igor IMPLEMENTOR....: COPIES TO......: Psergey, Timour CATEGORY.......: Server-RawIdeaBin TASK ID........: 119 (http://askmonty.org/worklog/?tid=119) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: There are complaints (see links below) about cases with non-mergeable VIEW (because it has a GROUP BY), a query that has restrictions on the grouped column, and poor performance that is caused by VIEW processing code ignoring the restriction. This WL is about addressing this issue. links to complaints: http://code.openark.org/blog/mysql/views-better-performance-with-condition-… http://www.mysqlperformanceblog.com/2010/05/19/a-workaround-for-the-perform… The target version is MariaDB 5.3+, because it has late optimization/execution for FROM-subqueries/non mergeable VIEWs, which makes it much more feasible to inject something into VIEW before it is optimized/executed. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Psergey): Push conditions down into non-mergeable VIEWs when possible (119)
by worklog-noreply＠askmonty.org 24 May '10

24 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Push conditions down into non-mergeable VIEWs when possible CREATION DATE..: Mon, 24 May 2010, 20:52 SUPERVISOR.....: Igor IMPLEMENTOR....: COPIES TO......: Psergey, Timour CATEGORY.......: Server-RawIdeaBin TASK ID........: 119 (http://askmonty.org/worklog/?tid=119) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: There are complaints (see links below) about cases with non-mergeable VIEW (because it has a GROUP BY), a query that has restrictions on the grouped column, and poor performance that is caused by VIEW processing code ignoring the restriction. This WL is about addressing this issue. links to complaints: http://code.openark.org/blog/mysql/views-better-performance-with-condition-… http://www.mysqlperformanceblog.com/2010/05/19/a-workaround-for-the-perform… The target version is MariaDB 5.3+, because it has late optimization/execution for FROM-subqueries/non mergeable VIEWs, which makes it much more feasible to inject something into VIEW before it is optimized/executed. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 2789: Subquery cache (as is) in file:///home/bell/maria/bzr/work-maria-5.3-scache/
by sanja＠askmonty.org 24 May '10

24 May '10

At file:///home/bell/maria/bzr/work-maria-5.3-scache/ ------------------------------------------------------------ revno: 2789 revision-id: sanja(a)askmonty.org-20100524172956-7b14x01aodizr3sq parent: sergii(a)pisem.net-20100510134608-oyi2vznyghgcrt0x committer: sanja(a)askmonty.org branch nick: work-maria-5.3-scache timestamp: Mon 2010-05-24 20:29:56 +0300 message: Subquery cache (as is) === modified file 'libmysqld/Makefile.am' --- a/libmysqld/Makefile.am 2010-03-20 12:01:47 +0000 +++ b/libmysqld/Makefile.am 2010-05-24 17:29:56 +0000 @@ -80,7 +80,8 @@ sql_tablespace.cc \ rpl_injector.cc my_user.c partition_info.cc \ sql_servers.cc event_parse_data.cc opt_table_elimination.cc \ - multi_range_read.cc opt_index_cond_pushdown.cc + multi_range_read.cc opt_index_cond_pushdown.cc \ + sql_subquery_cache.cc libmysqld_int_a_SOURCES= $(libmysqld_sources) nodist_libmysqld_int_a_SOURCES= $(libmysqlsources) $(sqlsources) === modified file 'sql/CMakeLists.txt' --- a/sql/CMakeLists.txt 2010-03-20 12:01:47 +0000 +++ b/sql/CMakeLists.txt 2010-05-24 17:29:56 +0000 @@ -78,7 +78,7 @@ rpl_rli.cc rpl_mi.cc sql_servers.cc sql_connect.cc scheduler.cc sql_profile.cc event_parse_data.cc opt_table_elimination.cc - ds_mrr.cc + ds_mrr.cc sql_subquery_cache.cc ${PROJECT_SOURCE_DIR}/sql/sql_yacc.cc ${PROJECT_SOURCE_DIR}/sql/sql_yacc.h ${PROJECT_SOURCE_DIR}/include/mysqld_error.h === modified file 'sql/Makefile.am' --- a/sql/Makefile.am 2010-03-20 12:01:47 +0000 +++ b/sql/Makefile.am 2010-05-24 17:29:56 +0000 @@ -80,7 +80,7 @@ event_data_objects.h event_scheduler.h \ sql_partition.h partition_info.h partition_element.h \ contributors.h sql_servers.h \ - multi_range_read.h + multi_range_read.h sql_subquery_cache.h mysqld_SOURCES = sql_lex.cc sql_handler.cc sql_partition.cc \ item.cc item_sum.cc item_buff.cc item_func.cc \ @@ -130,7 +130,7 @@ sql_servers.cc event_parse_data.cc \ opt_table_elimination.cc \ multi_range_read.cc \ - opt_index_cond_pushdown.cc + opt_index_cond_pushdown.cc sql_subquery_cache.cc nodist_mysqld_SOURCES = mini_client_errors.c pack.c client.c my_time.c my_user.c === modified file 'sql/item.cc' --- a/sql/item.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item.cc 2010-05-24 17:29:56 +0000 @@ -2273,6 +2273,13 @@ str->append(str_value); } +void Item_bool_cache::print(String *str, enum_query_type query_type) +{ + if (null_value) + str->append("NULL", 4); + else + Item_int::print(str, query_type); +} Item_uint::Item_uint(const char *str_arg, uint length): Item_int(str_arg, length) @@ -3646,12 +3653,17 @@ resolved_item->db_name : ""); const char *table_name= (resolved_item->table_name ? resolved_item->table_name : ""); + DBUG_ENTER("mark_as_dependent"); + DBUG_PRINT("enter", ("Field '%s.%s.%s in select %d resolved in %d", + db_name, table_name, + resolved_item->field_name, current->select_number, + last->select_number)); /* store pointer on SELECT_LEX from which item is dependent */ if (mark_item) mark_item->depended_from= last; if (current->mark_as_dependent(thd, last, /** resolved_item psergey-thu **/mark_item)) - return TRUE; + DBUG_RETURN(TRUE); if (thd->lex->describe & DESCRIBE_EXTENDED) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, @@ -3661,7 +3673,7 @@ resolved_item->field_name, current->select_number, last->select_number); } - return FALSE; + DBUG_RETURN(FALSE); } @@ -3698,6 +3710,7 @@ resolving) */ SELECT_LEX *previous_select= current_sel; + for (; previous_select->outer_select() != last_select; previous_select= previous_select->outer_select()) { @@ -3726,6 +3739,7 @@ mark_as_dependent(thd, last_select, current_sel, resolved_item, dependent); } + return; } @@ -4098,6 +4112,9 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); return 0; } } @@ -4113,7 +4130,9 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); - + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* A reference to a view field had been found and we substituted it instead of this Item (find_field_in_tables @@ -4215,6 +4234,10 @@ mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, rf, rf); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); + return 0; } else @@ -4222,6 +4245,9 @@ mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, this, (Item_ident*)*reference); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); if (last_checked_context->select_lex->having_fix_field) { Item_ref *rf; @@ -5973,6 +5999,9 @@ refer_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* view reference found, we substituted it instead of this Item, so can quit @@ -6023,6 +6052,9 @@ thd->change_item_tree(reference, fld); mark_as_dependent(thd, last_checked_context->select_lex, thd->lex->current_select, fld, fld); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + reference); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of @@ -6046,6 +6078,9 @@ DBUG_ASSERT(*ref && (*ref)->fixed); mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, this, this); + context->select_lex-> + register_dependency_item(last_checked_context->select_lex, + ref); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of === modified file 'sql/item.h' --- a/sql/item.h 2010-03-20 12:01:47 +0000 +++ b/sql/item.h 2010-05-24 17:29:56 +0000 @@ -1143,6 +1143,11 @@ { return Field::GEOM_GEOMETRY; }; String *check_well_formed_result(String *str, bool send_error= 0); bool eq_by_collation(Item *item, bool binary_cmp, CHARSET_INFO *cs); + + /** + Used to get reference on real item (not Item_ref) + */ + virtual Item **unref(Item **my_ref) { return my_ref; }; }; @@ -1922,8 +1927,31 @@ virtual void print(String *str, enum_query_type query_type); Item_num *neg (); uint decimal_precision() const { return max_length; } - bool check_partition_func_processor(uchar *bool_arg) { return FALSE;} - bool check_vcol_func_processor(uchar *arg) { return FALSE;} +}; + + +/** + Item represent TRUE/FALSE/NULL for subquery values +*/ + +class Item_bool_cache: public Item_int +{ +public: + Item_bool_cache(): Item_int(0, 1) + { + unsigned_flag= maybe_null= null_value= TRUE; + name= (char *)"bool chache"; + } + Item_bool_cache(my_bool val, my_bool null): Item_int(val, 1) + { + unsigned_flag= maybe_null= TRUE; + null_value= null; + name= (char *)"bool chache"; + } + Item *clone_item() { return new Item_bool_cache(value, null_value); } + uint decimal_precision() const { return 1; } + virtual void print(String *str, enum_query_type query_type); + void set(my_bool val, my_bool null) {value= test(val); null_value= null;} }; @@ -2479,6 +2507,11 @@ { return trace_unsupported_by_check_vcol_func_processor("ref"); } + + /** + Used to get reference on real item (not Item_ref) + */ + virtual Item **unref(Item **my_ref) {return (*ref)->unref(ref); }; }; @@ -3146,7 +3179,8 @@ example(0), used_table_map(0), cached_field(0), cached_field_type(MYSQL_TYPE_STRING), value_cached(0) { - fixed= 1; + fixed= 1; + maybe_null= 1; null_value= 1; } Item_cache(enum_field_types field_type_arg): @@ -3154,6 +3188,7 @@ value_cached(0) { fixed= 1; + maybe_null= 1; null_value= 1; } === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.cc 2010-05-24 17:29:56 +0000 @@ -1736,6 +1736,13 @@ used_tables_cache|= args[1]->used_tables(); not_null_tables_cache|= args[1]->not_null_tables(); const_item_cache&= args[1]->const_item(); + DBUG_ASSERT(scache == NULL); + if (args[0]->cols() ==1 && + thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE) + { + sub->depends_on.push_front((Item**)&cache); + scache= new Subquery_cache_tmptable(thd, sub->depends_on, &result); + } fixed= 1; return FALSE; } @@ -1744,10 +1751,26 @@ longlong Item_in_optimizer::val_int() { bool tmp; + DBUG_ENTER("Item_in_optimizer::val_int"); + DBUG_ASSERT(fixed == 1); cache->store(args[0]); cache->cache_value(); - + + /* check if result is in the cache */ + if (scache) + { + Subquery_cache_tmptable::result res; + Item *cached_value; + res= scache->check_value(&cached_value); + if (res == Subquery_cache_tmptable::HIT) + { + tmp= cached_value->val_int(); + null_value= cached_value->null_value; + DBUG_RETURN(tmp); + } + } + if (cache->null_value) { /* @@ -1818,11 +1841,18 @@ for (uint i= 0; i < ncols; i++) item_subs->set_cond_guard_var(i, TRUE); } - return 0; + DBUG_RETURN(0); } tmp= args[1]->val_bool_result(); null_value= args[1]->null_value; - return tmp; + + /* put result in the cache */ + if (scache) + { + result.set(tmp, null_value); + scache->put_value(&result); + } + DBUG_RETURN(tmp); } @@ -1839,6 +1869,11 @@ Item_bool_func::cleanup(); if (!save_cache) cache= 0; + if (scache) + { + delete scache; + scache= 0; + } DBUG_VOID_RETURN; } === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.h 2010-05-24 17:29:56 +0000 @@ -215,6 +215,7 @@ class Item_cache; +class Subquery_cache; #define UNKNOWN ((my_bool)-1) @@ -237,6 +238,10 @@ { protected: Item_cache *cache; + /* Subquery cache */ + Subquery_cache *scache; + /* result representation for the subquery cache */ + Item_bool_cache result; bool save_cache; /* Stores the value of "NULL IN (SELECT ...)" for uncorrelated subqueries: @@ -247,7 +252,7 @@ my_bool result_for_null_param; public: Item_in_optimizer(Item *a, Item_in_subselect *b): - Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), + Item_bool_func(a, my_reinterpret_cast(Item *)(b)), cache(0), scache(NULL), save_cache(0), result_for_null_param(UNKNOWN) {} bool fix_fields(THD *, Item **); === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.cc 2010-05-24 17:29:56 +0000 @@ -1,4 +1,4 @@ -/* Copyright (C) 2000 MySQL AB +/* Copyrigh (C) 2000 MySQL AB This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -34,11 +34,10 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), - engine(0), old_engine(0), used_tables_cache(0), have_to_be_excluded(0), - const_item_cache(1), - inside_first_fix_fields(0), done_first_fix_fields(FALSE), - eliminated(FALSE), - engine_changed(0), changed(0), is_correlated(FALSE) + engine(0), old_engine(0), scache(0), used_tables_cache(0), + have_to_be_excluded(0), const_item_cache(1), inside_first_fix_fields(0), + done_first_fix_fields(FALSE), eliminated(FALSE), engine_changed(0), + changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -116,6 +115,12 @@ } if (engine) engine->cleanup(); + depends_on.empty(); + if (scache) + { + delete scache; + scache= 0; + } reset(); value_assigned= 0; DBUG_VOID_RETURN; @@ -148,6 +153,8 @@ Item_subselect::~Item_subselect() { delete engine; + if (scache) + delete scache; } Item_subselect::trans_res @@ -746,9 +753,19 @@ void Item_singlerow_subselect::fix_length_and_dec() { + DBUG_ENTER("Item_singlerow_subselect::fix_length_and_dec"); if ((max_columns= engine->cols()) == 1) { + DBUG_PRINT("info", ("one, elements: %u flag %u", + (uint)depends_on.elements, + (uint)test(thd->variables.optimizer_switch & OPTIMIZER_SWITCH_SUBQUERY_CACHE))); engine->fix_length_and_dec(row= &value); + if (depends_on.elements && optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE)) + { + DBUG_ASSERT(scache == NULL); + scache= new Subquery_cache_tmptable(thd, depends_on, value); + DBUG_PRINT("info", ("cache: 0x%lx", (ulong) scache)); + } } else { @@ -765,6 +782,7 @@ */ if (engine->no_tables()) maybe_null= engine->may_be_null(); + DBUG_VOID_RETURN; } uint Item_singlerow_subselect::cols() @@ -797,77 +815,200 @@ exec(); } + +Item *Item_subselect::check_cache() +{ + DBUG_ENTER("Item_subselect::check_cache"); + if (scache) + { + Subquery_cache_tmptable::result res; + Item *cached_value; + res= scache->check_value(&cached_value); + if (res == Subquery_cache_tmptable::HIT) + DBUG_RETURN(cached_value); + } + DBUG_RETURN(NULL); +} + double Item_singlerow_subselect::val_real() { + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_real"); DBUG_ASSERT(fixed == 1); - if (!exec() && !value->null_value) + + if ((cached_value = check_cache())) + { + double res= cached_value->val_real(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_real(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_real()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } longlong Item_singlerow_subselect::val_int() { + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_int"); DBUG_ASSERT(fixed == 1); - if (!exec() && !value->null_value) + + if ((cached_value = check_cache())) + { + longlong res= cached_value->val_int(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_int(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_int()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } String *Item_singlerow_subselect::val_str(String *str) { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_str"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + String *res= cached_value->val_str(str); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_str(str); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_str(str)); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } my_decimal *Item_singlerow_subselect::val_decimal(my_decimal *decimal_value) { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_decimal"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_decimal *res= cached_value->val_decimal(decimal_value); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_decimal(decimal_value); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_decimal(decimal_value)); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } bool Item_singlerow_subselect::val_bool() { - if (!exec() && !value->null_value) + Item *cached_value; + bool err; + DBUG_ENTER("Item_singlerow_subselect::val_bool"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + bool res= cached_value->val_bool(); + if ((null_value= cached_value->null_value)) + { + reset(); + DBUG_RETURN(0); + } + else + DBUG_RETURN(res); + } + + if (!(err= exec()) && !value->null_value) { null_value= 0; - return value->val_bool(); + if (scache) + scache->put_value(value); + DBUG_RETURN(value->val_bool()); } else { reset(); - return 0; + DBUG_PRINT("info", ("error: %u", (uint)err)); + if (scache && !err) + scache->put_value(&const_null_value); + DBUG_RETURN(0); } } @@ -952,33 +1093,77 @@ void Item_exists_subselect::fix_length_and_dec() { + DBUG_ENTER("Item_exists_subselect::fix_length_and_dec"); decimals= 0; max_length= 1; max_columns= engine->cols(); /* We need only 1 row to determine existence */ unit->global_parameters->select_limit= new Item_int((int32) 1); + if (substype() == EXISTS_SUBS && depends_on.elements && + optimizer_flag(thd, OPTIMIZER_SWITCH_SUBQUERY_CACHE)) + { + DBUG_ASSERT(scache == NULL); + scache= new Subquery_cache_tmptable(thd, depends_on, &result); + DBUG_PRINT("info", ("cache: 0x%lx", (ulong) scache)); + } + DBUG_VOID_RETURN; } double Item_exists_subselect::val_real() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_int"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + double res= cached_value->val_real(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) { reset(); - return 0; - } - return (double) value; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN((double) value); } longlong Item_exists_subselect::val_int() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_real"); + DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + longlong res= cached_value->val_int(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + DBUG_ASSERT(fixed == 1); if (exec()) { reset(); - return 0; - } - return value; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN(value); } @@ -997,11 +1182,31 @@ String *Item_exists_subselect::val_str(String *str) { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_str"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + String *res= cached_value->val_str(str); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) + { reset(); + DBUG_RETURN(NULL); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + str->set((ulonglong)value,&my_charset_bin); - return str; + DBUG_RETURN(str); } @@ -1020,23 +1225,60 @@ my_decimal *Item_exists_subselect::val_decimal(my_decimal *decimal_value) { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_decvimal"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_decimal *res= cached_value->val_decimal(decimal_value); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) + { reset(); + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + int2my_decimal(E_DEC_FATAL_ERROR, value, 0, decimal_value); - return decimal_value; + DBUG_RETURN(decimal_value); } bool Item_exists_subselect::val_bool() { + Item *cached_value; + DBUG_ENTER("Item_exists_subselect::val_real"); DBUG_ASSERT(fixed == 1); + + if ((cached_value = check_cache())) + { + my_bool res= cached_value->val_bool(); + DBUG_ASSERT(!cached_value->null_value); + DBUG_RETURN(res); + } + if (exec()) { reset(); - return 0; - } - return value != 0; + DBUG_RETURN(0); + } + + if (scache) + { + result.set(value, FALSE); + scache->put_value(&result); + } + + DBUG_RETURN(value != 0); } === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.h 2010-05-24 17:29:56 +0000 @@ -27,6 +27,7 @@ class subselect_hash_sj_engine; class Item_bool_func2; class Cached_item; +class Subquery_cache; /* base class for subselects */ @@ -57,6 +58,10 @@ subselect_engine *engine; /* old engine if engine was changed */ subselect_engine *old_engine; + /* subquery cache */ + Subquery_cache *scache; + /* null consrtant for caching */ + Item_null const_null_value; /* cache of used external tables */ table_map used_tables_cache; /* allowed number of columns (1 for single value subqueries) */ @@ -67,7 +72,7 @@ bool have_to_be_excluded; /* cache of constant state */ bool const_item_cache; - + bool inside_first_fix_fields; bool done_first_fix_fields; public: @@ -88,13 +93,18 @@ */ List<Ref_to_outside> upper_refs; st_select_lex *parent_select; - - /* + + /** + List of items subquery depends on (externally resolved); + */ + List<Item*> depends_on; + + /* TRUE<=>Table Elimination has made it redundant to evaluate this select (and so it is not part of QEP, etc) - */ + */ bool eliminated; - + /* changed engine indicator */ bool engine_changed; /* subquery is transformed */ @@ -178,6 +188,8 @@ return trace_unsupported_by_check_vcol_func_processor("subselect"); } + Item *check_cache(); + /** Get the SELECT_LEX structure associated with this Item. @return the SELECT_LEX structure associated with this Item @@ -202,6 +214,7 @@ { protected: Item_cache *value, **row; + public: Item_singlerow_subselect(st_select_lex *select_lex); Item_singlerow_subselect() :Item_subselect(), value(0), row (0) {} @@ -268,6 +281,8 @@ { protected: bool value; /* value of this item (boolean: exists/not-exists) */ + /* result representation for the subquery cache */ + Item_bool_cache result; public: Item_exists_subselect(st_select_lex *select_lex); === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-03-20 12:01:47 +0000 +++ b/sql/mysql_priv.h 2010-05-24 17:29:56 +0000 @@ -568,12 +568,13 @@ #define OPTIMIZER_SWITCH_SEMIJOIN 256 #define OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE 512 #define OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN 1024 +#define OPTIMIZER_SWITCH_SUBQUERY_CACHE (1<<11) #ifdef DBUG_OFF -# define OPTIMIZER_SWITCH_LAST 2048 +# define OPTIMIZER_SWITCH_LAST (1<<12) #else -# define OPTIMIZER_SWITCH_TABLE_ELIMINATION 2048 -# define OPTIMIZER_SWITCH_LAST 4096 +# define OPTIMIZER_SWITCH_TABLE_ELIMINATION (1<<12) +# define OPTIMIZER_SWITCH_LAST (1<<13) #endif #ifdef DBUG_OFF @@ -588,7 +589,8 @@ OPTIMIZER_SWITCH_MATERIALIZATION | \ OPTIMIZER_SWITCH_SEMIJOIN | \ OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE|\ - OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN) + OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN|\ + OPTIMIZER_SWITCH_SUBQUERY_CACHE) #else # define OPTIMIZER_SWITCH_DEFAULT (OPTIMIZER_SWITCH_INDEX_MERGE | \ OPTIMIZER_SWITCH_INDEX_MERGE_UNION | \ @@ -601,7 +603,8 @@ OPTIMIZER_SWITCH_MATERIALIZATION | \ OPTIMIZER_SWITCH_SEMIJOIN | \ OPTIMIZER_SWITCH_PARTIAL_MATCH_ROWID_MERGE|\ - OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN) + OPTIMIZER_SWITCH_PARTIAL_MATCH_TABLE_SCAN|\ + OPTIMIZER_SWITCH_SUBQUERY_CACHE) #endif /* @@ -936,6 +939,7 @@ #ifdef MYSQL_SERVER #include "sql_servers.h" #include "opt_range.h" +#include "sql_subquery_cache.h" #ifdef HAVE_QUERY_CACHE struct Query_cache_query_flags @@ -1269,6 +1273,10 @@ Item *having, ORDER *proc_param, ulonglong select_type, select_result *result, SELECT_LEX_UNIT *unit, SELECT_LEX *select_lex); + +struct st_join_table *create_index_lookup_join_tab(TABLE *table); +int join_read_key2(THD *thd, struct st_join_table *tab, TABLE *table, + struct st_table_ref *table_ref); void free_underlaid_joins(THD *thd, SELECT_LEX *select); bool mysql_explain_union(THD *thd, SELECT_LEX_UNIT *unit, select_result *result); @@ -1288,6 +1296,7 @@ bool table_cant_handle_bit_fields, bool make_copy_field, uint convert_blob_length); +bool open_tmp_table(TABLE *table); void sp_prepare_create_field(THD *thd, Create_field *sql_field); int prepare_create_field(Create_field *sql_field, uint *blob_columns, === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2010-03-20 12:01:47 +0000 +++ b/sql/mysqld.cc 2010-05-24 17:29:56 +0000 @@ -305,6 +305,7 @@ "firstmatch","loosescan","materialization", "semijoin", "partial_match_rowid_merge", "partial_match_table_scan", + "subquery_cache", #ifndef DBUG_OFF "table_elimination", #endif @@ -325,6 +326,7 @@ sizeof("semijoin") - 1, sizeof("partial_match_rowid_merge") - 1, sizeof("partial_match_table_scan") - 1, + sizeof("subquery_cache") - 1, #ifndef DBUG_OFF sizeof("table_elimination") - 1, #endif @@ -404,8 +406,9 @@ static const char *optimizer_switch_str="index_merge=on,index_merge_union=on," "index_merge_sort_union=on," "index_merge_intersection=on," - "index_condition_pushdown=on" -#ifndef DBUG_OFF + "index_condition_pushdown=on," + "subquery_cache=on" +#ifndef DBUG_OFF ",table_elimination=on"; #else ; @@ -5872,7 +5875,9 @@ OPT_RECORD_RND_BUFFER, OPT_DIV_PRECINCREMENT, OPT_RELAY_LOG_SPACE_LIMIT, OPT_RELAY_LOG_PURGE, OPT_SLAVE_NET_TIMEOUT, OPT_SLAVE_COMPRESSED_PROTOCOL, OPT_SLOW_LAUNCH_TIME, - OPT_SLAVE_TRANS_RETRIES, OPT_READONLY, OPT_ROWID_MERGE_BUFF_SIZE, + OPT_SLAVE_TRANS_RETRIES, + OPT_SUBQUERY_CACHE, + OPT_READONLY, OPT_ROWID_MERGE_BUFF_SIZE, OPT_DEBUGGING, OPT_DEBUG_FLUSH, OPT_SORT_BUFFER, OPT_TABLE_OPEN_CACHE, OPT_TABLE_DEF_CACHE, OPT_THREAD_CONCURRENCY, OPT_THREAD_CACHE_SIZE, @@ -7164,7 +7169,7 @@ {"optimizer_switch", OPT_OPTIMIZER_SWITCH, "optimizer_switch=option=val[,option=val...], where option={index_merge, " "index_merge_union, index_merge_sort_union, index_merge_intersection, " - "index_condition_pushdown" + "index_condition_pushdown, subquery_cache" #ifndef DBUG_OFF ", table_elimination" #endif @@ -7868,6 +7873,8 @@ {"Ssl_version", (char*) &show_ssl_get_version, SHOW_FUNC}, #endif /* HAVE_OPENSSL */ {"Syncs", (char*) &my_sync_count, SHOW_LONG_NOFLUSH}, + {"Subquery_cache_hit", (char*) &subquery_cache_hit, SHOW_LONG}, + {"Subquery_cache_miss", (char*) &subquery_cache_miss, SHOW_LONG}, {"Table_locks_immediate", (char*) &locks_immediate, SHOW_LONG}, {"Table_locks_waited", (char*) &locks_waited, SHOW_LONG}, #ifdef HAVE_MMAP @@ -8006,6 +8013,7 @@ abort_loop= select_thread_in_use= signal_thread_in_use= 0; ready_to_exit= shutdown_in_progress= grant_option= 0; aborted_threads= aborted_connects= 0; + subquery_cache_miss= subquery_cache_hit= 0; delayed_insert_threads= delayed_insert_writes= delayed_rows_in_use= 0; delayed_insert_errors= thread_created= 0; specialflag= 0; === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_base.cc 2010-05-24 17:29:56 +0000 @@ -8062,6 +8062,10 @@ if (*conds) { thd->where="where clause"; + DBUG_EXECUTE("where", + print_where(*conds, + "WHERE in setup_conds", + QT_ORDINARY);); if ((!(*conds)->fixed && (*conds)->fix_fields(thd, conds)) || (*conds)->check_cols(1)) goto err_no_arena; === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.cc 2010-05-24 17:29:56 +0000 @@ -3020,6 +3020,7 @@ table_charset= 0; precomputed_group_by= 0; bit_fields_as_long= 0; + skip_create_table= 0; DBUG_VOID_RETURN; } === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.h 2010-05-24 17:29:56 +0000 @@ -2786,12 +2786,17 @@ that MEMORY tables cannot index BIT columns. */ bool bit_fields_as_long; + /* + Whether to create or postpone actual creation of this temporary table. + TRUE <=> create_tmp_table will create only the TABLE structure. + */ + bool skip_create_table; TMP_TABLE_PARAM() :copy_field(0), group_parts(0), group_length(0), group_null_parts(0), convert_blob_length(0), schema_table(0), precomputed_group_by(0), force_copy_fields(0), - bit_fields_as_long(0) + bit_fields_as_long(0), skip_create_table(0) {} ~TMP_TABLE_PARAM() { === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.cc 2010-05-24 17:29:56 +0000 @@ -1829,6 +1829,53 @@ } +/** + Registers reference on items on which the subqueries depends + + @param last pointer to last st_select_lex struct, before + which all st_select_lex have to be marked as + dependent + @param dependency reference on the item on which all this + subqueries depends + +*/ + +void st_select_lex::register_dependency_item(st_select_lex *last, + Item **dependency) +{ + SELECT_LEX *s= this; + DBUG_ENTER("st_select_lex::register_dependency_item"); + DBUG_ASSERT(this != last); + DBUG_ASSERT(*dependency); + dependency= (*dependency)->unref(dependency); + do + { + /* check duplicates */ + List_iterator_fast<Item*> li(s->master_unit()->item->depends_on); + Item **dep; + while ((dep= li++)) + { + if ((*dep)->eq(*dependency, FALSE)) + { + DBUG_PRINT("info", ("dependency %s already present", + ((*dependency)->name ? + (*dependency)->name : + "<no name>"))); + DBUG_VOID_RETURN; + } + } + + s->master_unit()->item->depends_on.push_back(dependency); + DBUG_PRINT("info", ("depends_on: Select: %d added: %s", + s->select_number, + ((*dependency)->name ? + (*dependency)->name : + "<no name>"))); + } while ((s= s->outer_select()) != last && s != 0); + DBUG_VOID_RETURN; +} + + /* st_select_lex_node::mark_as_dependent mark all st_select_lex struct from this to 'last' as dependent @@ -1843,7 +1890,7 @@ bool st_select_lex::mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency) { - + DBUG_ENTER("st_select_lex::mark_as_dependent"); DBUG_ASSERT(this != last); /* @@ -1872,11 +1919,11 @@ Item_subselect *subquery_expr= s->master_unit()->item; if (subquery_expr && subquery_expr->mark_as_dependent(thd, last, dependency)) - return TRUE; + DBUG_RETURN(TRUE); } while ((s= s->outer_select()) != last && s != 0); is_correlated= TRUE; this->master_unit()->item->is_correlated= TRUE; - return FALSE; + DBUG_RETURN(FALSE); } bool st_select_lex_node::set_braces(bool value) { return 1; } === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.h 2010-05-24 17:29:56 +0000 @@ -748,6 +748,7 @@ } bool mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency); + void register_dependency_item(st_select_lex *last, Item **dependency); bool set_braces(bool value); bool inc_in_sum_expr(); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-10 13:46:08 +0000 +++ b/sql/sql_select.cc 2010-05-24 17:29:56 +0000 @@ -151,7 +151,6 @@ static int join_read_system(JOIN_TAB *tab); static int join_read_const(JOIN_TAB *tab); static int join_read_key(JOIN_TAB *tab); -static int join_read_key2(JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref); static void join_read_key_unlock_row(st_join_table *tab); static int join_read_always_key(JOIN_TAB *tab); static int join_read_last_key(JOIN_TAB *tab); @@ -5209,7 +5208,7 @@ 'join->best_positions' contains a complete optimal extension of the current partial QEP. */ - DBUG_EXECUTE("opt", print_plan(join, join->tables, + DBUG_EXECUTE("opt", print_plan(join, n_tables, record_count, read_time, read_time, "optimal");); DBUG_RETURN(FALSE); @@ -7625,6 +7624,40 @@ /** + Creates and fills JOIN_TAB for index look up in temporary table + + @param table The table where to look up + + @return JOIN_TAB object or NULL in case of error +*/ + +JOIN_TAB *create_index_lookup_join_tab(TABLE *table) +{ + JOIN_TAB *tab; + DBUG_ENTER("create_index_lookup_join_tab"); + + if (!((tab= new JOIN_TAB))) + DBUG_RETURN(NULL); + tab->read_record.table= table; + tab->read_record.file=table->file; + /*tab->read_record.unlock_row= rr_unlock_row;*/ + tab->next_select=0; + tab->sorted= 1; + + table->status= STATUS_NO_RECORD; + tab->read_first_record= join_read_key; + /*tab->read_record.unlock_row= join_read_key_unlock_row;*/ + tab->read_record.read_record= join_no_more_records; + if (table->covering_keys.is_set(tab->ref.key) && + !table->no_keyread) + { + table->key_read=1; + table->file->extra(HA_EXTRA_KEYREAD); + } + DBUG_RETURN(tab); +} + +/** Give error if we some tables are done with a full join. This is used by multi_table_update and multi_table_delete when running @@ -10778,6 +10811,7 @@ case Item::REF_ITEM: case Item::NULL_ITEM: case Item::VARBIN_ITEM: + case Item::CACHE_ITEM: if (make_copy_field) { DBUG_ASSERT(((Item_result_field*)item)->result_field); @@ -11552,7 +11586,8 @@ &param->recinfo, select_options)) goto err; } - if (open_tmp_table(table)) + DBUG_PRINT("info", ("skip_create_table: %d", (int)param->skip_create_table)); + if (!param->skip_create_table && open_tmp_table(table)) goto err; thd->mem_root= mem_root_save; @@ -11700,16 +11735,17 @@ bool open_tmp_table(TABLE *table) { int error; + DBUG_ENTER("open_tmp_table"); if ((error= table->file->ha_open(table, table->s->table_name.str, O_RDWR, HA_OPEN_TMP_TABLE | HA_OPEN_INTERNAL_TABLE))) { table->file->print_error(error,MYF(0)); /* purecov: inspected */ table->db_stat=0; - return(1); + DBUG_RETURN(1); } (void) table->file->extra(HA_EXTRA_QUICK); /* Faster */ - return(0); + DBUG_RETURN(0); } @@ -12540,7 +12576,8 @@ else { /* Do index lookup in the materialized table */ - if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + if ((res= join_read_key2(join_tab->join->thd, join_tab, + sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); @@ -13323,61 +13360,61 @@ static int join_read_key(JOIN_TAB *tab) { - return join_read_key2(tab, tab->table, &tab->ref); + return join_read_key2(tab->join->thd, tab, tab->table, &tab->ref); } -/* +/* eq_ref access handler but generalized a bit to support TABLE and TABLE_REF not from the join_tab. See join_read_key for detailed synopsis. */ -static int -join_read_key2(JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref) +int join_read_key2(THD *thd, JOIN_TAB *tab, TABLE *table, TABLE_REF *table_ref) { int error; + DBUG_ENTER("join_read_key2"); if (!table->file->inited) { table->file->ha_index_init(table_ref->key, tab->sorted); } /* TODO: Why don't we do "Late NULLs Filtering" here? */ - if (cmp_buffer_with_ref(tab->join->thd, table, table_ref) || + if (cmp_buffer_with_ref(thd, table, table_ref) || (table->status & (STATUS_GARBAGE | STATUS_NO_PARENT | STATUS_NULL_ROW))) { if (table_ref->key_err) { table->status=STATUS_NOT_FOUND; - return -1; + DBUG_RETURN(-1); } /* Moving away from the current record. Unlock the row in the handler if it did not match the partial WHERE. */ - if (tab->ref.has_record && tab->ref.use_count == 0) + if (table_ref->has_record && table_ref->use_count == 0) { tab->read_record.file->unlock_row(); - tab->ref.has_record= FALSE; + table_ref->has_record= FALSE; } error=table->file->ha_index_read_map(table->record[0], table_ref->key_buff, make_prev_keypart_map(table_ref->key_parts), HA_READ_KEY_EXACT); if (error && error != HA_ERR_KEY_NOT_FOUND && error != HA_ERR_END_OF_FILE) - return report_error(table, error); + DBUG_RETURN(report_error(table, error)); if (! error) { - tab->ref.has_record= TRUE; - tab->ref.use_count= 1; + table_ref->has_record= TRUE; + table_ref->use_count= 1; } } else if (table->status == 0) { - DBUG_ASSERT(tab->ref.has_record); - tab->ref.use_count++; + DBUG_ASSERT(table_ref->has_record); + table_ref->use_count++; } table->null_row=0; - return table->status ? -1 : 0; + DBUG_RETURN(table->status ? -1 : 0); } === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-20 12:01:47 +0000 +++ b/sql/table.cc 2010-05-24 17:29:56 +0000 @@ -20,6 +20,7 @@ #include "sql_trigger.h" #include <m_ctype.h> #include "my_md5.h" +#include "my_bit.h" /* INFORMATION_SCHEMA name */ LEX_STRING INFORMATION_SCHEMA_NAME= {C_STRING_WITH_LEN("information_schema")}; @@ -5096,6 +5097,115 @@ file->column_bitmaps_signal(); } + +/** + @brief + Allocate space for keys + + @param key_count number of keys to allocate. + + @details + Allocate space enough to fit 'key_count' keys for this table. + + @return FALSE space was successfully allocated. + @return TRUE an error occur. +*/ + +bool TABLE::alloc_keys(uint key_count) +{ + DBUG_ASSERT(!s->keys); + key_info= s->key_info= (KEY*) my_malloc(sizeof(KEY)*key_count, MYF(0)); + max_keys= key_count; + return !(key_info); +} + + +/** + @brief Adds one key to a temporary table. + + @param key_parts bitmap of fields that take a part in the key. + @param key_name name of the key + + @details + Creates a key for this table from fields which corresponds the bits set to 1 + in the 'key_parts' bitmap. The 'key_name' name is given to the newly created + key. + + @return <0 an error occur. + @return >=0 number of newly added key. +*/ + +int TABLE::add_tmp_key(ulonglong key_parts, const char *key_name) +{ + DBUG_ASSERT(s->keys< max_keys); + + KEY* keyinfo; + Field **reg_field; + uint i; + bool key_start= TRUE; + uint key_part_count= my_count_bits(key_parts); + KEY_PART_INFO* key_part_info= + (KEY_PART_INFO*) my_malloc(sizeof(KEY_PART_INFO)* key_part_count, MYF(0)); + if (!key_part_info) + return -1; + keyinfo= key_info + s->keys; + keyinfo->key_part=key_part_info; + keyinfo->usable_key_parts=keyinfo->key_parts= key_part_count; + keyinfo->key_length=0; + keyinfo->algorithm= HA_KEY_ALG_UNDEF; + keyinfo->name= (char *)key_name; + keyinfo->flags= HA_GENERATED_KEY; + keyinfo->rec_per_key= (ulong*)my_malloc(sizeof(ulong)*key_part_count, MYF(0)); + if (!keyinfo->rec_per_key) + return -1; + bzero(keyinfo->rec_per_key, sizeof(ulong)*key_part_count); + for (i= 0, reg_field=field ; + *reg_field; + i++, reg_field++) + { + if (!(key_parts & (1 << i))) + continue; + if (key_start) + (*reg_field)->key_start.set_bit(s->keys); + key_start= FALSE; + (*reg_field)->part_of_key.set_bit(s->keys); + (*reg_field)->flags|= PART_KEY_FLAG; + key_part_info->null_bit= (*reg_field)->null_bit; + key_part_info->null_offset= (uint) ((*reg_field)->null_ptr - + (uchar*) record[0]); + key_part_info->field= *reg_field; + key_part_info->offset= (*reg_field)->offset(record[0]); + key_part_info->length= (uint16) (*reg_field)->pack_length(); + keyinfo->key_length+= key_part_info->length; + /* TODO: + The below method of computing the key format length of the + key part is a copy/paste from opt_range.cc, and table.cc. + This should be factored out, e.g. as a method of Field. + In addition it is not clear if any of the Field::*_length + methods is supposed to compute the same length. If so, it + might be reused. + */ + key_part_info->store_length= key_part_info->length; + + if ((*reg_field)->real_maybe_null()) + key_part_info->store_length+= HA_KEY_NULL_LENGTH; + if ((*reg_field)->type() == MYSQL_TYPE_BLOB || + (*reg_field)->real_type() == MYSQL_TYPE_VARCHAR) + key_part_info->store_length+= HA_KEY_BLOB_LENGTH; + + key_part_info->type= (uint8) (*reg_field)->key_type(); + key_part_info->key_type = + ((ha_base_keytype) key_part_info->type == HA_KEYTYPE_TEXT || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT1 || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT2) ? + 0 : FIELDFLAG_BINARY; + key_part_info++; + } + set_if_bigger(s->max_key_length, keyinfo->key_length); + return ++s->keys - 1; +} + + /** @brief Check if this is part of a MERGE table with attached children. === modified file 'sql/table.h' --- a/sql/table.h 2010-03-20 12:01:47 +0000 +++ b/sql/table.h 2010-05-24 17:29:56 +0000 @@ -781,6 +781,7 @@ uint temp_pool_slot; /* Used by intern temp tables */ uint status; /* What's in record[0] */ uint db_stat; /* mode of file as in handler.h */ + uint max_keys; /* Size of allocated key_info array. */ /* number of select if it is derived table */ uint derived_select_number; int current_lock; /* Type of lock on table */ @@ -914,6 +915,8 @@ inline bool needs_reopen_or_name_lock() { return s->version != refresh_version; } bool is_children_attached(void); + bool alloc_keys(uint key_count); + int add_tmp_key(ulonglong key_parts, const char *key_name); }; enum enum_schema_table_state === modified file 'storage/maria/ha_maria.cc' --- a/storage/maria/ha_maria.cc 2010-03-20 12:01:47 +0000 +++ b/storage/maria/ha_maria.cc 2010-05-24 17:29:56 +0000 @@ -995,6 +995,8 @@ { MARIA_HA *tmp= file; file= 0; + if (!tmp) + return 0; return maria_close(tmp); }

1 0

[Maria-developers] internals of page row format
by Igor Kozyrev 19 May '10

19 May '10

Could anyone please explain data representation, both on disk and in memory? Especially about BLOB fields What was the reasons to move hot data from OS file cache to MySQL memory? Is it great improvement of efficiency(to avoid kernel calls) or necessity for future transaction support? -- This message was sent from Launchpad by the user Igor Kozyrev (https://launchpad.net/~ikozyrev36) using the "Contact this team" link on the Maria developers team page to each member directly. For more information see https://help.launchpad.net/YourAccount/ContactingPeople

1 0

[Maria-developers] Segmented Key Cache - About section
by Daniel Bartholomew 19 May '10

19 May '10

Hello everyone, I'm working on the documentation for the Segmented Key Cache, one of the new features in MariaDB 5.2. The documentation page will have "about", "syntax", and "example" sections. I've based the "About" section on the High-Level Description from the worklog ( http://askmonty.org/worklog/Server-Sprint/?tid=85 ) and from the description in the mysys/mf_keycache.c file in the source. Here is my first draft. I've changed the wording a bit to help it flow better and I've changed "partition" to "segment" in keeping with the official name. ---------------------------------------------------------------------- == About Segmented Key Cache == A segmented key cache is a collection of structures for regular MyISAM key caches called key cache segments. Segmented key caches mitigate one of the major problem of the simple key cache: thread contention for key cache lock (mutex). With regular key caches, every call of a key cache interface function must acquire this lock. Threads compete for this lock even when the file and pages they have acquired shared locks for are in the key cache buffers. When working with a segmented key cache any key cache interface function that needs only one page has to acquire the key cache lock only for the segment the page is assigned to. This makes the chances for threads not having to compete for the same key cache lock better. Any page from a file can be placed into a buffer of only one segment. The number of the segment is calculated from the file number and the position of the page in the file, and it's always the same for the page. Pages are evenly distributed among segments. The idea and the original code of the segmented key cache was provided by Fredrik Nylander from Stardoll.com. The code was extensively reworked, improved, and eventually merged into MariaDB by Igor Babaev from Monty Program. ---------------------------------------------------------------------- If there are any factual errors with the above, let me know. Improvements and suggestions are also welcome. One thing I'm not happy with is this sentence: "Threads compete for this lock even when the file and pages they have acquired shared locks for are in the key cache buffers." The original sentence from the worklog reads: So threads compete for this lock even in the case when they have acquired shared locks for the file and pages they want read from are in the key cache buffers. ... which is very confusing to me. If anyone has a better or more accurate rewording, please send it my way. Also, there's a note in the source code (lines 5018-5023 of mysys/mf_keycache.c) which states: Unfortunately if we use a partitioned key cache with N partitions for B-tree indexes we can't say that the chances becomes N times less. The fact is that any index lookup operation requires reading from the root page that, for any index, is always ascribed to the same partition. To resolve this problem we should have employed more sophisticated mechanisms of working with root pages. Do any of you have any opinions on whether or not this should be mentioned somewhere in the documentation? The only issue I have with mentioning it is that if I do, the first question that comes to my mind is: "If it is not N-times less, how much less is it?" I don't have an answer to that question (and math isn't one of my strengths). Next up will be the "Syntax" section. Thanks. -- Daniel Bartholomew Monty Program - http://askmonty.org

3 6

Re: [Maria-developers] MySQL 5.1.46 diff of sql/sql_select.cc for review
by Kristian Nielsen 19 May '10

19 May '10

Igor Babaev <igor(a)askmonty.org> writes: > Here's my review. Thanks a lot for your help! I checked through all your points (few detailed comments below), all your proposed solutions seem reasonable to me. In summary, we should as you suggest: - Rework patches for bugs 39022, 48483, and 49324 as you described (1,7,8). - Revert bug 45640 patch (5) and instead apply your proposed patch. - Revert patches for bugs 51242 and 52336, and instead apply your proposed fix (15,18). - Reject (eg. revert in 5.1-release) patch for bug 39653 (2) - Do nothing for bugs 40277, 45195, 45989, 49902, 50995, and 51494 (3,4,6,10,14,16), as necessary changes were already pushed to 5.1-release. - Do nothing for bugs 49829, 50335, 50591, 50843, and 52177 (9,11,12,13,17), as the patches for those are ok. You proposed on the call yesterday that you could prepare a patch with these changes for our merge tree: lp:~maria-captains/maria/5.1-release Please do so. I will try to get hold of Monty and discuss with him, but I think he will agree with making your proposed changes. If you need a full review for your changes, you will probably need to ask someone else than me, as I am unfamiliar with this part of the code. Thanks, - Kristian. > 1. Bug #39022 (by Georgi Kodinov) > ---------------------------------- > > SYNOPSIS > Any call of SQL_SELECT::skip record ignore the fact that > during evaluation of an expression (item) an error may occur. > > CONCLUSION > The patch cannot be applied as it is, requires some rework. > > REASONS: > The patch is incomplete: > There are 5 calls of SQL_SELECT::skip_record in total: > two - in sql_select.cc, and three others in > filesort.cc, sql_delete.cc, sql_update.cc. > Only the calls in sql_select.cc are handled in an error aware > manner. The second call in sql_select is handled incorrectly: > the error occurred at the evaluation of the select condition > is caught only if the function returns false. > > POSSIBLE SOLUTION: > change the synopsis of SQL_SELECT::skip_record: > int SQL_SELECT::skip_record() > { > int rc= test(cond && !cond->val_int()); > if thd->is_error() rc=-1; > return rc; > } > > (thd must be added to SQL_SELECT) > > then after each call SQL_SELECT::retain_record > add error handling for the cases when > SQL_SELECT::skip_record returns -1: > int rc= 0; > if (!select || (rc= select->skip_record()) != 0) > { > if (rc < 0) > { > /* handle error returned by skip_record() */ > ... > } > ... > } Agree with your proposed solution. > 2. Bug #39653 (by Gleb Shchepa) > > SYNOPSIS > InnoDB covering primary index is used when using another > covering index is more beneficial. > > CONCLUSION > The patch must be rejected. > > REASONS: > The patch is based on completely wrong idea that any > covering secondary index is better than primary covering index > for scanning in InnoDB. > > Here an example demonstrating that it's not so: > CREATE TABLE t1 ( > a int, b int, c int, > PRIMARY KEY (a, b), > KEY idx (a,c) > ); > Both primary key and the secondary key here are covering for the query > SELECT a FROM t1 WHERE a BETWEEN 1000 and 5000; > Apparently scanning by the primary key will be faster here as it does > not use random seeks. > > The patch completely ignores non-InnoDB engines. > > POSSIBLE SOLUTION: > Cost based choice that takes into account that sequential access is > C times faster then random access. I am ok with rejecting the patch. It's hard to tell whether primary key or index will be better to use, in some cases secondary index may require no random seeks if it is not fragmented while primary key could be. In any case this seems a dangerous change in a stable release (user can force index of choice if he/she has more information about which will be best). > 3. Bug #40277 (by Davi Arnaut) > ------------------------------- > see the notes from Monty's review Ok, Monty already fixed this in 5.1-release. Agree. > 4. 45195 (by Sergey Glukhov) > ----------------------------- > SYNOPSIS > Reading uninitialized bytes from join cache buffer. > (Valgrind's complain) > > CONCLUSION > The patch should be accepted as it is > (Monty has some a comment on this patch though). Agree, Monty already pushed his fixes to 5.1-release. > 5. Bug 45640 (by Gleb Shchepa) > ------------------------------- > SYNOPSYS. > Building Item_ref objects of a wrong type for outer references used > in aliased expressions in a select with group by causes wrong > results. Group by expressions containing references to aliases may > cause wrong results. > > CONCLUSION. > The patch cannot be accepted as it is, requires a serious re-work. > > REASON. > Although the basic ideas behind the fix appear to be be valid their > implementation is quite clumsy: > -an unnecessary parameter is added to the function fix_inner_refs > -the info about the syntax context of a field referenced is passed > into Item_field::fix_fields in an unconventional and ugly manner > -the group expression are traversed for each reference of the list > of inner references > > POSSIBLE SOLUTION > See the patch at the very end of the post. Agree with proposed solution, using the patch at end. > 6. Bug #45989 (by Georgi Kodinov) > ---------------------------------- > This bug has been already fixed in MariaDB 5.1.44. > Our fix is correct, the fix by Georgi is not quite correct > (but not harmful). Agree, yes the better fix is already pushed. > 7. Bug #48483 (by Sergey Glukhov) > ---------------------------------- > (No public access) > > SYNOPSIS > Wrong calculation of table dependencies for join queries with > outer join operation causes a crash. > > CONCLUSION > The patch can be accepted with one change that matters > - if (!((prev_table->on_expr->used_tables() & ~RAND_TABLE_BIT) & > - ~prev_used_tables)) > + if (!((prev_table->on_expr->used_tables() & > + ~(OUTER_REF_BIT | RAND_TABLE_BIT)) & > + ~prev_used_tables)) I assume you mean OUTER_REF_TABLE_BIT here. Ok. > 8. Bug #49324 (by Georgi Kodinov) > ---------------------------------- > SYNOPSIS > With InnoDB a GROUP BY / ORDER BY can use an index extended by some > number of major components of the primary. The value of rec_per_key > for such extended indexes must be calculated in a special way. > > CONCLUSION > The patch could be accepted after changing the formula that calculates > the value of rec_per_key for an extended index: > > - rec_per_key= used_key_parts && > - used_key_parts <= keyinfo->key_parts ? > - keyinfo->rec_per_key[used_key_parts-1] : 1; > + int used_index_parts= keyinfo->key_parts; > + int used_pk_parts= 0; > + set_if_bigger(used_pk_parts, > + used_key_parts-used_index_parts); > + rec_per_key= keyinfo->rec_per_key[used_key_parts-1]; > + if (used_pk_parts) > + { > + KEY *pkinfo= tab->table->key_info+table->s->primary_key; > + rec_per_key*= pkinfo->rec_per_key[used_pk_parts-1]; > + rec_per_key/= pkinfo->rec_per_key[0]; > + } > > REASONS > The formula in the patch does not take into account how many > components of the of the primary key is used in the extended index. Ok (I don't understand the calculation, but not knowing the code I'm willing to take your word for it). > 9. Bug #49829 (by Staale Smedseng) > ---------------------------------- > SYNOPSYS > Compiler problems (warnings) for a platform > > CONCLUSION > THe patch is ok. Agree (it was kind of nice to see the explanation for these warnings, which I think I saw before but didn't know what meant). > 10. Bug #49902 (by Sergey Vojtovich) > ----------------------------------- > See the comments/suggestions from Monty's review Yes, this is pushed to 5.1-release (and I agree with Monty's comment). > 11. Bug #50335 (by Alexey Kopytov) > ----------------------------------- > (No public access) > > SYNOPSYS > Failure of a wrong assertion > > CONCLUSION > The patch is ok. Ok. > 12. Bug #50591 (by Sergey Glukhov) > ----------------------------------- > SYNOPSIS > Wrong result for a grouping query over a table with a BIT field > > CONCLUSION > The patch is ok. Ok. > 13. Bug #50843 (by Evgeny Potemkin) > ------------------------------------ > SYNOPSIS > Performance degradation problem when join cache + filesort > are used intead of a full index scan by a primary key. > > CONCLUSION > The patch looks ok. Ok. > 14. Bug #50995 (by Sergey Glukhov) > ------------------------------------ > SYNOPSIS > A badly formed list for conditions causes wrong query results. > > CONCLUSION > The patch looks ok for me. > See also Monty's recommendation from his review. Yes. As far as I can see, Monty pushed his changes from his review to 5.1-release. He did however not update the comments for eliminate_item_equal() as per his suggestion (maybe he forgot): "Note that we should update the function comment for eliminate_item_equal() as this can't return 0. (If it would, then other things would break, just look at how this function is used)." > 15. Bug #51242 (by Sergey Glukhov) > ----------------------------------- > SYNOPSYS > The conjuncts that become false after substitution of the constant > tables are ignored. > > CONCLUSION > The fix should be turned down (but not the test case). > > REASON > See the reasons for turning the fix for bug #52336 that is a > correction for this patch. > > POSSIBLE SOLUTION > See the solution for bug #52336. > 16. Bug #51494 (by Sergey Glukhov) > ----------------------------------- > SYNOPSIS > Crash with explain of a query with outer join > > CONCLUSION > The patch must be turned down. > > REASONS > The patch triggers bug #53334 - a failure of a base join > query for InnoDB. > The bug is actually fixed by the patch for bug #52177 > (see my comment for bug #53334). Yes, it is already reverted in our tree. > 17. Bug #52177 (by Sergey Glukhov) > ----------------------------------- > SYNOPSIS > Crash with exaplain for a query with an outer join. > > CONCLUSION > The fix is correct and the patch should be applied. > The patch also fixes the bug #51494. Ok. > 18. Bug #52336 (by Sergey Glukhov) > ----------------------------------- > SYNOPSYS > A crash caused by an invalid fix for bug #51242. > > CONCLUSION > The patch rather should be turned down. > > REASON. > The patch does not fix the real cause of the problem: > a wrong value is passed as a parameter in the call > Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, > used_tables, > used_tables); > The patch actually suggest a work-around that hides the bug. > This work-around simultaneously adds a new feature. > that catches impossible HAVINGs after constant table substitution. > Yet impossible WHEREs appeared after this optimization remain > uncaught. So the feature is introduced half-baked. > > POSSIBLE SOLUTION > - Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, > - used_tables, > - used_tables); > + Item* sort_table_cond= make_cond_for_table(curr_join->tmp_having, > + used_tables, > + (table_map) 0); Ok. This possible solution is not yet applied to our tree (we only discussed it so far).

1 0

[Maria-developers] MariaDB 5.2 source code tree tarball
by Daniel Bartholomew 19 May '10

19 May '10

All, I've created a tarball of a freshly branched MariaDB 5.2 source tree. I created the branch with: bzr branch lp:maria/5.2 mariadb-5.2 I created the tarball with: tar -czvf mariadb-5.2-repo.tar.gz mariadb-5.2/ I then saw to it that the tarball was uploaded to our mirrors. If you are having trouble with using bzr to branch the complete MariaDB source tree, using this tarball is an option. For example, see this bug on Launchpad: https://bugs.launchpad.net/bugs/407834 Links to the tarball (and some brief instructions) are here: http://askmonty.org/wiki/Getting_the_MariaDB_Source_Code#Source_Tree_Tarball I've tested the tree by downloading the tarball to my local machine, and it appears to work just fine. Let me know if you try to use it and it doesn't work for you. Thanks. -- Daniel Bartholomew Monty Program - http://askmonty.org

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (igor:2791)
by Igor Babaev 18 May '10

18 May '10

#At lp:maria/5.2 based on revid:igor@askmonty.org-20100513065914-yq9y2pbd9zn2fm7w 2791 Igor Babaev 2010-05-18 Fixed bugs in the backport of derived tables (mwl106). modified: mysql-test/r/derived_view.result mysql-test/r/table_elim.result sql/item_cmpfunc.cc sql/item_cmpfunc.h sql/sql_class.h sql/sql_select.cc sql/sql_select.h sql/sql_union.cc === modified file 'mysql-test/r/derived_view.result' --- a/mysql-test/r/derived_view.result 2010-04-29 21:10:39 +0000 +++ b/mysql-test/r/derived_view.result 2010-05-18 17:46:32 +0000 @@ -442,7 +442,7 @@ id select_type table type possible_keys 1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where 1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer Warnings: -Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` join `test`.`t1` where ((`test`.`t1`.`f1` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7) and ((`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7))) +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` join `test`.`t1` where ((`test`.`t1`.`f1` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7) and (`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7)) select * from (select * from (select * from t1 where f1 < 7 ) tt where f1 > 2 ) x === modified file 'mysql-test/r/table_elim.result' --- a/mysql-test/r/table_elim.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/table_elim.result 2010-05-18 17:46:32 +0000 @@ -117,58 +117,58 @@ t2 where id=f.id); This should use one table: explain select id from v1 where id=2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY f const PRIMARY PRIMARY 4 const 1 Using index +1 SIMPLE f const PRIMARY PRIMARY 4 const 1 Using index This should use one table: explain extended select id from v1 where id in (1,2,3,4); id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index +1 SIMPLE f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index Warnings: -Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` where (`f`.`id` in (1,2,3,4)) This should use facts and a1 tables: explain extended select id from v1 where attr1 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index +1 SIMPLE a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index Warnings: -Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t1` `a1` where ((`f`.`id` = `a1`.`id`) and (`a1`.`attr1` between 12 and 14)) This should use facts, a2 and its subquery: explain extended select id from v1 where attr2 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using where; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using index +1 SIMPLE a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using where; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using index 3 DEPENDENT SUBQUERY t2 ref PRIMARY PRIMARY 4 test.a2.id 2 100.00 Using index Warnings: -Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.a2.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t2` `a2` where ((`f`.`id` = `a2`.`id`) and (`a2`.`attr2` between 12 and 14) and (`a2`.`fromdate` = (select max(`test`.`t2`.`fromdate`) AS `MAX(fromdate)` from `test`.`t2` where (`test`.`t2`.`id` = `a2`.`id`)))) This should use one table: explain select id from v2 where id=2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY f const PRIMARY PRIMARY 4 const 1 Using index +1 SIMPLE f const PRIMARY PRIMARY 4 const 1 Using index This should use one table: explain extended select id from v2 where id in (1,2,3,4); id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index +1 SIMPLE f range PRIMARY PRIMARY 4 NULL 4 100.00 Using where; Using index Warnings: -Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` where (`f`.`id` in (1,2,3,4)) This should use facts and a1 tables: explain extended select id from v2 where attr1 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index +1 SIMPLE a1 range PRIMARY,attr1 attr1 5 NULL 2 100.00 Using index condition; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a1.id 1 100.00 Using index Warnings: -Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t1` `a1` where ((`f`.`id` = `a1`.`id`) and (`a1`.`attr1` between 12 and 14)) This should use facts, a2 and its subquery: explain extended select id from v2 where attr2 between 12 and 14; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using MRR -1 PRIMARY f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using where; Using index +1 SIMPLE a2 range PRIMARY,attr2 attr2 5 NULL 5 100.00 Using index condition; Using MRR +1 SIMPLE f eq_ref PRIMARY PRIMARY 4 test.a2.id 1 100.00 Using where; Using index 3 DEPENDENT SUBQUERY t2 ref PRIMARY PRIMARY 4 test.f.id 2 100.00 Using index Warnings: -Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #1 +Note 1276 Field or reference 'test.f.id' of SELECT #3 was resolved in SELECT #2 Note 1003 select `f`.`id` AS `id` from `test`.`t0` `f` join `test`.`t2` `a2` where ((`f`.`id` = `a2`.`id`) and (`a2`.`attr2` between 12 and 14) and (`a2`.`fromdate` = (select max(`test`.`t2`.`fromdate`) AS `MAX(fromdate)` from `test`.`t2` where (`test`.`t2`.`id` = `f`.`id`)))) drop view v1, v2; drop table t0, t1, t2; === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-04-29 21:10:39 +0000 +++ b/sql/item_cmpfunc.cc 2010-05-18 17:46:32 +0000 @@ -4232,8 +4232,20 @@ Item_cond::fix_fields(THD *thd, Item **r (item= *li.ref())->check_cols(1)) return TRUE; /* purecov: inspected */ used_tables_cache|= item->used_tables(); +#if 0 if (!item->const_item()) const_item_cache= FALSE; +#else + if (item->const_item()) + and_tables_cache= (table_map) 0; + else + { + table_map tmp_table_map= item->not_null_tables(); + not_null_tables_cache|= tmp_table_map; + and_tables_cache&= tmp_table_map; + const_item_cache= FALSE; + } +#endif with_sum_func= with_sum_func || item->with_sum_func; with_subselect|= item->with_subselect; @@ -4253,6 +4265,7 @@ Item_cond::eval_not_null_tables(uchar *o { Item *item; List_iterator<Item> li(list); + and_tables_cache= ~(table_map) 0; while ((item=li++)) { table_map tmp_table_map; === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-04-29 21:10:39 +0000 +++ b/sql/item_cmpfunc.h 2010-05-18 17:46:32 +0000 @@ -1778,19 +1778,6 @@ inline Item *and_conds(Item *a, Item *b) { if (!b) return a; if (!a) return b; - /* Try to minimize item tree by adding to already present AND functions. */ - if (a->type() == Item::COND_ITEM && - ((Item_cond*) a)->functype() == Item_func::COND_AND_FUNC) - { - ((Item_cond*)a)->add(b); - return a; - } - else if (b->type() == Item::COND_ITEM && - ((Item_cond*) b)->functype() == Item_func::COND_AND_FUNC) - { - ((Item_cond*)b)->add(a); - return b; - } return new Item_cond_and(a, b); } === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-04-29 21:10:39 +0000 +++ b/sql/sql_class.h 2010-05-18 17:46:32 +0000 @@ -2790,11 +2790,6 @@ public: */ bool bit_fields_as_long; - /* - Whether to create or postpone actual creation of this temporary table. - TRUE <=> create_tmp_table will create only the TABLE structure. - */ - bool skip_create_table; TMP_TABLE_PARAM() :copy_field(0), group_parts(0), group_length(0), group_null_parts(0), convert_blob_length(0), === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-05-12 04:09:58 +0000 +++ b/sql/sql_select.cc 2010-05-18 17:46:32 +0000 @@ -9845,8 +9845,9 @@ simplify_joins(JOIN *join, List<TABLE_LI { conds= and_conds(conds, table->on_expr); conds->top_level_item(); - if (!conds->fixed) - conds->fix_fields(join->thd, &conds); + /* conds is always a new item as both cond and on_expr existed */ + DBUG_ASSERT(!conds->fixed); + conds->fix_fields(join->thd, &conds); } else conds= table->on_expr; @@ -11035,7 +11036,7 @@ TABLE * create_tmp_table(THD *thd,TMP_TABLE_PARAM *param,List<Item> &fields, ORDER *group, bool distinct, bool save_sum_fields, ulonglong select_options, ha_rows rows_limit, - char *table_alias) + char *table_alias, bool do_not_open) { MEM_ROOT *mem_root_save, own_root; TABLE *table; @@ -11728,7 +11729,7 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA if (thd->is_fatal_error) // If end of memory goto err; /* purecov: inspected */ share->db_record_offset= 1; - if (!param->skip_create_table) + if (!do_not_open) { if (share->db_type() == TMP_ENGINE_HTON) { === modified file 'sql/sql_select.h' --- a/sql/sql_select.h 2010-04-29 21:10:39 +0000 +++ b/sql/sql_select.h 2010-05-18 17:46:32 +0000 @@ -1984,7 +1984,7 @@ void push_index_cond(JOIN_TAB *tab, uint TABLE *create_tmp_table(THD *thd,TMP_TABLE_PARAM *param,List<Item> &fields, ORDER *group, bool distinct, bool save_sum_fields, ulonglong select_options, ha_rows rows_limit, - char* alias); + char* alias, bool do_not_open=FALSE); void free_tmp_table(THD *thd, TABLE *entry); bool create_internal_tmp_table_from_heap(THD *thd, TABLE *table, ENGINE_COLUMNDEF *start_recinfo, === modified file 'sql/sql_union.cc' --- a/sql/sql_union.cc 2010-05-12 04:09:58 +0000 +++ b/sql/sql_union.cc 2010-05-18 17:46:32 +0000 @@ -126,12 +126,11 @@ select_union::create_result_table(THD *t tmp_table_param.init(); tmp_table_param.field_count= column_types->elements; tmp_table_param.bit_fields_as_long= bit_fields_as_long; - tmp_table_param.skip_create_table= !create_table; - if (! (table= create_tmp_table(thd_arg, &tmp_table_param, *column_types, (ORDER*) 0, is_union_distinct, 1, - options, HA_POS_ERROR, (char*) alias))) + options, HA_POS_ERROR, (char*) alias, + !create_table))) return TRUE; if (create_table) {

1 0

[Maria-developers] Segmented Key Cache - Syntax section
by Daniel Bartholomew 17 May '10

17 May '10

Hello everyone, Summarizing from my previous email: I'm working on the documentation for the Segmented Key Cache, one of the new features in MariaDB 5.2. My previous email was about the "About" section of the documentation. This email is about the "Syntax" section. For the syntax section, I see there is one new global variable defined for this feature and a new KEY_CACHES table in the information_schema database. Are there any other user-visible items which should be mentioned? Here is a first draft of the syntax section: ---------------------------------------------------------------------- == Segmented Key Cache Syntax == New global variable: key_cache_partitions, it sets the number of segments in a key cache. Valid values for this variable are whole numbers between 0 and 64. If the number of partitions is set to a number greater than 64 the number of partitions will be truncated to 64 and a warning will be issued. A value of '0' means the key cache is a regular (i.e. non-segmented) key cache. This is the default. Other global variables used when working with regular key caches also apply to segmented key caches: key_buffer_size, key_cache_age_threshold, key_cache_block_size, and key_cache_division_limit. See the MySQL manual for descriptions of these variables. http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html Statistics about the key cache can be found by looking at the KEY_CACHES table in the INFORMATION_SCHEMA database. Columns in this table are: * KEY_CACHE_NAME: The name of the key cache * PARTITIONS: total number of segments * PARTITION_NUMBER: segment number (set to NULL if a simple key cache) * FULL_SIZE: memory for cache buffers/auxiliary structures * BLOCK_SIZE: size of the blocks * USED_BLOCKS: number of currently used blocks * UNUSED_BLOCKS: number of currently unused blocks * DIRTY_BLOCKS: number of currently dirty blocks * READ_REQUESTS: number of read requests * READS: number of actual reads from files into buffers * WRITE_REQUESTS: number of write requests * WRITES: number of actual writes from buffers into files ---------------------------------------------------------------------- If there are any factual errors with the above, let me know. Improvements and suggestions are also welcome. Question for the developers: A key_cache_partitions value of '0' means the key cache will not be segmented, it will be a regular (or simple) key cache. A value of '1' means the key cache will be a segmented key cache with a single segment. Is there any benefit to having a "single-segment segmented key cache" compared to a regular "simple key cache" or are they practically the same thing? The key_cache_partitions variable also needs to be documented on the Server System Variables page (http://askmonty.org/wiki/Manual:Server_System_Variables) The entry will look something like this: ---------------------------------------------------------------------- * <code>key_cache_partitions</code> ** '''Description:''' The number of segments in a key cache. ** '''Commandline:''' <code>--key_cache_partitions=#</code> ** '''Scope:''' Global ** '''Dynamic:''' No ** '''Type:''' number ** '''Valid values:''' <code>0-64</code> ** '''Default value:''' <code>0</code> ''(non-segmented)'' * '''Introduced:''' MariaDB 5.2 ---------------------------------------------------------------------- Let me know if there is anything wrong with the above. My task now is to come up with some examples. I'll probably use the test cases for inspiration unless someone has an awesome segmented key cache example that they've been dying to share with me. :) Thanks. -- Daniel Bartholomew Monty Program - http://askmonty.org

3 3

[Maria-developers] Buildbot Status Updates
by Adam M. Dutko 17 May '10

17 May '10

All of the buildbots I administrate should now be back online (adutko-centos5-amd64, adutko-ultrasparc3 and mariadb-brs). Thank you for your patience. -Adam

1 0

Re: [Maria-developers] Creating .bzr tree for download
by Kristian Nielsen 17 May '10

17 May '10

Michael Widenius <michael.widenius(a)gmail.com> writes: > Daniel, can you create a 'initial' repository of MariaDB 5.2 and make a > .tar.gz file of it available on our download page. > We need this ASAP because we get complains from developers that they > can't use bzr to create download MariaDB source because it takes way > too long and often fails in the middle of the process. Since Colin asked on IRC, let me add that (I think) the root of this problem is this bug: https://bugs.launchpad.net/bzr/+bug/407834 (so I can confirm it is a real problem!) - Kristian.

1 0

[Maria-developers] Progress (by Knielsen): Use Buildbot to populate apt/yum repositories (117)
by worklog-noreply＠askmonty.org 17 May '10

17 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Use Buildbot to populate apt/yum repositories CREATION DATE..: Wed, 12 May 2010, 07:04 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 117 (http://askmonty.org/worklog/?tid=117) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 12 ESTIMATE.......: 8 (hours remain) ORIG. ESTIMATE.: 20 PROGRESS NOTES: -=-=(Knielsen - Mon, 17 May 2010, 08:48)=-=- Fixed the conflict on lucid with mysql-client-core-5.1. Figure out and document how to do the signing, discussions with OurDelta. Worked 8 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 12 May 2010, 21:20)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.21448 2010-05-12 21:20:34.000000000 +0000 +++ /tmp/wklog.117.new.21448 2010-05-12 21:20:34.000000000 +0000 @@ -1,5 +1,5 @@ -As for signing, I think it may be possible/best to do the signing outside of -buildbot, as a separate process. There are some advantages to this: +The signing of packages can be done outside of Buildbot, as a separate +process. There are some advantages to this: - Security: the private key can be kept less exposed when it is not included in the buildbot infrastructure. @@ -9,9 +9,6 @@ - Generally reducing the complexity of the buildbot setup. -This of course requires that it is possible to sign the packages after the -actual build. - ---- Here is how to sign the .rpms. @@ -42,20 +39,37 @@ ---- -For .deb, I *think* we are using secure apt, which does not actually sign the -packages, rather it signs the "Release" file which is created when the -repository is set up. So in this case again there is no problem doing the -signing outside of the build itself (in fact that is the way it must be). +For .deb, it is not the individual .deb that is signed, it is the +repository. Here is one way to generate a signed repository, using reprepro. -Found two tools that can help with building and signing apt repositories: -reprepro (seems to be the newest, recommended) and apt-ftparchive. +The ourdelta/bakery signing stuff needs to be copied to ~/.gnupg ----- +mkdir repo # or whatever +cd repo +mkdir conf +cat >conf/distributions <<END +Origin: MariaDB +Label: MariaDB +Codename: hardy +Architectures: amd64 +Components: mariadb-ourdelta +Description: MariaDB test Repository +SignWith: autosign(a)ourdelta.org +END +for i in `find /home/buildbot/debs/ -name '*.deb'` ; do reprepro --basedir=. +includedeb hardy $i ; done + +The corrosponding line for /etc/apt/sources.list: -ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen -suggested splitting up so we have this package ourselves, or maybe it can be -handled with replace/provide/conflict dependencies. + deb file:///home/buildbot/repo hardy mariadb-ourdelta + +This works for multiple distributions, by adding more sections to the +conf/distributions file. + +---- -ToDo: Figure out exactly what files/directory structure needs to be uploaded -(asked Peter, awaiting reply). +For the mysql-client-core-5.1 issue, the solution is to split the +mariadb-client-5.1 (and 5.2) package similarly into +mariadb-client-core-5.1. The mariadb-client-core-5.1 package then provides: +mysql-client-core-5.1. -=-=(Knielsen - Wed, 12 May 2010, 18:25)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.12634 2010-05-12 18:25:58.000000000 +0000 +++ /tmp/wklog.117.new.12634 2010-05-12 18:25:58.000000000 +0000 @@ -12,9 +12,35 @@ This of course requires that it is possible to sign the packages after the actual build. -For .rpm this seems to be easy (from reading, didn't try yet): +---- + +Here is how to sign the .rpms. + +Copy in the ourdelta/bakery signing stuff to ~/.gnupg and ~/.rpmmacros. + +Run + + rpm --addsign *.rpm + +That's all! This can be tested by creating a local yum repository: - rpm --addsign <packages> + createrepo <dir> + +(where <dir> contains the signed .rpms). Then create the file +/etc/yum.repos.d/localmaria.repo: + +[localmaria] +name=Local MariaDB repo +baseurl=file:///home/buildbot/rpms +gpgcheck=1 +enabled=1 +gpgkey=http://master.ourdelta.org/deb/ourdelta.gpg + +Now this should work to install MariaDB: + + sudo yum install MariaDB-server + +---- For .deb, I *think* we are using secure apt, which does not actually sign the packages, rather it signs the "Release" file which is created when the -=-=(Knielsen - Wed, 12 May 2010, 07:14)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.401 2010-05-12 07:14:27.000000000 +0000 +++ /tmp/wklog.117.new.401 2010-05-12 07:14:27.000000000 +0000 @@ -1 +1,35 @@ +As for signing, I think it may be possible/best to do the signing outside of +buildbot, as a separate process. There are some advantages to this: + + - Security: the private key can be kept less exposed when it is not included + in the buildbot infrastructure. + + - It is good to have one step of human intervention before actually signing + and releasing packages. + + - Generally reducing the complexity of the buildbot setup. + +This of course requires that it is possible to sign the packages after the +actual build. + +For .rpm this seems to be easy (from reading, didn't try yet): + + rpm --addsign <packages> + +For .deb, I *think* we are using secure apt, which does not actually sign the +packages, rather it signs the "Release" file which is created when the +repository is set up. So in this case again there is no problem doing the +signing outside of the build itself (in fact that is the way it must be). + +Found two tools that can help with building and signing apt repositories: +reprepro (seems to be the newest, recommended) and apt-ftparchive. + +---- + +ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen +suggested splitting up so we have this package ourselves, or maybe it can be +handled with replace/provide/conflict dependencies. + +ToDo: Figure out exactly what files/directory structure needs to be uploaded +(asked Peter, awaiting reply). -=-=(Knielsen - Wed, 12 May 2010, 07:06)=-=- Upgraded lucid VMs to the official release. Discussed with Arjen how to handle things. Did a lot of reading on how apt repositories work. Worked 4 hours and estimate 16 hours remain (original estimate unchanged). DESCRIPTION: Since the package building for MariaDB is now fully automated in Buildbot, it has been decided to use packages from Buildbot for the OurDelta apt and yum repositories. This worklog is about fixing/implementing anything that is missing to achieve this. - When doing a real release build, packages/repositories need to be signed, so that users will not get a warning about unauthenticated packages. This signing must only be done on official releases, not on daily builds (to avoid confusing one with the other). - Packages must be uploaded from the Buildbot host. The OurDelta infrastructure has a DropBox share that could be used for this, another option is to simply use rsync. - Ubuntu 10.04 "lucid" has been released, and we need to support that for packages, so the Buildbot VM for lucid must be upgraded to have the official release. - In Ubuntu 10.04, the official MySQL packages include a new package mysql-client-core, we currently have a conflict with this on install that we need to handle somehow. HIGH-LEVEL SPECIFICATION: The signing of packages can be done outside of Buildbot, as a separate process. There are some advantages to this: - Security: the private key can be kept less exposed when it is not included in the buildbot infrastructure. - It is good to have one step of human intervention before actually signing and releasing packages. - Generally reducing the complexity of the buildbot setup. ---- Here is how to sign the .rpms. Copy in the ourdelta/bakery signing stuff to ~/.gnupg and ~/.rpmmacros. Run rpm --addsign *.rpm That's all! This can be tested by creating a local yum repository: createrepo <dir> (where <dir> contains the signed .rpms). Then create the file /etc/yum.repos.d/localmaria.repo: [localmaria] name=Local MariaDB repo baseurl=file:///home/buildbot/rpms gpgcheck=1 enabled=1 gpgkey=http://master.ourdelta.org/deb/ourdelta.gpg Now this should work to install MariaDB: sudo yum install MariaDB-server ---- For .deb, it is not the individual .deb that is signed, it is the repository. Here is one way to generate a signed repository, using reprepro. The ourdelta/bakery signing stuff needs to be copied to ~/.gnupg mkdir repo # or whatever cd repo mkdir conf cat >conf/distributions <<END Origin: MariaDB Label: MariaDB Codename: hardy Architectures: amd64 Components: mariadb-ourdelta Description: MariaDB test Repository SignWith: autosign(a)ourdelta.org END for i in `find /home/buildbot/debs/ -name '*.deb'` ; do reprepro --basedir=. includedeb hardy $i ; done The corrosponding line for /etc/apt/sources.list: deb file:///home/buildbot/repo hardy mariadb-ourdelta This works for multiple distributions, by adding more sections to the conf/distributions file. ---- For the mysql-client-core-5.1 issue, the solution is to split the mariadb-client-5.1 (and 5.2) package similarly into mariadb-client-core-5.1. The mariadb-client-core-5.1 package then provides: mysql-client-core-5.1. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 17 May '10

17 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 30 ESTIMATE.......: 5 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress ------------------------------------------------------------ -=-=(View All Progress Notes, 30 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 17 May '10

17 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 30 ESTIMATE.......: 5 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress ------------------------------------------------------------ -=-=(View All Progress Notes, 30 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 17 May '10

17 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 30 ESTIMATE.......: 5 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress ------------------------------------------------------------ -=-=(View All Progress Notes, 30 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 17 May '10

17 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 30 ESTIMATE.......: 5 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Mon, 17 May 2010, 08:45)=-=- Merge with latest trunk and run Buildbot tests. Worked 1 hour and estimate 5 hours remain (original estimate unchanged). -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress ------------------------------------------------------------ -=-=(View All Progress Notes, 30 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Buildbot MariaDB->MariaDB upgrade testing (118)
by worklog-noreply＠askmonty.org 17 May '10

17 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Buildbot MariaDB->MariaDB upgrade testing CREATION DATE..: Wed, 12 May 2010, 13:48 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 118 (http://askmonty.org/worklog/?tid=118) VERSION........: STATUS.........: Complete PRIORITY.......: 60 WORKED HOURS...: 8 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: -=-=(Knielsen - Mon, 17 May 2010, 08:31)=-=- Fixed a problem with the upgrade test for 5.2. Using a 5.[0-9] wildcard doesn't work, as it tries to install both mariadb-server-5.1 and mariadb-server-5.2 at the same time! Running `apt-get upgrade` also does not work, as it will not test upgrade 5.1->5.2. Instead, grab the correct 5.x version to test install of from the bzr branch name. Found and fixed an upgrade dependency problem when upgrading to 5.2 (/usr/lib/mysql/plugins moved from libmysqlclient-dev in 5.1 to mariadb-server-5.2 in 5.2). Worked 3 hours and estimate 0 hours remain (original estimate increased by 3 hours). -=-=(Knielsen - Fri, 14 May 2010, 06:41)=-=- Version updated. --- /tmp/wklog.118.old.18369 2010-05-14 06:41:01.000000000 +0000 +++ /tmp/wklog.118.new.18369 2010-05-14 06:41:01.000000000 +0000 @@ -1 +1 @@ -Server-9.x + -=-=(Knielsen - Fri, 14 May 2010, 06:41)=-=- Status updated. --- /tmp/wklog.118.old.18369 2010-05-14 06:41:01.000000000 +0000 +++ /tmp/wklog.118.new.18369 2010-05-14 06:41:01.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Complete -=-=(Knielsen - Fri, 14 May 2010, 06:40)=-=- Installed the new virtual images, using 5.1.42 from OurDelta (for all except lucid, which is new and so has no 5.1.42 ourdelta package and also needed a fixed set of packages due to changes in the lucid MySQL packaging). Added a new upgrade2 step in the Buildbot configuration to test this. Updated Buildbot wiki documentation. Worked 5 hours and estimate 0 hours remain (original estimate decreased by 3 hours). DESCRIPTION: Create an additional test step for Buildbot to check that upgrading from one version of MariaDB to another works ok. We already have testing of upgrade from distro-official MySQL .debs to our MariaDB .debs. What we need is for each Debian/Ubuntu, a new KVM virtual image with MariaDB 5.1.42 (First GA release) pre-installed, just like the existing update test uses images with MySQL pre-installed. Then the Buildbot configuration must be updated to add another upgrade test step, just like the existing one but using the images with MariaDB pre-installed. Also, the setup of the new images must be added to the existing documentation: http://askmonty.org/wiki/BuildBot::package http://askmonty.org/wiki/BuildBot:vm-setup ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2857: fix includes in libevent to support vpath builds
by noreply＠launchpad.net 15 May '10

15 May '10

------------------------------------------------------------ revno: 2857 committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: maria-5.1 timestamp: Sat 2010-05-15 14:17:33 +0200 message: fix includes in libevent to support vpath builds modified: extra/libevent/Makefile.am -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

Re: [Maria-developers] [Merge] lp:~paul-mccullagh/maria/pbxt-1.0.11 into lp:maria
by Kristian Nielsen 14 May '10

14 May '10

Paul McCullagh <paul.mccullagh(a)primebase.org> writes: > All tests in the PBXT suite run through on Mac and Linux, except for one error under Linux, which is a bit weird (see below). > ------------------ > > pbxt.select_safe [ fail ] > Test ended at 2010-05-06 17:19:13 > > CURRENT_TEST: pbxt.select_safe > mysqltest: At line 19: query 'select 1 from t1,t1 as t2,t1 as t3' failed: 1104: The SELECT would examine more than MAX_JOIN_SIZE rows; check your WHERE and use SET SQL_BIG_SELECTS=1 or SET SQL_MAX_JOIN_SIZE=# if the SELECT is okay > > The result from queries just before the failure was: > drop table if exists t1; > SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=4, SQL_MAX_JOIN_SIZE=9; > create table t1 (a int auto_increment primary key, b char(20)); > insert into t1 values(1,"test"); > SELECT SQL_BUFFER_RESULT * from t1; > a b > 1 test > update t1 set b="a" whe Ok, I looked into this. So this test does a three-way cartesian self-join on a table with 2 rows (a total of 8 rows). And it sets SQL_MAX_JOIN_SIZE=9. And sometimes (apparently timing dependent), this test fails because the optimiser estimates more than 9 rows in the join. So the problem here is that the optimiser estimate is sometimes too big for SQL_MAX_JOIN_SIZE=9. What I found is that when the test case fails, the function ha_pbxt::info() returns records==3 for the table. When the test case succeeds, it returns records==2. So it's pretty clear that the server will throw an error when records==3 and SQL_MAX_JOIN_SIZE=9. What remains to consider is why the returned records value differs between test runs. The testcase actually first inserts a row, then deletes it, then inserts two more rows. I'm speculating that PBXT has some background cleanup thread or similar that causes a race between freeing up space from the first row and allocating space for the two new rows? So Paul, do you think returning records = 2 or 3 at random from ha_pbxt::info() is expected or not? If expected, one way to fix the problem is to create a new table before running the test: create table t2 like t1; insert into t2 select * from t1; analyze table t2; # PBXT: required to get the correct COUNT(*) select 1 from t2 as t1,t2,t2 as t3; Then there is no possibility for the insert+delete to take up a slot and cause a failure (I was not able to repeat the failure with this change). Alternatively, we can just increate the SQL_MAX_JOIN_SIZE value. Or alternatively, you may decide that the test is meaningless for PBXT due to imprecise statistics, and just remove it from select_safe.test. Just let me know what you prefer, and I'll change it as needed. - Kristian.

2 3

[Maria-developers] Patch to fix compiler Warnings and build failure in pbxt
by Michael Widenius 14 May '10

14 May '10

Hi! I have now merged and pushed the xtstat patch into MariaDB 5.1-release. Here is a patch that fixes some compiler warnings and a build failure when building in another directory. Paul, hope you can fix these also on your side. (The patch is already in 5.1-release). Regards, Monty === modified file 'storage/pbxt/bin/Makefile.am' --- storage/pbxt/bin/Makefile.am 2010-05-11 13:45:45 +0000 +++ storage/pbxt/bin/Makefile.am 2010-05-14 11:53:01 +0000 @@ -5,10 +5,10 @@ INCLUDES = -I$(top_srcdir)/include -I$(t -I$(top_srcdir)/storage/innobase/include \ -I$(top_srcdir)/sql \ -I$(srcdir) \ - -I../src + -I$(srcdir)/../src bin_PROGRAMS = xtstat xtstat_SOURCES = xtstat_xt.cc ../src/strutil_xt.cc -xtstat_LDADD = $(top_srcdir)/libmysql/libmysqlclient.la +xtstat_LDADD = $(top_builddir)/libmysql/libmysqlclient.la === modified file 'storage/pbxt/bin/xtstat_xt.cc' --- storage/pbxt/bin/xtstat_xt.cc 2010-05-11 13:45:45 +0000 +++ storage/pbxt/bin/xtstat_xt.cc 2010-05-14 10:59:41 +0000 @@ -93,7 +93,7 @@ struct Options { "Connection protocol to use: default/tcp/socket/pipe/memory", "default", MYSQL_PROTOCOL_DEFAULT, false }, { OPT_DISPLAY, 0, "display", OPT_HAS_VALUE, "Columns to display: use short names separated by |, partial match allowed", "time-msec,commt,row-ins,rec,ind,ilog,xlog,data,to,dirty", 0, false }, - { OPT_NONE, 0, NULL, 0, NULL, 0, false } + { OPT_NONE, 0, NULL, 0, NULL, NULL, 0, false } }; #ifdef XT_WIN === modified file 'storage/pbxt/src/datalog_xt.cc' --- storage/pbxt/src/datalog_xt.cc 2010-05-05 10:59:57 +0000 +++ storage/pbxt/src/datalog_xt.cc 2010-05-14 10:55:32 +0000 @@ -1249,7 +1249,7 @@ xtBool XTDataLogBuffer::dlb_write_thru_l */ dlb_data_log->dlf_log_eof += size; #ifdef DEBUG - if (log_offset + size > dlb_max_write_offset) + if ((ulonglong) (log_offset + size) > (ulonglong) dlb_max_write_offset) dlb_max_write_offset = log_offset + size; #endif dlb_flush_required = TRUE; @@ -1291,7 +1291,7 @@ xtBool XTDataLogBuffer::dlb_append_log(x if (!xt_pwrite_file(dlb_data_log->dlf_log_file, log_offset, size, data, &thread->st_statistics.st_data, thread)) return FAILED; #ifdef DEBUG - if (log_offset + size > dlb_max_write_offset) + if ((ulonglong) (log_offset + size) > (ulonglong) dlb_max_write_offset) dlb_max_write_offset = log_offset + size; #endif dlb_flush_required = TRUE; @@ -1734,8 +1734,8 @@ static xtBool dl_collect_garbage(XTThrea xtLogOffset src_log_offset; xtLogID curr_log_id; xtLogOffset curr_log_offset; - xtLogID dest_log_id; - xtLogOffset dest_log_offset; + xtLogID dest_log_id= 0; + xtLogOffset dest_log_offset= 0; off_t garbage_count = 0; memset(&cs, 0, sizeof(XTCompactorStateRec)); === modified file 'storage/pbxt/src/ha_pbxt.cc' --- storage/pbxt/src/ha_pbxt.cc 2010-05-12 14:27:18 +0000 +++ storage/pbxt/src/ha_pbxt.cc 2010-05-14 10:54:13 +0000 @@ -1609,7 +1609,7 @@ static int pbxt_prepare(handlerton *hton return err; } -static XTThreadPtr ha_temp_open_global_database(handlerton *hton, THD **ret_thd, int *temp_thread, char *thread_name, int *err) +static XTThreadPtr ha_temp_open_global_database(handlerton *hton, THD **ret_thd, int *temp_thread, const char *thread_name, int *err) { THD *thd; XTThreadPtr self = NULL; === modified file 'storage/pbxt/src/table_xt.cc' --- storage/pbxt/src/table_xt.cc 2010-05-06 12:42:28 +0000 +++ storage/pbxt/src/table_xt.cc 2010-05-14 10:57:14 +0000 @@ -1793,10 +1793,12 @@ xtPublic void xt_check_table(XTThreadPtr XTTableHPtr tab = ot->ot_table; xtRecordID prec_id; XTTabRecExtDPtr rec_buf = (XTTabRecExtDPtr) ot->ot_row_rbuffer; +#ifdef CHECK_TABLE_READ_DATA_LOG XTactExtRecEntryDRec ext_rec; size_t log_size; xtLogID log_id; xtLogOffset log_offset; +#endif xtRecordID rec_id; xtRecordID prev_rec_id; xtXactID xn_id;

2 1

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2856: Add a -nobuild argument to the script. Useful for building the zip file release with Express Edit...
by noreply＠launchpad.net 14 May '10

14 May '10

------------------------------------------------------------ revno: 2856 committer: Bo Thorsen <bo(a)askmonty.org> branch nick: trunk-work timestamp: Fri 2010-05-14 14:12:23 +0200 message: Add a -nobuild argument to the script. Useful for building the zip file release with Express Edition which doesn't have the devenv command modified: win/make_mariadb_win_dist -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

Re: [Maria-developers] Windows zip file
by Bo Thorsen 14 May '10

14 May '10

Den 14-05-2010 10:01, Henrik Ingo skrev: > (Adding maria-developers) > > On Wednesday 12 May 2010 16:10:36 Bo Thorsen wrote: >> Den 12-05-2010 15:04, Henrik Ingo skrev: >>> On Wednesday 12 May 2010 13:44:18 Michael Widenius wrote: >>>> Hi! >>>> >>>>>>>>> "Bo" == Bo Thorsen<bo(a)askmonty.org> writes: >>>> >>>> Bo> Hi Daniel (and devs), >>>> Bo> I have uploaded a windows 32 bit build zip file to >>>> Bo> >>>> /srv/www/vhosts/main/download/mariadb-5.1.44b/windows/mariadb-noinstall- >>>> 5.1 .44b-win32.zip. >>>> >>>> Bo> How do I get this mentioned on the download page? And will the >>>> mirrors Bo> pick the file up automatically? >>>> >>>> Bo> I'm not sure if we should add a warning about this zip file. It's >>>> the Bo> first one we have done in a while, something could have gone >>>> wrong with it. >>>> >>>> It would be good if we could add a warning like: >>>> >>>> We are actively working on a better Windows installation. This is our >>>> first try in this direction; It *should* work, but it if doesn't >>>> please report a bug about it so that we can fix it! >>> >>> Actually, the windows builds should be called beta for some time yet. >>> Please also change the zip filename to indicate that (and make sure it >>> says so on the download page). >> >> That's a change in what we usually do, since beta is only for the >> software, not the packaging. It would be odd to have both a 5.1.44b and >> a 5.1.44b-beta live at the site. People could be confused about what it >> is. It's actually already live without the beta in the filename. > > It's not about the packaging but the platform. If you compile our sources for > OS X, the sources are stable, but we never gave a guarantee that it will work > at all for OS X. Windows is the same, we are only now paying attention to it. > It's unreasonable to expect it will magically work just because we already do > stable releases for Linux and Solaris. I agree there is a risk that some of the code we have added between 5.1.39 and now can be broken on Windows. >> And the scripts haven't changed since we last released this, so I don't >> think it's a high risk. > > Ok, this is an argument I can buy. We've had one beta out, if no problems were > reported, we could move to stable. > > Even so, my personal preference is to call the windows release stable only > once we have a automated (buildbot based) system that outputs binaries without > manual steps - I consider that more reliable as a process. > > To do the manually built zip file fills a need our users have, but isn't yet > the end goal. The steps I do manually are exactly what the buildbot system will do. I just uncomment the lines that do visual studio build and run the scripts. At one of the first runs I did, I modified the script to do this: if [ "x_$1" != "x_-nobuild" ]; then win/configure-mariadb.sh cmake -G "Visual Studio 9 2008" devenv.com MySQL.sln /build RelWithDebInfo devenv.com MySQL.sln /build Debug fi The only modification here from the checked in script, is the if statement. When I run "sh win/make_mariadb_win_dist -nobuild", I just have to run the build lines manually before the script. Perhaps I should check this if into the lp:maria? Bo.

2 1

[Maria-developers] Updated (by Knielsen): Windows installer for MariaDB (55)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Windows installer for MariaDB CREATION DATE..: Wed, 14 Oct 2009, 00:07 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 55 (http://askmonty.org/worklog/?tid=55) VERSION........: Server-5.1 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Version updated. --- /tmp/wklog.55.old.18466 2010-05-14 06:45:38.000000000 +0000 +++ /tmp/wklog.55.new.18466 2010-05-14 06:45:38.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.1 -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Category updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Status updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Psergey - Sat, 17 Oct 2009, 00:03)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.19999 2009-10-17 00:03:11.000000000 +0300 +++ /tmp/wklog.55.new.19999 2009-10-17 00:03:11.000000000 +0300 @@ -79,5 +79,9 @@ 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: -http://askmonty.org/wiki/index.php/MariaDB_Logo + http://askmonty.org/wiki/index.php/MariaDB_Logo +* We should make both 32-bit installer and 64-bit installer (the + latter will be possible when we have 64-bit windows binaries) +* At this point we don't see a need to force a reboot after the installation. +* The installer should be Vista and Windows7-proof. -=-=(Psergey - Thu, 15 Oct 2009, 23:46)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.23061 2009-10-15 23:46:18.000000000 +0300 +++ /tmp/wklog.55.new.23061 2009-10-15 23:46:18.000000000 +0300 @@ -7,28 +7,25 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service -- Prompts the user for "essential" configuration options. Preliminary list - of "essential" options: +- Presents the user with GPL licence +- Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. - + * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are - busy. If they are, it asks to remove the previous installation first - and aborts. (that is: upgrades are not supported in step#1) + busy. If they are, offers to either change these parameters or abort the + installation (that is: no support for any kind of upgrades at this point) +- Copies installation files to appropriate destination +- Registers mysqld a service - Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) + (note: starting server manually requires write access to datadir, which + not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. @@ -53,7 +50,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. - TODO come up with options + TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22795 2009-10-15 23:40:36.000000000 +0300 +++ /tmp/wklog.55.new.22795 2009-10-15 23:40:36.000000000 +0300 @@ -7,7 +7,8 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Shows GPL Licence +- Copies files on installation +- Registers mysqld a service - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -15,27 +16,28 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * [perhaps] sql_mode setting. + * TODO come up with the final list. The criteria for inclusion are: + 1. ask for things that are essential to have a working setup as soon as + the installation is complete + 2. ask for things without answers for which the newbies can get into + trouble. -- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file (from a template) -- Sets up SQL root user with specified password +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - (note: will this work from any user, and on any OS? if not, this might - be omitted) - to start mysql client - - to edit the my.cnf file -- Registers MariaDB to start as a service with the specified parameters -- Registers MariaDB as installed software, sets up uninstaller + - to edit the my.cnf file. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the installation procedure will end up being). + on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22774 2009-10-15 23:40:16.000000000 +0300 +++ /tmp/wklog.55.new.22774 2009-10-15 23:40:16.000000000 +0300 @@ -15,13 +15,9 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. + * [perhaps] sql_mode setting. -- Copies files to destination directory +- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) @@ -29,6 +25,8 @@ - Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) + (note: will this work from any user, and on any OS? if not, this might + be omitted) - to start mysql client - to edit the my.cnf file - Registers MariaDB to start as a service with the specified parameters -=-=(Psergey - Thu, 15 Oct 2009, 23:38)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22618 2009-10-15 23:38:06.000000000 +0300 +++ /tmp/wklog.55.new.22618 2009-10-15 23:38:06.000000000 +0300 @@ -7,8 +7,7 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service +- Shows GPL Licence - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -22,22 +21,23 @@ 2. ask for things without answers for which the newbies can get into trouble. +- Copies files to destination directory - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Generates appropriate my.cnf file (from a template) +- Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - - to edit the my.cnf file. -- Registers MariaDB to start as a service with the specified parameters. -- Registers MariaDB as installed software, sets up uninstaller. + - to edit the my.cnf file +- Registers MariaDB to start as a service with the specified parameters +- Registers MariaDB as installed software, sets up uninstaller (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the final installation procedure will be). + on how complex and error-prone the installation procedure will end up being). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 16:34)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.3919 2009-10-15 16:34:22.000000000 +0300 +++ /tmp/wklog.55.new.3919 2009-10-15 16:34:22.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal. It can be found here: +* MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Bothorsen - Thu, 15 Oct 2009, 15:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.1396 2009-10-15 15:40:03.000000000 +0300 +++ /tmp/wklog.55.new.1396 2009-10-15 15:40:03.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal (see e.g. here: https://code.launchpad.net/maria) - Bo Thorsen has the latest revision of the picture in various formats. +* MySQL's logo is the seal. It can be found here: +http://askmonty.org/wiki/index.php/MariaDB_Logo ------------------------------------------------------------ -=-=(View All Progress Notes, 18 total)=-=- http://askmonty.org/worklog/index.pl?tid=55&nolimit=1 DESCRIPTION: We need Windows Installer package for MariaDB. HIGH-LEVEL SPECIFICATION: Not a spec so far but a list of points to consider: 1. Installer wishlist (user POV) -------------------------------- >From the user point of view: 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that - Presents the user with GPL licence - Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, offers to either change these parameters or abort the installation (that is: no support for any kind of upgrades at this point) - Copies installation files to appropriate destination - Registers mysqld a service - Generates appropriate my.cnf file - Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) (note: starting server manually requires write access to datadir, which not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. - Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Same as above but with handling of the case where MySQL has been already installed: - offer to replace MySQL. - upgrade the data directory (todo we should sort out if anything/what is needed for this). - Uninstall MySQL - Install MariaDB. 1.3 Step 3: Configuration wizard ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a list of things that one might want an installer to do but that are out of scope of this WL entry: * Calibration of my.cnf parameters based on available memory, number of CPUs, etc. 2. Installer wishlist (developer POV) ------------------------------------- * Some "installshield-like" tool that's easy to use (suggestion by Webyog: NSIS) * Installation procedure source should reside in MariaDB source repository * Installation procedure source file is better to be in human-readable text format. * It should be possible to automate creation of the installer package, in a way that can be run from buildbot (e.g. the installer package build process should print messages to its stdout) * Any suggestions on how can one automatically test the installation package? (for example, we'll want to start the installer, install, check that installation succeeded, then start the server, run some commands, then uninstall. Any ways to achieve that?) 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo * We should make both 32-bit installer and 64-bit installer (the latter will be possible when we have 64-bit windows binaries) * At this point we don't see a need to force a reboot after the installation. * The installer should be Vista and Windows7-proof. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Windows installer for MariaDB (55)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Windows installer for MariaDB CREATION DATE..: Wed, 14 Oct 2009, 00:07 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 55 (http://askmonty.org/worklog/?tid=55) VERSION........: Server-5.1 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Version updated. --- /tmp/wklog.55.old.18466 2010-05-14 06:45:38.000000000 +0000 +++ /tmp/wklog.55.new.18466 2010-05-14 06:45:38.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.1 -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Category updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Status updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Psergey - Sat, 17 Oct 2009, 00:03)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.19999 2009-10-17 00:03:11.000000000 +0300 +++ /tmp/wklog.55.new.19999 2009-10-17 00:03:11.000000000 +0300 @@ -79,5 +79,9 @@ 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: -http://askmonty.org/wiki/index.php/MariaDB_Logo + http://askmonty.org/wiki/index.php/MariaDB_Logo +* We should make both 32-bit installer and 64-bit installer (the + latter will be possible when we have 64-bit windows binaries) +* At this point we don't see a need to force a reboot after the installation. +* The installer should be Vista and Windows7-proof. -=-=(Psergey - Thu, 15 Oct 2009, 23:46)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.23061 2009-10-15 23:46:18.000000000 +0300 +++ /tmp/wklog.55.new.23061 2009-10-15 23:46:18.000000000 +0300 @@ -7,28 +7,25 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service -- Prompts the user for "essential" configuration options. Preliminary list - of "essential" options: +- Presents the user with GPL licence +- Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. - + * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are - busy. If they are, it asks to remove the previous installation first - and aborts. (that is: upgrades are not supported in step#1) + busy. If they are, offers to either change these parameters or abort the + installation (that is: no support for any kind of upgrades at this point) +- Copies installation files to appropriate destination +- Registers mysqld a service - Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) + (note: starting server manually requires write access to datadir, which + not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. @@ -53,7 +50,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. - TODO come up with options + TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22795 2009-10-15 23:40:36.000000000 +0300 +++ /tmp/wklog.55.new.22795 2009-10-15 23:40:36.000000000 +0300 @@ -7,7 +7,8 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Shows GPL Licence +- Copies files on installation +- Registers mysqld a service - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -15,27 +16,28 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * [perhaps] sql_mode setting. + * TODO come up with the final list. The criteria for inclusion are: + 1. ask for things that are essential to have a working setup as soon as + the installation is complete + 2. ask for things without answers for which the newbies can get into + trouble. -- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file (from a template) -- Sets up SQL root user with specified password +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - (note: will this work from any user, and on any OS? if not, this might - be omitted) - to start mysql client - - to edit the my.cnf file -- Registers MariaDB to start as a service with the specified parameters -- Registers MariaDB as installed software, sets up uninstaller + - to edit the my.cnf file. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the installation procedure will end up being). + on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22774 2009-10-15 23:40:16.000000000 +0300 +++ /tmp/wklog.55.new.22774 2009-10-15 23:40:16.000000000 +0300 @@ -15,13 +15,9 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. + * [perhaps] sql_mode setting. -- Copies files to destination directory +- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) @@ -29,6 +25,8 @@ - Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) + (note: will this work from any user, and on any OS? if not, this might + be omitted) - to start mysql client - to edit the my.cnf file - Registers MariaDB to start as a service with the specified parameters -=-=(Psergey - Thu, 15 Oct 2009, 23:38)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22618 2009-10-15 23:38:06.000000000 +0300 +++ /tmp/wklog.55.new.22618 2009-10-15 23:38:06.000000000 +0300 @@ -7,8 +7,7 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service +- Shows GPL Licence - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -22,22 +21,23 @@ 2. ask for things without answers for which the newbies can get into trouble. +- Copies files to destination directory - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Generates appropriate my.cnf file (from a template) +- Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - - to edit the my.cnf file. -- Registers MariaDB to start as a service with the specified parameters. -- Registers MariaDB as installed software, sets up uninstaller. + - to edit the my.cnf file +- Registers MariaDB to start as a service with the specified parameters +- Registers MariaDB as installed software, sets up uninstaller (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the final installation procedure will be). + on how complex and error-prone the installation procedure will end up being). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 16:34)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.3919 2009-10-15 16:34:22.000000000 +0300 +++ /tmp/wklog.55.new.3919 2009-10-15 16:34:22.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal. It can be found here: +* MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Bothorsen - Thu, 15 Oct 2009, 15:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.1396 2009-10-15 15:40:03.000000000 +0300 +++ /tmp/wklog.55.new.1396 2009-10-15 15:40:03.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal (see e.g. here: https://code.launchpad.net/maria) - Bo Thorsen has the latest revision of the picture in various formats. +* MySQL's logo is the seal. It can be found here: +http://askmonty.org/wiki/index.php/MariaDB_Logo ------------------------------------------------------------ -=-=(View All Progress Notes, 18 total)=-=- http://askmonty.org/worklog/index.pl?tid=55&nolimit=1 DESCRIPTION: We need Windows Installer package for MariaDB. HIGH-LEVEL SPECIFICATION: Not a spec so far but a list of points to consider: 1. Installer wishlist (user POV) -------------------------------- >From the user point of view: 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that - Presents the user with GPL licence - Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, offers to either change these parameters or abort the installation (that is: no support for any kind of upgrades at this point) - Copies installation files to appropriate destination - Registers mysqld a service - Generates appropriate my.cnf file - Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) (note: starting server manually requires write access to datadir, which not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. - Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Same as above but with handling of the case where MySQL has been already installed: - offer to replace MySQL. - upgrade the data directory (todo we should sort out if anything/what is needed for this). - Uninstall MySQL - Install MariaDB. 1.3 Step 3: Configuration wizard ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a list of things that one might want an installer to do but that are out of scope of this WL entry: * Calibration of my.cnf parameters based on available memory, number of CPUs, etc. 2. Installer wishlist (developer POV) ------------------------------------- * Some "installshield-like" tool that's easy to use (suggestion by Webyog: NSIS) * Installation procedure source should reside in MariaDB source repository * Installation procedure source file is better to be in human-readable text format. * It should be possible to automate creation of the installer package, in a way that can be run from buildbot (e.g. the installer package build process should print messages to its stdout) * Any suggestions on how can one automatically test the installation package? (for example, we'll want to start the installer, install, check that installation succeeded, then start the server, run some commands, then uninstall. Any ways to achieve that?) 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo * We should make both 32-bit installer and 64-bit installer (the latter will be possible when we have 64-bit windows binaries) * At this point we don't see a need to force a reboot after the installation. * The installer should be Vista and Windows7-proof. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Windows installer for MariaDB (55)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Windows installer for MariaDB CREATION DATE..: Wed, 14 Oct 2009, 00:07 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 55 (http://askmonty.org/worklog/?tid=55) VERSION........: Server-5.1 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Version updated. --- /tmp/wklog.55.old.18466 2010-05-14 06:45:38.000000000 +0000 +++ /tmp/wklog.55.new.18466 2010-05-14 06:45:38.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.1 -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Category updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Status updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Psergey - Sat, 17 Oct 2009, 00:03)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.19999 2009-10-17 00:03:11.000000000 +0300 +++ /tmp/wklog.55.new.19999 2009-10-17 00:03:11.000000000 +0300 @@ -79,5 +79,9 @@ 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: -http://askmonty.org/wiki/index.php/MariaDB_Logo + http://askmonty.org/wiki/index.php/MariaDB_Logo +* We should make both 32-bit installer and 64-bit installer (the + latter will be possible when we have 64-bit windows binaries) +* At this point we don't see a need to force a reboot after the installation. +* The installer should be Vista and Windows7-proof. -=-=(Psergey - Thu, 15 Oct 2009, 23:46)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.23061 2009-10-15 23:46:18.000000000 +0300 +++ /tmp/wklog.55.new.23061 2009-10-15 23:46:18.000000000 +0300 @@ -7,28 +7,25 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service -- Prompts the user for "essential" configuration options. Preliminary list - of "essential" options: +- Presents the user with GPL licence +- Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. - + * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are - busy. If they are, it asks to remove the previous installation first - and aborts. (that is: upgrades are not supported in step#1) + busy. If they are, offers to either change these parameters or abort the + installation (that is: no support for any kind of upgrades at this point) +- Copies installation files to appropriate destination +- Registers mysqld a service - Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) + (note: starting server manually requires write access to datadir, which + not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. @@ -53,7 +50,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. - TODO come up with options + TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22795 2009-10-15 23:40:36.000000000 +0300 +++ /tmp/wklog.55.new.22795 2009-10-15 23:40:36.000000000 +0300 @@ -7,7 +7,8 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Shows GPL Licence +- Copies files on installation +- Registers mysqld a service - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -15,27 +16,28 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * [perhaps] sql_mode setting. + * TODO come up with the final list. The criteria for inclusion are: + 1. ask for things that are essential to have a working setup as soon as + the installation is complete + 2. ask for things without answers for which the newbies can get into + trouble. -- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file (from a template) -- Sets up SQL root user with specified password +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - (note: will this work from any user, and on any OS? if not, this might - be omitted) - to start mysql client - - to edit the my.cnf file -- Registers MariaDB to start as a service with the specified parameters -- Registers MariaDB as installed software, sets up uninstaller + - to edit the my.cnf file. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the installation procedure will end up being). + on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22774 2009-10-15 23:40:16.000000000 +0300 +++ /tmp/wklog.55.new.22774 2009-10-15 23:40:16.000000000 +0300 @@ -15,13 +15,9 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. + * [perhaps] sql_mode setting. -- Copies files to destination directory +- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) @@ -29,6 +25,8 @@ - Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) + (note: will this work from any user, and on any OS? if not, this might + be omitted) - to start mysql client - to edit the my.cnf file - Registers MariaDB to start as a service with the specified parameters -=-=(Psergey - Thu, 15 Oct 2009, 23:38)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22618 2009-10-15 23:38:06.000000000 +0300 +++ /tmp/wklog.55.new.22618 2009-10-15 23:38:06.000000000 +0300 @@ -7,8 +7,7 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service +- Shows GPL Licence - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -22,22 +21,23 @@ 2. ask for things without answers for which the newbies can get into trouble. +- Copies files to destination directory - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Generates appropriate my.cnf file (from a template) +- Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - - to edit the my.cnf file. -- Registers MariaDB to start as a service with the specified parameters. -- Registers MariaDB as installed software, sets up uninstaller. + - to edit the my.cnf file +- Registers MariaDB to start as a service with the specified parameters +- Registers MariaDB as installed software, sets up uninstaller (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the final installation procedure will be). + on how complex and error-prone the installation procedure will end up being). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 16:34)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.3919 2009-10-15 16:34:22.000000000 +0300 +++ /tmp/wklog.55.new.3919 2009-10-15 16:34:22.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal. It can be found here: +* MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Bothorsen - Thu, 15 Oct 2009, 15:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.1396 2009-10-15 15:40:03.000000000 +0300 +++ /tmp/wklog.55.new.1396 2009-10-15 15:40:03.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal (see e.g. here: https://code.launchpad.net/maria) - Bo Thorsen has the latest revision of the picture in various formats. +* MySQL's logo is the seal. It can be found here: +http://askmonty.org/wiki/index.php/MariaDB_Logo ------------------------------------------------------------ -=-=(View All Progress Notes, 18 total)=-=- http://askmonty.org/worklog/index.pl?tid=55&nolimit=1 DESCRIPTION: We need Windows Installer package for MariaDB. HIGH-LEVEL SPECIFICATION: Not a spec so far but a list of points to consider: 1. Installer wishlist (user POV) -------------------------------- >From the user point of view: 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that - Presents the user with GPL licence - Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, offers to either change these parameters or abort the installation (that is: no support for any kind of upgrades at this point) - Copies installation files to appropriate destination - Registers mysqld a service - Generates appropriate my.cnf file - Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) (note: starting server manually requires write access to datadir, which not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. - Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Same as above but with handling of the case where MySQL has been already installed: - offer to replace MySQL. - upgrade the data directory (todo we should sort out if anything/what is needed for this). - Uninstall MySQL - Install MariaDB. 1.3 Step 3: Configuration wizard ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a list of things that one might want an installer to do but that are out of scope of this WL entry: * Calibration of my.cnf parameters based on available memory, number of CPUs, etc. 2. Installer wishlist (developer POV) ------------------------------------- * Some "installshield-like" tool that's easy to use (suggestion by Webyog: NSIS) * Installation procedure source should reside in MariaDB source repository * Installation procedure source file is better to be in human-readable text format. * It should be possible to automate creation of the installer package, in a way that can be run from buildbot (e.g. the installer package build process should print messages to its stdout) * Any suggestions on how can one automatically test the installation package? (for example, we'll want to start the installer, install, check that installation succeeded, then start the server, run some commands, then uninstall. Any ways to achieve that?) 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo * We should make both 32-bit installer and 64-bit installer (the latter will be possible when we have 64-bit windows binaries) * At this point we don't see a need to force a reboot after the installation. * The installer should be Vista and Windows7-proof. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Windows installer for MariaDB (55)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Windows installer for MariaDB CREATION DATE..: Wed, 14 Oct 2009, 00:07 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 55 (http://askmonty.org/worklog/?tid=55) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Category updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Status updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Psergey - Sat, 17 Oct 2009, 00:03)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.19999 2009-10-17 00:03:11.000000000 +0300 +++ /tmp/wklog.55.new.19999 2009-10-17 00:03:11.000000000 +0300 @@ -79,5 +79,9 @@ 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: -http://askmonty.org/wiki/index.php/MariaDB_Logo + http://askmonty.org/wiki/index.php/MariaDB_Logo +* We should make both 32-bit installer and 64-bit installer (the + latter will be possible when we have 64-bit windows binaries) +* At this point we don't see a need to force a reboot after the installation. +* The installer should be Vista and Windows7-proof. -=-=(Psergey - Thu, 15 Oct 2009, 23:46)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.23061 2009-10-15 23:46:18.000000000 +0300 +++ /tmp/wklog.55.new.23061 2009-10-15 23:46:18.000000000 +0300 @@ -7,28 +7,25 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service -- Prompts the user for "essential" configuration options. Preliminary list - of "essential" options: +- Presents the user with GPL licence +- Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. - + * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are - busy. If they are, it asks to remove the previous installation first - and aborts. (that is: upgrades are not supported in step#1) + busy. If they are, offers to either change these parameters or abort the + installation (that is: no support for any kind of upgrades at this point) +- Copies installation files to appropriate destination +- Registers mysqld a service - Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) + (note: starting server manually requires write access to datadir, which + not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. @@ -53,7 +50,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. - TODO come up with options + TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22795 2009-10-15 23:40:36.000000000 +0300 +++ /tmp/wklog.55.new.22795 2009-10-15 23:40:36.000000000 +0300 @@ -7,7 +7,8 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Shows GPL Licence +- Copies files on installation +- Registers mysqld a service - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -15,27 +16,28 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * [perhaps] sql_mode setting. + * TODO come up with the final list. The criteria for inclusion are: + 1. ask for things that are essential to have a working setup as soon as + the installation is complete + 2. ask for things without answers for which the newbies can get into + trouble. -- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file (from a template) -- Sets up SQL root user with specified password +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - (note: will this work from any user, and on any OS? if not, this might - be omitted) - to start mysql client - - to edit the my.cnf file -- Registers MariaDB to start as a service with the specified parameters -- Registers MariaDB as installed software, sets up uninstaller + - to edit the my.cnf file. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the installation procedure will end up being). + on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22774 2009-10-15 23:40:16.000000000 +0300 +++ /tmp/wklog.55.new.22774 2009-10-15 23:40:16.000000000 +0300 @@ -15,13 +15,9 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. + * [perhaps] sql_mode setting. -- Copies files to destination directory +- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) @@ -29,6 +25,8 @@ - Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) + (note: will this work from any user, and on any OS? if not, this might + be omitted) - to start mysql client - to edit the my.cnf file - Registers MariaDB to start as a service with the specified parameters -=-=(Psergey - Thu, 15 Oct 2009, 23:38)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22618 2009-10-15 23:38:06.000000000 +0300 +++ /tmp/wklog.55.new.22618 2009-10-15 23:38:06.000000000 +0300 @@ -7,8 +7,7 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service +- Shows GPL Licence - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -22,22 +21,23 @@ 2. ask for things without answers for which the newbies can get into trouble. +- Copies files to destination directory - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Generates appropriate my.cnf file (from a template) +- Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - - to edit the my.cnf file. -- Registers MariaDB to start as a service with the specified parameters. -- Registers MariaDB as installed software, sets up uninstaller. + - to edit the my.cnf file +- Registers MariaDB to start as a service with the specified parameters +- Registers MariaDB as installed software, sets up uninstaller (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the final installation procedure will be). + on how complex and error-prone the installation procedure will end up being). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 16:34)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.3919 2009-10-15 16:34:22.000000000 +0300 +++ /tmp/wklog.55.new.3919 2009-10-15 16:34:22.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal. It can be found here: +* MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Bothorsen - Thu, 15 Oct 2009, 15:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.1396 2009-10-15 15:40:03.000000000 +0300 +++ /tmp/wklog.55.new.1396 2009-10-15 15:40:03.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal (see e.g. here: https://code.launchpad.net/maria) - Bo Thorsen has the latest revision of the picture in various formats. +* MySQL's logo is the seal. It can be found here: +http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Psergey - Thu, 15 Oct 2009, 15:22)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.466 2009-10-15 15:22:05.000000000 +0300 +++ /tmp/wklog.55.new.466 2009-10-15 15:22:05.000000000 +0300 @@ -16,7 +16,7 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * <TODO come up with the final list. The criteria for inclusion are: + * TODO come up with the final list. The criteria for inclusion are: 1. ask for things that are essential to have a working setup as soon as the installation is complete 2. ask for things without answers for which the newbies can get into @@ -25,11 +25,14 @@ - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - to edit the my.cnf file. -- Registers MariaDB as installed, sets up uninstaller. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to @@ -54,9 +57,10 @@ 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This is a list of things that one might want an installer to do but that are out -of scope of this WL entry. -* Calibration of my.cnf parameters based on available memory, number of CPUs, etc. +This is a list of things that one might want an installer to do but that are +out of scope of this WL entry: +* Calibration of my.cnf parameters based on available memory, number of CPUs, + etc. 2. Installer wishlist (developer POV) ------------------------------------------------------------ -=-=(View All Progress Notes, 17 total)=-=- http://askmonty.org/worklog/index.pl?tid=55&nolimit=1 DESCRIPTION: We need Windows Installer package for MariaDB. HIGH-LEVEL SPECIFICATION: Not a spec so far but a list of points to consider: 1. Installer wishlist (user POV) -------------------------------- >From the user point of view: 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that - Presents the user with GPL licence - Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, offers to either change these parameters or abort the installation (that is: no support for any kind of upgrades at this point) - Copies installation files to appropriate destination - Registers mysqld a service - Generates appropriate my.cnf file - Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) (note: starting server manually requires write access to datadir, which not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. - Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Same as above but with handling of the case where MySQL has been already installed: - offer to replace MySQL. - upgrade the data directory (todo we should sort out if anything/what is needed for this). - Uninstall MySQL - Install MariaDB. 1.3 Step 3: Configuration wizard ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a list of things that one might want an installer to do but that are out of scope of this WL entry: * Calibration of my.cnf parameters based on available memory, number of CPUs, etc. 2. Installer wishlist (developer POV) ------------------------------------- * Some "installshield-like" tool that's easy to use (suggestion by Webyog: NSIS) * Installation procedure source should reside in MariaDB source repository * Installation procedure source file is better to be in human-readable text format. * It should be possible to automate creation of the installer package, in a way that can be run from buildbot (e.g. the installer package build process should print messages to its stdout) * Any suggestions on how can one automatically test the installation package? (for example, we'll want to start the installer, install, check that installation succeeded, then start the server, run some commands, then uninstall. Any ways to achieve that?) 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo * We should make both 32-bit installer and 64-bit installer (the latter will be possible when we have 64-bit windows binaries) * At this point we don't see a need to force a reboot after the installation. * The installer should be Vista and Windows7-proof. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Windows installer for MariaDB (55)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Windows installer for MariaDB CREATION DATE..: Wed, 14 Oct 2009, 00:07 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 55 (http://askmonty.org/worklog/?tid=55) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Category updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Status updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Psergey - Sat, 17 Oct 2009, 00:03)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.19999 2009-10-17 00:03:11.000000000 +0300 +++ /tmp/wklog.55.new.19999 2009-10-17 00:03:11.000000000 +0300 @@ -79,5 +79,9 @@ 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: -http://askmonty.org/wiki/index.php/MariaDB_Logo + http://askmonty.org/wiki/index.php/MariaDB_Logo +* We should make both 32-bit installer and 64-bit installer (the + latter will be possible when we have 64-bit windows binaries) +* At this point we don't see a need to force a reboot after the installation. +* The installer should be Vista and Windows7-proof. -=-=(Psergey - Thu, 15 Oct 2009, 23:46)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.23061 2009-10-15 23:46:18.000000000 +0300 +++ /tmp/wklog.55.new.23061 2009-10-15 23:46:18.000000000 +0300 @@ -7,28 +7,25 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service -- Prompts the user for "essential" configuration options. Preliminary list - of "essential" options: +- Presents the user with GPL licence +- Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. - + * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are - busy. If they are, it asks to remove the previous installation first - and aborts. (that is: upgrades are not supported in step#1) + busy. If they are, offers to either change these parameters or abort the + installation (that is: no support for any kind of upgrades at this point) +- Copies installation files to appropriate destination +- Registers mysqld a service - Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) + (note: starting server manually requires write access to datadir, which + not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. @@ -53,7 +50,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. - TODO come up with options + TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22795 2009-10-15 23:40:36.000000000 +0300 +++ /tmp/wklog.55.new.22795 2009-10-15 23:40:36.000000000 +0300 @@ -7,7 +7,8 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Shows GPL Licence +- Copies files on installation +- Registers mysqld a service - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -15,27 +16,28 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * [perhaps] sql_mode setting. + * TODO come up with the final list. The criteria for inclusion are: + 1. ask for things that are essential to have a working setup as soon as + the installation is complete + 2. ask for things without answers for which the newbies can get into + trouble. -- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file (from a template) -- Sets up SQL root user with specified password +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - (note: will this work from any user, and on any OS? if not, this might - be omitted) - to start mysql client - - to edit the my.cnf file -- Registers MariaDB to start as a service with the specified parameters -- Registers MariaDB as installed software, sets up uninstaller + - to edit the my.cnf file. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the installation procedure will end up being). + on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22774 2009-10-15 23:40:16.000000000 +0300 +++ /tmp/wklog.55.new.22774 2009-10-15 23:40:16.000000000 +0300 @@ -15,13 +15,9 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. + * [perhaps] sql_mode setting. -- Copies files to destination directory +- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) @@ -29,6 +25,8 @@ - Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) + (note: will this work from any user, and on any OS? if not, this might + be omitted) - to start mysql client - to edit the my.cnf file - Registers MariaDB to start as a service with the specified parameters -=-=(Psergey - Thu, 15 Oct 2009, 23:38)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22618 2009-10-15 23:38:06.000000000 +0300 +++ /tmp/wklog.55.new.22618 2009-10-15 23:38:06.000000000 +0300 @@ -7,8 +7,7 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service +- Shows GPL Licence - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -22,22 +21,23 @@ 2. ask for things without answers for which the newbies can get into trouble. +- Copies files to destination directory - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Generates appropriate my.cnf file (from a template) +- Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - - to edit the my.cnf file. -- Registers MariaDB to start as a service with the specified parameters. -- Registers MariaDB as installed software, sets up uninstaller. + - to edit the my.cnf file +- Registers MariaDB to start as a service with the specified parameters +- Registers MariaDB as installed software, sets up uninstaller (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the final installation procedure will be). + on how complex and error-prone the installation procedure will end up being). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 16:34)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.3919 2009-10-15 16:34:22.000000000 +0300 +++ /tmp/wklog.55.new.3919 2009-10-15 16:34:22.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal. It can be found here: +* MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Bothorsen - Thu, 15 Oct 2009, 15:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.1396 2009-10-15 15:40:03.000000000 +0300 +++ /tmp/wklog.55.new.1396 2009-10-15 15:40:03.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal (see e.g. here: https://code.launchpad.net/maria) - Bo Thorsen has the latest revision of the picture in various formats. +* MySQL's logo is the seal. It can be found here: +http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Psergey - Thu, 15 Oct 2009, 15:22)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.466 2009-10-15 15:22:05.000000000 +0300 +++ /tmp/wklog.55.new.466 2009-10-15 15:22:05.000000000 +0300 @@ -16,7 +16,7 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * <TODO come up with the final list. The criteria for inclusion are: + * TODO come up with the final list. The criteria for inclusion are: 1. ask for things that are essential to have a working setup as soon as the installation is complete 2. ask for things without answers for which the newbies can get into @@ -25,11 +25,14 @@ - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - to edit the my.cnf file. -- Registers MariaDB as installed, sets up uninstaller. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to @@ -54,9 +57,10 @@ 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This is a list of things that one might want an installer to do but that are out -of scope of this WL entry. -* Calibration of my.cnf parameters based on available memory, number of CPUs, etc. +This is a list of things that one might want an installer to do but that are +out of scope of this WL entry: +* Calibration of my.cnf parameters based on available memory, number of CPUs, + etc. 2. Installer wishlist (developer POV) ------------------------------------------------------------ -=-=(View All Progress Notes, 17 total)=-=- http://askmonty.org/worklog/index.pl?tid=55&nolimit=1 DESCRIPTION: We need Windows Installer package for MariaDB. HIGH-LEVEL SPECIFICATION: Not a spec so far but a list of points to consider: 1. Installer wishlist (user POV) -------------------------------- >From the user point of view: 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that - Presents the user with GPL licence - Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, offers to either change these parameters or abort the installation (that is: no support for any kind of upgrades at this point) - Copies installation files to appropriate destination - Registers mysqld a service - Generates appropriate my.cnf file - Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) (note: starting server manually requires write access to datadir, which not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. - Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Same as above but with handling of the case where MySQL has been already installed: - offer to replace MySQL. - upgrade the data directory (todo we should sort out if anything/what is needed for this). - Uninstall MySQL - Install MariaDB. 1.3 Step 3: Configuration wizard ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a list of things that one might want an installer to do but that are out of scope of this WL entry: * Calibration of my.cnf parameters based on available memory, number of CPUs, etc. 2. Installer wishlist (developer POV) ------------------------------------- * Some "installshield-like" tool that's easy to use (suggestion by Webyog: NSIS) * Installation procedure source should reside in MariaDB source repository * Installation procedure source file is better to be in human-readable text format. * It should be possible to automate creation of the installer package, in a way that can be run from buildbot (e.g. the installer package build process should print messages to its stdout) * Any suggestions on how can one automatically test the installation package? (for example, we'll want to start the installer, install, check that installation succeeded, then start the server, run some commands, then uninstall. Any ways to achieve that?) 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo * We should make both 32-bit installer and 64-bit installer (the latter will be possible when we have 64-bit windows binaries) * At this point we don't see a need to force a reboot after the installation. * The installer should be Vista and Windows7-proof. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Windows installer for MariaDB (55)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Windows installer for MariaDB CREATION DATE..: Wed, 14 Oct 2009, 00:07 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 55 (http://askmonty.org/worklog/?tid=55) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Category updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Knielsen - Fri, 14 May 2010, 06:45)=-=- Status updated. --- /tmp/wklog.55.old.18457 2010-05-14 06:45:28.000000000 +0000 +++ /tmp/wklog.55.new.18457 2010-05-14 06:45:28.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Psergey - Sat, 17 Oct 2009, 00:03)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.19999 2009-10-17 00:03:11.000000000 +0300 +++ /tmp/wklog.55.new.19999 2009-10-17 00:03:11.000000000 +0300 @@ -79,5 +79,9 @@ 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: -http://askmonty.org/wiki/index.php/MariaDB_Logo + http://askmonty.org/wiki/index.php/MariaDB_Logo +* We should make both 32-bit installer and 64-bit installer (the + latter will be possible when we have 64-bit windows binaries) +* At this point we don't see a need to force a reboot after the installation. +* The installer should be Vista and Windows7-proof. -=-=(Psergey - Thu, 15 Oct 2009, 23:46)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.23061 2009-10-15 23:46:18.000000000 +0300 +++ /tmp/wklog.55.new.23061 2009-10-15 23:46:18.000000000 +0300 @@ -7,28 +7,25 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service -- Prompts the user for "essential" configuration options. Preliminary list - of "essential" options: +- Presents the user with GPL licence +- Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. - + * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are - busy. If they are, it asks to remove the previous installation first - and aborts. (that is: upgrades are not supported in step#1) + busy. If they are, offers to either change these parameters or abort the + installation (that is: no support for any kind of upgrades at this point) +- Copies installation files to appropriate destination +- Registers mysqld a service - Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) + (note: starting server manually requires write access to datadir, which + not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. @@ -53,7 +50,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. - TODO come up with options + TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22795 2009-10-15 23:40:36.000000000 +0300 +++ /tmp/wklog.55.new.22795 2009-10-15 23:40:36.000000000 +0300 @@ -7,7 +7,8 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Shows GPL Licence +- Copies files on installation +- Registers mysqld a service - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -15,27 +16,28 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * [perhaps] sql_mode setting. + * TODO come up with the final list. The criteria for inclusion are: + 1. ask for things that are essential to have a working setup as soon as + the installation is complete + 2. ask for things without answers for which the newbies can get into + trouble. -- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file (from a template) -- Sets up SQL root user with specified password +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - (note: will this work from any user, and on any OS? if not, this might - be omitted) - to start mysql client - - to edit the my.cnf file -- Registers MariaDB to start as a service with the specified parameters -- Registers MariaDB as installed software, sets up uninstaller + - to edit the my.cnf file. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the installation procedure will end up being). + on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 23:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22774 2009-10-15 23:40:16.000000000 +0300 +++ /tmp/wklog.55.new.22774 2009-10-15 23:40:16.000000000 +0300 @@ -15,13 +15,9 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * TODO come up with the final list. The criteria for inclusion are: - 1. ask for things that are essential to have a working setup as soon as - the installation is complete - 2. ask for things without answers for which the newbies can get into - trouble. + * [perhaps] sql_mode setting. -- Copies files to destination directory +- Copies files to destination directories - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) @@ -29,6 +25,8 @@ - Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) + (note: will this work from any user, and on any OS? if not, this might + be omitted) - to start mysql client - to edit the my.cnf file - Registers MariaDB to start as a service with the specified parameters -=-=(Psergey - Thu, 15 Oct 2009, 23:38)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.22618 2009-10-15 23:38:06.000000000 +0300 +++ /tmp/wklog.55.new.22618 2009-10-15 23:38:06.000000000 +0300 @@ -7,8 +7,7 @@ 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that -- Copies files on installation -- Registers mysqld a service +- Shows GPL Licence - Prompts the user for "essential" configuration options. Preliminary list of "essential" options: * Install directory @@ -22,22 +21,23 @@ 2. ask for things without answers for which the newbies can get into trouble. +- Copies files to destination directory - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) -- Generates appropriate my.cnf file -- Sets up SQL user with specified password +- Generates appropriate my.cnf file (from a template) +- Sets up SQL root user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - - to edit the my.cnf file. -- Registers MariaDB to start as a service with the specified parameters. -- Registers MariaDB as installed software, sets up uninstaller. + - to edit the my.cnf file +- Registers MariaDB to start as a service with the specified parameters +- Registers MariaDB as installed software, sets up uninstaller (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends - on how complex and error-prone the final installation procedure will be). + on how complex and error-prone the installation procedure will end up being). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -=-=(Psergey - Thu, 15 Oct 2009, 16:34)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.3919 2009-10-15 16:34:22.000000000 +0300 +++ /tmp/wklog.55.new.3919 2009-10-15 16:34:22.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal. It can be found here: +* MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Bothorsen - Thu, 15 Oct 2009, 15:40)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.1396 2009-10-15 15:40:03.000000000 +0300 +++ /tmp/wklog.55.new.1396 2009-10-15 15:40:03.000000000 +0300 @@ -81,6 +81,6 @@ 3. Other notes -------------- -* MySQL's logo is the seal (see e.g. here: https://code.launchpad.net/maria) - Bo Thorsen has the latest revision of the picture in various formats. +* MySQL's logo is the seal. It can be found here: +http://askmonty.org/wiki/index.php/MariaDB_Logo -=-=(Psergey - Thu, 15 Oct 2009, 15:22)=-=- High-Level Specification modified. --- /tmp/wklog.55.old.466 2009-10-15 15:22:05.000000000 +0300 +++ /tmp/wklog.55.new.466 2009-10-15 15:22:05.000000000 +0300 @@ -16,7 +16,7 @@ 14-10-2009 about data directory, Vista and UAC) * root password * default character set - * <TODO come up with the final list. The criteria for inclusion are: + * TODO come up with the final list. The criteria for inclusion are: 1. ask for things that are essential to have a working setup as soon as the installation is complete 2. ask for things without answers for which the newbies can get into @@ -25,11 +25,14 @@ - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, it asks to remove the previous installation first and aborts. (that is: upgrades are not supported in step#1) +- Generates appropriate my.cnf file +- Sets up SQL user with specified password - Adds start menu items - to start the server manually (with --console) - to start mysql client - to edit the my.cnf file. -- Registers MariaDB as installed, sets up uninstaller. +- Registers MariaDB to start as a service with the specified parameters. +- Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to @@ -54,9 +57,10 @@ 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This is a list of things that one might want an installer to do but that are out -of scope of this WL entry. -* Calibration of my.cnf parameters based on available memory, number of CPUs, etc. +This is a list of things that one might want an installer to do but that are +out of scope of this WL entry: +* Calibration of my.cnf parameters based on available memory, number of CPUs, + etc. 2. Installer wishlist (developer POV) ------------------------------------------------------------ -=-=(View All Progress Notes, 17 total)=-=- http://askmonty.org/worklog/index.pl?tid=55&nolimit=1 DESCRIPTION: We need Windows Installer package for MariaDB. HIGH-LEVEL SPECIFICATION: Not a spec so far but a list of points to consider: 1. Installer wishlist (user POV) -------------------------------- >From the user point of view: 1.1 Step 1: simple installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An installer package that - Presents the user with GPL licence - Prompts the user for "essential" configuration options: * Install directory - Data directory (see email from Peter Laursen on maria-developers@ dated 14-10-2009 about data directory, Vista and UAC) * root password * default character set * [possibly] sql_mode setting. - Checks if the target installation directory, TCP port, or named pipe are busy. If they are, offers to either change these parameters or abort the installation (that is: no support for any kind of upgrades at this point) - Copies installation files to appropriate destination - Registers mysqld a service - Generates appropriate my.cnf file - Sets up SQL root user with the specified password - Adds start menu items - to start the server manually (with --console) (note: starting server manually requires write access to datadir, which not all users will have. what to do?) - to start mysql client - to edit the my.cnf file. - Registers MariaDB to start as a service with the specified parameters. - Registers MariaDB as installed software, sets up uninstaller. (TODO: should the uninstaller the datadir or leave it? (or ask the user?)) - Creates installation log, and in case of any failures presents the log to the user and requests to file it as a bug (How far we should go here depends on how complex and error-prone the final installation procedure will be). 1.2 Step 2: Upgrades from MySQL or MariaDB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Same as above but with handling of the case where MySQL has been already installed: - offer to replace MySQL. - upgrade the data directory (todo we should sort out if anything/what is needed for this). - Uninstall MySQL - Install MariaDB. 1.3 Step 3: Configuration wizard ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Installer should include configuration wizard that's similar to what MySQL installer does. TODO come up with a list of options to set. 1.4 Items not on the wishlist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a list of things that one might want an installer to do but that are out of scope of this WL entry: * Calibration of my.cnf parameters based on available memory, number of CPUs, etc. 2. Installer wishlist (developer POV) ------------------------------------- * Some "installshield-like" tool that's easy to use (suggestion by Webyog: NSIS) * Installation procedure source should reside in MariaDB source repository * Installation procedure source file is better to be in human-readable text format. * It should be possible to automate creation of the installer package, in a way that can be run from buildbot (e.g. the installer package build process should print messages to its stdout) * Any suggestions on how can one automatically test the installation package? (for example, we'll want to start the installer, install, check that installation succeeded, then start the server, run some commands, then uninstall. Any ways to achieve that?) 3. Other notes -------------- * MariaDB's logo is the seal. It can be found here: http://askmonty.org/wiki/index.php/MariaDB_Logo * We should make both 32-bit installer and 64-bit installer (the latter will be possible when we have 64-bit windows binaries) * At this point we don't see a need to force a reboot after the installation. * The installer should be Vista and Windows7-proof. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Implement mysql-test output parser for Buildbot (22)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Implement mysql-test output parser for Buildbot CREATION DATE..: Thu, 21 May 2009, 22:19 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 22 (http://askmonty.org/worklog/?tid=22) VERSION........: STATUS.........: Complete PRIORITY.......: 60 WORKED HOURS...: 30 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 30 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:44)=-=- Status updated. --- /tmp/wklog.22.old.18407 2010-05-14 06:44:28.000000000 +0000 +++ /tmp/wklog.22.new.18407 2010-05-14 06:44:28.000000000 +0000 @@ -1 +1 @@ -Assigned +Complete -=-=(Knielsen - Fri, 14 May 2010, 06:44)=-=- This was done long time ago. But I lost records of how much time was spent on it. Worked 30 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Guest - Fri, 26 Jun 2009, 11:34)=-=- Status updated. --- /tmp/wklog.22.old.29004 2009-06-26 11:34:41.000000000 +0300 +++ /tmp/wklog.22.new.29004 2009-06-26 11:34:41.000000000 +0300 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Thu, 21 May 2009, 22:33)=-=- High Level Description modified. --- /tmp/wklog.22.old.27679 2009-05-21 22:33:35.000000000 +0300 +++ /tmp/wklog.22.new.27679 2009-05-21 22:33:35.000000000 +0300 @@ -9,3 +9,6 @@ http://djmitche.github.com/buildbot/docs/0.7.10/#Writing-New-BuildSteps +Later, once we get the infrastructure to write Buildbot results into a MySQL +database, we want to extend this to also insert into the database all test +failures and the mysqltest failure output (for cross-reference search). DESCRIPTION: Like in Pushbuild at MySQL AB, we want to have buildbot parse the output of mysql-test-run for test errors so we can display on the front page the name of any tests that failed. The parser can also count the number of tests completed so far so Buildbot can provide more accurate completion estimates. Buildbot already has support for plugging in such modules. See eg. http://djmitche.github.com/buildbot/docs/0.7.10/#Writing-New-BuildSteps Later, once we get the infrastructure to write Buildbot results into a MySQL database, we want to extend this to also insert into the database all test failures and the mysqltest failure output (for cross-reference search). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Implement mysql-test output parser for Buildbot (22)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Implement mysql-test output parser for Buildbot CREATION DATE..: Thu, 21 May 2009, 22:19 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 22 (http://askmonty.org/worklog/?tid=22) VERSION........: STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 30 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 30 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:44)=-=- This was done long time ago. But I lost records of how much time was spent on it. Worked 30 hours and estimate 0 hours remain (original estimate unchanged). -=-=(Guest - Fri, 26 Jun 2009, 11:34)=-=- Status updated. --- /tmp/wklog.22.old.29004 2009-06-26 11:34:41.000000000 +0300 +++ /tmp/wklog.22.new.29004 2009-06-26 11:34:41.000000000 +0300 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Knielsen - Thu, 21 May 2009, 22:33)=-=- High Level Description modified. --- /tmp/wklog.22.old.27679 2009-05-21 22:33:35.000000000 +0300 +++ /tmp/wklog.22.new.27679 2009-05-21 22:33:35.000000000 +0300 @@ -9,3 +9,6 @@ http://djmitche.github.com/buildbot/docs/0.7.10/#Writing-New-BuildSteps +Later, once we get the infrastructure to write Buildbot results into a MySQL +database, we want to extend this to also insert into the database all test +failures and the mysqltest failure output (for cross-reference search). DESCRIPTION: Like in Pushbuild at MySQL AB, we want to have buildbot parse the output of mysql-test-run for test errors so we can display on the front page the name of any tests that failed. The parser can also count the number of tests completed so far so Buildbot can provide more accurate completion estimates. Buildbot already has support for plugging in such modules. See eg. http://djmitche.github.com/buildbot/docs/0.7.10/#Writing-New-BuildSteps Later, once we get the infrastructure to write Buildbot results into a MySQL database, we want to extend this to also insert into the database all test failures and the mysqltest failure output (for cross-reference search). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Buildbot upgrade test from MariaDB 5.1.42->newest (108)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Buildbot upgrade test from MariaDB 5.1.42->newest CREATION DATE..: Fri, 19 Mar 2010, 07:48 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Other TASK ID........: 108 (http://askmonty.org/worklog/?tid=108) VERSION........: STATUS.........: Cancelled PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 25 (hours remain) ORIG. ESTIMATE.: 25 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:43)=-=- Version updated. --- /tmp/wklog.108.old.18388 2010-05-14 06:43:08.000000000 +0000 +++ /tmp/wklog.108.new.18388 2010-05-14 06:43:08.000000000 +0000 @@ -1 +1 @@ -Server-9.x + -=-=(Knielsen - Fri, 14 May 2010, 06:43)=-=- Status updated. --- /tmp/wklog.108.old.18388 2010-05-14 06:43:08.000000000 +0000 +++ /tmp/wklog.108.new.18388 2010-05-14 06:43:08.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Cancelled -=-=(Knielsen - Fri, 14 May 2010, 06:42)=-=- This task was done in MWL#118 (I forgot about this one, so added a new, duplicate task). Reported zero hours worked. Estimate unchanged. -=-=(Knielsen - Fri, 19 Mar 2010, 07:52)=-=- Low Level Design modified. --- /tmp/wklog.108.old.3290 2010-03-19 07:52:59.000000000 +0000 +++ /tmp/wklog.108.new.3290 2010-03-19 07:52:59.000000000 +0000 @@ -1 +1,11 @@ +Tasks needed to do this: + + - For all the .deb kvm builders, set up a new virtual machine image. Based on + -serial or -install, pre-install the appropriate MariaDB 5.1.42 package + with some data. Similar to how the -upgrade images are set up with MySQL + pre-installed. + + - Add a test step in the Buildbot configuration that runs another upgrade + test, similar to the existing upgrade test, but with the images with + MariaDB preinstalled rather than MySQL. DESCRIPTION: Buildbot currently does automatic upgrade test for .debs from the official MySQL 5.0/5.1 package to the newest MariaDB package. In addition to this, we need an upgrade test from older MariaDB release. Probably 5.1.42, the first stable release, is the one to use. This is particularly important for the 5.2/5.3 trees, to test that upgrade from 5.1->5.2/5.3 works. LOW-LEVEL DESIGN: Tasks needed to do this: - For all the .deb kvm builders, set up a new virtual machine image. Based on -serial or -install, pre-install the appropriate MariaDB 5.1.42 package with some data. Similar to how the -upgrade images are set up with MySQL pre-installed. - Add a test step in the Buildbot configuration that runs another upgrade test, similar to the existing upgrade test, but with the images with MariaDB preinstalled rather than MySQL. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Buildbot upgrade test from MariaDB 5.1.42->newest (108)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Buildbot upgrade test from MariaDB 5.1.42->newest CREATION DATE..: Fri, 19 Mar 2010, 07:48 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Other TASK ID........: 108 (http://askmonty.org/worklog/?tid=108) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 25 (hours remain) ORIG. ESTIMATE.: 25 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:42)=-=- This task was done in MWL#118 (I forgot about this one, so added a new, duplicate task). Reported zero hours worked. Estimate unchanged. -=-=(Knielsen - Fri, 19 Mar 2010, 07:52)=-=- Low Level Design modified. --- /tmp/wklog.108.old.3290 2010-03-19 07:52:59.000000000 +0000 +++ /tmp/wklog.108.new.3290 2010-03-19 07:52:59.000000000 +0000 @@ -1 +1,11 @@ +Tasks needed to do this: + + - For all the .deb kvm builders, set up a new virtual machine image. Based on + -serial or -install, pre-install the appropriate MariaDB 5.1.42 package + with some data. Similar to how the -upgrade images are set up with MySQL + pre-installed. + + - Add a test step in the Buildbot configuration that runs another upgrade + test, similar to the existing upgrade test, but with the images with + MariaDB preinstalled rather than MySQL. DESCRIPTION: Buildbot currently does automatic upgrade test for .debs from the official MySQL 5.0/5.1 package to the newest MariaDB package. In addition to this, we need an upgrade test from older MariaDB release. Probably 5.1.42, the first stable release, is the one to use. This is particularly important for the 5.2/5.3 trees, to test that upgrade from 5.1->5.2/5.3 works. LOW-LEVEL DESIGN: Tasks needed to do this: - For all the .deb kvm builders, set up a new virtual machine image. Based on -serial or -install, pre-install the appropriate MariaDB 5.1.42 package with some data. Similar to how the -upgrade images are set up with MySQL pre-installed. - Add a test step in the Buildbot configuration that runs another upgrade test, similar to the existing upgrade test, but with the images with MariaDB preinstalled rather than MySQL. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Buildbot MariaDB->MariaDB upgrade testing (118)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Buildbot MariaDB->MariaDB upgrade testing CREATION DATE..: Wed, 12 May 2010, 13:48 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 118 (http://askmonty.org/worklog/?tid=118) VERSION........: STATUS.........: Complete PRIORITY.......: 60 WORKED HOURS...: 5 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:41)=-=- Version updated. --- /tmp/wklog.118.old.18369 2010-05-14 06:41:01.000000000 +0000 +++ /tmp/wklog.118.new.18369 2010-05-14 06:41:01.000000000 +0000 @@ -1 +1 @@ -Server-9.x + -=-=(Knielsen - Fri, 14 May 2010, 06:41)=-=- Status updated. --- /tmp/wklog.118.old.18369 2010-05-14 06:41:01.000000000 +0000 +++ /tmp/wklog.118.new.18369 2010-05-14 06:41:01.000000000 +0000 @@ -1 +1 @@ -Un-Assigned +Complete -=-=(Knielsen - Fri, 14 May 2010, 06:40)=-=- Installed the new virtual images, using 5.1.42 from OurDelta (for all except lucid, which is new and so has no 5.1.42 ourdelta package and also needed a fixed set of packages due to changes in the lucid MySQL packaging). Added a new upgrade2 step in the Buildbot configuration to test this. Updated Buildbot wiki documentation. Worked 5 hours and estimate 0 hours remain (original estimate decreased by 3 hours). DESCRIPTION: Create an additional test step for Buildbot to check that upgrading from one version of MariaDB to another works ok. We already have testing of upgrade from distro-official MySQL .debs to our MariaDB .debs. What we need is for each Debian/Ubuntu, a new KVM virtual image with MariaDB 5.1.42 (First GA release) pre-installed, just like the existing update test uses images with MySQL pre-installed. Then the Buildbot configuration must be updated to add another upgrade test step, just like the existing one but using the images with MariaDB pre-installed. Also, the setup of the new images must be added to the existing documentation: http://askmonty.org/wiki/BuildBot::package http://askmonty.org/wiki/BuildBot:vm-setup ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Buildbot MariaDB->MariaDB upgrade testing (118)
by worklog-noreply＠askmonty.org 14 May '10

14 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Buildbot MariaDB->MariaDB upgrade testing CREATION DATE..: Wed, 12 May 2010, 13:48 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Other TASK ID........: 118 (http://askmonty.org/worklog/?tid=118) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 5 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 May 2010, 06:40)=-=- Installed the new virtual images, using 5.1.42 from OurDelta (for all except lucid, which is new and so has no 5.1.42 ourdelta package and also needed a fixed set of packages due to changes in the lucid MySQL packaging). Added a new upgrade2 step in the Buildbot configuration to test this. Updated Buildbot wiki documentation. Worked 5 hours and estimate 0 hours remain (original estimate decreased by 3 hours). DESCRIPTION: Create an additional test step for Buildbot to check that upgrading from one version of MariaDB to another works ok. We already have testing of upgrade from distro-official MySQL .debs to our MariaDB .debs. What we need is for each Debian/Ubuntu, a new KVM virtual image with MariaDB 5.1.42 (First GA release) pre-installed, just like the existing update test uses images with MySQL pre-installed. Then the Buildbot configuration must be updated to add another upgrade test step, just like the existing one but using the images with MariaDB pre-installed. Also, the setup of the new images must be added to the existing documentation: http://askmonty.org/wiki/BuildBot::package http://askmonty.org/wiki/BuildBot:vm-setup ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Summary of discussion about MariaDB future and release plans
by Kristian Nielsen 13 May '10

13 May '10

Hi, With all the european MariaDB people stuck in California, there is an unexpected opportunity for extra face-to-face meetups. So today I had the opportunity to enjoy lunch at a nice sushi place in Santa Clara with Henrik, Timour, Peter, Vadim, and Sanja. Here are a couple of key points from the discussion: 1. We need to consider carefully how we handle keeping the MariaDB product stable for production use. Peter had the point that the critical thing is avoiding behaviour change. This means that if some version of MariaDB works sufficiently different that the user need to work on their application to upgrade, it is a blocker to adoption. Different in this context means different from any previous version of MariaDB, but it _also_ means any difference from MySQL! I think I can see that this is actually more important than mere bugs/regressions. If a bug slips through, we can fix it, and if we can fix it quick enough the user may even be happy with us. But if the behaviour is qualitatively different (and such difference can not easily be turned off), that is a real problem for the user. We discussed a lot around this for the subquery optimisations. This is a "scary feature", as it has the very real potential to change execution plans, occasionally for the worse. As Timour pointed out, there is for each new optimiser feature a server variable to enable/disable it. So we need to very clearly state in the release notes, under "upgrading to MariaDB x.y", that there are optimiser changes, and give the exact list of variable settings that will make the optimiser run in "MySQL 5.1 mode". For other features this is simpler. For storage engines, just disable the engine and it will have no impact. For possibly replication plugins, again don't use the plugin and there should be no difference in behaviour. But again, we need to very carefully think about this and communicate/document this. For other features again we might need to re-think this point, making sure we do not change behaviour in a non reversible way. I think this is a good observation from those with a deep knowledge of the operational side of Databases. 2. On a note related to stability, I think we need to carefully avoid the mistakes from the MySQL release model. Basically, we need to have regular releases (6-12 month cycles). This is all-important! Much more important than any single feature, however big. We must _never_ push a feature into a tree if it is not ready. Better make an empty release! So this means it is actually wrong to say that subqueries will be in MariaDB 5.3. That should not be the plan. The subqueries will be in the first release made after they are ready. Which maybe 5.3, may be something else. I think we very nearly made the same mistake with 5.2. We had a feature list, and while some of them were "rolling", we also had features that were decided in advance that they _must_ be pushed at a certain date. I strongly believe this is wrong. We should have made _all_ features rolling. Unless we are extremely careful with this, sooner or later there will be sufficient pressure that we will push unfinished stuff into a release, causing _all_ of that release to be a failure. By waiting with the unfinished feature, only that feature will "fail", and by being careful with releases, the next release will be close anyway. (Yes, "finished" cannot mean bug free. But it can mean "as good as we can reasonably make it".) 3. The final point I took from the discussion was related to version numbering. We in the MariaDB team of course know that we rock, and that we will change the world tomorrow, or at least as soon as planes will take us back to Europe :-). But the reality is that for now, we are an appendix to MySQL with MariaDB 5.1 and 5.2. It thus can be argued that this should be reflected in the version numbering, to avoid confusion. Peter made the point that he would like to see MariaDB 5.1 and 5.2 versioning be made so that it is clear that there is a base MySQL version plus some well-defined set of changes. (This is how XtraDB release numbering works). So we currently have MariaDB 5.1.44, which is equivalent to MySQL 5.1.44 plus some set of safe patches. But MariaDB 5.2.0 is actually the same, just with some additional, also safe patches. So rather than go to MariaDB 5.2, we could imagine something like this: MariaDB 5.1.44-1 Current MariaDB 5.1.44 MariaDB 5.1.44-2 Current MariaDB 5.2.0 MariaDB 5.1.44-1.1 In case we need to do a bugfix for 5.1.44-1 MariaDB 5.1.46-1 Next time we merge MySQL 5.1.46 into MariaDB -1 MariaDB 5.1.46-2 Next time we merge MySQL 5.1.46 into MariaDB -2 I need to think more before I fully make up my mind about these points one way or the other, but I think these are in any case interesting points to consider. - Kristian.

7 17

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (igor:2790)
by Igor Babaev 13 May '10

13 May '10

#At lp:maria/5.2 based on revid:igor@askmonty.org-20100512040958-yc654eq2zdfo8l8m 2790 Igor Babaev 2010-05-12 Cleanup modified: sql/sql_derived.cc === modified file 'sql/sql_derived.cc' --- a/sql/sql_derived.cc 2010-05-12 04:09:58 +0000 +++ b/sql/sql_derived.cc 2010-05-13 06:59:14 +0000 @@ -159,12 +159,7 @@ mysql_handle_single_derived(LEX *lex, TA uint phase_flag= DT_INIT << phase; if (phase_flag > phases) break; -#if 0 - if (!(phases & phase_flag) || - derived->merged_for_insert && phase_flag != DT_REINIT) -#else if (!(phases & phase_flag)) -#endif continue; /* Skip derived tables to which the phase isn't applicable. */ if (phase_flag != DT_PREPARE && @@ -605,19 +600,11 @@ bool mysql_derived_prepare(THD *thd, LEX bool res= FALSE; // Skip already prepared views/DT -#if 0 - if (!unit || unit->prepared || derived->merged_for_insert) -#else if (!unit || unit->prepared) -#endif DBUG_RETURN(FALSE); /* It's a target view for an INSERT, create field translation only. */ -#if 0 - if (derived->skip_prepare_derived && !derived->is_multitable()) -#else if (derived->merged_for_insert) -#endif { res= derived->create_field_translation(thd); DBUG_RETURN(res);

1 0

[Maria-developers] Updated (by Knielsen): Use Buildbot to populate apt/yum repositories (117)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Use Buildbot to populate apt/yum repositories CREATION DATE..: Wed, 12 May 2010, 07:04 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 117 (http://askmonty.org/worklog/?tid=117) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 4 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 20 PROGRESS NOTES: -=-=(Knielsen - Wed, 12 May 2010, 21:20)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.21448 2010-05-12 21:20:34.000000000 +0000 +++ /tmp/wklog.117.new.21448 2010-05-12 21:20:34.000000000 +0000 @@ -1,5 +1,5 @@ -As for signing, I think it may be possible/best to do the signing outside of -buildbot, as a separate process. There are some advantages to this: +The signing of packages can be done outside of Buildbot, as a separate +process. There are some advantages to this: - Security: the private key can be kept less exposed when it is not included in the buildbot infrastructure. @@ -9,9 +9,6 @@ - Generally reducing the complexity of the buildbot setup. -This of course requires that it is possible to sign the packages after the -actual build. - ---- Here is how to sign the .rpms. @@ -42,20 +39,37 @@ ---- -For .deb, I *think* we are using secure apt, which does not actually sign the -packages, rather it signs the "Release" file which is created when the -repository is set up. So in this case again there is no problem doing the -signing outside of the build itself (in fact that is the way it must be). +For .deb, it is not the individual .deb that is signed, it is the +repository. Here is one way to generate a signed repository, using reprepro. -Found two tools that can help with building and signing apt repositories: -reprepro (seems to be the newest, recommended) and apt-ftparchive. +The ourdelta/bakery signing stuff needs to be copied to ~/.gnupg ----- +mkdir repo # or whatever +cd repo +mkdir conf +cat >conf/distributions <<END +Origin: MariaDB +Label: MariaDB +Codename: hardy +Architectures: amd64 +Components: mariadb-ourdelta +Description: MariaDB test Repository +SignWith: autosign(a)ourdelta.org +END +for i in `find /home/buildbot/debs/ -name '*.deb'` ; do reprepro --basedir=. +includedeb hardy $i ; done + +The corrosponding line for /etc/apt/sources.list: -ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen -suggested splitting up so we have this package ourselves, or maybe it can be -handled with replace/provide/conflict dependencies. + deb file:///home/buildbot/repo hardy mariadb-ourdelta + +This works for multiple distributions, by adding more sections to the +conf/distributions file. + +---- -ToDo: Figure out exactly what files/directory structure needs to be uploaded -(asked Peter, awaiting reply). +For the mysql-client-core-5.1 issue, the solution is to split the +mariadb-client-5.1 (and 5.2) package similarly into +mariadb-client-core-5.1. The mariadb-client-core-5.1 package then provides: +mysql-client-core-5.1. -=-=(Knielsen - Wed, 12 May 2010, 18:25)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.12634 2010-05-12 18:25:58.000000000 +0000 +++ /tmp/wklog.117.new.12634 2010-05-12 18:25:58.000000000 +0000 @@ -12,9 +12,35 @@ This of course requires that it is possible to sign the packages after the actual build. -For .rpm this seems to be easy (from reading, didn't try yet): +---- + +Here is how to sign the .rpms. + +Copy in the ourdelta/bakery signing stuff to ~/.gnupg and ~/.rpmmacros. + +Run + + rpm --addsign *.rpm + +That's all! This can be tested by creating a local yum repository: - rpm --addsign <packages> + createrepo <dir> + +(where <dir> contains the signed .rpms). Then create the file +/etc/yum.repos.d/localmaria.repo: + +[localmaria] +name=Local MariaDB repo +baseurl=file:///home/buildbot/rpms +gpgcheck=1 +enabled=1 +gpgkey=http://master.ourdelta.org/deb/ourdelta.gpg + +Now this should work to install MariaDB: + + sudo yum install MariaDB-server + +---- For .deb, I *think* we are using secure apt, which does not actually sign the packages, rather it signs the "Release" file which is created when the -=-=(Knielsen - Wed, 12 May 2010, 07:14)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.401 2010-05-12 07:14:27.000000000 +0000 +++ /tmp/wklog.117.new.401 2010-05-12 07:14:27.000000000 +0000 @@ -1 +1,35 @@ +As for signing, I think it may be possible/best to do the signing outside of +buildbot, as a separate process. There are some advantages to this: + + - Security: the private key can be kept less exposed when it is not included + in the buildbot infrastructure. + + - It is good to have one step of human intervention before actually signing + and releasing packages. + + - Generally reducing the complexity of the buildbot setup. + +This of course requires that it is possible to sign the packages after the +actual build. + +For .rpm this seems to be easy (from reading, didn't try yet): + + rpm --addsign <packages> + +For .deb, I *think* we are using secure apt, which does not actually sign the +packages, rather it signs the "Release" file which is created when the +repository is set up. So in this case again there is no problem doing the +signing outside of the build itself (in fact that is the way it must be). + +Found two tools that can help with building and signing apt repositories: +reprepro (seems to be the newest, recommended) and apt-ftparchive. + +---- + +ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen +suggested splitting up so we have this package ourselves, or maybe it can be +handled with replace/provide/conflict dependencies. + +ToDo: Figure out exactly what files/directory structure needs to be uploaded +(asked Peter, awaiting reply). -=-=(Knielsen - Wed, 12 May 2010, 07:06)=-=- Upgraded lucid VMs to the official release. Discussed with Arjen how to handle things. Did a lot of reading on how apt repositories work. Worked 4 hours and estimate 16 hours remain (original estimate unchanged). DESCRIPTION: Since the package building for MariaDB is now fully automated in Buildbot, it has been decided to use packages from Buildbot for the OurDelta apt and yum repositories. This worklog is about fixing/implementing anything that is missing to achieve this. - When doing a real release build, packages/repositories need to be signed, so that users will not get a warning about unauthenticated packages. This signing must only be done on official releases, not on daily builds (to avoid confusing one with the other). - Packages must be uploaded from the Buildbot host. The OurDelta infrastructure has a DropBox share that could be used for this, another option is to simply use rsync. - Ubuntu 10.04 "lucid" has been released, and we need to support that for packages, so the Buildbot VM for lucid must be upgraded to have the official release. - In Ubuntu 10.04, the official MySQL packages include a new package mysql-client-core, we currently have a conflict with this on install that we need to handle somehow. HIGH-LEVEL SPECIFICATION: The signing of packages can be done outside of Buildbot, as a separate process. There are some advantages to this: - Security: the private key can be kept less exposed when it is not included in the buildbot infrastructure. - It is good to have one step of human intervention before actually signing and releasing packages. - Generally reducing the complexity of the buildbot setup. ---- Here is how to sign the .rpms. Copy in the ourdelta/bakery signing stuff to ~/.gnupg and ~/.rpmmacros. Run rpm --addsign *.rpm That's all! This can be tested by creating a local yum repository: createrepo <dir> (where <dir> contains the signed .rpms). Then create the file /etc/yum.repos.d/localmaria.repo: [localmaria] name=Local MariaDB repo baseurl=file:///home/buildbot/rpms gpgcheck=1 enabled=1 gpgkey=http://master.ourdelta.org/deb/ourdelta.gpg Now this should work to install MariaDB: sudo yum install MariaDB-server ---- For .deb, it is not the individual .deb that is signed, it is the repository. Here is one way to generate a signed repository, using reprepro. The ourdelta/bakery signing stuff needs to be copied to ~/.gnupg mkdir repo # or whatever cd repo mkdir conf cat >conf/distributions <<END Origin: MariaDB Label: MariaDB Codename: hardy Architectures: amd64 Components: mariadb-ourdelta Description: MariaDB test Repository SignWith: autosign(a)ourdelta.org END for i in `find /home/buildbot/debs/ -name '*.deb'` ; do reprepro --basedir=. includedeb hardy $i ; done The corrosponding line for /etc/apt/sources.list: deb file:///home/buildbot/repo hardy mariadb-ourdelta This works for multiple distributions, by adding more sections to the conf/distributions file. ---- For the mysql-client-core-5.1 issue, the solution is to split the mariadb-client-5.1 (and 5.2) package similarly into mariadb-client-core-5.1. The mariadb-client-core-5.1 package then provides: mysql-client-core-5.1. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Use Buildbot to populate apt/yum repositories (117)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Use Buildbot to populate apt/yum repositories CREATION DATE..: Wed, 12 May 2010, 07:04 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 117 (http://askmonty.org/worklog/?tid=117) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 4 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 20 PROGRESS NOTES: -=-=(Knielsen - Wed, 12 May 2010, 18:25)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.12634 2010-05-12 18:25:58.000000000 +0000 +++ /tmp/wklog.117.new.12634 2010-05-12 18:25:58.000000000 +0000 @@ -12,9 +12,35 @@ This of course requires that it is possible to sign the packages after the actual build. -For .rpm this seems to be easy (from reading, didn't try yet): +---- + +Here is how to sign the .rpms. + +Copy in the ourdelta/bakery signing stuff to ~/.gnupg and ~/.rpmmacros. + +Run + + rpm --addsign *.rpm + +That's all! This can be tested by creating a local yum repository: - rpm --addsign <packages> + createrepo <dir> + +(where <dir> contains the signed .rpms). Then create the file +/etc/yum.repos.d/localmaria.repo: + +[localmaria] +name=Local MariaDB repo +baseurl=file:///home/buildbot/rpms +gpgcheck=1 +enabled=1 +gpgkey=http://master.ourdelta.org/deb/ourdelta.gpg + +Now this should work to install MariaDB: + + sudo yum install MariaDB-server + +---- For .deb, I *think* we are using secure apt, which does not actually sign the packages, rather it signs the "Release" file which is created when the -=-=(Knielsen - Wed, 12 May 2010, 07:14)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.401 2010-05-12 07:14:27.000000000 +0000 +++ /tmp/wklog.117.new.401 2010-05-12 07:14:27.000000000 +0000 @@ -1 +1,35 @@ +As for signing, I think it may be possible/best to do the signing outside of +buildbot, as a separate process. There are some advantages to this: + + - Security: the private key can be kept less exposed when it is not included + in the buildbot infrastructure. + + - It is good to have one step of human intervention before actually signing + and releasing packages. + + - Generally reducing the complexity of the buildbot setup. + +This of course requires that it is possible to sign the packages after the +actual build. + +For .rpm this seems to be easy (from reading, didn't try yet): + + rpm --addsign <packages> + +For .deb, I *think* we are using secure apt, which does not actually sign the +packages, rather it signs the "Release" file which is created when the +repository is set up. So in this case again there is no problem doing the +signing outside of the build itself (in fact that is the way it must be). + +Found two tools that can help with building and signing apt repositories: +reprepro (seems to be the newest, recommended) and apt-ftparchive. + +---- + +ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen +suggested splitting up so we have this package ourselves, or maybe it can be +handled with replace/provide/conflict dependencies. + +ToDo: Figure out exactly what files/directory structure needs to be uploaded +(asked Peter, awaiting reply). -=-=(Knielsen - Wed, 12 May 2010, 07:06)=-=- Upgraded lucid VMs to the official release. Discussed with Arjen how to handle things. Did a lot of reading on how apt repositories work. Worked 4 hours and estimate 16 hours remain (original estimate unchanged). DESCRIPTION: Since the package building for MariaDB is now fully automated in Buildbot, it has been decided to use packages from Buildbot for the OurDelta apt and yum repositories. This worklog is about fixing/implementing anything that is missing to achieve this. - When doing a real release build, packages/repositories need to be signed, so that users will not get a warning about unauthenticated packages. This signing must only be done on official releases, not on daily builds (to avoid confusing one with the other). - Packages must be uploaded from the Buildbot host. The OurDelta infrastructure has a DropBox share that could be used for this, another option is to simply use rsync. - Ubuntu 10.04 "lucid" has been released, and we need to support that for packages, so the Buildbot VM for lucid must be upgraded to have the official release. - In Ubuntu 10.04, the official MySQL packages include a new package mysql-client-core, we currently have a conflict with this on install that we need to handle somehow. HIGH-LEVEL SPECIFICATION: As for signing, I think it may be possible/best to do the signing outside of buildbot, as a separate process. There are some advantages to this: - Security: the private key can be kept less exposed when it is not included in the buildbot infrastructure. - It is good to have one step of human intervention before actually signing and releasing packages. - Generally reducing the complexity of the buildbot setup. This of course requires that it is possible to sign the packages after the actual build. ---- Here is how to sign the .rpms. Copy in the ourdelta/bakery signing stuff to ~/.gnupg and ~/.rpmmacros. Run rpm --addsign *.rpm That's all! This can be tested by creating a local yum repository: createrepo <dir> (where <dir> contains the signed .rpms). Then create the file /etc/yum.repos.d/localmaria.repo: [localmaria] name=Local MariaDB repo baseurl=file:///home/buildbot/rpms gpgcheck=1 enabled=1 gpgkey=http://master.ourdelta.org/deb/ourdelta.gpg Now this should work to install MariaDB: sudo yum install MariaDB-server ---- For .deb, I *think* we are using secure apt, which does not actually sign the packages, rather it signs the "Release" file which is created when the repository is set up. So in this case again there is no problem doing the signing outside of the build itself (in fact that is the way it must be). Found two tools that can help with building and signing apt repositories: reprepro (seems to be the newest, recommended) and apt-ftparchive. ---- ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen suggested splitting up so we have this package ourselves, or maybe it can be handled with replace/provide/conflict dependencies. ToDo: Figure out exactly what files/directory structure needs to be uploaded (asked Peter, awaiting reply). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Knielsen): Buildbot MariaDB->MariaDB upgrade testing (118)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Buildbot MariaDB->MariaDB upgrade testing CREATION DATE..: Wed, 12 May 2010, 13:48 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Other TASK ID........: 118 (http://askmonty.org/worklog/?tid=118) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 8 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: DESCRIPTION: Create an additional test step for Buildbot to check that upgrading from one version of MariaDB to another works ok. We already have testing of upgrade from distro-official MySQL .debs to our MariaDB .debs. What we need is for each Debian/Ubuntu, a new KVM virtual image with MariaDB 5.1.42 (First GA release) pre-installed, just like the existing update test uses images with MySQL pre-installed. Then the Buildbot configuration must be updated to add another upgrade test step, just like the existing one but using the images with MariaDB pre-installed. Also, the setup of the new images must be added to the existing documentation: http://askmonty.org/wiki/BuildBot::package http://askmonty.org/wiki/BuildBot:vm-setup ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2855: Build perl scripts in the correct directory
by noreply＠launchpad.net 12 May '10

12 May '10

------------------------------------------------------------ revno: 2855 committer: Bo Thorsen <bo(a)askmonty.org> branch nick: trunk-work timestamp: Wed 2010-05-12 14:33:10 +0200 message: Build perl scripts in the correct directory modified: scripts/CMakeLists.txt -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] Patch to generate perl scripts in the right place
by Bo Thorsen 12 May '10

12 May '10

As discussed with Kristian Nielsen on IRC. Bo.

1 0

[Maria-developers] Updated (by Knielsen): Use Buildbot to populate apt/yum repositories (117)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Use Buildbot to populate apt/yum repositories CREATION DATE..: Wed, 12 May 2010, 07:04 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 117 (http://askmonty.org/worklog/?tid=117) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 4 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 20 PROGRESS NOTES: -=-=(Knielsen - Wed, 12 May 2010, 07:14)=-=- High-Level Specification modified. --- /tmp/wklog.117.old.401 2010-05-12 07:14:27.000000000 +0000 +++ /tmp/wklog.117.new.401 2010-05-12 07:14:27.000000000 +0000 @@ -1 +1,35 @@ +As for signing, I think it may be possible/best to do the signing outside of +buildbot, as a separate process. There are some advantages to this: + + - Security: the private key can be kept less exposed when it is not included + in the buildbot infrastructure. + + - It is good to have one step of human intervention before actually signing + and releasing packages. + + - Generally reducing the complexity of the buildbot setup. + +This of course requires that it is possible to sign the packages after the +actual build. + +For .rpm this seems to be easy (from reading, didn't try yet): + + rpm --addsign <packages> + +For .deb, I *think* we are using secure apt, which does not actually sign the +packages, rather it signs the "Release" file which is created when the +repository is set up. So in this case again there is no problem doing the +signing outside of the build itself (in fact that is the way it must be). + +Found two tools that can help with building and signing apt repositories: +reprepro (seems to be the newest, recommended) and apt-ftparchive. + +---- + +ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen +suggested splitting up so we have this package ourselves, or maybe it can be +handled with replace/provide/conflict dependencies. + +ToDo: Figure out exactly what files/directory structure needs to be uploaded +(asked Peter, awaiting reply). -=-=(Knielsen - Wed, 12 May 2010, 07:06)=-=- Upgraded lucid VMs to the official release. Discussed with Arjen how to handle things. Did a lot of reading on how apt repositories work. Worked 4 hours and estimate 16 hours remain (original estimate unchanged). DESCRIPTION: Since the package building for MariaDB is now fully automated in Buildbot, it has been decided to use packages from Buildbot for the OurDelta apt and yum repositories. This worklog is about fixing/implementing anything that is missing to achieve this. - When doing a real release build, packages/repositories need to be signed, so that users will not get a warning about unauthenticated packages. This signing must only be done on official releases, not on daily builds (to avoid confusing one with the other). - Packages must be uploaded from the Buildbot host. The OurDelta infrastructure has a DropBox share that could be used for this, another option is to simply use rsync. - Ubuntu 10.04 "lucid" has been released, and we need to support that for packages, so the Buildbot VM for lucid must be upgraded to have the official release. - In Ubuntu 10.04, the official MySQL packages include a new package mysql-client-core, we currently have a conflict with this on install that we need to handle somehow. HIGH-LEVEL SPECIFICATION: As for signing, I think it may be possible/best to do the signing outside of buildbot, as a separate process. There are some advantages to this: - Security: the private key can be kept less exposed when it is not included in the buildbot infrastructure. - It is good to have one step of human intervention before actually signing and releasing packages. - Generally reducing the complexity of the buildbot setup. This of course requires that it is possible to sign the packages after the actual build. For .rpm this seems to be easy (from reading, didn't try yet): rpm --addsign <packages> For .deb, I *think* we are using secure apt, which does not actually sign the packages, rather it signs the "Release" file which is created when the repository is set up. So in this case again there is no problem doing the signing outside of the build itself (in fact that is the way it must be). Found two tools that can help with building and signing apt repositories: reprepro (seems to be the newest, recommended) and apt-ftparchive. ---- ToDO: Figure out how to handle the mysql-client-core issue on lucid. Arjen suggested splitting up so we have this package ourselves, or maybe it can be handled with replace/provide/conflict dependencies. ToDo: Figure out exactly what files/directory structure needs to be uploaded (asked Peter, awaiting reply). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Use Buildbot to populate apt/yum repositories (117)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Use Buildbot to populate apt/yum repositories CREATION DATE..: Wed, 12 May 2010, 07:04 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 117 (http://askmonty.org/worklog/?tid=117) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 4 ESTIMATE.......: 16 (hours remain) ORIG. ESTIMATE.: 20 PROGRESS NOTES: -=-=(Knielsen - Wed, 12 May 2010, 07:06)=-=- Upgraded lucid VMs to the official release. Discussed with Arjen how to handle things. Did a lot of reading on how apt repositories work. Worked 4 hours and estimate 16 hours remain (original estimate unchanged). DESCRIPTION: Since the package building for MariaDB is now fully automated in Buildbot, it has been decided to use packages from Buildbot for the OurDelta apt and yum repositories. This worklog is about fixing/implementing anything that is missing to achieve this. - When doing a real release build, packages/repositories need to be signed, so that users will not get a warning about unauthenticated packages. This signing must only be done on official releases, not on daily builds (to avoid confusing one with the other). - Packages must be uploaded from the Buildbot host. The OurDelta infrastructure has a DropBox share that could be used for this, another option is to simply use rsync. - Ubuntu 10.04 "lucid" has been released, and we need to support that for packages, so the Buildbot VM for lucid must be upgraded to have the official release. - In Ubuntu 10.04, the official MySQL packages include a new package mysql-client-core, we currently have a conflict with this on install that we need to handle somehow. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Knielsen): Use Buildbot to populate apt/yum repositories (117)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Use Buildbot to populate apt/yum repositories CREATION DATE..: Wed, 12 May 2010, 07:04 SUPERVISOR.....: Knielsen IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Other TASK ID........: 117 (http://askmonty.org/worklog/?tid=117) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 20 (hours remain) ORIG. ESTIMATE.: 20 PROGRESS NOTES: DESCRIPTION: Since the package building for MariaDB is now fully automated in Buildbot, it has been decided to use packages from Buildbot for the OurDelta apt and yum repositories. This worklog is about fixing/implementing anything that is missing to achieve this. - When doing a real release build, packages/repositories need to be signed, so that users will not get a warning about unauthenticated packages. This signing must only be done on official releases, not on daily builds (to avoid confusing one with the other). - Packages must be uploaded from the Buildbot host. The OurDelta infrastructure has a DropBox share that could be used for this, another option is to simply use rsync. - Ubuntu 10.04 "lucid" has been released, and we need to support that for packages, so the Buildbot VM for lucid must be upgraded to have the official release. - In Ubuntu 10.04, the official MySQL packages include a new package mysql-client-core, we currently have a conflict with this on install that we need to handle somehow. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 28 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 12 May '10

12 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 28 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Wed, 12 May 2010, 06:41)=-=- Started work on a Quilt patch series, refactoring the binlog code to prepare for implementing the group commit, and working on the design of group commit in parallel. Found and fixed several problems in error handling when writing to binlog. Removed redundant table map version locking. Split binlog writing into two parts in preparations for group commit. When ready to write to the binlog, threads enter a queue, and the first thread in the queue handles the binlog writing for everyone. When it obtains the LOCK_log, it first loops over all threads, executing the first part of binlog writing (the write(2) syscall essentially). It then runs the second part (fsync(2) essentially) only once, and then wakes up the remaining threads in the queue. Still to be done: Finish the proof-of-concept group commit patch, by 1) implementing the prepare_fast() and commit_fast() callbacks in handler.cc 2) move the binlog thread enqueue from log_xid() to binlog_prepare_fast(), 3) move fast part of InnoDB commit to innobase_commit_fast(), removing the prepare_commit_mutex(). Write up the final design in this worklog. Evaluate the design to see if we can do better/different. Think about possible next steps, such as releasing innodb row locks early (in innobase_prepare_fast), and doing crash recovery by replaying transactions from the binlog (removing the need for engine durability and 2 of 3 fsync() in commit). Worked 28 hours and estimate 0 hours remain (original estimate increased by 28 hours). -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (igor:2788)
by Igor Babaev 12 May '10

12 May '10

#At lp:maria/5.2 based on revid:psergey@askmonty.org-20100329200940-9ikx6gpww0gtsx00 2788 Igor Babaev 2010-05-11 [merge] Merge 5.1-release -> 5.3 added: mysql-test/include/min_null_cond.inc mysql-test/include/not_binlog_format_row.inc mysql-test/include/view_alias.inc mysql-test/r/bug39022.result mysql-test/r/innodb_bug47621.result mysql-test/r/log_tables_upgrade.result mysql-test/r/no_binlog.result mysql-test/r/partition_debug_sync.result mysql-test/r/plugin_not_embedded.result mysql-test/r/view_alias.result mysql-test/std_data/binlog_savepoint.000001 mysql-test/std_data/bug46565.ARZ mysql-test/std_data/bug46565.frm mysql-test/std_data/bug48265.frm mysql-test/std_data/bug48449.frm mysql-test/std_data/bug49823.CSM mysql-test/std_data/bug49823.CSV mysql-test/std_data/bug49823.frm mysql-test/suite/binlog/t/binlog_index-master.opt mysql-test/suite/engines/ mysql-test/suite/engines/README mysql-test/suite/engines/funcs/ mysql-test/suite/engines/funcs/r/ mysql-test/suite/engines/funcs/r/ai_init_alter_table.result mysql-test/suite/engines/funcs/r/ai_init_create_table.result mysql-test/suite/engines/funcs/r/ai_init_insert.result mysql-test/suite/engines/funcs/r/ai_init_insert_id.result mysql-test/suite/engines/funcs/r/ai_overflow_error.result mysql-test/suite/engines/funcs/r/ai_reset_by_truncate.result mysql-test/suite/engines/funcs/r/ai_sql_auto_is_null.result mysql-test/suite/engines/funcs/r/an_calendar.result mysql-test/suite/engines/funcs/r/an_number.result mysql-test/suite/engines/funcs/r/an_string.result mysql-test/suite/engines/funcs/r/comment_column.result mysql-test/suite/engines/funcs/r/comment_column2.result mysql-test/suite/engines/funcs/r/comment_table.result mysql-test/suite/engines/funcs/r/crash_manycolumns_number.result mysql-test/suite/engines/funcs/r/crash_manycolumns_string.result mysql-test/suite/engines/funcs/r/crash_manyindexes_number.result mysql-test/suite/engines/funcs/r/crash_manyindexes_string.result mysql-test/suite/engines/funcs/r/crash_manytables_number.result mysql-test/suite/engines/funcs/r/crash_manytables_string.result mysql-test/suite/engines/funcs/r/date_function.result mysql-test/suite/engines/funcs/r/datetime_function.result mysql-test/suite/engines/funcs/r/db_alter_character_set.result mysql-test/suite/engines/funcs/r/db_alter_character_set_collate.result mysql-test/suite/engines/funcs/r/db_alter_collate_ascii.result mysql-test/suite/engines/funcs/r/db_alter_collate_utf8.result mysql-test/suite/engines/funcs/r/db_create_character_set.result mysql-test/suite/engines/funcs/r/db_create_character_set_collate.result mysql-test/suite/engines/funcs/r/db_create_drop.result mysql-test/suite/engines/funcs/r/db_create_error.result mysql-test/suite/engines/funcs/r/db_create_error_reserved.result mysql-test/suite/engines/funcs/r/db_create_if_not_exists.result mysql-test/suite/engines/funcs/r/db_drop_error.result mysql-test/suite/engines/funcs/r/db_use_error.result mysql-test/suite/engines/funcs/r/de_autoinc.result mysql-test/suite/engines/funcs/r/de_calendar_range.result mysql-test/suite/engines/funcs/r/de_ignore.result mysql-test/suite/engines/funcs/r/de_limit.result mysql-test/suite/engines/funcs/r/de_multi_db_table.result mysql-test/suite/engines/funcs/r/de_multi_db_table_using.result mysql-test/suite/engines/funcs/r/de_multi_table.result mysql-test/suite/engines/funcs/r/de_multi_table_using.result mysql-test/suite/engines/funcs/r/de_number_range.result mysql-test/suite/engines/funcs/r/de_quick.result mysql-test/suite/engines/funcs/r/de_string_range.result mysql-test/suite/engines/funcs/r/de_truncate.result mysql-test/suite/engines/funcs/r/de_truncate_autoinc.result mysql-test/suite/engines/funcs/r/fu_aggregate_avg_number.result mysql-test/suite/engines/funcs/r/fu_aggregate_count_number.result mysql-test/suite/engines/funcs/r/fu_aggregate_max_number.result mysql-test/suite/engines/funcs/r/fu_aggregate_max_subquery.result mysql-test/suite/engines/funcs/r/fu_aggregate_min_number.result mysql-test/suite/engines/funcs/r/fu_aggregate_sum_number.result mysql-test/suite/engines/funcs/r/general_no_data.result mysql-test/suite/engines/funcs/r/general_not_null.result mysql-test/suite/engines/funcs/r/general_null.result mysql-test/suite/engines/funcs/r/in_calendar_2_unique_constraints_duplicate_update.result mysql-test/suite/engines/funcs/r/in_calendar_pk_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_calendar_pk_constraint_error.result mysql-test/suite/engines/funcs/r/in_calendar_pk_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_calendar_unique_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_calendar_unique_constraint_error.result mysql-test/suite/engines/funcs/r/in_calendar_unique_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_enum_null.result mysql-test/suite/engines/funcs/r/in_enum_null_boundary_error.result mysql-test/suite/engines/funcs/r/in_enum_null_large_error.result mysql-test/suite/engines/funcs/r/in_insert_select.result mysql-test/suite/engines/funcs/r/in_insert_select_autoinc.result mysql-test/suite/engines/funcs/r/in_insert_select_unique_violation.result mysql-test/suite/engines/funcs/r/in_lob_boundary_error.result mysql-test/suite/engines/funcs/r/in_multicolumn_calendar_pk_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_multicolumn_calendar_pk_constraint_error.result mysql-test/suite/engines/funcs/r/in_multicolumn_calendar_pk_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_multicolumn_calendar_unique_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_multicolumn_calendar_unique_constraint_error.result mysql-test/suite/engines/funcs/r/in_multicolumn_calendar_unique_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_multicolumn_number_pk_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_multicolumn_number_pk_constraint_error.result mysql-test/suite/engines/funcs/r/in_multicolumn_number_pk_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_multicolumn_number_unique_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_multicolumn_number_unique_constraint_error.result mysql-test/suite/engines/funcs/r/in_multicolumn_number_unique_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_multicolumn_string_pk_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_multicolumn_string_pk_constraint_error.result mysql-test/suite/engines/funcs/r/in_multicolumn_string_pk_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_multicolumn_string_unique_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_multicolumn_string_unique_constraint_error.result mysql-test/suite/engines/funcs/r/in_multicolumn_string_unique_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_number_2_unique_constraints_duplicate_update.result mysql-test/suite/engines/funcs/r/in_number_boundary_error.result mysql-test/suite/engines/funcs/r/in_number_decimal_boundary_error.result mysql-test/suite/engines/funcs/r/in_number_length.result mysql-test/suite/engines/funcs/r/in_number_null.result mysql-test/suite/engines/funcs/r/in_number_pk_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_number_pk_constraint_error.result mysql-test/suite/engines/funcs/r/in_number_pk_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_number_unique_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_number_unique_constraint_error.result mysql-test/suite/engines/funcs/r/in_number_unique_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_set_null.result mysql-test/suite/engines/funcs/r/in_set_null_boundary_error.result mysql-test/suite/engines/funcs/r/in_set_null_large.result mysql-test/suite/engines/funcs/r/in_string_2_unique_constraints_duplicate_update.result mysql-test/suite/engines/funcs/r/in_string_boundary_error.result mysql-test/suite/engines/funcs/r/in_string_not_null.result mysql-test/suite/engines/funcs/r/in_string_null.result mysql-test/suite/engines/funcs/r/in_string_pk_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_string_pk_constraint_error.result mysql-test/suite/engines/funcs/r/in_string_pk_constraint_ignore.result mysql-test/suite/engines/funcs/r/in_string_set_enum_fail.result mysql-test/suite/engines/funcs/r/in_string_unique_constraint_duplicate_update.result mysql-test/suite/engines/funcs/r/in_string_unique_constraint_error.result mysql-test/suite/engines/funcs/r/in_string_unique_constraint_ignore.result mysql-test/suite/engines/funcs/r/ix_drop.result mysql-test/suite/engines/funcs/r/ix_drop_error.result mysql-test/suite/engines/funcs/r/ix_index_decimals.result mysql-test/suite/engines/funcs/r/ix_index_lob.result mysql-test/suite/engines/funcs/r/ix_index_non_string.result mysql-test/suite/engines/funcs/r/ix_index_string.result mysql-test/suite/engines/funcs/r/ix_index_string_length.result mysql-test/suite/engines/funcs/r/ix_unique_decimals.result mysql-test/suite/engines/funcs/r/ix_unique_lob.result mysql-test/suite/engines/funcs/r/ix_unique_non_string.result mysql-test/suite/engines/funcs/r/ix_unique_string.result mysql-test/suite/engines/funcs/r/ix_unique_string_length.result mysql-test/suite/engines/funcs/r/ix_using_order.result mysql-test/suite/engines/funcs/r/jp_comment_column.result mysql-test/suite/engines/funcs/r/jp_comment_older_compatibility1.result mysql-test/suite/engines/funcs/r/jp_comment_table.result mysql-test/suite/engines/funcs/r/ld_all_number_string_calendar_types.result mysql-test/suite/engines/funcs/r/ld_bit.result mysql-test/suite/engines/funcs/r/ld_enum_set.result mysql-test/suite/engines/funcs/r/ld_less_columns.result mysql-test/suite/engines/funcs/r/ld_more_columns_truncated.result mysql-test/suite/engines/funcs/r/ld_null.result mysql-test/suite/engines/funcs/r/ld_quote.result mysql-test/suite/engines/funcs/r/ld_simple.result mysql-test/suite/engines/funcs/r/ld_starting.result mysql-test/suite/engines/funcs/r/ld_unique_error1.result mysql-test/suite/engines/funcs/r/ld_unique_error1_local.result mysql-test/suite/engines/funcs/r/ld_unique_error2.result mysql-test/suite/engines/funcs/r/ld_unique_error2_local.result mysql-test/suite/engines/funcs/r/ld_unique_error3.result mysql-test/suite/engines/funcs/r/ld_unique_error3_local.result mysql-test/suite/engines/funcs/r/ps_number_length.result mysql-test/suite/engines/funcs/r/ps_number_null.result mysql-test/suite/engines/funcs/r/ps_string_not_null.result mysql-test/suite/engines/funcs/r/ps_string_null.result mysql-test/suite/engines/funcs/r/re_number_range.result mysql-test/suite/engines/funcs/r/re_number_range_set.result mysql-test/suite/engines/funcs/r/re_number_select.result mysql-test/suite/engines/funcs/r/re_string_range.result mysql-test/suite/engines/funcs/r/re_string_range_set.result mysql-test/suite/engines/funcs/r/rpl000010.result mysql-test/suite/engines/funcs/r/rpl000011.result mysql-test/suite/engines/funcs/r/rpl000013.result mysql-test/suite/engines/funcs/r/rpl000017.result mysql-test/suite/engines/funcs/r/rpl_000015.result mysql-test/suite/engines/funcs/r/rpl_LD_INFILE.result mysql-test/suite/engines/funcs/r/rpl_REDIRECT.result mysql-test/suite/engines/funcs/r/rpl_alter.result mysql-test/suite/engines/funcs/r/rpl_alter_db.result mysql-test/suite/engines/funcs/r/rpl_bit.result mysql-test/suite/engines/funcs/r/rpl_bit_npk.result mysql-test/suite/engines/funcs/r/rpl_change_master.result mysql-test/suite/engines/funcs/r/rpl_create_database.result mysql-test/suite/engines/funcs/r/rpl_do_grant.result mysql-test/suite/engines/funcs/r/rpl_drop.result mysql-test/suite/engines/funcs/r/rpl_drop_db.result mysql-test/suite/engines/funcs/r/rpl_dual_pos_advance.result mysql-test/suite/engines/funcs/r/rpl_empty_master_crash.result mysql-test/suite/engines/funcs/r/rpl_err_ignoredtable.result mysql-test/suite/engines/funcs/r/rpl_flushlog_loop.result mysql-test/suite/engines/funcs/r/rpl_free_items.result mysql-test/suite/engines/funcs/r/rpl_get_lock.result mysql-test/suite/engines/funcs/r/rpl_ignore_grant.result mysql-test/suite/engines/funcs/r/rpl_ignore_revoke.result mysql-test/suite/engines/funcs/r/rpl_ignore_table_update.result mysql-test/suite/engines/funcs/r/rpl_init_slave.result mysql-test/suite/engines/funcs/r/rpl_insert.result mysql-test/suite/engines/funcs/r/rpl_insert_select.result mysql-test/suite/engines/funcs/r/rpl_loaddata2.result mysql-test/suite/engines/funcs/r/rpl_loaddata_m.result mysql-test/suite/engines/funcs/r/rpl_loaddata_s.result mysql-test/suite/engines/funcs/r/rpl_loaddatalocal.result mysql-test/suite/engines/funcs/r/rpl_loadfile.result mysql-test/suite/engines/funcs/r/rpl_log_pos.result mysql-test/suite/engines/funcs/r/rpl_many_optimize.result mysql-test/suite/engines/funcs/r/rpl_master_pos_wait.result mysql-test/suite/engines/funcs/r/rpl_misc_functions.result mysql-test/suite/engines/funcs/r/rpl_multi_delete.result mysql-test/suite/engines/funcs/r/rpl_multi_delete2.result mysql-test/suite/engines/funcs/r/rpl_multi_update4.result mysql-test/suite/engines/funcs/r/rpl_ps.result mysql-test/suite/engines/funcs/r/rpl_rbr_to_sbr.result mysql-test/suite/engines/funcs/r/rpl_relayspace.result mysql-test/suite/engines/funcs/r/rpl_replicate_ignore_db.result mysql-test/suite/engines/funcs/r/rpl_row_NOW.result mysql-test/suite/engines/funcs/r/rpl_row_USER.result mysql-test/suite/engines/funcs/r/rpl_row_drop.result mysql-test/suite/engines/funcs/r/rpl_row_func001.result mysql-test/suite/engines/funcs/r/rpl_row_inexist_tbl.result mysql-test/suite/engines/funcs/r/rpl_row_max_relay_size.result mysql-test/suite/engines/funcs/r/rpl_row_reset_slave.result mysql-test/suite/engines/funcs/r/rpl_row_sp001.result mysql-test/suite/engines/funcs/r/rpl_row_sp005.result mysql-test/suite/engines/funcs/r/rpl_row_sp008.result mysql-test/suite/engines/funcs/r/rpl_row_sp009.result mysql-test/suite/engines/funcs/r/rpl_row_sp010.result mysql-test/suite/engines/funcs/r/rpl_row_sp011.result mysql-test/suite/engines/funcs/r/rpl_row_sp012.result mysql-test/suite/engines/funcs/r/rpl_row_stop_middle.result mysql-test/suite/engines/funcs/r/rpl_row_trig001.result mysql-test/suite/engines/funcs/r/rpl_row_trig002.result mysql-test/suite/engines/funcs/r/rpl_row_trig003.result mysql-test/suite/engines/funcs/r/rpl_row_until.result mysql-test/suite/engines/funcs/r/rpl_row_view01.result mysql-test/suite/engines/funcs/r/rpl_server_id1.result mysql-test/suite/engines/funcs/r/rpl_server_id2.result mysql-test/suite/engines/funcs/r/rpl_session_var.result mysql-test/suite/engines/funcs/r/rpl_sf.result mysql-test/suite/engines/funcs/r/rpl_skip_error.result mysql-test/suite/engines/funcs/r/rpl_slave_status.result mysql-test/suite/engines/funcs/r/rpl_sp.result mysql-test/suite/engines/funcs/r/rpl_sp004.result mysql-test/suite/engines/funcs/r/rpl_sp_effects.result mysql-test/suite/engines/funcs/r/rpl_start_stop_slave.result mysql-test/suite/engines/funcs/r/rpl_stm_max_relay_size.result mysql-test/suite/engines/funcs/r/rpl_stm_mystery22.result mysql-test/suite/engines/funcs/r/rpl_stm_no_op.result mysql-test/suite/engines/funcs/r/rpl_stm_reset_slave.result mysql-test/suite/engines/funcs/r/rpl_switch_stm_row_mixed.result mysql-test/suite/engines/funcs/r/rpl_temp_table.result mysql-test/suite/engines/funcs/r/rpl_temporary.result mysql-test/suite/engines/funcs/r/rpl_trigger.result mysql-test/suite/engines/funcs/r/rpl_trunc_temp.result mysql-test/suite/engines/funcs/r/rpl_user_variables.result mysql-test/suite/engines/funcs/r/rpl_variables.result mysql-test/suite/engines/funcs/r/rpl_view.result mysql-test/suite/engines/funcs/r/se_join_cross.result mysql-test/suite/engines/funcs/r/se_join_default.result mysql-test/suite/engines/funcs/r/se_join_inner.result mysql-test/suite/engines/funcs/r/se_join_left.result mysql-test/suite/engines/funcs/r/se_join_left_outer.result mysql-test/suite/engines/funcs/r/se_join_natural_left.result mysql-test/suite/engines/funcs/r/se_join_natural_left_outer.result mysql-test/suite/engines/funcs/r/se_join_natural_right.result mysql-test/suite/engines/funcs/r/se_join_natural_right_outer.result mysql-test/suite/engines/funcs/r/se_join_right.result mysql-test/suite/engines/funcs/r/se_join_right_outer.result mysql-test/suite/engines/funcs/r/se_join_straight.result mysql-test/suite/engines/funcs/r/se_rowid.result mysql-test/suite/engines/funcs/r/se_string_distinct.result mysql-test/suite/engines/funcs/r/se_string_from.result mysql-test/suite/engines/funcs/r/se_string_groupby.result mysql-test/suite/engines/funcs/r/se_string_having.result mysql-test/suite/engines/funcs/r/se_string_limit.result mysql-test/suite/engines/funcs/r/se_string_orderby.result mysql-test/suite/engines/funcs/r/se_string_union.result mysql-test/suite/engines/funcs/r/se_string_where.result mysql-test/suite/engines/funcs/r/se_string_where_and.result mysql-test/suite/engines/funcs/r/se_string_where_or.result mysql-test/suite/engines/funcs/r/sf_alter.result mysql-test/suite/engines/funcs/r/sf_cursor.result mysql-test/suite/engines/funcs/r/sf_simple1.result mysql-test/suite/engines/funcs/r/sp_alter.result mysql-test/suite/engines/funcs/r/sp_cursor.result mysql-test/suite/engines/funcs/r/sp_simple1.result mysql-test/suite/engines/funcs/r/sq_all.result mysql-test/suite/engines/funcs/r/sq_any.result mysql-test/suite/engines/funcs/r/sq_corr.result mysql-test/suite/engines/funcs/r/sq_error.result mysql-test/suite/engines/funcs/r/sq_exists.result mysql-test/suite/engines/funcs/r/sq_from.result mysql-test/suite/engines/funcs/r/sq_in.result mysql-test/suite/engines/funcs/r/sq_row.result mysql-test/suite/engines/funcs/r/sq_scalar.result mysql-test/suite/engines/funcs/r/sq_some.result mysql-test/suite/engines/funcs/r/ta_2part_column_to_pk.result mysql-test/suite/engines/funcs/r/ta_2part_diff_string_to_pk.result mysql-test/suite/engines/funcs/r/ta_2part_diff_to_pk.result mysql-test/suite/engines/funcs/r/ta_2part_string_to_pk.result mysql-test/suite/engines/funcs/r/ta_3part_column_to_pk.result mysql-test/suite/engines/funcs/r/ta_3part_string_to_pk.result mysql-test/suite/engines/funcs/r/ta_add_column.result mysql-test/suite/engines/funcs/r/ta_add_column2.result mysql-test/suite/engines/funcs/r/ta_add_column_first.result mysql-test/suite/engines/funcs/r/ta_add_column_first2.result mysql-test/suite/engines/funcs/r/ta_add_column_middle.result mysql-test/suite/engines/funcs/r/ta_add_column_middle2.result mysql-test/suite/engines/funcs/r/ta_add_string.result mysql-test/suite/engines/funcs/r/ta_add_string2.result mysql-test/suite/engines/funcs/r/ta_add_string_first.result mysql-test/suite/engines/funcs/r/ta_add_string_first2.result mysql-test/suite/engines/funcs/r/ta_add_string_middle.result mysql-test/suite/engines/funcs/r/ta_add_string_middle2.result mysql-test/suite/engines/funcs/r/ta_add_string_unique_index.result mysql-test/suite/engines/funcs/r/ta_add_unique_index.result mysql-test/suite/engines/funcs/r/ta_column_from_unsigned.result mysql-test/suite/engines/funcs/r/ta_column_from_zerofill.result mysql-test/suite/engines/funcs/r/ta_column_to_index.result mysql-test/suite/engines/funcs/r/ta_column_to_not_null.result mysql-test/suite/engines/funcs/r/ta_column_to_null.result mysql-test/suite/engines/funcs/r/ta_column_to_pk.result mysql-test/suite/engines/funcs/r/ta_column_to_unsigned.result mysql-test/suite/engines/funcs/r/ta_column_to_zerofill.result mysql-test/suite/engines/funcs/r/ta_drop_column.result mysql-test/suite/engines/funcs/r/ta_drop_index.result mysql-test/suite/engines/funcs/r/ta_drop_pk_autoincrement.result mysql-test/suite/engines/funcs/r/ta_drop_pk_number.result mysql-test/suite/engines/funcs/r/ta_drop_pk_string.result mysql-test/suite/engines/funcs/r/ta_drop_string_index.result mysql-test/suite/engines/funcs/r/ta_orderby.result mysql-test/suite/engines/funcs/r/ta_rename.result mysql-test/suite/engines/funcs/r/ta_set_drop_default.result mysql-test/suite/engines/funcs/r/ta_string_drop_column.result mysql-test/suite/engines/funcs/r/ta_string_to_index.result mysql-test/suite/engines/funcs/r/ta_string_to_not_null.result mysql-test/suite/engines/funcs/r/ta_string_to_null.result mysql-test/suite/engines/funcs/r/ta_string_to_pk.result mysql-test/suite/engines/funcs/r/tc_column_autoincrement.result mysql-test/suite/engines/funcs/r/tc_column_comment.result mysql-test/suite/engines/funcs/r/tc_column_comment_string.result mysql-test/suite/engines/funcs/r/tc_column_default_decimal.result mysql-test/suite/engines/funcs/r/tc_column_default_number.result mysql-test/suite/engines/funcs/r/tc_column_default_string.result mysql-test/suite/engines/funcs/r/tc_column_enum.result mysql-test/suite/engines/funcs/r/tc_column_enum_long.result mysql-test/suite/engines/funcs/r/tc_column_key.result mysql-test/suite/engines/funcs/r/tc_column_key_length.result mysql-test/suite/engines/funcs/r/tc_column_length.result mysql-test/suite/engines/funcs/r/tc_column_length_decimals.result mysql-test/suite/engines/funcs/r/tc_column_length_zero.result mysql-test/suite/engines/funcs/r/tc_column_not_null.result mysql-test/suite/engines/funcs/r/tc_column_null.result mysql-test/suite/engines/funcs/r/tc_column_primary_key_number.result mysql-test/suite/engines/funcs/r/tc_column_primary_key_string.result mysql-test/suite/engines/funcs/r/tc_column_serial.result mysql-test/suite/engines/funcs/r/tc_column_set.result mysql-test/suite/engines/funcs/r/tc_column_set_long.result mysql-test/suite/engines/funcs/r/tc_column_unique_key.result mysql-test/suite/engines/funcs/r/tc_column_unique_key_string.result mysql-test/suite/engines/funcs/r/tc_column_unsigned.result mysql-test/suite/engines/funcs/r/tc_column_zerofill.result mysql-test/suite/engines/funcs/r/tc_drop_table.result mysql-test/suite/engines/funcs/r/tc_multicolumn_different.result mysql-test/suite/engines/funcs/r/tc_multicolumn_same.result mysql-test/suite/engines/funcs/r/tc_multicolumn_same_string.result mysql-test/suite/engines/funcs/r/tc_partition_analyze.result mysql-test/suite/engines/funcs/r/tc_partition_change_from_range_to_hash_key.result mysql-test/suite/engines/funcs/r/tc_partition_check.result mysql-test/suite/engines/funcs/r/tc_partition_hash.result mysql-test/suite/engines/funcs/r/tc_partition_hash_date_function.result mysql-test/suite/engines/funcs/r/tc_partition_key.result mysql-test/suite/engines/funcs/r/tc_partition_linear_key.result mysql-test/suite/engines/funcs/r/tc_partition_list_directory.result mysql-test/suite/engines/funcs/r/tc_partition_list_error.result mysql-test/suite/engines/funcs/r/tc_partition_optimize.result mysql-test/suite/engines/funcs/r/tc_partition_rebuild.result mysql-test/suite/engines/funcs/r/tc_partition_remove.result mysql-test/suite/engines/funcs/r/tc_partition_reorg_divide.result mysql-test/suite/engines/funcs/r/tc_partition_reorg_hash_key.result mysql-test/suite/engines/funcs/r/tc_partition_reorg_merge.result mysql-test/suite/engines/funcs/r/tc_partition_repair.result mysql-test/suite/engines/funcs/r/tc_partition_sub1.result mysql-test/suite/engines/funcs/r/tc_partition_sub2.result mysql-test/suite/engines/funcs/r/tc_partition_value.result mysql-test/suite/engines/funcs/r/tc_partition_value_error.result mysql-test/suite/engines/funcs/r/tc_partition_value_specific.result mysql-test/suite/engines/funcs/r/tc_rename.result mysql-test/suite/engines/funcs/r/tc_rename_across_database.result mysql-test/suite/engines/funcs/r/tc_rename_error.result mysql-test/suite/engines/funcs/r/tc_structure_comment.result mysql-test/suite/engines/funcs/r/tc_structure_create_like.result mysql-test/suite/engines/funcs/r/tc_structure_create_like_string.result mysql-test/suite/engines/funcs/r/tc_structure_create_select.result mysql-test/suite/engines/funcs/r/tc_structure_create_select_string.result mysql-test/suite/engines/funcs/r/tc_structure_string_comment.result mysql-test/suite/engines/funcs/r/tc_temporary_column.result mysql-test/suite/engines/funcs/r/tc_temporary_column_length.result mysql-test/suite/engines/funcs/r/time_function.result mysql-test/suite/engines/funcs/r/tr_all_type_triggers.result mysql-test/suite/engines/funcs/r/tr_delete.result mysql-test/suite/engines/funcs/r/tr_delete_new_error.result mysql-test/suite/engines/funcs/r/tr_insert.result mysql-test/suite/engines/funcs/r/tr_insert_after_error.result mysql-test/suite/engines/funcs/r/tr_insert_old_error.result mysql-test/suite/engines/funcs/r/tr_update.result mysql-test/suite/engines/funcs/r/tr_update_after_error.result mysql-test/suite/engines/funcs/r/up_calendar_range.result mysql-test/suite/engines/funcs/r/up_ignore.result mysql-test/suite/engines/funcs/r/up_limit.result mysql-test/suite/engines/funcs/r/up_multi_db_table.result mysql-test/suite/engines/funcs/r/up_multi_table.result mysql-test/suite/engines/funcs/r/up_nullcheck.result mysql-test/suite/engines/funcs/r/up_number_range.result mysql-test/suite/engines/funcs/r/up_string_range.result mysql-test/suite/engines/funcs/t/ mysql-test/suite/engines/funcs/t/ai_init_alter_table.test mysql-test/suite/engines/funcs/t/ai_init_create_table.test mysql-test/suite/engines/funcs/t/ai_init_insert.test mysql-test/suite/engines/funcs/t/ai_init_insert_id.test mysql-test/suite/engines/funcs/t/ai_overflow_error.test mysql-test/suite/engines/funcs/t/ai_reset_by_truncate.test mysql-test/suite/engines/funcs/t/ai_sql_auto_is_null.test mysql-test/suite/engines/funcs/t/an_calendar.test mysql-test/suite/engines/funcs/t/an_number.test mysql-test/suite/engines/funcs/t/an_string.test mysql-test/suite/engines/funcs/t/comment_column.test mysql-test/suite/engines/funcs/t/comment_column2.test mysql-test/suite/engines/funcs/t/comment_table.test mysql-test/suite/engines/funcs/t/crash_manycolumns_number.test mysql-test/suite/engines/funcs/t/crash_manycolumns_string.test mysql-test/suite/engines/funcs/t/crash_manyindexes_number.test mysql-test/suite/engines/funcs/t/crash_manyindexes_string.test mysql-test/suite/engines/funcs/t/crash_manytables_number.test mysql-test/suite/engines/funcs/t/crash_manytables_string.test mysql-test/suite/engines/funcs/t/data1.inc mysql-test/suite/engines/funcs/t/data2.inc mysql-test/suite/engines/funcs/t/date_function.test mysql-test/suite/engines/funcs/t/datetime_function.test mysql-test/suite/engines/funcs/t/db_alter_character_set.test mysql-test/suite/engines/funcs/t/db_alter_character_set_collate.test mysql-test/suite/engines/funcs/t/db_alter_collate_ascii.test mysql-test/suite/engines/funcs/t/db_alter_collate_utf8.test mysql-test/suite/engines/funcs/t/db_create_character_set.test mysql-test/suite/engines/funcs/t/db_create_character_set_collate.test mysql-test/suite/engines/funcs/t/db_create_drop.test mysql-test/suite/engines/funcs/t/db_create_error.test mysql-test/suite/engines/funcs/t/db_create_error_reserved.test mysql-test/suite/engines/funcs/t/db_create_if_not_exists.test mysql-test/suite/engines/funcs/t/db_drop_error.test mysql-test/suite/engines/funcs/t/db_use_error.test mysql-test/suite/engines/funcs/t/de_autoinc.test mysql-test/suite/engines/funcs/t/de_calendar_range.test mysql-test/suite/engines/funcs/t/de_ignore.test mysql-test/suite/engines/funcs/t/de_limit.test mysql-test/suite/engines/funcs/t/de_multi_db_table.test mysql-test/suite/engines/funcs/t/de_multi_db_table_using.test mysql-test/suite/engines/funcs/t/de_multi_table.test mysql-test/suite/engines/funcs/t/de_multi_table_using.test mysql-test/suite/engines/funcs/t/de_number_range.test mysql-test/suite/engines/funcs/t/de_quick.test mysql-test/suite/engines/funcs/t/de_string_range.test mysql-test/suite/engines/funcs/t/de_truncate.test mysql-test/suite/engines/funcs/t/de_truncate_autoinc.test mysql-test/suite/engines/funcs/t/disabled.def mysql-test/suite/engines/funcs/t/fu_aggregate_avg_number.test mysql-test/suite/engines/funcs/t/fu_aggregate_count_number.test mysql-test/suite/engines/funcs/t/fu_aggregate_max_number.test mysql-test/suite/engines/funcs/t/fu_aggregate_max_subquery.test mysql-test/suite/engines/funcs/t/fu_aggregate_min_number.test mysql-test/suite/engines/funcs/t/fu_aggregate_sum_number.test mysql-test/suite/engines/funcs/t/general_no_data.test mysql-test/suite/engines/funcs/t/general_not_null.test mysql-test/suite/engines/funcs/t/general_null.test mysql-test/suite/engines/funcs/t/in_calendar_2_unique_constraints_duplicate_update.test mysql-test/suite/engines/funcs/t/in_calendar_pk_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_calendar_pk_constraint_error.test mysql-test/suite/engines/funcs/t/in_calendar_pk_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_calendar_unique_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_calendar_unique_constraint_error.test mysql-test/suite/engines/funcs/t/in_calendar_unique_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_enum_null.test mysql-test/suite/engines/funcs/t/in_enum_null_boundary_error.test mysql-test/suite/engines/funcs/t/in_enum_null_large_error.test mysql-test/suite/engines/funcs/t/in_insert_select.test mysql-test/suite/engines/funcs/t/in_insert_select_autoinc.test mysql-test/suite/engines/funcs/t/in_insert_select_unique_violation.test mysql-test/suite/engines/funcs/t/in_lob_boundary_error.test mysql-test/suite/engines/funcs/t/in_multicolumn_calendar_pk_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_multicolumn_calendar_pk_constraint_error.test mysql-test/suite/engines/funcs/t/in_multicolumn_calendar_pk_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_multicolumn_calendar_unique_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_multicolumn_calendar_unique_constraint_error.test mysql-test/suite/engines/funcs/t/in_multicolumn_calendar_unique_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_multicolumn_number_pk_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_multicolumn_number_pk_constraint_error.test mysql-test/suite/engines/funcs/t/in_multicolumn_number_pk_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_multicolumn_number_unique_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_multicolumn_number_unique_constraint_error.test mysql-test/suite/engines/funcs/t/in_multicolumn_number_unique_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_multicolumn_string_pk_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_multicolumn_string_pk_constraint_error.test mysql-test/suite/engines/funcs/t/in_multicolumn_string_pk_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_multicolumn_string_unique_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_multicolumn_string_unique_constraint_error.test mysql-test/suite/engines/funcs/t/in_multicolumn_string_unique_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_number_2_unique_constraints_duplicate_update.test mysql-test/suite/engines/funcs/t/in_number_boundary_error.test mysql-test/suite/engines/funcs/t/in_number_decimal_boundary_error.test mysql-test/suite/engines/funcs/t/in_number_length.test mysql-test/suite/engines/funcs/t/in_number_null.test mysql-test/suite/engines/funcs/t/in_number_pk_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_number_pk_constraint_error.test mysql-test/suite/engines/funcs/t/in_number_pk_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_number_unique_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_number_unique_constraint_error.test mysql-test/suite/engines/funcs/t/in_number_unique_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_set_null.test mysql-test/suite/engines/funcs/t/in_set_null_boundary_error.test mysql-test/suite/engines/funcs/t/in_set_null_large.test mysql-test/suite/engines/funcs/t/in_string_2_unique_constraints_duplicate_update.test mysql-test/suite/engines/funcs/t/in_string_boundary_error.test mysql-test/suite/engines/funcs/t/in_string_not_null.test mysql-test/suite/engines/funcs/t/in_string_null.test mysql-test/suite/engines/funcs/t/in_string_pk_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_string_pk_constraint_error.test mysql-test/suite/engines/funcs/t/in_string_pk_constraint_ignore.test mysql-test/suite/engines/funcs/t/in_string_set_enum_fail.test mysql-test/suite/engines/funcs/t/in_string_unique_constraint_duplicate_update.test mysql-test/suite/engines/funcs/t/in_string_unique_constraint_error.test mysql-test/suite/engines/funcs/t/in_string_unique_constraint_ignore.test mysql-test/suite/engines/funcs/t/ix_drop.test mysql-test/suite/engines/funcs/t/ix_drop_error.test mysql-test/suite/engines/funcs/t/ix_index_decimals.test mysql-test/suite/engines/funcs/t/ix_index_lob.test mysql-test/suite/engines/funcs/t/ix_index_non_string.test mysql-test/suite/engines/funcs/t/ix_index_string.test mysql-test/suite/engines/funcs/t/ix_index_string_length.test mysql-test/suite/engines/funcs/t/ix_unique_decimals.test mysql-test/suite/engines/funcs/t/ix_unique_lob.test mysql-test/suite/engines/funcs/t/ix_unique_non_string.test mysql-test/suite/engines/funcs/t/ix_unique_string.test mysql-test/suite/engines/funcs/t/ix_unique_string_length.test mysql-test/suite/engines/funcs/t/ix_using_order.test mysql-test/suite/engines/funcs/t/jp_comment_column.test mysql-test/suite/engines/funcs/t/jp_comment_older_compatibility1.test mysql-test/suite/engines/funcs/t/jp_comment_table.test mysql-test/suite/engines/funcs/t/ld_all_number_string_calendar_types.test mysql-test/suite/engines/funcs/t/ld_bit.test mysql-test/suite/engines/funcs/t/ld_enum_set.test mysql-test/suite/engines/funcs/t/ld_less_columns.test mysql-test/suite/engines/funcs/t/ld_more_columns_truncated.test mysql-test/suite/engines/funcs/t/ld_null.test mysql-test/suite/engines/funcs/t/ld_quote.test mysql-test/suite/engines/funcs/t/ld_simple.test mysql-test/suite/engines/funcs/t/ld_starting.test mysql-test/suite/engines/funcs/t/ld_unique_error1.test mysql-test/suite/engines/funcs/t/ld_unique_error1_local.test mysql-test/suite/engines/funcs/t/ld_unique_error2.test mysql-test/suite/engines/funcs/t/ld_unique_error2_local.test mysql-test/suite/engines/funcs/t/ld_unique_error3.test mysql-test/suite/engines/funcs/t/ld_unique_error3_local.test mysql-test/suite/engines/funcs/t/load_bit.inc mysql-test/suite/engines/funcs/t/load_enum_set.inc mysql-test/suite/engines/funcs/t/load_less_columns.inc mysql-test/suite/engines/funcs/t/load_more_columns.inc mysql-test/suite/engines/funcs/t/load_null.inc mysql-test/suite/engines/funcs/t/load_null2.inc mysql-test/suite/engines/funcs/t/load_quote.inc mysql-test/suite/engines/funcs/t/load_simple.inc mysql-test/suite/engines/funcs/t/load_starting.inc mysql-test/suite/engines/funcs/t/load_unique_error1.inc mysql-test/suite/engines/funcs/t/load_unique_error2.inc mysql-test/suite/engines/funcs/t/load_unique_error3.inc mysql-test/suite/engines/funcs/t/ps_number_length.test mysql-test/suite/engines/funcs/t/ps_number_null.test mysql-test/suite/engines/funcs/t/ps_string_not_null.test mysql-test/suite/engines/funcs/t/ps_string_null.test mysql-test/suite/engines/funcs/t/re_number_range.test mysql-test/suite/engines/funcs/t/re_number_range_set.test mysql-test/suite/engines/funcs/t/re_number_select.test mysql-test/suite/engines/funcs/t/re_string_range.test mysql-test/suite/engines/funcs/t/re_string_range_set.test mysql-test/suite/engines/funcs/t/rpl000010-slave.opt mysql-test/suite/engines/funcs/t/rpl000010.test mysql-test/suite/engines/funcs/t/rpl000011.test mysql-test/suite/engines/funcs/t/rpl000013.test mysql-test/suite/engines/funcs/t/rpl000017-slave.opt mysql-test/suite/engines/funcs/t/rpl000017.test mysql-test/suite/engines/funcs/t/rpl_000015.test mysql-test/suite/engines/funcs/t/rpl_LD_INFILE.test mysql-test/suite/engines/funcs/t/rpl_REDIRECT.test mysql-test/suite/engines/funcs/t/rpl_alter.test mysql-test/suite/engines/funcs/t/rpl_alter_db.test mysql-test/suite/engines/funcs/t/rpl_bit.test mysql-test/suite/engines/funcs/t/rpl_bit_npk.test mysql-test/suite/engines/funcs/t/rpl_change_master.test mysql-test/suite/engines/funcs/t/rpl_create_database-master.opt mysql-test/suite/engines/funcs/t/rpl_create_database-slave.opt mysql-test/suite/engines/funcs/t/rpl_create_database.test mysql-test/suite/engines/funcs/t/rpl_do_grant.test mysql-test/suite/engines/funcs/t/rpl_drop.test mysql-test/suite/engines/funcs/t/rpl_drop_db.test mysql-test/suite/engines/funcs/t/rpl_dual_pos_advance-master.opt mysql-test/suite/engines/funcs/t/rpl_dual_pos_advance.test mysql-test/suite/engines/funcs/t/rpl_empty_master_crash-master.opt mysql-test/suite/engines/funcs/t/rpl_empty_master_crash.test mysql-test/suite/engines/funcs/t/rpl_err_ignoredtable-slave.opt mysql-test/suite/engines/funcs/t/rpl_err_ignoredtable.test mysql-test/suite/engines/funcs/t/rpl_flushlog_loop.test mysql-test/suite/engines/funcs/t/rpl_free_items-slave.opt mysql-test/suite/engines/funcs/t/rpl_free_items.test mysql-test/suite/engines/funcs/t/rpl_get_lock.test mysql-test/suite/engines/funcs/t/rpl_ignore_grant-slave.opt mysql-test/suite/engines/funcs/t/rpl_ignore_grant.test mysql-test/suite/engines/funcs/t/rpl_ignore_revoke-slave.opt mysql-test/suite/engines/funcs/t/rpl_ignore_revoke.test mysql-test/suite/engines/funcs/t/rpl_ignore_table_update-slave.opt mysql-test/suite/engines/funcs/t/rpl_ignore_table_update.test mysql-test/suite/engines/funcs/t/rpl_init_slave-slave.opt mysql-test/suite/engines/funcs/t/rpl_init_slave.test mysql-test/suite/engines/funcs/t/rpl_insert.test mysql-test/suite/engines/funcs/t/rpl_insert_select.test mysql-test/suite/engines/funcs/t/rpl_loaddata2.test mysql-test/suite/engines/funcs/t/rpl_loaddata_m-master.opt mysql-test/suite/engines/funcs/t/rpl_loaddata_m.test mysql-test/suite/engines/funcs/t/rpl_loaddata_s-slave.opt mysql-test/suite/engines/funcs/t/rpl_loaddata_s.test mysql-test/suite/engines/funcs/t/rpl_loaddatalocal.test mysql-test/suite/engines/funcs/t/rpl_loadfile.test mysql-test/suite/engines/funcs/t/rpl_log_pos.test mysql-test/suite/engines/funcs/t/rpl_many_optimize.test mysql-test/suite/engines/funcs/t/rpl_master_pos_wait.test mysql-test/suite/engines/funcs/t/rpl_misc_functions.test mysql-test/suite/engines/funcs/t/rpl_multi_delete-slave.opt mysql-test/suite/engines/funcs/t/rpl_multi_delete.test mysql-test/suite/engines/funcs/t/rpl_multi_delete2-slave.opt mysql-test/suite/engines/funcs/t/rpl_multi_delete2.test mysql-test/suite/engines/funcs/t/rpl_multi_update4-slave.opt mysql-test/suite/engines/funcs/t/rpl_multi_update4.test mysql-test/suite/engines/funcs/t/rpl_ps.test mysql-test/suite/engines/funcs/t/rpl_rbr_to_sbr.test mysql-test/suite/engines/funcs/t/rpl_relayspace-slave.opt mysql-test/suite/engines/funcs/t/rpl_relayspace.test mysql-test/suite/engines/funcs/t/rpl_replicate_ignore_db-slave.opt mysql-test/suite/engines/funcs/t/rpl_replicate_ignore_db.test mysql-test/suite/engines/funcs/t/rpl_row_NOW.test mysql-test/suite/engines/funcs/t/rpl_row_USER.test mysql-test/suite/engines/funcs/t/rpl_row_drop.test mysql-test/suite/engines/funcs/t/rpl_row_func001.test mysql-test/suite/engines/funcs/t/rpl_row_inexist_tbl-slave.opt mysql-test/suite/engines/funcs/t/rpl_row_inexist_tbl.test mysql-test/suite/engines/funcs/t/rpl_row_max_relay_size.test mysql-test/suite/engines/funcs/t/rpl_row_reset_slave.test mysql-test/suite/engines/funcs/t/rpl_row_sp001.test mysql-test/suite/engines/funcs/t/rpl_row_sp005.test mysql-test/suite/engines/funcs/t/rpl_row_sp008.test mysql-test/suite/engines/funcs/t/rpl_row_sp009.test mysql-test/suite/engines/funcs/t/rpl_row_sp010.test mysql-test/suite/engines/funcs/t/rpl_row_sp011.test mysql-test/suite/engines/funcs/t/rpl_row_sp012.test mysql-test/suite/engines/funcs/t/rpl_row_stop_middle.test mysql-test/suite/engines/funcs/t/rpl_row_trig001.test mysql-test/suite/engines/funcs/t/rpl_row_trig002.test mysql-test/suite/engines/funcs/t/rpl_row_trig003.test mysql-test/suite/engines/funcs/t/rpl_row_until.test mysql-test/suite/engines/funcs/t/rpl_row_view01.test mysql-test/suite/engines/funcs/t/rpl_server_id1.test mysql-test/suite/engines/funcs/t/rpl_server_id2-slave.opt mysql-test/suite/engines/funcs/t/rpl_server_id2.test mysql-test/suite/engines/funcs/t/rpl_session_var.test mysql-test/suite/engines/funcs/t/rpl_sf.test mysql-test/suite/engines/funcs/t/rpl_skip_error-slave.opt mysql-test/suite/engines/funcs/t/rpl_skip_error.test mysql-test/suite/engines/funcs/t/rpl_slave_status.test mysql-test/suite/engines/funcs/t/rpl_sp-master.opt mysql-test/suite/engines/funcs/t/rpl_sp-slave.opt mysql-test/suite/engines/funcs/t/rpl_sp.test mysql-test/suite/engines/funcs/t/rpl_sp004.test mysql-test/suite/engines/funcs/t/rpl_sp_effects-master.opt mysql-test/suite/engines/funcs/t/rpl_sp_effects-slave.opt mysql-test/suite/engines/funcs/t/rpl_sp_effects.test mysql-test/suite/engines/funcs/t/rpl_start_stop_slave.test mysql-test/suite/engines/funcs/t/rpl_stm_max_relay_size.test mysql-test/suite/engines/funcs/t/rpl_stm_mystery22.test mysql-test/suite/engines/funcs/t/rpl_stm_no_op.test mysql-test/suite/engines/funcs/t/rpl_stm_reset_slave.test mysql-test/suite/engines/funcs/t/rpl_switch_stm_row_mixed.test mysql-test/suite/engines/funcs/t/rpl_temp_table.test mysql-test/suite/engines/funcs/t/rpl_temporary.test mysql-test/suite/engines/funcs/t/rpl_trigger.test mysql-test/suite/engines/funcs/t/rpl_trunc_temp.test mysql-test/suite/engines/funcs/t/rpl_user_variables.test mysql-test/suite/engines/funcs/t/rpl_variables-master.opt mysql-test/suite/engines/funcs/t/rpl_variables.test mysql-test/suite/engines/funcs/t/rpl_view-slave.opt mysql-test/suite/engines/funcs/t/rpl_view.test mysql-test/suite/engines/funcs/t/se_join_cross.test mysql-test/suite/engines/funcs/t/se_join_default.test mysql-test/suite/engines/funcs/t/se_join_inner.test mysql-test/suite/engines/funcs/t/se_join_left.test mysql-test/suite/engines/funcs/t/se_join_left_outer.test mysql-test/suite/engines/funcs/t/se_join_natural_left.test mysql-test/suite/engines/funcs/t/se_join_natural_left_outer.test mysql-test/suite/engines/funcs/t/se_join_natural_right.test mysql-test/suite/engines/funcs/t/se_join_natural_right_outer.test mysql-test/suite/engines/funcs/t/se_join_right.test mysql-test/suite/engines/funcs/t/se_join_right_outer.test mysql-test/suite/engines/funcs/t/se_join_straight.test mysql-test/suite/engines/funcs/t/se_rowid.test mysql-test/suite/engines/funcs/t/se_string_distinct.test mysql-test/suite/engines/funcs/t/se_string_from.test mysql-test/suite/engines/funcs/t/se_string_groupby.test mysql-test/suite/engines/funcs/t/se_string_having.test mysql-test/suite/engines/funcs/t/se_string_limit.test mysql-test/suite/engines/funcs/t/se_string_orderby.test mysql-test/suite/engines/funcs/t/se_string_union.test mysql-test/suite/engines/funcs/t/se_string_where.test mysql-test/suite/engines/funcs/t/se_string_where_and.test mysql-test/suite/engines/funcs/t/se_string_where_or.test mysql-test/suite/engines/funcs/t/sf_alter.test mysql-test/suite/engines/funcs/t/sf_cursor.test mysql-test/suite/engines/funcs/t/sf_simple1.test mysql-test/suite/engines/funcs/t/sp_alter.test mysql-test/suite/engines/funcs/t/sp_cursor.test mysql-test/suite/engines/funcs/t/sp_simple1.test mysql-test/suite/engines/funcs/t/sq_all.test mysql-test/suite/engines/funcs/t/sq_any.test mysql-test/suite/engines/funcs/t/sq_corr.test mysql-test/suite/engines/funcs/t/sq_error.test mysql-test/suite/engines/funcs/t/sq_exists.test mysql-test/suite/engines/funcs/t/sq_from.test mysql-test/suite/engines/funcs/t/sq_in.test mysql-test/suite/engines/funcs/t/sq_row.test mysql-test/suite/engines/funcs/t/sq_scalar.test mysql-test/suite/engines/funcs/t/sq_some.test mysql-test/suite/engines/funcs/t/ta_2part_column_to_pk.test mysql-test/suite/engines/funcs/t/ta_2part_diff_string_to_pk.test mysql-test/suite/engines/funcs/t/ta_2part_diff_to_pk.test mysql-test/suite/engines/funcs/t/ta_2part_string_to_pk.test mysql-test/suite/engines/funcs/t/ta_3part_column_to_pk.test mysql-test/suite/engines/funcs/t/ta_3part_string_to_pk.test mysql-test/suite/engines/funcs/t/ta_add_column.test mysql-test/suite/engines/funcs/t/ta_add_column2.test mysql-test/suite/engines/funcs/t/ta_add_column_first.test mysql-test/suite/engines/funcs/t/ta_add_column_first2.test mysql-test/suite/engines/funcs/t/ta_add_column_middle.test mysql-test/suite/engines/funcs/t/ta_add_column_middle2.test mysql-test/suite/engines/funcs/t/ta_add_string.test mysql-test/suite/engines/funcs/t/ta_add_string2.test mysql-test/suite/engines/funcs/t/ta_add_string_first.test mysql-test/suite/engines/funcs/t/ta_add_string_first2.test mysql-test/suite/engines/funcs/t/ta_add_string_middle.test mysql-test/suite/engines/funcs/t/ta_add_string_middle2.test mysql-test/suite/engines/funcs/t/ta_add_string_unique_index.test mysql-test/suite/engines/funcs/t/ta_add_unique_index.test mysql-test/suite/engines/funcs/t/ta_column_from_unsigned.test mysql-test/suite/engines/funcs/t/ta_column_from_zerofill.test mysql-test/suite/engines/funcs/t/ta_column_to_index.test mysql-test/suite/engines/funcs/t/ta_column_to_not_null.test mysql-test/suite/engines/funcs/t/ta_column_to_null.test mysql-test/suite/engines/funcs/t/ta_column_to_pk.test mysql-test/suite/engines/funcs/t/ta_column_to_unsigned.test mysql-test/suite/engines/funcs/t/ta_column_to_zerofill.test mysql-test/suite/engines/funcs/t/ta_drop_column.test mysql-test/suite/engines/funcs/t/ta_drop_index.test mysql-test/suite/engines/funcs/t/ta_drop_pk_autoincrement.test mysql-test/suite/engines/funcs/t/ta_drop_pk_number.test mysql-test/suite/engines/funcs/t/ta_drop_pk_string.test mysql-test/suite/engines/funcs/t/ta_drop_string_index.test mysql-test/suite/engines/funcs/t/ta_orderby.test mysql-test/suite/engines/funcs/t/ta_rename.test mysql-test/suite/engines/funcs/t/ta_set_drop_default.test mysql-test/suite/engines/funcs/t/ta_string_drop_column.test mysql-test/suite/engines/funcs/t/ta_string_to_index.test mysql-test/suite/engines/funcs/t/ta_string_to_not_null.test mysql-test/suite/engines/funcs/t/ta_string_to_null.test mysql-test/suite/engines/funcs/t/ta_string_to_pk.test mysql-test/suite/engines/funcs/t/tc_column_autoincrement.test mysql-test/suite/engines/funcs/t/tc_column_comment.test mysql-test/suite/engines/funcs/t/tc_column_comment_string.test mysql-test/suite/engines/funcs/t/tc_column_default_decimal.test mysql-test/suite/engines/funcs/t/tc_column_default_number.test mysql-test/suite/engines/funcs/t/tc_column_default_string.test mysql-test/suite/engines/funcs/t/tc_column_enum.test mysql-test/suite/engines/funcs/t/tc_column_enum_long.test mysql-test/suite/engines/funcs/t/tc_column_key.test mysql-test/suite/engines/funcs/t/tc_column_key_length.test mysql-test/suite/engines/funcs/t/tc_column_length.test mysql-test/suite/engines/funcs/t/tc_column_length_decimals.test mysql-test/suite/engines/funcs/t/tc_column_length_zero.test mysql-test/suite/engines/funcs/t/tc_column_not_null.test mysql-test/suite/engines/funcs/t/tc_column_null.test mysql-test/suite/engines/funcs/t/tc_column_primary_key_number.test mysql-test/suite/engines/funcs/t/tc_column_primary_key_string.test mysql-test/suite/engines/funcs/t/tc_column_serial.test mysql-test/suite/engines/funcs/t/tc_column_set.test mysql-test/suite/engines/funcs/t/tc_column_set_long.test mysql-test/suite/engines/funcs/t/tc_column_unique_key.test mysql-test/suite/engines/funcs/t/tc_column_unique_key_string.test mysql-test/suite/engines/funcs/t/tc_column_unsigned.test mysql-test/suite/engines/funcs/t/tc_column_zerofill.test mysql-test/suite/engines/funcs/t/tc_drop_table.test mysql-test/suite/engines/funcs/t/tc_multicolumn_different.test mysql-test/suite/engines/funcs/t/tc_multicolumn_same.test mysql-test/suite/engines/funcs/t/tc_multicolumn_same_string.test mysql-test/suite/engines/funcs/t/tc_partition_analyze.test mysql-test/suite/engines/funcs/t/tc_partition_change_from_range_to_hash_key.test mysql-test/suite/engines/funcs/t/tc_partition_check.test mysql-test/suite/engines/funcs/t/tc_partition_hash.test mysql-test/suite/engines/funcs/t/tc_partition_hash_date_function.test mysql-test/suite/engines/funcs/t/tc_partition_key.test mysql-test/suite/engines/funcs/t/tc_partition_linear_key.test mysql-test/suite/engines/funcs/t/tc_partition_list_directory.test mysql-test/suite/engines/funcs/t/tc_partition_list_error.test mysql-test/suite/engines/funcs/t/tc_partition_optimize.test mysql-test/suite/engines/funcs/t/tc_partition_rebuild.test mysql-test/suite/engines/funcs/t/tc_partition_remove.test mysql-test/suite/engines/funcs/t/tc_partition_reorg_divide.test mysql-test/suite/engines/funcs/t/tc_partition_reorg_hash_key.test mysql-test/suite/engines/funcs/t/tc_partition_reorg_merge.test mysql-test/suite/engines/funcs/t/tc_partition_repair.test mysql-test/suite/engines/funcs/t/tc_partition_sub1.test mysql-test/suite/engines/funcs/t/tc_partition_sub2.test mysql-test/suite/engines/funcs/t/tc_partition_value.test mysql-test/suite/engines/funcs/t/tc_partition_value_error.test mysql-test/suite/engines/funcs/t/tc_partition_value_specific.test mysql-test/suite/engines/funcs/t/tc_rename.test mysql-test/suite/engines/funcs/t/tc_rename_across_database.test mysql-test/suite/engines/funcs/t/tc_rename_error.test mysql-test/suite/engines/funcs/t/tc_structure_comment.test mysql-test/suite/engines/funcs/t/tc_structure_create_like.test mysql-test/suite/engines/funcs/t/tc_structure_create_like_string.test mysql-test/suite/engines/funcs/t/tc_structure_create_select.test mysql-test/suite/engines/funcs/t/tc_structure_create_select_string.test mysql-test/suite/engines/funcs/t/tc_structure_string_comment.test mysql-test/suite/engines/funcs/t/tc_temporary_column.test mysql-test/suite/engines/funcs/t/tc_temporary_column_length.test mysql-test/suite/engines/funcs/t/time_function.test mysql-test/suite/engines/funcs/t/tr_all_type_triggers.test mysql-test/suite/engines/funcs/t/tr_delete.test mysql-test/suite/engines/funcs/t/tr_delete_new_error.test mysql-test/suite/engines/funcs/t/tr_insert.test mysql-test/suite/engines/funcs/t/tr_insert_after_error.test mysql-test/suite/engines/funcs/t/tr_insert_old_error.test mysql-test/suite/engines/funcs/t/tr_update.test mysql-test/suite/engines/funcs/t/tr_update_after_error.test mysql-test/suite/engines/funcs/t/up_calendar_range.test mysql-test/suite/engines/funcs/t/up_ignore.test mysql-test/suite/engines/funcs/t/up_limit.test mysql-test/suite/engines/funcs/t/up_multi_db_table.test mysql-test/suite/engines/funcs/t/up_multi_table.test mysql-test/suite/engines/funcs/t/up_nullcheck.test mysql-test/suite/engines/funcs/t/up_number_range.test mysql-test/suite/engines/funcs/t/up_string_range.test mysql-test/suite/engines/funcs/t/wait_show_pattern.inc mysql-test/suite/engines/funcs/t/wait_slave_status.inc mysql-test/suite/engines/iuds/ mysql-test/suite/engines/iuds/r/ mysql-test/suite/engines/iuds/r/delete_decimal.result mysql-test/suite/engines/iuds/r/delete_time.result mysql-test/suite/engines/iuds/r/delete_year.result mysql-test/suite/engines/iuds/r/insert_calendar.result mysql-test/suite/engines/iuds/r/insert_decimal.result mysql-test/suite/engines/iuds/r/insert_number.result mysql-test/suite/engines/iuds/r/insert_time.result mysql-test/suite/engines/iuds/r/insert_year.result mysql-test/suite/engines/iuds/r/strings_charsets_update_delete.result mysql-test/suite/engines/iuds/r/strings_update_delete.result mysql-test/suite/engines/iuds/r/type_bit_iuds.result mysql-test/suite/engines/iuds/r/update_decimal.result mysql-test/suite/engines/iuds/r/update_delete_calendar.result mysql-test/suite/engines/iuds/r/update_delete_number.result mysql-test/suite/engines/iuds/r/update_time.result mysql-test/suite/engines/iuds/r/update_year.result mysql-test/suite/engines/iuds/t/ mysql-test/suite/engines/iuds/t/delete_decimal.test mysql-test/suite/engines/iuds/t/delete_time.test mysql-test/suite/engines/iuds/t/delete_year.test mysql-test/suite/engines/iuds/t/disabled.def mysql-test/suite/engines/iuds/t/hindi.txt mysql-test/suite/engines/iuds/t/insert_calendar.test mysql-test/suite/engines/iuds/t/insert_decimal.test mysql-test/suite/engines/iuds/t/insert_number.test mysql-test/suite/engines/iuds/t/insert_time.test mysql-test/suite/engines/iuds/t/insert_year.test mysql-test/suite/engines/iuds/t/sample.txt mysql-test/suite/engines/iuds/t/strings_charsets_update_delete.test mysql-test/suite/engines/iuds/t/strings_update_delete.test mysql-test/suite/engines/iuds/t/type_bit_iuds.test mysql-test/suite/engines/iuds/t/update_decimal.test mysql-test/suite/engines/iuds/t/update_delete_calendar.test mysql-test/suite/engines/iuds/t/update_delete_number.test mysql-test/suite/engines/iuds/t/update_time.test mysql-test/suite/engines/iuds/t/update_year.test mysql-test/suite/engines/rr_trx/ mysql-test/suite/engines/rr_trx/check_consistency.sql mysql-test/suite/engines/rr_trx/include/ mysql-test/suite/engines/rr_trx/include/check_for_error_rollback.inc mysql-test/suite/engines/rr_trx/include/check_for_error_rollback_skip.inc mysql-test/suite/engines/rr_trx/include/check_repeatable_read_all_columns.inc mysql-test/suite/engines/rr_trx/include/record_query_all_columns.inc mysql-test/suite/engines/rr_trx/include/rr_init.test mysql-test/suite/engines/rr_trx/init_innodb.txt mysql-test/suite/engines/rr_trx/r/ mysql-test/suite/engines/rr_trx/r/init_innodb.result mysql-test/suite/engines/rr_trx/r/rr_c_count_not_zero.result mysql-test/suite/engines/rr_trx/r/rr_c_stats.result mysql-test/suite/engines/rr_trx/r/rr_i_40-44.result mysql-test/suite/engines/rr_trx/r/rr_id_3.result mysql-test/suite/engines/rr_trx/r/rr_id_900.result mysql-test/suite/engines/rr_trx/r/rr_insert_select_2.result mysql-test/suite/engines/rr_trx/r/rr_iud_rollback-multi-50.result mysql-test/suite/engines/rr_trx/r/rr_replace_7-8.result mysql-test/suite/engines/rr_trx/r/rr_s_select-uncommitted.result mysql-test/suite/engines/rr_trx/r/rr_sc_select-limit-nolimit_4.result mysql-test/suite/engines/rr_trx/r/rr_sc_select-same_2.result mysql-test/suite/engines/rr_trx/r/rr_sc_sum_total.result mysql-test/suite/engines/rr_trx/r/rr_u_10-19.result mysql-test/suite/engines/rr_trx/r/rr_u_10-19_nolimit.result mysql-test/suite/engines/rr_trx/r/rr_u_4.result mysql-test/suite/engines/rr_trx/run.txt mysql-test/suite/engines/rr_trx/run_stress_tx_rr.pl mysql-test/suite/engines/rr_trx/t/ mysql-test/suite/engines/rr_trx/t/init_innodb.test mysql-test/suite/engines/rr_trx/t/rr_c_count_not_zero.test mysql-test/suite/engines/rr_trx/t/rr_c_stats.test mysql-test/suite/engines/rr_trx/t/rr_i_40-44.test mysql-test/suite/engines/rr_trx/t/rr_id_3.test mysql-test/suite/engines/rr_trx/t/rr_id_900.test mysql-test/suite/engines/rr_trx/t/rr_insert_select_2.test mysql-test/suite/engines/rr_trx/t/rr_iud_rollback-multi-50.test mysql-test/suite/engines/rr_trx/t/rr_replace_7-8.test mysql-test/suite/engines/rr_trx/t/rr_s_select-uncommitted.test mysql-test/suite/engines/rr_trx/t/rr_sc_select-limit-nolimit_4.test mysql-test/suite/engines/rr_trx/t/rr_sc_select-same_2.test mysql-test/suite/engines/rr_trx/t/rr_sc_sum_total.test mysql-test/suite/engines/rr_trx/t/rr_u_10-19.test mysql-test/suite/engines/rr_trx/t/rr_u_10-19_nolimit.test mysql-test/suite/engines/rr_trx/t/rr_u_4.test mysql-test/suite/innodb/r/innodb_bug47622.result mysql-test/suite/innodb/r/innodb_bug51378.result mysql-test/suite/innodb/t/innodb_bug47622.test mysql-test/suite/innodb/t/innodb_bug51378.test mysql-test/suite/pbxt/r/pbxt_xa.result mysql-test/suite/pbxt/t/multi_statement-master.opt mysql-test/suite/pbxt/t/pbxt_xa.test mysql-test/suite/pbxt/t/suite.opt mysql-test/suite/rpl/r/rpl_show_slave_running.result mysql-test/suite/rpl/r/rpl_slow_query_log.result mysql-test/suite/rpl/r/rpl_stm_sql_mode.result mysql-test/suite/rpl/r/rpl_typeconv_innodb.result mysql-test/suite/rpl/t/rpl_begin_commit_rollback-master.opt mysql-test/suite/rpl/t/rpl_show_slave_running.test mysql-test/suite/rpl/t/rpl_slow_query_log-slave.opt mysql-test/suite/rpl/t/rpl_slow_query_log.test mysql-test/suite/rpl/t/rpl_stm_sql_mode.test mysql-test/suite/rpl/t/rpl_typeconv-slave.opt mysql-test/suite/rpl/t/rpl_typeconv_innodb.test mysql-test/t/bug39022.test mysql-test/t/innodb_bug47621.test mysql-test/t/log_tables_upgrade.test mysql-test/t/no_binlog.test mysql-test/t/partition_debug_sync.test mysql-test/t/plugin_not_embedded-master.opt mysql-test/t/plugin_not_embedded.test mysql-test/t/view_alias.test storage/innodb_plugin/include/ut0rbt.h storage/innodb_plugin/ut/ut0rbt.c storage/xtradb/build/ storage/xtradb/build/debian/ storage/xtradb/build/debian/README.Maintainer storage/xtradb/build/debian/additions/ storage/xtradb/build/debian/additions/Docs__Images__Makefile.in storage/xtradb/build/debian/additions/Docs__Makefile.in storage/xtradb/build/debian/additions/debian-start storage/xtradb/build/debian/additions/debian-start.inc.sh storage/xtradb/build/debian/additions/echo_stderr storage/xtradb/build/debian/additions/innotop/ storage/xtradb/build/debian/additions/innotop/InnoDBParser.pm storage/xtradb/build/debian/additions/innotop/changelog.innotop storage/xtradb/build/debian/additions/innotop/innotop storage/xtradb/build/debian/additions/innotop/innotop.1 storage/xtradb/build/debian/additions/msql2mysql.1 storage/xtradb/build/debian/additions/my.cnf storage/xtradb/build/debian/additions/my_print_defaults.1 storage/xtradb/build/debian/additions/myisam_ftdump.1 storage/xtradb/build/debian/additions/myisamchk.1 storage/xtradb/build/debian/additions/myisamlog.1 storage/xtradb/build/debian/additions/myisampack.1 storage/xtradb/build/debian/additions/mysql-server.lintian-overrides storage/xtradb/build/debian/additions/mysql_config.1 storage/xtradb/build/debian/additions/mysql_convert_table_format.1 storage/xtradb/build/debian/additions/mysql_find_rows.1 storage/xtradb/build/debian/additions/mysql_fix_extensions.1 storage/xtradb/build/debian/additions/mysql_install_db.1 storage/xtradb/build/debian/additions/mysql_secure_installation.1 storage/xtradb/build/debian/additions/mysql_setpermission.1 storage/xtradb/build/debian/additions/mysql_tableinfo.1 storage/xtradb/build/debian/additions/mysql_waitpid.1 storage/xtradb/build/debian/additions/mysqlbinlog.1 storage/xtradb/build/debian/additions/mysqlbug.1 storage/xtradb/build/debian/additions/mysqlcheck.1 storage/xtradb/build/debian/additions/mysqld_safe_syslog.cnf storage/xtradb/build/debian/additions/mysqldumpslow.1 storage/xtradb/build/debian/additions/mysqlimport.1 storage/xtradb/build/debian/additions/mysqlmanager.1 storage/xtradb/build/debian/additions/mysqlreport storage/xtradb/build/debian/additions/mysqlreport.1 storage/xtradb/build/debian/additions/mysqltest.1 storage/xtradb/build/debian/additions/pack_isam.1 storage/xtradb/build/debian/additions/resolve_stack_dump.1 storage/xtradb/build/debian/additions/resolveip.1 storage/xtradb/build/debian/changelog storage/xtradb/build/debian/compat storage/xtradb/build/debian/control storage/xtradb/build/debian/copyright storage/xtradb/build/debian/libpercona-xtradb-client-dev.README.Maintainer storage/xtradb/build/debian/libpercona-xtradb-client-dev.dirs storage/xtradb/build/debian/libpercona-xtradb-client-dev.docs storage/xtradb/build/debian/libpercona-xtradb-client-dev.examples storage/xtradb/build/debian/libpercona-xtradb-client-dev.files storage/xtradb/build/debian/libpercona-xtradb-client-dev.links storage/xtradb/build/debian/libpercona-xtradb-client16.dirs storage/xtradb/build/debian/libpercona-xtradb-client16.docs storage/xtradb/build/debian/libpercona-xtradb-client16.files storage/xtradb/build/debian/libpercona-xtradb-client16.postinst storage/xtradb/build/debian/patches/ storage/xtradb/build/debian/patches/00list storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Images_Makefile.in.dpatch storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Makefile.in.dpatch storage/xtradb/build/debian/patches/33_scripts__mysql_create_system_tables__no_test.dpatch storage/xtradb/build/debian/patches/38_scripts__mysqld_safe.sh__signals.dpatch storage/xtradb/build/debian/patches/41_scripts__mysql_install_db.sh__no_test.dpatch storage/xtradb/build/debian/patches/44_scripts__mysql_config__libs.dpatch storage/xtradb/build/debian/patches/50_mysql-test__db_test.dpatch storage/xtradb/build/debian/patches/60_percona_support.dpatch storage/xtradb/build/debian/percona-xtradb-client-5.1.README.Debian storage/xtradb/build/debian/percona-xtradb-client-5.1.dirs storage/xtradb/build/debian/percona-xtradb-client-5.1.docs storage/xtradb/build/debian/percona-xtradb-client-5.1.files storage/xtradb/build/debian/percona-xtradb-client-5.1.links storage/xtradb/build/debian/percona-xtradb-client-5.1.lintian-overrides storage/xtradb/build/debian/percona-xtradb-client-5.1.menu storage/xtradb/build/debian/percona-xtradb-common.dirs storage/xtradb/build/debian/percona-xtradb-common.files storage/xtradb/build/debian/percona-xtradb-common.lintian-overrides storage/xtradb/build/debian/percona-xtradb-common.postrm storage/xtradb/build/debian/percona-xtradb-server-5.1.NEWS storage/xtradb/build/debian/percona-xtradb-server-5.1.README.Debian storage/xtradb/build/debian/percona-xtradb-server-5.1.config storage/xtradb/build/debian/percona-xtradb-server-5.1.dirs storage/xtradb/build/debian/percona-xtradb-server-5.1.docs storage/xtradb/build/debian/percona-xtradb-server-5.1.files storage/xtradb/build/debian/percona-xtradb-server-5.1.links storage/xtradb/build/debian/percona-xtradb-server-5.1.lintian-overrides storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.paranoid storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.server storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.workstation storage/xtradb/build/debian/percona-xtradb-server-5.1.mysql.init storage/xtradb/build/debian/percona-xtradb-server-5.1.percona-xtradb-server.logrotate storage/xtradb/build/debian/percona-xtradb-server-5.1.postinst storage/xtradb/build/debian/percona-xtradb-server-5.1.postrm storage/xtradb/build/debian/percona-xtradb-server-5.1.preinst storage/xtradb/build/debian/percona-xtradb-server-5.1.prerm storage/xtradb/build/debian/percona-xtradb-server-5.1.templates storage/xtradb/build/debian/po/ storage/xtradb/build/debian/po/POTFILES.in storage/xtradb/build/debian/po/ar.po storage/xtradb/build/debian/po/ca.po storage/xtradb/build/debian/po/cs.po storage/xtradb/build/debian/po/da.po storage/xtradb/build/debian/po/de.po storage/xtradb/build/debian/po/es.po storage/xtradb/build/debian/po/eu.po storage/xtradb/build/debian/po/fr.po storage/xtradb/build/debian/po/gl.po storage/xtradb/build/debian/po/it.po storage/xtradb/build/debian/po/ja.po storage/xtradb/build/debian/po/nb.po storage/xtradb/build/debian/po/nl.po storage/xtradb/build/debian/po/pt.po storage/xtradb/build/debian/po/pt_BR.po storage/xtradb/build/debian/po/ro.po storage/xtradb/build/debian/po/ru.po storage/xtradb/build/debian/po/sv.po storage/xtradb/build/debian/po/templates.pot storage/xtradb/build/debian/po/tr.po storage/xtradb/build/debian/rules storage/xtradb/build/debian/source.lintian-overrides storage/xtradb/build/debian/watch storage/xtradb/build/percona-sql.spec renamed: mysql-test/r/variables+c.result => mysql-test/r/variables_community.result mysql-test/t/variables+c.test => mysql-test/t/variables_community.test modified: BUILD/SETUP.sh BUILD/compile-solaris-sparc COPYING INSTALL-SOURCE INSTALL-WIN-SOURCE client/mysql.cc client/mysql_upgrade.c client/mysqladmin.cc client/mysqlbinlog.cc client/mysqlcheck.c client/mysqldump.c client/mysqlimport.c client/mysqlshow.c client/mysqlslap.c client/mysqltest.cc cmd-line-utils/libedit/filecomplete.c cmd-line-utils/readline/rlmbutil.h extra/comp_err.c extra/libevent/event-internal.h extra/libevent/kqueue.c extra/yassl/include/yassl_error.hpp extra/yassl/src/ssl.cpp extra/yassl/src/yassl_error.cpp extra/yassl/taocrypt/src/algebra.cpp include/my_sys.h include/mysql/plugin.h libmysql/libmysql.c man/comp_err.1 man/innochecksum.1 man/make_win_bin_dist.1 man/msql2mysql.1 man/my_print_defaults.1 man/myisam_ftdump.1 man/myisamchk.1 man/myisamlog.1 man/myisampack.1 man/mysql-stress-test.pl.1 man/mysql-test-run.pl.1 man/mysql.1 man/mysql.server.1 man/mysql_client_test.1 man/mysql_config.1 man/mysql_convert_table_format.1 man/mysql_find_rows.1 man/mysql_fix_extensions.1 man/mysql_fix_privilege_tables.1 man/mysql_install_db.1 man/mysql_secure_installation.1 man/mysql_setpermission.1 man/mysql_tzinfo_to_sql.1 man/mysql_upgrade.1 man/mysql_waitpid.1 man/mysql_zap.1 man/mysqlaccess.1 man/mysqladmin.1 man/mysqlbinlog.1 man/mysqlbug.1 man/mysqlcheck.1 man/mysqld.8 man/mysqld_multi.1 man/mysqld_safe.1 man/mysqldump.1 man/mysqldumpslow.1 man/mysqlhotcopy.1 man/mysqlimport.1 man/mysqlmanager.8 man/mysqlshow.1 man/mysqlslap.1 man/mysqltest.1 man/ndbd.8 man/ndbd_redo_log_reader.1 man/ndbmtd.8 man/perror.1 man/replace.1 man/resolve_stack_dump.1 man/resolveip.1 mysql-test/Makefile.am mysql-test/collections/default.daily mysql-test/collections/default.push mysql-test/extra/rpl_tests/rpl_get_master_version_and_clock.test mysql-test/extra/rpl_tests/rpl_insert_id_pk.test mysql-test/extra/rpl_tests/rpl_loaddata.test mysql-test/extra/rpl_tests/rpl_tmp_table_and_DDL.test mysql-test/include/default_mysqld.cnf mysql-test/include/maria_empty_logs.inc mysql-test/include/mtr_warnings.sql mysql-test/include/test_fieldsize.inc mysql-test/lib/My/ConfigFactory.pm mysql-test/lib/My/SafeProcess.pm mysql-test/lib/My/SafeProcess/safe_process.cc mysql-test/lib/My/SafeProcess/safe_process_win.cc mysql-test/lib/mtr_cases.pm mysql-test/lib/mtr_gprof.pl mysql-test/lib/mtr_misc.pl mysql-test/lib/mtr_report.pm mysql-test/lib/mtr_stress.pl mysql-test/lib/v1/mtr_stress.pl mysql-test/lib/v1/mysql-test-run.pl mysql-test/mysql-stress-test.pl mysql-test/mysql-test-run.pl mysql-test/r/archive.result mysql-test/r/backup.result mysql-test/r/bigint.result mysql-test/r/compare.result mysql-test/r/csv.result mysql-test/r/ctype_ldml.result mysql-test/r/default.result mysql-test/r/delete.result mysql-test/r/explain.result mysql-test/r/fulltext.result mysql-test/r/func_concat.result mysql-test/r/func_gconcat.result mysql-test/r/func_str.result mysql-test/r/func_time.result mysql-test/r/gis-rtree.result mysql-test/r/grant.result mysql-test/r/group_by.result mysql-test/r/group_min_max.result mysql-test/r/handler_myisam.result mysql-test/r/having.result mysql-test/r/information_schema.result mysql-test/r/innodb-autoinc.result mysql-test/r/innodb_mysql.result mysql-test/r/join.result mysql-test/r/join_outer.result mysql-test/r/join_outer_jcl6.result mysql-test/r/loaddata.result* mysql-test/r/log_state.result mysql-test/r/merge.result mysql-test/r/metadata.result mysql-test/r/multi_update.result mysql-test/r/myisam.result mysql-test/r/mysqlbinlog.result mysql-test/r/mysqlbinlog_row_innodb.result mysql-test/r/mysqltest.result mysql-test/r/partition.result mysql-test/r/partition_error.result mysql-test/r/partition_innodb.result mysql-test/r/partition_pruning.result mysql-test/r/partition_range.result mysql-test/r/ps.result mysql-test/r/query_cache_with_views.result mysql-test/r/select.result mysql-test/r/select_jcl6.result mysql-test/r/show_check.result mysql-test/r/skip_name_resolve.result mysql-test/r/sp-bugs.result mysql-test/r/sp-error.result mysql-test/r/sp.result mysql-test/r/sp_notembedded.result mysql-test/r/sp_trans.result mysql-test/r/subselect.result mysql-test/r/subselect3.result mysql-test/r/symlink.result mysql-test/r/table_elim.result mysql-test/r/trigger.result mysql-test/r/type_bit.result mysql-test/r/type_blob.result mysql-test/r/type_date.result mysql-test/r/type_timestamp.result mysql-test/r/type_year.result mysql-test/r/union.result mysql-test/r/update.result mysql-test/r/variables.result mysql-test/r/view.result mysql-test/r/view_grant.result mysql-test/r/warnings.result mysql-test/r/xa.result mysql-test/suite/binlog/r/binlog_index.result mysql-test/suite/binlog/r/binlog_innodb_row.result mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result mysql-test/suite/binlog/r/binlog_tmp_table.result mysql-test/suite/binlog/t/binlog_index.test mysql-test/suite/binlog/t/binlog_innodb_row.test mysql-test/suite/binlog/t/binlog_tmp_table.test mysql-test/suite/federated/federated.result mysql-test/suite/federated/federated.test mysql-test/suite/funcs_1/datadict/processlist_priv.inc mysql-test/suite/funcs_1/r/is_columns_is.result mysql-test/suite/funcs_1/r/is_tables_is.result mysql-test/suite/innodb/r/innodb-index.result mysql-test/suite/innodb/r/innodb_bug44571.result mysql-test/suite/innodb/t/innodb-consistent.test mysql-test/suite/innodb/t/innodb-index.test mysql-test/suite/innodb/t/innodb_bug44571.test mysql-test/suite/maria/t/maria-recovery-bitmap.test mysql-test/suite/parts/inc/partition_auto_increment.inc mysql-test/suite/parts/r/partition_auto_increment_archive.result mysql-test/suite/parts/r/partition_auto_increment_blackhole.result mysql-test/suite/parts/r/partition_auto_increment_innodb.result mysql-test/suite/parts/r/partition_auto_increment_maria.result mysql-test/suite/parts/r/partition_auto_increment_memory.result mysql-test/suite/parts/r/partition_auto_increment_myisam.result mysql-test/suite/parts/r/partition_auto_increment_ndb.result mysql-test/suite/parts/t/rpl_partition.test mysql-test/suite/pbxt/r/default.result mysql-test/suite/pbxt/r/func_str.result mysql-test/suite/pbxt/r/group_min_max.result mysql-test/suite/pbxt/r/join_nested.result mysql-test/suite/pbxt/r/multi_statement.result mysql-test/suite/pbxt/r/mysqlshow.result mysql-test/suite/pbxt/r/negation_elimination.result mysql-test/suite/pbxt/r/null.result mysql-test/suite/pbxt/r/order_by.result mysql-test/suite/pbxt/r/pbxt_ref_int.result mysql-test/suite/pbxt/r/range.result mysql-test/suite/pbxt/r/type_timestamp.result mysql-test/suite/pbxt/t/status.test mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result mysql-test/suite/rpl/r/rpl_do_grant.result mysql-test/suite/rpl/r/rpl_events.result mysql-test/suite/rpl/r/rpl_get_master_version_and_clock.result mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result mysql-test/suite/rpl/r/rpl_optimize.result mysql-test/suite/rpl/r/rpl_row_create_table.result mysql-test/suite/rpl/r/rpl_sp.result mysql-test/suite/rpl/t/disabled.def mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test mysql-test/suite/rpl/t/rpl_do_grant.test mysql-test/suite/rpl/t/rpl_events.test mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test mysql-test/suite/rpl/t/rpl_loaddata_symlink.test mysql-test/suite/rpl/t/rpl_name_const.test mysql-test/suite/rpl/t/rpl_optimize.test mysql-test/suite/rpl/t/rpl_row_basic_11bugs.test mysql-test/suite/rpl/t/rpl_row_create_table.test mysql-test/suite/rpl/t/rpl_row_trig003.test mysql-test/suite/rpl/t/rpl_slave_skip.test mysql-test/suite/sys_vars/r/log_basic.result mysql-test/suite/sys_vars/r/log_bin_trust_routine_creators_basic.result mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_32.result mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_64.result mysql-test/suite/sys_vars/r/slow_query_log_func.result mysql-test/suite/sys_vars/t/innodb_table_locks_func.test mysql-test/suite/sys_vars/t/slow_query_log_func.test mysql-test/suite/sys_vars/t/sql_low_priority_updates_func.test mysql-test/t/archive.test mysql-test/t/bigint.test mysql-test/t/bug47671-master.opt mysql-test/t/csv.test mysql-test/t/ctype_latin1_de-master.opt mysql-test/t/ctype_ldml.test mysql-test/t/ctype_ucs2_def-master.opt mysql-test/t/delete.test mysql-test/t/explain.test mysql-test/t/fulltext.test mysql-test/t/func_concat.test mysql-test/t/func_gconcat.test mysql-test/t/func_str.test mysql-test/t/func_time.test mysql-test/t/gis-rtree.test mysql-test/t/grant.test mysql-test/t/group_by.test mysql-test/t/group_min_max.test mysql-test/t/handler_myisam.test mysql-test/t/having.test mysql-test/t/innodb-autoinc.test mysql-test/t/innodb_bug38231.test mysql-test/t/innodb_mysql.test mysql-test/t/join.test mysql-test/t/join_outer.test mysql-test/t/loaddata.test mysql-test/t/merge.test mysql-test/t/metadata.test mysql-test/t/multi_update.test mysql-test/t/myisam.test mysql-test/t/mysql_upgrade.test mysql-test/t/mysqlbinlog.test mysql-test/t/mysqltest.test mysql-test/t/partition.test mysql-test/t/partition_error.test mysql-test/t/partition_innodb.test mysql-test/t/partition_innodb_semi_consistent.test mysql-test/t/partition_pruning.test mysql-test/t/partition_range.test mysql-test/t/query_cache_with_views.test mysql-test/t/skip_name_resolve.test mysql-test/t/sp-bugs.test mysql-test/t/sp_notembedded.test mysql-test/t/subselect.test mysql-test/t/symlink.test mysql-test/t/trigger.test mysql-test/t/type_bit.test mysql-test/t/type_date.test mysql-test/t/type_year.test mysql-test/t/udf.test mysql-test/t/update.test mysql-test/t/variables.test mysql-test/t/view.test mysql-test/t/view_grant.test mysql-test/t/xa.test mysys/charset.c mysys/default.c mysys/mf_keycache.c mysys/mf_pack.c mysys/my_gethostbyname.c mysys/my_init.c scripts/fill_help_tables.sql scripts/mysql_system_tables_fix.sql scripts/mysqld_multi.sh server-tools/instance-manager/options.cc sql-common/client.c sql/debug_sync.cc sql/debug_sync.h sql/events.cc sql/field.cc sql/field.h sql/field_conv.cc sql/ha_partition.cc sql/handler.cc sql/hash_filo.cc sql/item.cc sql/item.h sql/item_cmpfunc.cc sql/item_cmpfunc.h sql/item_create.cc sql/item_create.h sql/item_func.cc sql/item_row.cc sql/item_row.h sql/item_strfunc.cc sql/item_strfunc.h sql/item_subselect.cc sql/item_sum.cc sql/item_sum.h sql/item_timefunc.cc sql/log.cc sql/log_event.cc sql/log_event.h sql/log_event_old.cc sql/mf_iocache.cc sql/mysql_priv.h sql/mysqld.cc sql/net_serv.cc sql/opt_range.cc sql/opt_sum.cc sql/partition_info.cc sql/protocol.cc sql/repl_failsafe.cc sql/rpl_utility.cc sql/rpl_utility.h sql/set_var.cc sql/share/errmsg.txt sql/slave.cc sql/sp.cc sql/sp_cache.cc sql/sp_head.cc sql/sql_acl.cc sql/sql_base.cc sql/sql_class.cc sql/sql_class.h sql/sql_delete.cc sql/sql_insert.cc sql/sql_lex.cc sql/sql_lex.h sql/sql_load.cc sql/sql_parse.cc sql/sql_partition.cc sql/sql_plugin.cc sql/sql_profile.cc sql/sql_repl.cc sql/sql_select.cc sql/sql_select.h sql/sql_show.cc sql/sql_table.cc sql/sql_trigger.cc sql/sql_update.cc sql/sql_view.cc sql/sql_yacc.yy sql/table.cc sql/table.h storage/archive/ha_archive.cc storage/csv/ha_tina.cc storage/example/ha_example.h storage/federated/ha_federated.cc storage/federated/ha_federated.h storage/innobase/buf/buf0buf.c storage/innobase/buf/buf0rea.c storage/innobase/handler/ha_innodb.cc storage/innobase/include/buf0rea.h storage/innobase/lock/lock0lock.c storage/innobase/os/os0file.c storage/innobase/plug.in.disabled storage/innobase/row/row0sel.c storage/innobase/trx/trx0sys.c storage/innodb_plugin/CMakeLists.txt storage/innodb_plugin/ChangeLog storage/innodb_plugin/Makefile.am storage/innodb_plugin/btr/btr0btr.c storage/innodb_plugin/btr/btr0cur.c storage/innodb_plugin/btr/btr0pcur.c storage/innodb_plugin/buf/buf0buddy.c storage/innodb_plugin/buf/buf0buf.c storage/innodb_plugin/buf/buf0flu.c storage/innodb_plugin/buf/buf0lru.c storage/innodb_plugin/buf/buf0rea.c storage/innodb_plugin/dict/dict0boot.c storage/innodb_plugin/dict/dict0crea.c storage/innodb_plugin/dict/dict0dict.c storage/innodb_plugin/dict/dict0load.c storage/innodb_plugin/dict/dict0mem.c storage/innodb_plugin/fil/fil0fil.c storage/innodb_plugin/fsp/fsp0fsp.c storage/innodb_plugin/ha/ha0ha.c storage/innodb_plugin/ha/hash0hash.c storage/innodb_plugin/handler/ha_innodb.cc storage/innodb_plugin/handler/ha_innodb.h storage/innodb_plugin/handler/handler0alter.cc storage/innodb_plugin/ibuf/ibuf0ibuf.c storage/innodb_plugin/include/btr0btr.h storage/innodb_plugin/include/btr0btr.ic storage/innodb_plugin/include/btr0cur.h storage/innodb_plugin/include/btr0pcur.h storage/innodb_plugin/include/btr0pcur.ic storage/innodb_plugin/include/buf0buf.h storage/innodb_plugin/include/buf0buf.ic storage/innodb_plugin/include/buf0flu.h storage/innodb_plugin/include/data0type.ic storage/innodb_plugin/include/dict0boot.h storage/innodb_plugin/include/dict0mem.h storage/innodb_plugin/include/fil0fil.h storage/innodb_plugin/include/hash0hash.h storage/innodb_plugin/include/hash0hash.ic storage/innodb_plugin/include/lock0lock.h storage/innodb_plugin/include/log0log.h storage/innodb_plugin/include/log0log.ic storage/innodb_plugin/include/log0recv.h storage/innodb_plugin/include/mem0dbg.h storage/innodb_plugin/include/mem0dbg.ic storage/innodb_plugin/include/mem0mem.h storage/innodb_plugin/include/mem0mem.ic storage/innodb_plugin/include/mtr0mtr.ic storage/innodb_plugin/include/os0file.h storage/innodb_plugin/include/que0que.h storage/innodb_plugin/include/que0que.ic storage/innodb_plugin/include/row0mysql.h storage/innodb_plugin/include/row0sel.h storage/innodb_plugin/include/srv0srv.h storage/innodb_plugin/include/sync0rw.h storage/innodb_plugin/include/sync0sync.h storage/innodb_plugin/include/trx0rseg.h storage/innodb_plugin/include/trx0sys.h storage/innodb_plugin/include/trx0trx.h storage/innodb_plugin/include/trx0types.h storage/innodb_plugin/include/univ.i storage/innodb_plugin/include/ut0rnd.ic storage/innodb_plugin/lock/lock0lock.c storage/innodb_plugin/log/log0log.c storage/innodb_plugin/log/log0recv.c storage/innodb_plugin/mem/mem0dbg.c storage/innodb_plugin/mem/mem0mem.c storage/innodb_plugin/os/os0file.c storage/innodb_plugin/page/page0page.c storage/innodb_plugin/plug.in.disabled storage/innodb_plugin/rem/rem0rec.c storage/innodb_plugin/row/row0ins.c storage/innodb_plugin/row/row0merge.c storage/innodb_plugin/row/row0mysql.c storage/innodb_plugin/row/row0row.c storage/innodb_plugin/row/row0sel.c storage/innodb_plugin/row/row0umod.c storage/innodb_plugin/row/row0upd.c storage/innodb_plugin/srv/srv0srv.c storage/innodb_plugin/srv/srv0start.c storage/innodb_plugin/sync/sync0sync.c storage/innodb_plugin/trx/trx0i_s.c storage/innodb_plugin/trx/trx0rec.c storage/innodb_plugin/trx/trx0rseg.c storage/innodb_plugin/trx/trx0sys.c storage/innodb_plugin/trx/trx0trx.c storage/maria/ma_loghandler.c storage/maria/ma_search.c storage/maria/maria_def.h storage/myisam/ft_boolean_search.c storage/myisam/ft_stopwords.c storage/myisam/ha_myisam.cc storage/myisam/mi_check.c storage/myisam/mi_delete_all.c storage/myisam/mi_delete_table.c storage/myisam/mi_dynrec.c storage/myisam/mi_extra.c storage/myisam/mi_locking.c storage/myisam/mi_open.c storage/myisam/mi_page.c storage/myisam/mi_rnext.c storage/myisam/mi_write.c storage/myisam/myisamdef.h storage/myisam/rt_index.c storage/myisam/rt_split.c storage/myisam/sort.c storage/myisammrg/ha_myisammrg.cc storage/myisammrg/myrg_open.c storage/pbxt/ChangeLog storage/pbxt/src/backup_xt.cc storage/pbxt/src/cache_xt.cc storage/pbxt/src/cache_xt.h storage/pbxt/src/database_xt.cc storage/pbxt/src/database_xt.h storage/pbxt/src/datadic_xt.cc storage/pbxt/src/datadic_xt.h storage/pbxt/src/datalog_xt.cc storage/pbxt/src/filesys_xt.h storage/pbxt/src/ha_pbxt.cc storage/pbxt/src/index_xt.cc storage/pbxt/src/index_xt.h storage/pbxt/src/lock_xt.cc storage/pbxt/src/lock_xt.h storage/pbxt/src/locklist_xt.cc storage/pbxt/src/myxt_xt.cc storage/pbxt/src/pbms_enabled.cc storage/pbxt/src/pthread_xt.cc storage/pbxt/src/pthread_xt.h storage/pbxt/src/restart_xt.cc storage/pbxt/src/restart_xt.h storage/pbxt/src/strutil_xt.cc storage/pbxt/src/tabcache_xt.cc storage/pbxt/src/tabcache_xt.h storage/pbxt/src/table_xt.cc storage/pbxt/src/table_xt.h storage/pbxt/src/thread_xt.cc storage/pbxt/src/thread_xt.h storage/pbxt/src/trace_xt.cc storage/pbxt/src/trace_xt.h storage/pbxt/src/xaction_xt.cc storage/pbxt/src/xaction_xt.h storage/pbxt/src/xactlog_xt.cc storage/pbxt/src/xactlog_xt.h storage/pbxt/src/xt_defs.h storage/xtradb/btr/btr0btr.c storage/xtradb/btr/btr0cur.c storage/xtradb/btr/btr0pcur.c storage/xtradb/btr/btr0sea.c storage/xtradb/buf/buf0buddy.c storage/xtradb/buf/buf0buf.c storage/xtradb/buf/buf0flu.c storage/xtradb/buf/buf0rea.c storage/xtradb/dict/dict0dict.c storage/xtradb/dict/dict0mem.c storage/xtradb/fil/fil0fil.c storage/xtradb/fsp/fsp0fsp.c storage/xtradb/handler/ha_innodb.cc storage/xtradb/handler/ha_innodb.h storage/xtradb/handler/i_s.cc storage/xtradb/handler/i_s.h storage/xtradb/handler/innodb_patch_info.h storage/xtradb/include/btr0btr.ic storage/xtradb/include/buf0buddy.h storage/xtradb/include/buf0buf.h storage/xtradb/include/buf0buf.ic storage/xtradb/include/buf0types.h storage/xtradb/include/dict0dict.h storage/xtradb/include/dict0mem.h storage/xtradb/include/fil0fil.h storage/xtradb/include/fsp0types.h storage/xtradb/include/fut0fut.ic storage/xtradb/include/ha_prototypes.h storage/xtradb/include/page0cur.h storage/xtradb/include/page0page.h storage/xtradb/include/page0page.ic storage/xtradb/include/page0types.h storage/xtradb/include/srv0srv.h storage/xtradb/include/trx0sys.h storage/xtradb/include/univ.i storage/xtradb/include/ut0lst.h storage/xtradb/include/ut0rnd.h storage/xtradb/include/ut0rnd.ic storage/xtradb/lock/lock0lock.c storage/xtradb/log/log0log.c storage/xtradb/log/log0recv.c storage/xtradb/mtr/mtr0log.c storage/xtradb/page/page0cur.c storage/xtradb/page/page0zip.c storage/xtradb/row/row0ins.c storage/xtradb/row/row0merge.c storage/xtradb/row/row0sel.c storage/xtradb/srv/srv0srv.c storage/xtradb/srv/srv0start.c storage/xtradb/sync/sync0sync.c storage/xtradb/trx/trx0i_s.c storage/xtradb/trx/trx0trx.c strings/ctype-ucs2.c strings/ctype-utf8.c support-files/compiler_warnings.supp support-files/mysql.spec.sh tests/mysql_client_test.c unittest/mysys/waiting_threads-t.c Diff too large for email (1027113 lines, the limit is 1000000).

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (igor:2789)
by Igor Babaev 12 May '10

12 May '10

#At lp:maria/5.2 based on revid:igor@askmonty.org-20100429211039-rp1mza3xjeqd4t1w 2789 Igor Babaev 2010-05-11 Fixed several bugs in the backport code (mwl#106). modified: mysql-test/r/innodb_lock_wait_timeout_1.result mysql-test/r/lock_multi_bug38499.result mysql-test/r/ps_ddl.result mysql-test/t/lock_multi_bug38499.test sql/sql_base.cc sql/sql_delete.cc sql/sql_derived.cc sql/sql_select.cc sql/sql_union.cc sql/table.cc === modified file 'mysql-test/r/innodb_lock_wait_timeout_1.result' --- a/mysql-test/r/innodb_lock_wait_timeout_1.result 2009-11-12 11:43:33 +0000 +++ b/mysql-test/r/innodb_lock_wait_timeout_1.result 2010-05-12 04:09:58 +0000 @@ -104,7 +104,7 @@ id 1 select_type PRIMARY table <derived2> type ALL -possible_keys NULL +possible_keys key0 key NULL key_len NULL ref NULL @@ -308,7 +308,7 @@ id 1 select_type PRIMARY table <derived2> type ALL -possible_keys NULL +possible_keys key0 key NULL key_len NULL ref NULL === modified file 'mysql-test/r/lock_multi_bug38499.result' --- a/mysql-test/r/lock_multi_bug38499.result 2009-08-28 21:49:16 +0000 +++ b/mysql-test/r/lock_multi_bug38499.result 2010-05-12 04:09:58 +0000 @@ -2,7 +2,9 @@ SET @odl_sync_frm = @@global.sync_frm; SET @@global.sync_frm = OFF; DROP TABLE IF EXISTS t1; CREATE TABLE t1( a INT, b INT ); +CREATE TABLE t2( a INT, b INT ); INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3), (4, 4); +INSERT INTO t2 VALUES (1, 1), (2, 2), (3, 3), (4, 4); # 1. test regular tables # 1.1. test altering of columns that multiupdate doesn't use # 1.1.1. normal mode @@ -18,5 +20,5 @@ ALTER TABLE t1 ADD COLUMN a INT; # 2.2. test altering of columns that multiupdate uses # 2.2.1. normal mode # 2.2.2. PS mode -DROP TABLE t1; +DROP TABLE t1,t2; SET @@global.sync_frm = @odl_sync_frm; === modified file 'mysql-test/r/ps_ddl.result' --- a/mysql-test/r/ps_ddl.result 2010-01-16 07:44:24 +0000 +++ b/mysql-test/r/ps_ddl.result 2010-05-12 04:09:58 +0000 @@ -1507,12 +1507,12 @@ create view v_27690_1 as select A.a, A.b execute stmt; a b a b 1 1 1 1 -2 2 1 1 -1 1 1 1 -2 2 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 +1 1 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 call p_verify_reprepare_count(1); SUCCESS @@ -1520,12 +1520,12 @@ SUCCESS execute stmt; a b a b 1 1 1 1 -2 2 1 1 -1 1 1 1 -2 2 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 +1 1 1 1 1 1 2 2 +2 2 1 1 2 2 2 2 call p_verify_reprepare_count(0); SUCCESS === modified file 'mysql-test/t/lock_multi_bug38499.test' --- a/mysql-test/t/lock_multi_bug38499.test 2009-08-28 21:49:16 +0000 +++ b/mysql-test/t/lock_multi_bug38499.test 2010-05-12 04:09:58 +0000 @@ -16,7 +16,9 @@ connect (writer,localhost,root,,); DROP TABLE IF EXISTS t1; --enable_warnings CREATE TABLE t1( a INT, b INT ); +CREATE TABLE t2( a INT, b INT ); INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3), (4, 4); +INSERT INTO t2 VALUES (1, 1), (2, 2), (3, 3), (4, 4); --echo # 1. test regular tables --echo # 1.1. test altering of columns that multiupdate doesn't use @@ -28,7 +30,7 @@ while ($i) { --dec $i --connection writer - send UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0; + send UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0; --connection locker ALTER TABLE t1 ADD COLUMN (c INT); @@ -41,7 +43,7 @@ while ($i) { --echo # 1.1.2. PS mode --connection writer -PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0'; +PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0'; let $i = 100; while ($i) { @@ -75,7 +77,7 @@ while ($i) { UPDATE t1 SET a=b; --connection writer ---send UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0; +--send UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0; --connection locker --error 0,ER_CANT_DROP_FIELD_OR_KEY @@ -100,7 +102,7 @@ while ($i) { UPDATE t1 SET a=b; --connection writer - PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t1 t1i) d SET a = 0 WHERE 1=0'; + PREPARE stmt FROM 'UPDATE t1, (SELECT 1 FROM t2 t1i) d SET a = 0 WHERE 1=0'; --send EXECUTE stmt --connection locker @@ -210,7 +212,7 @@ while ($i) { } --enable_query_log --connection default -DROP TABLE t1; +DROP TABLE t1,t2; # Close connections === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_base.cc 2010-05-12 04:09:58 +0000 @@ -7642,7 +7642,8 @@ bool setup_tables(THD *thd, Name_resolut TABLE_LIST *first_select_table= (select_insert ? tables->next_local: 0); - SELECT_LEX *select_lex= thd->lex->current_select; + SELECT_LEX *select_lex= select_insert ? &thd->lex->select_lex : + thd->lex->current_select; if (select_lex->first_cond_optimization) { leaves.empty(); === modified file 'sql/sql_delete.cc' --- a/sql/sql_delete.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_delete.cc 2010-05-12 04:09:58 +0000 @@ -397,6 +397,12 @@ cleanup: query_cache_invalidate3(thd, table_list, 1); } + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } + delete select; transactional_table= table->file->has_transactions(); === modified file 'sql/sql_derived.cc' --- a/sql/sql_derived.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_derived.cc 2010-05-12 04:09:58 +0000 @@ -159,8 +159,12 @@ mysql_handle_single_derived(LEX *lex, TA uint phase_flag= DT_INIT << phase; if (phase_flag > phases) break; +#if 0 if (!(phases & phase_flag) || derived->merged_for_insert && phase_flag != DT_REINIT) +#else + if (!(phases & phase_flag)) +#endif continue; /* Skip derived tables to which the phase isn't applicable. */ if (phase_flag != DT_PREPARE && @@ -476,11 +480,27 @@ bool mysql_derived_merge_for_insert(THD derived->table= table; derived->schema_table= ((TABLE_LIST*)dt_select->table_list.first)->schema_table; - derived->select_lex->leaf_tables.push_back(tl); + if (!derived->merged) + { + Query_arena *arena, backup; + arena= thd->activate_stmt_arena_if_needed(&backup); // For easier test + derived->select_lex->leaf_tables.push_back(tl); + derived->nested_join= (NESTED_JOIN*) thd->calloc(sizeof(NESTED_JOIN)); + if (derived->nested_join) + { + derived->wrap_into_nested_join(tl->select_lex->top_join_list); + derived->get_unit()->exclude_level(); + } + if (arena) + thd->restore_active_arena(arena, &backup); + derived->merged= TRUE; + if (!derived->nested_join) + return TRUE; + } } else { - if (mysql_derived_merge(thd, lex, derived)) + if (!derived->merged_for_insert && mysql_derived_merge(thd, lex, derived)) return TRUE; } derived->merged_for_insert= TRUE; @@ -585,11 +605,19 @@ bool mysql_derived_prepare(THD *thd, LEX bool res= FALSE; // Skip already prepared views/DT +#if 0 if (!unit || unit->prepared || derived->merged_for_insert) +#else + if (!unit || unit->prepared) +#endif DBUG_RETURN(FALSE); /* It's a target view for an INSERT, create field translation only. */ +#if 0 if (derived->skip_prepare_derived && !derived->is_multitable()) +#else + if (derived->merged_for_insert) +#endif { res= derived->create_field_translation(thd); DBUG_RETURN(res); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_select.cc 2010-05-12 04:09:58 +0000 @@ -7836,7 +7836,7 @@ void JOIN_TAB::cleanup() bool JOIN_TAB::preread_init() { TABLE_LIST *derived= table->pos_in_table_list; - if (!derived->is_materialized_derived()) + if (!derived || !derived->is_materialized_derived()) { preread_init_done= TRUE; return FALSE; @@ -9923,12 +9923,14 @@ simplify_joins(JOIN *join, List<TABLE_LI { TABLE_LIST *tbl; List_iterator<TABLE_LIST> it(nested_join->join_list); + List<TABLE_LIST> repl_list; while ((tbl= it++)) { tbl->embedding= table->embedding; tbl->join_list= table->join_list; + repl_list.push_back(tbl); } - li.replace(nested_join->join_list); + li.replace(repl_list); /* Need to update the name resolution table chain when flattening joins */ fix_name_res= TRUE; table= *li.ref(); === modified file 'sql/sql_union.cc' --- a/sql/sql_union.cc 2010-04-29 21:10:39 +0000 +++ b/sql/sql_union.cc 2010-05-12 04:09:58 +0000 @@ -394,7 +394,7 @@ bool st_select_lex_unit::prepare(THD *th if (union_result->create_result_table(thd, &types, test(union_distinct), create_options, "", FALSE, TRUE)) goto err; - if (!lex_select_save->first_cond_optimization) + if (fake_select_lex && !fake_select_lex->first_cond_optimization) { save_tablenr= result_table_list.tablenr_exec; save_map= result_table_list.map_exec; @@ -403,7 +403,7 @@ bool st_select_lex_unit::prepare(THD *th result_table_list.db= (char*) ""; result_table_list.table_name= result_table_list.alias= (char*) "union"; result_table_list.table= table= union_result->table; - if (!lex_select_save->first_cond_optimization) + if (fake_select_lex && !fake_select_lex->first_cond_optimization) { result_table_list.tablenr_exec= save_tablenr; result_table_list.map_exec= save_map; === modified file 'sql/table.cc' --- a/sql/table.cc 2010-04-29 21:10:39 +0000 +++ b/sql/table.cc 2010-05-12 04:09:58 +0000 @@ -5638,7 +5638,7 @@ bool TABLE_LIST::handle_derived(struct s @return 0 when it's not a derived table/view. */ -inline st_select_lex_unit *TABLE_LIST::get_unit() +st_select_lex_unit *TABLE_LIST::get_unit() { return (view ? &view->unit : derived); } @@ -5652,7 +5652,7 @@ inline st_select_lex_unit *TABLE_LIST::g @return 0 when it's not a derived table. */ -inline st_select_lex *TABLE_LIST::get_single_select() +st_select_lex *TABLE_LIST::get_single_select() { SELECT_LEX_UNIT *unit= get_unit(); return (unit ? unit->first_select() : 0);

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (knielsen:2758)
by knielsen＠knielsen-hq.org 11 May '10

11 May '10

#At lp:maria/5.2 2758 knielsen(a)knielsen-hq.org 2010-05-11 Fix Windows ^M line ending. modified: sql/slave.h === modified file 'sql/slave.h' --- a/sql/slave.h 2010-03-22 07:34:28 +0000 +++ b/sql/slave.h 2010-05-11 13:24:37 +0000 @@ -106,7 +106,7 @@ extern MYSQL_PLUGIN_IMPORT char *relay_l extern char *opt_relay_logname, *opt_relaylog_index_name; extern my_bool opt_skip_slave_start, opt_reckless_slave; extern my_bool opt_log_slave_updates; -extern my_bool opt_replicate_annotate_rows_events; +extern my_bool opt_replicate_annotate_rows_events; extern ulonglong relay_log_space_limit; /*

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2854: Automerge MariaDB 5.1.44b into trunk.
by noreply＠launchpad.net 11 May '10

11 May '10

Merge authors: Kristian Nielsen (knielsen) ------------------------------------------------------------ revno: 2854 [merge] committer: knielsen(a)knielsen-hq.org branch nick: mariadb-5.1 timestamp: Tue 2010-05-11 13:28:14 +0200 message: Automerge MariaDB 5.1.44b into trunk. modified: configure.in mysql-test/r/grant.result mysql-test/t/grant.test sql/mysql_priv.h sql/partition_info.cc sql/sql_parse.cc sql/sql_table.cc sql/sql_yacc.yy sql/table.cc tests/mysql_client_test.c -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2853: Removed extra } that caused script to fail with syntax error
by noreply＠launchpad.net 10 May '10

10 May '10

------------------------------------------------------------ revno: 2853 committer: Michael Widenius <monty(a)askmonty.org> branch nick: maria-5.1 timestamp: Mon 2010-05-10 21:23:16 +0300 message: Removed extra } that caused script to fail with syntax error modified: scripts/mysqld_multi.sh -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] Windows installer
by Bo Thorsen 10 May '10

10 May '10

Hi everyone, I have been looking at getting our binary windows distribution back on track. One of the options would be to create the zip file like we did earlier. Another way would be to create a proper windows installer. For this, there are several options: NSIS, WIX, etc. I spent a bit of time investigating the CPack parts of CMake. And during the investigation, I managed to pretty much write an entire installer. It was quite easy. The way this works is to add INSTALL instructions in the CMakeFiles.txt, plus a bit of extra information for building the installer package. You can see this in the patch I have attached. With the patch applied, you have to install NSIS (http://nsis.sourceforge.net) and add it to the path. Build MariaDB in release, and run "cpack" in the MariaDB tree. It's NSIS based because this seems to be the one cpack works best with. The question is what direction to continue in. I'd appreciate some feedback on this, because I'm not certain if it's the right way to go. It has been pretty easy so far, so I'm pretty happy to continue with it. IMHO, the most important thing not implemented in this installer yet is to set up MariaDB as a service. I'm going to focus on getting Windows running in KVM for our buildbot system now. And then I'll get back to this later. Cheers, Bo Thorsen.

6 9

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2852: bugfix from mysql-5.1, apparently lost in a merge
by noreply＠launchpad.net 10 May '10

10 May '10

------------------------------------------------------------ revno: 2852 committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: maria-5.1 timestamp: Mon 2010-05-10 16:23:08 +0200 message: bugfix from mysql-5.1, apparently lost in a merge modified: sql/sql_select.cc -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] Updated (by Serg): innodb statistics in the slow log (115)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: innodb statistics in the slow log CREATION DATE..: Sun, 25 Apr 2010, 16:35 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 115 (http://askmonty.org/worklog/?tid=115) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:36)=-=- Category updated. --- /tmp/wklog.115.old.7130 2010-05-10 14:36:08.000000000 +0000 +++ /tmp/wklog.115.new.7130 2010-05-10 14:36:08.000000000 +0000 @@ -1 +1 @@ -Server-Sprint +Server-BackLog -=-=(Serg - Mon, 10 May 2010, 14:36)=-=- Version updated. --- /tmp/wklog.115.old.7130 2010-05-10 14:36:08.000000000 +0000 +++ /tmp/wklog.115.new.7130 2010-05-10 14:36:08.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: We need to find some way for innoDB/XtraDB specific information to appear in the slow log. Same effect as in Percona patches, but a with cleaner implementation. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): innodb statistics in the slow log (115)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: innodb statistics in the slow log CREATION DATE..: Sun, 25 Apr 2010, 16:35 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 115 (http://askmonty.org/worklog/?tid=115) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:36)=-=- Category updated. --- /tmp/wklog.115.old.7130 2010-05-10 14:36:08.000000000 +0000 +++ /tmp/wklog.115.new.7130 2010-05-10 14:36:08.000000000 +0000 @@ -1 +1 @@ -Server-Sprint +Server-BackLog -=-=(Serg - Mon, 10 May 2010, 14:36)=-=- Version updated. --- /tmp/wklog.115.old.7130 2010-05-10 14:36:08.000000000 +0000 +++ /tmp/wklog.115.new.7130 2010-05-10 14:36:08.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: We need to find some way for innoDB/XtraDB specific information to appear in the slow log. Same effect as in Percona patches, but a with cleaner implementation. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): insert ignore ha_extra hint (114)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: insert ignore ha_extra hint CREATION DATE..: Sun, 25 Apr 2010, 16:32 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 114 (http://askmonty.org/worklog/?tid=114) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Category updated. --- /tmp/wklog.114.old.7124 2010-05-10 14:35:59.000000000 +0000 +++ /tmp/wklog.114.new.7124 2010-05-10 14:35:59.000000000 +0000 @@ -1 +1 @@ -Server-Sprint +Server-BackLog -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Version updated. --- /tmp/wklog.114.old.7124 2010-05-10 14:35:59.000000000 +0000 +++ /tmp/wklog.114.new.7124 2010-05-10 14:35:59.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: There is HA_EXTRA_WRITE_CAN_REPLACE hint that tells the engine that the following ::write_row() calls are part of the REPLACE statement, not INSERT. With this knowledge the engine can execute the replace internally, deleting the conflicting row in the ::write_row() method instead of returning an error. We need a similar HA_EXTRA_WRITE_CAN_IGNORE hint to allow engines to optimize INSERT IGNORE in a similar way. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): insert ignore ha_extra hint (114)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: insert ignore ha_extra hint CREATION DATE..: Sun, 25 Apr 2010, 16:32 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 114 (http://askmonty.org/worklog/?tid=114) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Category updated. --- /tmp/wklog.114.old.7124 2010-05-10 14:35:59.000000000 +0000 +++ /tmp/wklog.114.new.7124 2010-05-10 14:35:59.000000000 +0000 @@ -1 +1 @@ -Server-Sprint +Server-BackLog -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Version updated. --- /tmp/wklog.114.old.7124 2010-05-10 14:35:59.000000000 +0000 +++ /tmp/wklog.114.new.7124 2010-05-10 14:35:59.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: There is HA_EXTRA_WRITE_CAN_REPLACE hint that tells the engine that the following ::write_row() calls are part of the REPLACE statement, not INSERT. With this knowledge the engine can execute the replace internally, deleting the conflicting row in the ::write_row() method instead of returning an error. We need a similar HA_EXTRA_WRITE_CAN_IGNORE hint to allow engines to optimize INSERT IGNORE in a similar way. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): support many clustered keys per table (113)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: support many clustered keys per table CREATION DATE..: Sun, 25 Apr 2010, 16:23 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 113 (http://askmonty.org/worklog/?tid=113) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Category updated. --- /tmp/wklog.113.old.7118 2010-05-10 14:35:50.000000000 +0000 +++ /tmp/wklog.113.new.7118 2010-05-10 14:35:50.000000000 +0000 @@ -1 +1 @@ -Server-Sprint +Server-BackLog -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Version updated. --- /tmp/wklog.113.old.7105 2010-05-10 14:35:25.000000000 +0000 +++ /tmp/wklog.113.new.7105 2010-05-10 14:35:25.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: The server now assumes that there can be only one clustered key per table, and only primary key can be clustered. It's not true for certain storage engines, we need to remove this limitation ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): support many clustered keys per table (113)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: support many clustered keys per table CREATION DATE..: Sun, 25 Apr 2010, 16:23 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 113 (http://askmonty.org/worklog/?tid=113) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Category updated. --- /tmp/wklog.113.old.7118 2010-05-10 14:35:50.000000000 +0000 +++ /tmp/wklog.113.new.7118 2010-05-10 14:35:50.000000000 +0000 @@ -1 +1 @@ -Server-Sprint +Server-BackLog -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Version updated. --- /tmp/wklog.113.old.7105 2010-05-10 14:35:25.000000000 +0000 +++ /tmp/wklog.113.new.7105 2010-05-10 14:35:25.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: The server now assumes that there can be only one clustered key per table, and only primary key can be clustered. It's not true for certain storage engines, we need to remove this limitation ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): support many clustered keys per table (113)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: support many clustered keys per table CREATION DATE..: Sun, 25 Apr 2010, 16:23 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 113 (http://askmonty.org/worklog/?tid=113) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Version updated. --- /tmp/wklog.113.old.7105 2010-05-10 14:35:25.000000000 +0000 +++ /tmp/wklog.113.new.7105 2010-05-10 14:35:25.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: The server now assumes that there can be only one clustered key per table, and only primary key can be clustered. It's not true for certain storage engines, we need to remove this limitation ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): support many clustered keys per table (113)
by worklog-noreply＠askmonty.org 10 May '10

10 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: support many clustered keys per table CREATION DATE..: Sun, 25 Apr 2010, 16:23 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 113 (http://askmonty.org/worklog/?tid=113) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 10 May 2010, 14:35)=-=- Version updated. --- /tmp/wklog.113.old.7105 2010-05-10 14:35:25.000000000 +0000 +++ /tmp/wklog.113.new.7105 2010-05-10 14:35:25.000000000 +0000 @@ -1 +1 @@ -Server-9.x +Server-5.3 DESCRIPTION: The server now assumes that there can be only one clustered key per table, and only primary key can be clustered. It's not true for certain storage engines, we need to remove this limitation ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2860)
by knielsen＠knielsen-hq.org 10 May '10

10 May '10

#At lp:maria 2860 knielsen(a)knielsen-hq.org 2010-05-10 [merge] Automerge MariaDB 5.1.44b release. modified: mysql-test/r/grant.result mysql-test/t/grant.test sql/mysql_priv.h sql/partition_info.cc sql/sql_parse.cc sql/sql_table.cc sql/sql_yacc.yy sql/table.cc tests/mysql_client_test.c === modified file 'mysql-test/r/grant.result' --- a/mysql-test/r/grant.result 2010-01-29 10:42:31 +0000 +++ b/mysql-test/r/grant.result 2010-05-09 19:30:06 +0000 @@ -1413,3 +1413,19 @@ DROP USER 'user1'; DROP USER 'user1'@'localhost'; DROP USER 'user2'; DROP DATABASE db1; +CREATE DATABASE db1; +CREATE DATABASE db2; +GRANT SELECT ON db1.* to 'testbug'@localhost; +USE db2; +CREATE TABLE t1 (a INT); +USE test; +SELECT * FROM `../db2/tb2`; +ERROR 42S02: Table 'db1.../db2/tb2' doesn't exist +SELECT * FROM `../db2`.tb2; +ERROR 42000: SELECT command denied to user 'testbug'@'localhost' for table 'tb2' +SELECT * FROM `#mysql50#/../db2/tb2`; +ERROR 42S02: Table 'db1.#mysql50#/../db2/tb2' doesn't exist +DROP USER 'testbug'@localhost; +DROP TABLE db2.t1; +DROP DATABASE db1; +DROP DATABASE db2; === modified file 'mysql-test/t/grant.test' --- a/mysql-test/t/grant.test 2010-01-29 10:42:31 +0000 +++ b/mysql-test/t/grant.test 2010-05-09 19:30:06 +0000 @@ -1525,5 +1525,30 @@ DROP USER 'user1'@'localhost'; DROP USER 'user2'; DROP DATABASE db1; + +# +# Bug #53371: COM_FIELD_LIST can be abused to bypass table level grants. +# + +CREATE DATABASE db1; +CREATE DATABASE db2; +GRANT SELECT ON db1.* to 'testbug'@localhost; +USE db2; +CREATE TABLE t1 (a INT); +USE test; +connect (con1,localhost,testbug,,db1); +--error ER_NO_SUCH_TABLE +SELECT * FROM `../db2/tb2`; +--error ER_TABLEACCESS_DENIED_ERROR +SELECT * FROM `../db2`.tb2; +--error ER_NO_SUCH_TABLE +SELECT * FROM `#mysql50#/../db2/tb2`; +connection default; +disconnect con1; +DROP USER 'testbug'@localhost; +DROP TABLE db2.t1; +DROP DATABASE db1; +DROP DATABASE db2; + # Wait till we reached the initial number of concurrent sessions --source include/wait_until_count_sessions.inc === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-04-28 12:52:24 +0000 +++ b/sql/mysql_priv.h 2010-05-10 07:34:49 +0000 @@ -2289,7 +2289,7 @@ void update_create_info_from_table(HA_CR int rename_file_ext(const char * from,const char * to,const char * ext); bool check_db_name(LEX_STRING *db); bool check_column_name(const char *name); -bool check_table_name(const char *name, uint length); +bool check_table_name(const char *name, uint length, bool check_for_path_chars); char *get_field(MEM_ROOT *mem, Field *field); bool get_field(MEM_ROOT *mem, Field *field, class String *res); int wild_case_compare(CHARSET_INFO *cs, const char *str,const char *wildstr); === modified file 'sql/partition_info.cc' --- a/sql/partition_info.cc 2009-12-03 11:19:05 +0000 +++ b/sql/partition_info.cc 2010-05-09 19:30:06 +0000 @@ -972,7 +972,7 @@ bool partition_info::check_partition_inf part_elem->engine_type= default_engine_type; } if (check_table_name(part_elem->partition_name, - strlen(part_elem->partition_name))) + strlen(part_elem->partition_name), FALSE)) { my_error(ER_WRONG_PARTITION_NAME, MYF(0)); goto end; @@ -990,7 +990,7 @@ bool partition_info::check_partition_inf { sub_elem= sub_it++; if (check_table_name(sub_elem->partition_name, - strlen(sub_elem->partition_name))) + strlen(sub_elem->partition_name), FALSE)) { my_error(ER_WRONG_PARTITION_NAME, MYF(0)); goto end; === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-04-30 04:23:39 +0000 +++ b/sql/sql_parse.cc 2010-05-10 07:34:49 +0000 @@ -1334,6 +1334,11 @@ bool dispatch_command(enum enum_server_c system_charset_info, packet, db_length, thd->charset(), &dummy_errors); db_buff[db_length]= '\0'; + if (check_table_name(db_buff, db_length, FALSE)) + { + my_error(ER_WRONG_TABLE_NAME, MYF(0), db_buff); + break; + } table_list.alias= table_list.table_name= db_buff; if (!(fields= (char *) thd->memdup(wildcard, query_length + 1))) break; @@ -6298,7 +6303,7 @@ TABLE_LIST *st_select_lex::add_table_to_ DBUG_RETURN(0); // End of memory alias_str= alias ? alias->str : table->table.str; if (!test(table_options & TL_OPTION_ALIAS) && - check_table_name(table->table.str, table->table.length)) + check_table_name(table->table.str, table->table.length, FALSE)) { my_error(ER_WRONG_TABLE_NAME, MYF(0), table->table.str); DBUG_RETURN(0); === modified file 'sql/sql_table.cc' --- a/sql/sql_table.cc 2010-04-28 12:52:24 +0000 +++ b/sql/sql_table.cc 2010-05-10 07:34:49 +0000 @@ -435,7 +435,21 @@ uint tablename_to_filename(const char *f DBUG_PRINT("enter", ("from '%s'", from)); if ((length= check_n_cut_mysql50_prefix(from, to, to_length))) + { + /* + Check if the name supplied is a valid mysql 5.0 name and + make the name a zero length string if it's not. + Note that just returning zero length is not enough : + a lot of places don't check the return value and expect + a zero terminated string. + */ + if (check_table_name(to, length, TRUE)) + { + to[0]= 0; + length= 0; + } DBUG_RETURN(length); + } length= strconvert(system_charset_info, from, &my_charset_filename, to, to_length, &errors); if (check_if_legal_tablename(to) && === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2010-04-28 12:52:24 +0000 +++ b/sql/sql_yacc.yy 2010-05-10 07:34:49 +0000 @@ -6149,7 +6149,7 @@ alter_list_item: { MYSQL_YYABORT; } - if (check_table_name($3->table.str,$3->table.length) || + if (check_table_name($3->table.str,$3->table.length, FALSE) || ($3->db.str && check_db_name(&$3->db))) { my_error(ER_WRONG_TABLE_NAME, MYF(0), $3->table.str); === modified file 'sql/table.cc' --- a/sql/table.cc 2010-04-28 12:52:24 +0000 +++ b/sql/table.cc 2010-05-10 07:34:49 +0000 @@ -494,6 +494,19 @@ inline bool is_system_table_name(const c } +/** + Check if a string contains path elements +*/ + +static inline bool has_disabled_path_chars(const char *str) +{ + for (; *str; str++) + if (*str == FN_EXTCHAR || *str == '/' || *str == '\\' || *str == '~' || *str == '@') + return TRUE; + return FALSE; +} + + /* Read table definition from a binary / text based .frm file @@ -548,7 +561,8 @@ int open_table_def(THD *thd, TABLE_SHARE This kind of tables must have been opened only by the my_open() above. */ - if (strchr(share->table_name.str, '@') || + if (has_disabled_path_chars(share->table_name.str) || + has_disabled_path_chars(share->db.str) || !strncmp(share->db.str, MYSQL50_TABLE_NAME_PREFIX, MYSQL50_TABLE_NAME_PREFIX_LENGTH) || !strncmp(share->table_name.str, MYSQL50_TABLE_NAME_PREFIX, @@ -2718,7 +2732,6 @@ bool check_db_name(LEX_STRING *org_name) (name_length > NAME_CHAR_LEN)); /* purecov: inspected */ } - /* Allow anything as a table name, as long as it doesn't contain an ' ' at the end @@ -2726,7 +2739,7 @@ bool check_db_name(LEX_STRING *org_name) */ -bool check_table_name(const char *name, uint length) +bool check_table_name(const char *name, uint length, bool check_for_path_chars) { uint name_length= 0; // name length in symbols const char *end= name+length; @@ -2753,6 +2766,9 @@ bool check_table_name(const char *name, continue; } } + if (check_for_path_chars && + (*name == '/' || *name == '\\' || *name == '~' || *name == FN_EXTCHAR)) + return 1; #endif name++; name_length++; === modified file 'tests/mysql_client_test.c' --- a/tests/mysql_client_test.c 2010-01-11 13:15:28 +0000 +++ b/tests/mysql_client_test.c 2010-05-09 19:30:06 +0000 @@ -18092,6 +18092,50 @@ static void test_bug44495() DBUG_VOID_RETURN; } +static void test_bug53371() +{ + int rc; + MYSQL_RES *result; + + myheader("test_bug53371"); + + rc= mysql_query(mysql, "DROP TABLE IF EXISTS t1"); + myquery(rc); + rc= mysql_query(mysql, "DROP DATABASE IF EXISTS bug53371"); + myquery(rc); + rc= mysql_query(mysql, "DROP USER 'testbug'@localhost"); + + rc= mysql_query(mysql, "CREATE TABLE t1 (a INT)"); + myquery(rc); + rc= mysql_query(mysql, "CREATE DATABASE bug53371"); + myquery(rc); + rc= mysql_query(mysql, "GRANT SELECT ON bug53371.* to 'testbug'@localhost"); + myquery(rc); + + rc= mysql_change_user(mysql, "testbug", NULL, "bug53371"); + myquery(rc); + + rc= mysql_query(mysql, "SHOW COLUMNS FROM client_test_db.t1"); + DIE_UNLESS(rc); + DIE_UNLESS(mysql_errno(mysql) == 1142); + + result= mysql_list_fields(mysql, "../client_test_db/t1", NULL); + DIE_IF(result); + + result= mysql_list_fields(mysql, "#mysql50#/../client_test_db/t1", NULL); + DIE_IF(result); + + rc= mysql_change_user(mysql, opt_user, opt_password, current_db); + myquery(rc); + rc= mysql_query(mysql, "DROP TABLE t1"); + myquery(rc); + rc= mysql_query(mysql, "DROP DATABASE bug53371"); + myquery(rc); + rc= mysql_query(mysql, "DROP USER 'testbug'@localhost"); + myquery(rc); +} + + /* Read and parse arguments and MySQL options from my.cnf */ @@ -18401,6 +18445,7 @@ static struct my_tests_st my_tests[]= { { "test_bug30472", test_bug30472 }, { "test_bug20023", test_bug20023 }, { "test_bug45010", test_bug45010 }, + { "test_bug53371", test_bug53371 }, { "test_bug31418", test_bug31418 }, { "test_bug31669", test_bug31669 }, { "test_bug28386", test_bug28386 },

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2859)
by knielsen＠knielsen-hq.org 10 May '10

10 May '10

#At lp:maria 2859 knielsen(a)knielsen-hq.org 2010-05-10 Suppress a safemutex warning pending fix of MBug#578117. modified: mysql-test/suite/pbxt/r/pbxt_xa.result mysql-test/suite/pbxt/t/pbxt_xa.test === modified file 'mysql-test/suite/pbxt/r/pbxt_xa.result' --- a/mysql-test/suite/pbxt/r/pbxt_xa.result 2010-03-24 22:12:39 +0000 +++ b/mysql-test/suite/pbxt/r/pbxt_xa.result 2010-05-10 07:29:30 +0000 @@ -1,4 +1,5 @@ drop table if exists t1, t2; +CALL mtr.add_suppression("Found wrong usage of mutex 'LOCK_sync' and 'LOCK_active'"); CREATE TABLE t1 (a INT PRIMARY KEY) ENGINE=innodb; CREATE TABLE t2 (b INT PRIMARY KEY) ENGINE=pbxt; BEGIN; === modified file 'mysql-test/suite/pbxt/t/pbxt_xa.test' --- a/mysql-test/suite/pbxt/t/pbxt_xa.test 2010-03-24 22:12:39 +0000 +++ b/mysql-test/suite/pbxt/t/pbxt_xa.test 2010-05-10 07:29:30 +0000 @@ -4,6 +4,11 @@ drop table if exists t1, t2; --enable_warnings +# This warning is indication of a real bug, MBug#578117. +# But it is not a regression, so we suppress it to get a clean test run. +# This suppression must be removed as part of MBug#578117 fix. +CALL mtr.add_suppression("Found wrong usage of mutex 'LOCK_sync' and 'LOCK_active'"); + # # bug lp:544173, xa crash with two 2pc-capable storage engines without binlog #

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2834) Bug#53371
by knielsen＠knielsen-hq.org 09 May '10

09 May '10

#At lp:maria 2834 knielsen(a)knielsen-hq.org 2010-05-09 Cherry-pick fix for Bug#53371, security hole with bypassing grants using special path in db/table names. Bump MariaDB version for security fix release. modified: configure.in mysql-test/r/grant.result mysql-test/t/grant.test sql/mysql_priv.h sql/partition_info.cc sql/sql_parse.cc sql/sql_table.cc sql/sql_yacc.yy sql/table.cc tests/mysql_client_test.c === modified file 'configure.in' --- a/configure.in 2010-04-29 07:57:25 +0000 +++ b/configure.in 2010-05-09 19:30:06 +0000 @@ -7,7 +7,7 @@ AC_PREREQ(2.59) # Remember to also update version.c in ndb. # When changing major version number please also check switch statement # in mysqlbinlog::check_master_version(). -AC_INIT([MariaDB Server], [5.1.44a-MariaDB], [], [mysql]) +AC_INIT([MariaDB Server], [5.1.44b-MariaDB], [], [mysql]) AC_CONFIG_SRCDIR([sql/mysqld.cc]) AC_CANONICAL_SYSTEM # USTAR format gives us the possibility to store longer path names in === modified file 'mysql-test/r/grant.result' --- a/mysql-test/r/grant.result 2010-01-29 10:42:31 +0000 +++ b/mysql-test/r/grant.result 2010-05-09 19:30:06 +0000 @@ -1413,3 +1413,19 @@ DROP USER 'user1'; DROP USER 'user1'@'localhost'; DROP USER 'user2'; DROP DATABASE db1; +CREATE DATABASE db1; +CREATE DATABASE db2; +GRANT SELECT ON db1.* to 'testbug'@localhost; +USE db2; +CREATE TABLE t1 (a INT); +USE test; +SELECT * FROM `../db2/tb2`; +ERROR 42S02: Table 'db1.../db2/tb2' doesn't exist +SELECT * FROM `../db2`.tb2; +ERROR 42000: SELECT command denied to user 'testbug'@'localhost' for table 'tb2' +SELECT * FROM `#mysql50#/../db2/tb2`; +ERROR 42S02: Table 'db1.#mysql50#/../db2/tb2' doesn't exist +DROP USER 'testbug'@localhost; +DROP TABLE db2.t1; +DROP DATABASE db1; +DROP DATABASE db2; === modified file 'mysql-test/t/grant.test' --- a/mysql-test/t/grant.test 2010-01-29 10:42:31 +0000 +++ b/mysql-test/t/grant.test 2010-05-09 19:30:06 +0000 @@ -1525,5 +1525,30 @@ DROP USER 'user1'@'localhost'; DROP USER 'user2'; DROP DATABASE db1; + +# +# Bug #53371: COM_FIELD_LIST can be abused to bypass table level grants. +# + +CREATE DATABASE db1; +CREATE DATABASE db2; +GRANT SELECT ON db1.* to 'testbug'@localhost; +USE db2; +CREATE TABLE t1 (a INT); +USE test; +connect (con1,localhost,testbug,,db1); +--error ER_NO_SUCH_TABLE +SELECT * FROM `../db2/tb2`; +--error ER_TABLEACCESS_DENIED_ERROR +SELECT * FROM `../db2`.tb2; +--error ER_NO_SUCH_TABLE +SELECT * FROM `#mysql50#/../db2/tb2`; +connection default; +disconnect con1; +DROP USER 'testbug'@localhost; +DROP TABLE db2.t1; +DROP DATABASE db1; +DROP DATABASE db2; + # Wait till we reached the initial number of concurrent sessions --source include/wait_until_count_sessions.inc === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-03-04 08:03:07 +0000 +++ b/sql/mysql_priv.h 2010-05-09 19:30:06 +0000 @@ -2300,7 +2300,7 @@ void update_create_info_from_table(HA_CR int rename_file_ext(const char * from,const char * to,const char * ext); bool check_db_name(LEX_STRING *db); bool check_column_name(const char *name); -bool check_table_name(const char *name, uint length); +bool check_table_name(const char *name, uint length, bool check_for_path_chars); char *get_field(MEM_ROOT *mem, Field *field); bool get_field(MEM_ROOT *mem, Field *field, class String *res); int wild_case_compare(CHARSET_INFO *cs, const char *str,const char *wildstr); === modified file 'sql/partition_info.cc' --- a/sql/partition_info.cc 2009-12-03 11:19:05 +0000 +++ b/sql/partition_info.cc 2010-05-09 19:30:06 +0000 @@ -972,7 +972,7 @@ bool partition_info::check_partition_inf part_elem->engine_type= default_engine_type; } if (check_table_name(part_elem->partition_name, - strlen(part_elem->partition_name))) + strlen(part_elem->partition_name), FALSE)) { my_error(ER_WRONG_PARTITION_NAME, MYF(0)); goto end; @@ -990,7 +990,7 @@ bool partition_info::check_partition_inf { sub_elem= sub_it++; if (check_table_name(sub_elem->partition_name, - strlen(sub_elem->partition_name))) + strlen(sub_elem->partition_name), FALSE)) { my_error(ER_WRONG_PARTITION_NAME, MYF(0)); goto end; === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-04-29 07:57:25 +0000 +++ b/sql/sql_parse.cc 2010-05-09 19:30:06 +0000 @@ -1334,6 +1334,11 @@ bool dispatch_command(enum enum_server_c system_charset_info, packet, db_length, thd->charset(), &dummy_errors); db_buff[db_length]= '\0'; + if (check_table_name(db_buff, db_length, FALSE)) + { + my_error(ER_WRONG_TABLE_NAME, MYF(0), db_buff); + break; + } table_list.alias= table_list.table_name= db_buff; if (!(fields= (char *) thd->memdup(wildcard, query_length + 1))) break; @@ -6298,7 +6303,7 @@ TABLE_LIST *st_select_lex::add_table_to_ DBUG_RETURN(0); // End of memory alias_str= alias ? alias->str : table->table.str; if (!test(table_options & TL_OPTION_ALIAS) && - check_table_name(table->table.str, table->table.length)) + check_table_name(table->table.str, table->table.length, FALSE)) { my_error(ER_WRONG_TABLE_NAME, MYF(0), table->table.str); DBUG_RETURN(0); === modified file 'sql/sql_table.cc' --- a/sql/sql_table.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_table.cc 2010-05-09 19:30:06 +0000 @@ -434,7 +434,21 @@ uint tablename_to_filename(const char *f DBUG_PRINT("enter", ("from '%s'", from)); if ((length= check_n_cut_mysql50_prefix(from, to, to_length))) + { + /* + Check if the name supplied is a valid mysql 5.0 name and + make the name a zero length string if it's not. + Note that just returning zero length is not enough : + a lot of places don't check the return value and expect + a zero terminated string. + */ + if (check_table_name(to, length, TRUE)) + { + to[0]= 0; + length= 0; + } DBUG_RETURN(length); + } length= strconvert(system_charset_info, from, &my_charset_filename, to, to_length, &errors); if (check_if_legal_tablename(to) && === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2010-03-04 08:03:07 +0000 +++ b/sql/sql_yacc.yy 2010-05-09 19:30:06 +0000 @@ -6149,7 +6149,7 @@ alter_list_item: { MYSQL_YYABORT; } - if (check_table_name($3->table.str,$3->table.length) || + if (check_table_name($3->table.str,$3->table.length, FALSE) || ($3->db.str && check_db_name(&$3->db))) { my_error(ER_WRONG_TABLE_NAME, MYF(0), $3->table.str); === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-10 10:32:14 +0000 +++ b/sql/table.cc 2010-05-09 19:30:06 +0000 @@ -494,6 +494,19 @@ inline bool is_system_table_name(const c } +/** + Check if a string contains path elements +*/ + +static inline bool has_disabled_path_chars(const char *str) +{ + for (; *str; str++) + if (*str == FN_EXTCHAR || *str == '/' || *str == '\\' || *str == '~' || *str == '@') + return TRUE; + return FALSE; +} + + /* Read table definition from a binary / text based .frm file @@ -548,7 +561,8 @@ int open_table_def(THD *thd, TABLE_SHARE This kind of tables must have been opened only by the my_open() above. */ - if (strchr(share->table_name.str, '@') || + if (has_disabled_path_chars(share->table_name.str) || + has_disabled_path_chars(share->db.str) || !strncmp(share->db.str, MYSQL50_TABLE_NAME_PREFIX, MYSQL50_TABLE_NAME_PREFIX_LENGTH) || !strncmp(share->table_name.str, MYSQL50_TABLE_NAME_PREFIX, @@ -2718,7 +2732,6 @@ bool check_db_name(LEX_STRING *org_name) (name_length > NAME_CHAR_LEN)); /* purecov: inspected */ } - /* Allow anything as a table name, as long as it doesn't contain an ' ' at the end @@ -2726,7 +2739,7 @@ bool check_db_name(LEX_STRING *org_name) */ -bool check_table_name(const char *name, uint length) +bool check_table_name(const char *name, uint length, bool check_for_path_chars) { uint name_length= 0; // name length in symbols const char *end= name+length; @@ -2753,6 +2766,9 @@ bool check_table_name(const char *name, continue; } } + if (check_for_path_chars && + (*name == '/' || *name == '\\' || *name == '~' || *name == FN_EXTCHAR)) + return 1; #endif name++; name_length++; === modified file 'tests/mysql_client_test.c' --- a/tests/mysql_client_test.c 2010-01-11 13:15:28 +0000 +++ b/tests/mysql_client_test.c 2010-05-09 19:30:06 +0000 @@ -18092,6 +18092,50 @@ static void test_bug44495() DBUG_VOID_RETURN; } +static void test_bug53371() +{ + int rc; + MYSQL_RES *result; + + myheader("test_bug53371"); + + rc= mysql_query(mysql, "DROP TABLE IF EXISTS t1"); + myquery(rc); + rc= mysql_query(mysql, "DROP DATABASE IF EXISTS bug53371"); + myquery(rc); + rc= mysql_query(mysql, "DROP USER 'testbug'@localhost"); + + rc= mysql_query(mysql, "CREATE TABLE t1 (a INT)"); + myquery(rc); + rc= mysql_query(mysql, "CREATE DATABASE bug53371"); + myquery(rc); + rc= mysql_query(mysql, "GRANT SELECT ON bug53371.* to 'testbug'@localhost"); + myquery(rc); + + rc= mysql_change_user(mysql, "testbug", NULL, "bug53371"); + myquery(rc); + + rc= mysql_query(mysql, "SHOW COLUMNS FROM client_test_db.t1"); + DIE_UNLESS(rc); + DIE_UNLESS(mysql_errno(mysql) == 1142); + + result= mysql_list_fields(mysql, "../client_test_db/t1", NULL); + DIE_IF(result); + + result= mysql_list_fields(mysql, "#mysql50#/../client_test_db/t1", NULL); + DIE_IF(result); + + rc= mysql_change_user(mysql, opt_user, opt_password, current_db); + myquery(rc); + rc= mysql_query(mysql, "DROP TABLE t1"); + myquery(rc); + rc= mysql_query(mysql, "DROP DATABASE bug53371"); + myquery(rc); + rc= mysql_query(mysql, "DROP USER 'testbug'@localhost"); + myquery(rc); +} + + /* Read and parse arguments and MySQL options from my.cnf */ @@ -18401,6 +18445,7 @@ static struct my_tests_st my_tests[]= { { "test_bug30472", test_bug30472 }, { "test_bug20023", test_bug20023 }, { "test_bug45010", test_bug45010 }, + { "test_bug53371", test_bug53371 }, { "test_bug31418", test_bug31418 }, { "test_bug31669", test_bug31669 }, { "test_bug28386", test_bug28386 },

1 0

[Maria-developers] BuildBot Outage
by Adam M. Dutko 08 May '10

08 May '10

The following buildbots will be unavailable until May 12th: adutko-centos5-amd64 adutko-ultrasparc3 mariadb-brs Sorry for the inconvenience. -Adam

1 0

Re: [Maria-developers] [Drizzle-discuss] Learning Storage Engine API
by Venu Kalyan 08 May '10

08 May '10

Hi You should check this link, as it has everything on storage engine API: http://forge.mysql.com/wiki/MySQL_Internals_Custom_Engine Also; you should check the example storage engine source code (storage/example) for bootstrapping, so that you can get your dummy storage engine ready in few minutes. Once you start, it will be real fun and easy to start though... Have fun Venu On Mon, Apr 19, 2010 at 1:35 PM, Igor K <igor175(a)gmail.com> wrote: > Dear all, > > I am new to the community and would like to ask a question: > > What is the best approach, in your opinion, to start learning storage > engine API? So that one can write a dummy storage engine in no time. > > Advices from the experts as well as various experiences from life are > welcomed. > > Igor Kozachenko > University of California, Berkeley > > _______________________________________________ > Mailing list: https://launchpad.net/~drizzle-discuss<https://launchpad.net/%7Edrizzle-discuss> > Post to : drizzle-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~drizzle-discuss<https://launchpad.net/%7Edrizzle-discuss> > More help : https://help.launchpad.net/ListHelp >

4 4

[Maria-developers] MariaDB Release Process
by Daniel Bartholomew 05 May '10

05 May '10

FYI: I just updated the Release Process page on the wiki. I removed some outdated information and added some new information (such as sending a note to maria-docs(a)lists.launchpad.net when a tree is ready for release instead of to docs(a)askmonty.org) http://askmonty.org/wiki/Release_Process Take a look if you are interested and let me know of any errors or omissions. Thanks. -- Daniel Bartholomew Monty Program - http://askmonty.org

1 0

Re: [Maria-developers] [Commits] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2851)
by Sergei Golubchik 05 May '10

05 May '10

Hi, knielsen! On May 05, knielsen(a)knielsen-hq.org wrote: > #At lp:maria > > 2851 knielsen(a)knielsen-hq.org 2010-05-05 > Change commit mails to go to commits(a)mariadb.org > modified: > .bzr-mysql/default.conf > > === modified file '.bzr-mysql/default.conf' > --- a/.bzr-mysql/default.conf 2010-03-04 08:03:07 +0000 > +++ b/.bzr-mysql/default.conf 2010-05-05 12:58:26 +0000 > @@ -1,6 +1,6 @@ > [MYSQL] > tree_location = lp:maria > -post_commit_to = maria-developers(a)lists.launchpad.net > +post_commit_to = commits(a)mariadb.org > post_commit_url = lp:maria > tree_name = maria > project_name = "MariaDB 5.1, with Maria 1.5" Don't bother chaning it - it only affects internal MySQL bzr plugin, we do not (and can not) use it. Regards, Sergei

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 May '10

05 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 29 ESTIMATE.......: 6 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 29 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 May '10

05 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 29 ESTIMATE.......: 6 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 29 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 May '10

05 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 29 ESTIMATE.......: 6 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 29 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Knielsen): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 May '10

05 May '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Code-Review PRIORITY.......: 60 WORKED HOURS...: 29 ESTIMATE.......: 6 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Knielsen - Wed, 05 May 2010, 13:53)=-=- Review of fixes to first review done. No new issues found. Worked 2 hours and estimate 6 hours remain (original estimate unchanged). -=-=(Knielsen - Fri, 23 Apr 2010, 12:51)=-=- Status updated. --- /tmp/wklog.47.old.28747 2010-04-23 12:51:36.000000000 +0000 +++ /tmp/wklog.47.new.28747 2010-04-23 12:51:36.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Knielsen - Tue, 06 Apr 2010, 15:26)=-=- Code review (mailed to maria-developers@). Worked 7 hours and estimate 8 hours remain (original estimate unchanged). -=-=(Knielsen - Tue, 06 Apr 2010, 15:25)=-=- Status updated. --- /tmp/wklog.47.old.12734 2010-04-06 15:25:54.000000000 +0000 +++ /tmp/wklog.47.new.12734 2010-04-06 15:25:54.000000000 +0000 @@ -1 +1 @@ -Code-Review +In-Progress -=-=(Knielsen - Mon, 29 Mar 2010, 10:59)=-=- Status updated. --- /tmp/wklog.47.old.27790 2010-03-29 10:59:53.000000000 +0000 +++ /tmp/wklog.47.new.27790 2010-03-29 10:59:53.000000000 +0000 @@ -1 +1 @@ -In-Progress +Code-Review -=-=(Alexi - Thu, 18 Feb 2010, 19:29)=-=- Worked 20 hours (alexi) Worked 20 hours and estimate 15 hours remain (original estimate unchanged). -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 29 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] commits@mariadb.org list
by Daniel Bartholomew 05 May '10

05 May '10

All, In order to make the maria-developers list more conversational, we've created a commits(a)mariadb.org mailing list for commit emails. If you want to see commit emails, subscribe to the commits list by sending an email with "subscribe" as the subject (no quotes) to: commits-request(a)mariadb.org If subscribing via email isn't your thing, you can subscribe to the list here: http://lists.askmonty.org/cgi-bin/mailman/listinfo/commits Maria Captains: After subscribing, please change the destination email address you have configured for the bzr email plugin in ~/.bazaar/bazaar.conf (or ~/.bazaar/locations.conf) to "commits(a)mariadb.org". Thanks! -- Daniel Bartholomew Monty Program - http://askmonty.org

2 1

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2851: Change commit mails to go to commits@mariadb.org
by noreply＠launchpad.net 05 May '10

05 May '10

------------------------------------------------------------ revno: 2851 committer: knielsen(a)knielsen-hq.org branch nick: mariadb-5.1 timestamp: Wed 2010-05-05 14:58:26 +0200 message: Change commit mails to go to commits(a)mariadb.org modified: .bzr-mysql/default.conf -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] New Maria Docs list
by Daniel Bartholomew 05 May '10

05 May '10

All, I've created a new "Maria Docs" Launchpad team and mailing list for documentation-related discussions (including things like documentation for current and upcoming releases of MariaDB, planning and discussion of the askmonty.org knowledgebase, and so on). The group is open to everyone. https://launchpad.net/~maria-docs Thanks! -- Daniel Bartholomew Monty Program - http://askmonty.org

1 0

Re: [Maria-developers] Route commit messages to separate list
by Daniel Bartholomew 03 May '10

03 May '10

On Mon, 3 May 2010 13:12:46 +0300 Henrik Ingo <hingo(a)askmonty.org> wrote: Henrik> Hi Colin Henrik> Henrik> You're working on moving commit messages to a separate list Henrik> from maria- developers. Henrik> Henrik> Since you're traveling, do you want to toss the ball to Daniel Henrik> to do that? You have the info on how to do it though. I've actually been working on this with Sergei. Colin, if you have any feedback, let me know! Thanks. -- Daniel Bartholomew Monty Program - http://askmonty.org

1 0

[Maria-developers] Please check optimiser regression in MySQL 5.1.46
by Kristian Nielsen 03 May '10

03 May '10

Hi Igor, Timour, Sergey, Can one of you please check this patch from MySQL 5.1.46? It is a commit to fix http://bugs.mysql.com/bug.php?id=51494 This patch introduces a regression: http://bugs.mysql.com/bug.php?id=53334 If I revert the patch, the regression disappears. And interestingly, the included test case does not fail even when the fix is reverted (go figure...) So it would be good if one of you could check the patch and check what is wrong with it, and if a different fix for Bug#51494 is needed. (needed to complete merge of MySQL 5.1.46). Thanks, - Kristian. ------------------------------------------------------------ revno: 3407.1.1 revision-id: sergey.glukhov(a)sun.com-20100319060102-57ykzjf4pc93avy1 parent: omer(a)mysql.com-20100318064207-l3ap0mpxt510b4n3 committer: Sergey Glukhov <Sergey.Glukhov(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2010-03-19 10:01:02 +0400 message: Bug#51494 crash with join, explain and 'sounds like' operator The crash happens because of discrepancy between values of conts_tables and join->const_table_map(make_join_statisctics). Calculation of conts_tables used condition with HA_STATS_RECORDS_IS_EXACT flag check. Calculation of join->const_table_map does not use this flag check. In case of MERGE table without union with index the table does not become const table and thus join_read_const_table() is not called for the table. join->const_table_map supposes this table is const and later in make_join_select this table is used for making&calculation const condition. As table record buffer is not populated it leads to crash. The fix is adding a check if an engine supports HA_STATS_RECORDS_IS_EXACT flag before updating join->const_table_map. diff: === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2010-03-14 16:01:45 +0000 +++ sql/sql_select.cc 2010-03-19 06:01:02 +0000 @@ -2943,7 +2943,8 @@ s->quick=select->quick; s->needed_reg=select->needed_reg; select->quick=0; - if (records == 0 && s->table->reginfo.impossible_range) + if (records == 0 && s->table->reginfo.impossible_range && + (s->table->file->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT)) { /* Impossible WHERE or ON expression === modified file 'mysql-test/r/merge.result' --- mysql-test/r/merge.result 2010-03-03 10:49:03 +0000 +++ mysql-test/r/merge.result 2010-03-19 06:01:02 +0000 @@ -2286,4 +2286,16 @@ DROP TABLE m1; DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`; +# +# Bug#51494c rash with join, explain and 'sounds like' operator +# +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables +DROP TABLE t2, t1; End of 5.1 tests === modified file 'mysql-test/t/merge.test' --- mysql-test/t/merge.test 2010-03-03 10:49:03 +0000 +++ mysql-test/t/merge.test 2010-03-19 06:01:02 +0000 @@ -1690,4 +1690,19 @@ DROP TABLE `test@1`.`t@1`; DROP DATABASE `test@1`; +--echo # +--echo # Bug#51494c rash with join, explain and 'sounds like' operator +--echo # + +CREATE TABLE t1 (a INT) ENGINE=MYISAM; +INSERT INTO t1 VALUES(1); + +CREATE TABLE t2 (b INT NOT NULL,c INT,d INT,e BLOB NOT NULL, +KEY idx0 (d, c)) ENGINE=MERGE; + +EXPLAIN SELECT * FROM t1 NATURAL RIGHT JOIN +t2 WHERE b SOUNDS LIKE e AND d = 1; + +DROP TABLE t2, t1; + --echo End of 5.1 tests

2 3

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2858) Bug#53334
by knielsen＠knielsen-hq.org 03 May '10

03 May '10

#At lp:maria 2858 knielsen(a)knielsen-hq.org 2010-05-03 Fixed bug #53334. The fix actually reverts the change introduced by the patch for bug 51494 The fact is that the patch for bug 52177 fixes bug 51194 as well. modified: mysql-test/r/innodb_mysql.result mysql-test/t/innodb_mysql.test sql/sql_select.cc === modified file 'mysql-test/r/innodb_mysql.result' --- a/mysql-test/r/innodb_mysql.result 2010-04-28 12:52:24 +0000 +++ b/mysql-test/r/innodb_mysql.result 2010-05-03 08:44:39 +0000 @@ -2350,4 +2350,34 @@ Null Index_type BTREE Comment DROP TABLE t1; +# +# Bug #53334: wrong result for outer join with impossible ON condition +# (see the same test case for MyISAM in join.test) +# +create table t1 (id int primary key); +create table t2 (id int); +insert into t1 values (75); +insert into t1 values (79); +insert into t1 values (78); +insert into t1 values (77); +replace into t1 values (76); +replace into t1 values (76); +insert into t1 values (104); +insert into t1 values (103); +insert into t1 values (102); +insert into t1 values (101); +insert into t1 values (105); +insert into t1 values (106); +insert into t1 values (107); +insert into t2 values (107),(75),(1000); +select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 +where t2.id=75 and t1.id is null; +id id +NULL 75 +explain select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 +where t2.id=75 and t1.id is null; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 const PRIMARY NULL NULL NULL 1 Impossible ON condition +1 SIMPLE t2 ALL NULL NULL NULL NULL 3 Using where +drop table t1,t2; End of 5.1 tests === modified file 'mysql-test/t/innodb_mysql.test' --- a/mysql-test/t/innodb_mysql.test 2010-04-28 12:52:24 +0000 +++ b/mysql-test/t/innodb_mysql.test 2010-05-03 08:44:39 +0000 @@ -591,4 +591,35 @@ ALTER TABLE t1 DROP INDEX k, ADD UNIQUE DROP TABLE t1; +--echo # +--echo # Bug #53334: wrong result for outer join with impossible ON condition +--echo # (see the same test case for MyISAM in join.test) +--echo # + +create table t1 (id int primary key); +create table t2 (id int); + +insert into t1 values (75); +insert into t1 values (79); +insert into t1 values (78); +insert into t1 values (77); +replace into t1 values (76); +replace into t1 values (76); +insert into t1 values (104); +insert into t1 values (103); +insert into t1 values (102); +insert into t1 values (101); +insert into t1 values (105); +insert into t1 values (106); +insert into t1 values (107); + +insert into t2 values (107),(75),(1000); + +select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 + where t2.id=75 and t1.id is null; +explain select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 + where t2.id=75 and t1.id is null; + +drop table t1,t2; + --echo End of 5.1 tests === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-04-28 12:52:24 +0000 +++ b/sql/sql_select.cc 2010-05-03 08:44:39 +0000 @@ -3008,8 +3008,7 @@ make_join_statistics(JOIN *join, TABLE_L s->quick=select->quick; s->needed_reg=select->needed_reg; select->quick=0; - if (records == 0 && s->table->reginfo.impossible_range && - (s->table->file->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT)) + if (records == 0 && s->table->reginfo.impossible_range) { /* Impossible WHERE or ON expression

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2857)
by knielsen＠knielsen-hq.org 02 May '10

02 May '10

#At lp:maria 2857 knielsen(a)knielsen-hq.org 2010-05-02 5.1.46 after-merge fixes: must FLUSH TABLES before copying .frm. modified: mysql-test/r/myisam.result mysql-test/t/myisam.test === modified file 'mysql-test/r/myisam.result' --- a/mysql-test/r/myisam.result 2010-04-28 12:52:24 +0000 +++ b/mysql-test/r/myisam.result 2010-05-02 06:47:28 +0000 @@ -2379,6 +2379,7 @@ DROP TABLE t1; CREATE TABLE t1(a INT, b BIT(1)); INSERT INTO t1 VALUES(1, 0), (2, 1); CREATE TABLE t2 SELECT * FROM t1; +FLUSH TABLES; CHECKSUM TABLE t1 EXTENDED; Table Checksum test.t1 3775188275 === modified file 'mysql-test/t/myisam.test' --- a/mysql-test/t/myisam.test 2010-04-28 12:52:24 +0000 +++ b/mysql-test/t/myisam.test 2010-05-02 06:47:28 +0000 @@ -1641,6 +1641,7 @@ DROP TABLE t1; CREATE TABLE t1(a INT, b BIT(1)); INSERT INTO t1 VALUES(1, 0), (2, 1); CREATE TABLE t2 SELECT * FROM t1; +FLUSH TABLES; --copy_file $MYSQLD_DATADIR/test/t1.frm $MYSQLD_DATADIR/test/t3.frm --copy_file $MYSQLD_DATADIR/test/t1.MYD $MYSQLD_DATADIR/test/t3.MYD --copy_file $MYSQLD_DATADIR/test/t1.MYI $MYSQLD_DATADIR/test/t3.MYI

1 0

[Maria-developers] PBXT test failure in MySQL 5.1.46 merge
by Kristian Nielsen 30 Apr '10

30 Apr '10

Hi Paul, I'm merging MySQL 5.1.46 into MariaDB. But I hit a test failure in PBXT, see below. It is wrong result from a SELECT. Interestingly, the same SELECT in the main test suite seems to work correctly. The code is in lp:~maria-captains/maria/5.1-release Can you maybe take a look, in case it is a PBXT issue? Probably also one of our optimiser people should take a look to see if it is a server problem. It's a bit strange that it seems to be pbxt specific, I'm wondering if there was a subtle change of the storage engine API introduced or something? Test failure link and output below. - Kristian. http://buildbot.askmonty.org/buildbot/reports/cross_reference#branch=5.1-re… Test case pbxt.join pbxt.join w4 [ fail ] Test ended at 2010-04-28 22:47:39 CURRENT_TEST: pbxt.join --- /space3/buildbot/makedist/build/install/mysql-test/suite/pbxt/r/join.result 2010-04-28 22:12:50.000000000 +0200 +++ /space3/buildbot/makedist/build/install/mysql-test/suite/pbxt/r/join.reject 2010-04-28 22:47:39.000000000 +0200 @@ -59,14 +59,13 @@ 107 1 select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 where t2.id=75 and t1.id is null; id id -NULL 75 explain select t1.id,t2.id from t2 left join t1 on t1.id>=74 and t1.id<=0 where t2.id=75 and t1.id is null; id select_type table type possible_keys key key_len ref rows Extra -1 SIMPLE t1 const PRIMARY NULL NULL NULL 1 Impossible ON condition -1 SIMPLE t2 ALL NULL NULL NULL NULL 3 Using where +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables explain select t1.id, t2.id from t1, t2 where t2.id = t1.id and t1.id <0 and t1.id > 0; id select_type table type possible_keys key key_len ref rows Extra -1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables +1 SIMPLE t2 ALL NULL NULL NULL NULL 3 Using where +1 SIMPLE t1 eq_ref PRIMARY PRIMARY 4 test.t2.id 1 Using index drop table t1,t2; CREATE TABLE t1 ( id int(11) NOT NULL auto_increment, mysqltest: Result length mismatch

2 5

[Maria-developers] Rev 2786: create table options bug: in http://bazaar.launchpad.net/~maria-captains/maria/5.2/
by serg＠askmonty.org 30 Apr '10

30 Apr '10

At http://bazaar.launchpad.net/~maria-captains/maria/5.2/ ------------------------------------------------------------ revno: 2786 revision-id: sergii(a)pisem.net-20100430200927-ns4ykuzr2l760bdc parent: sergii(a)pisem.net-20100430200435-1wr1ghgs9ahga0wg committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: 5.2 timestamp: Fri 2010-04-30 22:09:27 +0200 message: create table options bug: parse_engine_table_options() was only called when there was at least option with a non-default value. otherwise it was not called and option structure was not allocated at all. NULL pointer dereference in ::open(). === modified file 'sql/table.cc' --- a/sql/table.cc 2010-04-08 12:10:05 +0000 +++ b/sql/table.cc 2010-04-30 20:09:27 +0000 @@ -1633,10 +1633,11 @@ static int open_binary_frm(THD *thd, TAB if (share->db_create_options & HA_OPTION_TEXT_CREATE_OPTIONS) { DBUG_ASSERT(options_len); - if (engine_table_options_frm_read(options, options_len, share) || - parse_engine_table_options(thd, handler_file->ht, share)) + if (engine_table_options_frm_read(options, options_len, share)) goto free_and_err; } + if (parse_engine_table_options(thd, handler_file->ht, share)) + goto free_and_err; my_free(buff, MYF(MY_ALLOW_ZERO_PTR)); if (share->found_next_number_field) === modified file 'storage/example/ha_example.cc' --- a/storage/example/ha_example.cc 2010-04-30 10:12:25 +0000 +++ b/storage/example/ha_example.cc 2010-04-30 20:09:27 +0000 @@ -367,6 +367,15 @@ int ha_example::open(const char *name, i DBUG_RETURN(1); thr_lock_data_init(&share->lock,&lock,NULL); + example_table_options_struct *options= + (example_table_options_struct *)table->s->option_struct; + + DBUG_ASSERT(options); + DBUG_PRINT("info", ("strparam: '%-.64s' ullparam: %llu enumparam: %u "\ + "boolparam: %u", + (options->strparam ? options->strparam : "<NULL>"), + options->ullparam, options->enumparam, options->boolparam)); + DBUG_RETURN(0); }

1 0

[Maria-developers] Rev 2785: create table options bug: in http://bazaar.launchpad.net/~maria-captains/maria/5.2/
by serg＠askmonty.org 30 Apr '10

30 Apr '10

At http://bazaar.launchpad.net/~maria-captains/maria/5.2/ ------------------------------------------------------------ revno: 2785 revision-id: sergii(a)pisem.net-20100430200435-1wr1ghgs9ahga0wg parent: sergii(a)pisem.net-20100430101225-dhby6azvhpef1vme committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: 5.2 timestamp: Fri 2010-04-30 22:04:35 +0200 message: create table options bug: alter table does not reset HA_OPTION_TEXT_CREATE_OPTIONS when the last option value is removed === modified file 'mysql-test/r/plugin.result' --- a/mysql-test/r/plugin.result 2010-04-30 10:12:25 +0000 +++ b/mysql-test/r/plugin.result 2010-04-30 20:04:35 +0000 @@ -111,6 +111,12 @@ Table Create Table t1 CREATE TABLE `t1` ( `a` int(11) DEFAULT NULL ) ENGINE=EXAMPLE DEFAULT CHARSET=latin1 `ULL`=4660 +ALTER TABLE t1 ULL=DEFAULT; +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `a` int(11) DEFAULT NULL +) ENGINE=EXAMPLE DEFAULT CHARSET=latin1 DROP TABLE t1; SET @@SQL_MODE=@OLD_SQL_MODE; select 1; === modified file 'mysql-test/t/plugin.test' --- a/mysql-test/t/plugin.test 2010-04-30 10:12:25 +0000 +++ b/mysql-test/t/plugin.test 2010-04-30 20:04:35 +0000 @@ -119,6 +119,10 @@ CREATE TABLE t1 (a int) ENGINE=example U CREATE TABLE t1 (a int) ENGINE=example ULL=0x1234; SHOW CREATE TABLE t1; + +ALTER TABLE t1 ULL=DEFAULT; +SHOW CREATE TABLE t1; + DROP TABLE t1; SET @@SQL_MODE=@OLD_SQL_MODE; === modified file 'sql/unireg.cc' --- a/sql/unireg.cc 2010-04-08 12:10:05 +0000 +++ b/sql/unireg.cc 2010-04-30 20:04:35 +0000 @@ -197,6 +197,8 @@ bool mysql_create_frm(THD *thd, const ch create_info->table_options|= HA_OPTION_TEXT_CREATE_OPTIONS; create_info->extra_size+= (options_len + 4); } + else + create_info->table_options&= ~HA_OPTION_TEXT_CREATE_OPTIONS; if ((file=create_frm(thd, file_name, db, table, reclength, fileinfo, create_info, keys)) < 0)

1 0

[Maria-developers] Rev 2784: small changes to WL#43: in http://bazaar.launchpad.net/~maria-captains/maria/5.2/
by serg＠askmonty.org 30 Apr '10

30 Apr '10

At http://bazaar.launchpad.net/~maria-captains/maria/5.2/ ------------------------------------------------------------ revno: 2784 revision-id: sergii(a)pisem.net-20100430101225-dhby6azvhpef1vme parent: sergii(a)pisem.net-20100408210307-j82aap23b14h7j3t committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: 5.2 timestamp: Fri 2010-04-30 12:12:25 +0200 message: small changes to WL#43: consistency: don't use "index" and "key" interchangeably => rename "key" to "index" consistency: all option types are logical, besides ULL => rename ULL to NUMBER don't accept floats where integers are expected accept hexadecimal where integers are expected === modified file 'mysql-test/r/plugin.result' --- a/mysql-test/r/plugin.result 2010-04-08 17:19:01 +0000 +++ b/mysql-test/r/plugin.result 2010-04-30 10:12:25 +0000 @@ -101,6 +101,17 @@ drop table t1; SET SQL_MODE=''; CREATE TABLE t1 (a int) ENGINE=example ULL=10000000000000000000 one_or_two='ttt' YESNO=SSS; ERROR HY000: Incorrect value '10000000000000000000' for option 'ULL' +CREATE TABLE t1 (a int) ENGINE=example ULL=10.00; +ERROR 42000: Only integers allowed as number here near '10.00' at line 1 +CREATE TABLE t1 (a int) ENGINE=example ULL=1e2; +ERROR 42000: Only integers allowed as number here near '1e2' at line 1 +CREATE TABLE t1 (a int) ENGINE=example ULL=0x1234; +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `a` int(11) DEFAULT NULL +) ENGINE=EXAMPLE DEFAULT CHARSET=latin1 `ULL`=4660 +DROP TABLE t1; SET @@SQL_MODE=@OLD_SQL_MODE; select 1; 1 === modified file 'mysql-test/t/plugin.test' --- a/mysql-test/t/plugin.test 2010-04-08 17:19:01 +0000 +++ b/mysql-test/t/plugin.test 2010-04-30 10:12:25 +0000 @@ -110,6 +110,17 @@ drop table t1; SET SQL_MODE=''; --error ER_BAD_OPTION_VALUE CREATE TABLE t1 (a int) ENGINE=example ULL=10000000000000000000 one_or_two='ttt' YESNO=SSS; + +--error ER_PARSE_ERROR +CREATE TABLE t1 (a int) ENGINE=example ULL=10.00; + +--error ER_PARSE_ERROR +CREATE TABLE t1 (a int) ENGINE=example ULL=1e2; + +CREATE TABLE t1 (a int) ENGINE=example ULL=0x1234; +SHOW CREATE TABLE t1; +DROP TABLE t1; + SET @@SQL_MODE=@OLD_SQL_MODE; # === modified file 'sql/handler.h' --- a/sql/handler.h 2010-04-08 12:10:05 +0000 +++ b/sql/handler.h 2010-04-30 10:12:25 +0000 @@ -555,19 +555,19 @@ struct handler_log_file_data { /* Definitions for engine-specific table/field/index options in the CREATE TABLE. - Options are declared with HA_*OPTION_* macros (HA_TOPTION_ULL, HA_FOPTION_ENUM, - HA_KOPTION_STRING, etc). + Options are declared with HA_*OPTION_* macros (HA_TOPTION_NUMBER, + HA_FOPTION_ENUM, HA_IOPTION_STRING, etc). Every macros takes the option name, and the name of the underlying field of the appropriate C structure. The "appropriate C structure" is ha_table_option_struct for table level options, ha_field_option_struct for field level options, - ha_key_option_struct for key level options. The engine either + ha_index_option_struct for key level options. The engine either defines a structure of this name, or uses #define's to map these "appropriate" names to the actual structure type name. ULL options use a ulonglong as the backing store. - HA_*OPTION_ULL() takes the option name, the structure field name, + HA_*OPTION_NUMBER() takes the option name, the structure field name, the default value for the option, min, max, and blk_siz values. STRING options use a char* as a backing store. @@ -595,7 +595,7 @@ enum ha_option_type { HA_OPTION_TYPE_ULL HA_OPTION_TYPE_ENUM, /* uint */ HA_OPTION_TYPE_BOOL}; /* bool */ -#define HA_xOPTION_ULL(name, struc, field, def, min, max, blk_siz) \ +#define HA_xOPTION_NUMBER(name, struc, field, def, min, max, blk_siz) \ { HA_OPTION_TYPE_ULL, name, sizeof(name)-1, \ offsetof(struc, field), def, min, max, blk_siz, 0 } #define HA_xOPTION_STRING(name, struc, field) \ @@ -610,8 +610,8 @@ enum ha_option_type { HA_OPTION_TYPE_ULL offsetof(struc, field), def, 0, 1, 0, 0 } #define HA_xOPTION_END { HA_OPTION_TYPE_ULL, 0, 0, 0, 0, 0, 0, 0, 0 } -#define HA_TOPTION_ULL(name, field, def, min, max, blk_siz) \ - HA_xOPTION_ULL(name, ha_table_option_struct, field, def, min, max, blk_siz) +#define HA_TOPTION_NUMBER(name, field, def, min, max, blk_siz) \ + HA_xOPTION_NUMBER(name, ha_table_option_struct, field, def, min, max, blk_siz) #define HA_TOPTION_STRING(name, field) \ HA_xOPTION_STRING(name, ha_table_option_struct, field) #define HA_TOPTION_ENUM(name, field, values, def) \ @@ -620,8 +620,8 @@ enum ha_option_type { HA_OPTION_TYPE_ULL HA_xOPTION_BOOL(name, ha_table_option_struct, field, def) #define HA_TOPTION_END HA_xOPTION_END -#define HA_FOPTION_ULL(name, field, def, min, max, blk_siz) \ - HA_xOPTION_ULL(name, ha_field_option_struct, field, def, min, max, blk_siz) +#define HA_FOPTION_NUMBER(name, field, def, min, max, blk_siz) \ + HA_xOPTION_NUMBER(name, ha_field_option_struct, field, def, min, max, blk_siz) #define HA_FOPTION_STRING(name, field) \ HA_xOPTION_STRING(name, ha_field_option_struct, field) #define HA_FOPTION_ENUM(name, field, values, def) \ @@ -630,15 +630,15 @@ enum ha_option_type { HA_OPTION_TYPE_ULL HA_xOPTION_BOOL(name, ha_field_option_struct, field, def) #define HA_FOPTION_END HA_xOPTION_END -#define HA_KOPTION_ULL(name, field, def, min, max, blk_siz) \ - HA_xOPTION_ULL(name, ha_key_option_struct, field, def, min, max, blk_siz) -#define HA_KOPTION_STRING(name, field) \ - HA_xOPTION_STRING(name, ha_key_option_struct, field) -#define HA_KOPTION_ENUM(name, field, values, def) \ - HA_xOPTION_ENUM(name, ha_key_option_struct, field, values, def) -#define HA_KOPTION_BOOL(name, field, values, def) \ - HA_xOPTION_BOOL(name, ha_key_option_struct, field, values, def) -#define HA_KOPTION_END HA_xOPTION_END +#define HA_IOPTION_NUMBER(name, field, def, min, max, blk_siz) \ + HA_xOPTION_NUMBER(name, ha_index_option_struct, field, def, min, max, blk_siz) +#define HA_IOPTION_STRING(name, field) \ + HA_xOPTION_STRING(name, ha_index_option_struct, field) +#define HA_IOPTION_ENUM(name, field, values, def) \ + HA_xOPTION_ENUM(name, ha_index_option_struct, field, values, def) +#define HA_IOPTION_BOOL(name, field, values, def) \ + HA_xOPTION_BOOL(name, ha_index_option_struct, field, values, def) +#define HA_IOPTION_END HA_xOPTION_END typedef struct st_ha_create_table_option { enum ha_option_type type; @@ -1060,7 +1060,7 @@ typedef struct st_ha_create_information /* the following three are only for ALTER TABLE, check_if_incompatible_data() */ void *option_struct; ///< structure with parsed table options void **fileds_option_struct; ///< array of field option structures - void **keys_option_struct; ///< array of key option structures + void **indexes_option_struct; ///< array of index option structures } HA_CREATE_INFO; === modified file 'sql/sql_table.cc' --- a/sql/sql_table.cc 2010-04-08 12:10:05 +0000 +++ b/sql/sql_table.cc 2010-04-30 10:12:25 +0000 @@ -5783,7 +5783,7 @@ compare_tables(TABLE *table, if ((create_info->fileds_option_struct= (void**)thd->calloc(sizeof(void*) * table->s->fields)) == NULL || - (create_info->keys_option_struct= + (create_info->indexes_option_struct= (void**)thd->calloc(sizeof(void*) * table->s->keys)) == NULL) DBUG_RETURN(1); @@ -5984,7 +5984,7 @@ compare_tables(TABLE *table, else { DBUG_ASSERT(i < table->s->keys); - create_info->keys_option_struct[i]= new_key->option_struct; + create_info->indexes_option_struct[i]= new_key->option_struct; } } === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2010-04-08 12:10:05 +0000 +++ b/sql/sql_yacc.yy 2010-04-30 10:12:25 +0000 @@ -4769,7 +4769,7 @@ create_table_option: engine_option_value($1, $3, false, &Lex->create_info.option_list, &Lex->option_list_last); } - | IDENT_sys equal ulonglong_num + | IDENT_sys equal real_ulonglong_num { new (YYTHD->mem_root) engine_option_value($1, $3, &Lex->create_info.option_list, @@ -5434,7 +5434,7 @@ attribute: engine_option_value($1, $3, false, &Lex->option_list, &Lex->option_list_last); } - | IDENT_sys equal ulonglong_num + | IDENT_sys equal real_ulonglong_num { new (YYTHD->mem_root) engine_option_value($1, $3, &Lex->option_list, @@ -5746,7 +5746,7 @@ all_key_opt: engine_option_value($1, $3, false, &Lex->option_list, &Lex->option_list_last); } - | IDENT_sys equal ulonglong_num + | IDENT_sys equal real_ulonglong_num { new (YYTHD->mem_root) engine_option_value($1, $3, &Lex->option_list, @@ -9472,6 +9472,7 @@ ulonglong_num: real_ulonglong_num: NUM { int error; $$= (ulonglong) my_strtoll10($1.str, (char**) 0, &error); } | ULONGLONG_NUM { int error; $$= (ulonglong) my_strtoll10($1.str, (char**) 0, &error); } + | HEX_NUM { $$= strtoull($1.str, (char**) 0, 16); } | LONG_NUM { int error; $$= (ulonglong) my_strtoll10($1.str, (char**) 0, &error); } | dec_num_error { MYSQL_YYABORT; } ; === modified file 'storage/example/ha_example.cc' --- a/storage/example/ha_example.cc 2010-04-08 12:10:05 +0000 +++ b/storage/example/ha_example.cc 2010-04-30 10:12:25 +0000 @@ -151,7 +151,7 @@ ha_create_table_option example_table_opt range of values 0..UINT_MAX32, and a "block size" of 10 (any value must be divisible by 10). */ - HA_TOPTION_ULL("ULL", ullparam, UINT_MAX32, 0, UINT_MAX32, 10), + HA_TOPTION_NUMBER("ULL", ullparam, UINT_MAX32, 0, UINT_MAX32, 10), /* one option that takes an arbitrary string */

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2856)
by knielsen＠knielsen-hq.org 30 Apr '10

30 Apr '10

#At lp:maria 2856 knielsen(a)knielsen-hq.org 2010-04-30 [merge] Merge 5.1.44a security release + fix a couple compiler warnings. modified: storage/xtradb/fil/fil0fil.c strings/ctype-utf8.c === modified file 'storage/xtradb/fil/fil0fil.c' --- a/storage/xtradb/fil/fil0fil.c 2010-04-28 14:35:00 +0000 +++ b/storage/xtradb/fil/fil0fil.c 2010-04-30 04:23:39 +0000 @@ -631,7 +631,7 @@ fil_node_open_file( fil_system_t* system, /*!< in: tablespace memory cache */ fil_space_t* space) /*!< in: space */ { - ib_int64_t size_bytes; + ib_uint64_t size_bytes; ulint size_low; ulint size_high; ibool ret; @@ -672,8 +672,8 @@ fil_node_open_file( os_file_get_size(node->handle, &size_low, &size_high); - size_bytes = (((ib_int64_t)size_high) << 32) - + (ib_int64_t)size_low; + size_bytes = (((ib_uint64_t)size_high) << 32) + + (ib_uint64_t)size_low; #ifdef UNIV_HOTBACKUP if (space->id == 0) { node->size = (ulint) (size_bytes / UNIV_PAGE_SIZE); @@ -3443,7 +3443,7 @@ fil_load_single_table_tablespace( ulint flags; ulint size_low; ulint size_high; - ib_int64_t size; + ib_uint64_t size; #ifdef UNIV_HOTBACKUP fil_space_t* space; #endif @@ -3563,7 +3563,7 @@ fil_load_single_table_tablespace( /* Every .ibd file is created >= 4 pages in size. Smaller files cannot be ok. */ - size = (((ib_int64_t)size_high) << 32) + (ib_int64_t)size_low; + size = (((ib_uint64_t)size_high) << 32) + (ib_uint64_t)size_low; #ifndef UNIV_HOTBACKUP if (size < FIL_IBD_FILE_INITIAL_SIZE * UNIV_PAGE_SIZE) { fprintf(stderr, @@ -4789,13 +4789,16 @@ ibool fil_area_is_exist( /*==============*/ ulint space_id, /*!< in: space id */ - ulint zip_size, /*!< in: compressed page size in bytes; + ulint zip_size __attribute__((unused)), + /*!< in: compressed page size in bytes; 0 for uncompressed pages */ ulint block_offset, /*!< in: offset in number of blocks */ - ulint byte_offset, /*!< in: remainder of offset in bytes; in + ulint byte_offset __attribute__((unused)), + /*!< in: remainder of offset in bytes; in aio this must be divisible by the OS block size */ - ulint len) /*!< in: how many bytes to read or write; this + ulint len __attribute__((unused))) + /*!< in: how many bytes to read or write; this must not cross a file boundary; in aio this must be a block size multiple */ { === modified file 'strings/ctype-utf8.c' --- a/strings/ctype-utf8.c 2010-03-30 12:36:49 +0000 +++ b/strings/ctype-utf8.c 2010-04-30 04:23:39 +0000 @@ -4116,6 +4116,10 @@ my_wc_mb_filename(CHARSET_INFO *cs __att { int code; char hex[]= "0123456789abcdef"; + + if (s >= e) + return MY_CS_TOOSMALL; + if (wc < 128 && filename_safe_char[wc]) { *s= (uchar) wc;

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (igor:2788)
by Igor Babaev 29 Apr '10

29 Apr '10

#At lp:maria/5.2 based on revid:psergey@askmonty.org-20100329200940-9ikx6gpww0gtsx00 2788 Igor Babaev 2010-04-29 The main patch for mwl#106. added: mysql-test/r/derived_view.result mysql-test/t/derived_view.test modified: mysql-test/r/derived.result mysql-test/r/explain.result mysql-test/r/func_str.result mysql-test/r/index_merge_myisam.result mysql-test/r/information_schema.result mysql-test/r/innodb_mysql.result mysql-test/r/myisam_mrr.result mysql-test/r/ps.result mysql-test/r/strict.result mysql-test/r/subselect.result mysql-test/r/subselect3.result mysql-test/r/subselect3_jcl6.result mysql-test/r/subselect_no_mat.result mysql-test/r/subselect_no_opts.result mysql-test/r/subselect_no_semijoin.result mysql-test/r/view.result mysql-test/r/view_grant.result sql/field.cc sql/field.h sql/handler.cc sql/item.cc sql/item.h sql/item_cmpfunc.cc sql/item_cmpfunc.h sql/item_func.cc sql/item_func.h sql/item_subselect.cc sql/item_subselect.h sql/mysql_priv.h sql/opt_range.cc sql/opt_subselect.cc sql/opt_sum.cc sql/records.cc sql/sp_head.cc sql/sql_acl.cc sql/sql_base.cc sql/sql_bitmap.h sql/sql_cache.cc sql/sql_class.cc sql/sql_class.h sql/sql_cursor.cc sql/sql_delete.cc sql/sql_derived.cc sql/sql_help.cc sql/sql_insert.cc sql/sql_lex.cc sql/sql_lex.h sql/sql_list.h sql/sql_load.cc sql/sql_olap.cc sql/sql_parse.cc sql/sql_prepare.cc sql/sql_select.cc sql/sql_select.h sql/sql_show.cc sql/sql_table.cc sql/sql_union.cc sql/sql_update.cc sql/sql_view.cc sql/sql_yacc.yy sql/table.cc sql/table.h === modified file 'mysql-test/r/derived.result' --- a/mysql-test/r/derived.result 2009-07-11 18:44:29 +0000 +++ b/mysql-test/r/derived.result 2010-04-29 21:10:39 +0000 @@ -57,9 +57,8 @@ a b a b 3 c 3 c explain select * from t1 as x1, (select * from t1) as x2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY x1 ALL NULL NULL NULL NULL 4 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 Using join buffer -2 DERIVED t1 ALL NULL NULL NULL NULL 4 +1 SIMPLE x1 ALL NULL NULL NULL NULL 4 +1 SIMPLE t1 ALL NULL NULL NULL NULL 4 Using join buffer drop table if exists t2,t3; select * from (select 1) as a; 1 @@ -91,7 +90,7 @@ a b 2 b explain select * from (select * from t1 union select * from t1) a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 3 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 8 2 DERIVED t1 ALL NULL NULL NULL NULL 4 3 UNION t1 ALL NULL NULL NULL NULL 4 NULL UNION RESULT <union2,3> ALL NULL NULL NULL NULL NULL @@ -113,9 +112,8 @@ a b 3 c explain select * from (select t1.*, t2.a as t2a from t1,t2 where t1.a=t2.a) t1; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t2 system NULL NULL NULL NULL 1 -2 DERIVED t1 ALL NULL NULL NULL NULL 4 Using where +1 SIMPLE t2 system NULL NULL NULL NULL 1 +1 SIMPLE t1 ALL NULL NULL NULL NULL 4 Using where drop table t1, t2; create table t1(a int not null, t char(8), index(a)); SELECT * FROM (SELECT * FROM t1) as b ORDER BY a ASC LIMIT 0,20; @@ -142,8 +140,7 @@ a t 20 20 explain select count(*) from t1 as tt1, (select * from t1) as tt2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Select tables optimized away -2 DERIVED t1 ALL NULL NULL NULL NULL 10000 +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Select tables optimized away drop table t1; SELECT * FROM (SELECT (SELECT * FROM (SELECT 1 as a) as a )) as b; (SELECT * FROM (SELECT 1 as a) as a ) @@ -172,30 +169,30 @@ insert into t1 values (NULL, 'a', 1), (N insert into t2 values (1, 100), (1, 101), (1, 102), (2, 100), (2, 103), (2, 104), (3, 101), (3, 102), (3, 105); SELECT STRAIGHT_JOIN d.pla_id, m2.mat_id FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; pla_id mat_id -100 1 -101 1 102 1 -103 2 +101 1 +100 1 104 2 +103 2 105 3 SELECT STRAIGHT_JOIN d.pla_id, m2.test FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; pla_id test -100 1 -101 1 102 1 -103 2 +101 1 +100 1 104 2 +103 2 105 3 explain SELECT STRAIGHT_JOIN d.pla_id, m2.mat_id FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY m2 ALL NULL NULL NULL NULL 9 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 6 Using where; Using join buffer +1 PRIMARY <derived2> ref key0 key0 7 test.m2.matintnum 2 Using where 2 DERIVED mp ALL NULL NULL NULL NULL 9 Using temporary; Using filesort 2 DERIVED m1 eq_ref PRIMARY PRIMARY 3 test.mp.mat_id 1 explain SELECT STRAIGHT_JOIN d.pla_id, m2.test FROM t1 m2 INNER JOIN (SELECT mp.pla_id, MIN(m1.matintnum) AS matintnum FROM t2 mp INNER JOIN t1 m1 ON mp.mat_id=m1.mat_id GROUP BY mp.pla_id) d ON d.matintnum=m2.matintnum; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY m2 ALL NULL NULL NULL NULL 9 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 6 Using where; Using join buffer +1 PRIMARY <derived2> ref key0 key0 7 test.m2.matintnum 2 Using where 2 DERIVED mp ALL NULL NULL NULL NULL 9 Using temporary; Using filesort 2 DERIVED m1 eq_ref PRIMARY PRIMARY 3 test.mp.mat_id 1 drop table t1,t2; @@ -230,9 +227,8 @@ count(*) 2 explain select count(*) from t1 INNER JOIN (SELECT A.E1, A.E2, A.E3 FROM t1 AS A WHERE A.E3 = (SELECT MAX(B.E3) FROM t1 AS B WHERE A.E2 = B.E2)) AS THEMAX ON t1.E1 = THEMAX.E2 AND t1.E1 = t1.E2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 -1 PRIMARY t1 eq_ref PRIMARY PRIMARY 4 THEMAX.E2 1 Using where -2 DERIVED A ALL NULL NULL NULL NULL 2 Using where +1 SIMPLE A ALL NULL NULL NULL NULL 2 Using where +1 SIMPLE t1 eq_ref PRIMARY PRIMARY 4 test.A.E2 1 Using where 3 DEPENDENT SUBQUERY B ALL NULL NULL NULL NULL 2 Using where drop table t1; create table t1 (a int); @@ -245,8 +241,8 @@ a a 2 2 explain select * from ( select * from t1 union select * from t1) a,(select * from t1 union select * from t1) b; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 -1 PRIMARY <derived4> ALL NULL NULL NULL NULL 2 Using join buffer +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 +1 PRIMARY <derived4> ALL NULL NULL NULL NULL 4 Using join buffer 4 DERIVED t1 ALL NULL NULL NULL NULL 2 5 UNION t1 ALL NULL NULL NULL NULL 2 NULL UNION RESULT <union4,5> ALL NULL NULL NULL NULL NULL @@ -311,7 +307,7 @@ a 7.0000 b 3.5000 explain SELECT s.name, AVG(s.val) AS median FROM (SELECT x.name, x.val FROM t1 x, t1 y WHERE x.name=y.name GROUP BY x.name, x.val HAVING SUM(y.val <= x.val) >= COUNT(*)/2 AND SUM(y.val >= x.val) >= COUNT(*)/2) AS s GROUP BY s.name; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 3 Using temporary; Using filesort +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 289 Using temporary; Using filesort 2 DERIVED x ALL NULL NULL NULL NULL 17 Using temporary; Using filesort 2 DERIVED y ALL NULL NULL NULL NULL 17 Using where; Using join buffer drop table t1; @@ -322,8 +318,7 @@ id select_type table type possible_keys 1 SIMPLE t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index explain select a from (select a from t2 where a>1) tt; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index +1 SIMPLE t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index drop table t2; CREATE TABLE `t1` ( `itemid` int(11) NOT NULL default '0', `grpid` varchar(15) NOT NULL default '', `vendor` int(11) NOT NULL default '0', `date_` date NOT NULL default '0000-00-00', `price` decimal(12,2) NOT NULL default '0.00', PRIMARY KEY (`itemid`,`grpid`,`vendor`,`date_`), KEY `itemid` (`itemid`,`vendor`), KEY `itemid_2` (`itemid`,`date_`)); insert into t1 values (128, 'rozn', 2, curdate(), 10), === added file 'mysql-test/r/derived_view.result' --- a/mysql-test/r/derived_view.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/derived_view.result 2010-04-29 21:10:39 +0000 @@ -0,0 +1,570 @@ +drop table if exists t1,t2; +drop view if exists v1,v2,v3,v4; +create table t1(f1 int, f11 int); +create table t2(f2 int, f22 int); +insert into t1 values(1,1),(2,2),(3,3),(5,5),(9,9),(7,7); +insert into t1 values(17,17),(13,13),(11,11),(15,15),(19,19); +insert into t2 values(1,1),(3,3),(2,2),(4,4),(8,8),(6,6); +insert into t2 values(12,12),(14,14),(10,10),(18,18),(16,16); +Tests: +for merged derived tables +explain for simple derived +explain select * from (select * from t1) tt; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 +select * from (select * from t1) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +9 9 +7 7 +17 17 +13 13 +11 11 +15 15 +19 19 +explain for multitable derived +explain extended select * from (select * from t1 join t2 on f1=f2) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t1` join `test`.`t2` where (`test`.`t2`.`f2` = `test`.`t1`.`f1`) +select * from (select * from t1 join t2 on f1=f2) tt; +f1 f11 f2 f22 +1 1 1 1 +3 3 3 3 +2 2 2 2 +explain for derived with where +explain extended +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f11` = 2) and (`test`.`t1`.`f1` in (2,3))) +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +f1 f11 +2 2 +join of derived +explain extended +select * from (select * from t1 where f1 in (2,3)) tt join +(select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` join `test`.`t1` where ((`test`.`t1`.`f1` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` in (1,2)) and (`test`.`t1`.`f1` in (2,3))) +select * from (select * from t1 where f1 in (2,3)) tt join +(select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; +f1 f11 f1 f11 +2 2 2 2 +flush status; +explain extended +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f11` = 2) and (`test`.`t1`.`f1` in (2,3))) +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +f1 f11 +2 2 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 12 +for merged views +create view v1 as select * from t1; +create view v2 as select * from t1 join t2 on f1=f2; +create view v3 as select * from t1 where f1 in (2,3); +create view v4 as select * from t2 where f2 in (2,3); +explain for simple views +explain extended select * from v1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` +select * from v1; +f1 f11 +1 1 +2 2 +3 3 +5 5 +9 9 +7 7 +17 17 +13 13 +11 11 +15 15 +19 19 +explain for multitable views +explain extended select * from v2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t1` join `test`.`t2` where (`test`.`t2`.`f2` = `test`.`t1`.`f1`) +select * from v2; +f1 f11 f2 f22 +1 1 1 1 +3 3 3 3 +2 2 2 2 +explain for views with where +explain extended select * from v3 where f11 in (1,3); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f11` in (1,3)) and (`test`.`t1`.`f1` in (2,3))) +select * from v3 where f11 in (1,3); +f1 f11 +3 3 +explain for joined views +explain extended +select * from v3 join v4 on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t1` join `test`.`t2` where ((`test`.`t2`.`f2` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` in (2,3)) and (`test`.`t1`.`f1` in (2,3))) +select * from v3 join v4 on f1=f2; +f1 f11 f2 f22 +3 3 3 3 +2 2 2 2 +flush status; +explain extended select * from v4 where f2 in (1,3); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` where ((`test`.`t2`.`f2` in (1,3)) and (`test`.`t2`.`f2` in (2,3))) +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from v4 where f2 in (1,3); +f2 f22 +3 3 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 12 +for materialized derived tables +explain for simple derived +explain extended select * from (select * from t1 group by f1) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` group by `test`.`t1`.`f1`) `tt` +select * from (select * from t1 having f1=f1) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +9 9 +7 7 +17 17 +13 13 +11 11 +15 15 +19 19 +explain showing created indexes +explain extended +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 100.00 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 100.00 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`tt`.`f2` AS `f2`,`tt`.`f22` AS `f22` from `test`.`t1` join (select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` group by `test`.`t2`.`f2`) `tt` where (`tt`.`f2` = `test`.`t1`.`f1`) +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +explain showing late materialization +flush status; +explain select * from t1 join (select * from t2 group by f2) tt on f1=f2; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 Using temporary; Using filesort +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 11 +Handler_read_next 3 +Handler_read_prev 0 +Handler_read_rnd 11 +Handler_read_rnd_next 36 +for materialized views +drop view v1,v2,v3; +create view v1 as select * from t1 group by f1; +create view v2 as select * from t2 group by f2; +create view v3 as select t1.f1,t1.f11 from t1 join t1 as t11 where t1.f1=t11.f1 +having t1.f1<100; +explain for simple derived +explain extended select * from v1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`v1` +select * from v1; +f1 f11 +1 1 +2 2 +3 3 +5 5 +7 7 +9 9 +11 11 +13 13 +15 15 +17 17 +19 19 +explain showing created indexes +explain extended select * from t1 join v2 on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 100.00 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 100.00 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`v2`.`f2` AS `f2`,`v2`.`f22` AS `f22` from `test`.`t1` join `test`.`v2` where (`v2`.`f2` = `test`.`t1`.`f1`) +select * from t1 join v2 on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +explain extended +select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 100.00 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 10 100.00 Using where +1 PRIMARY <derived3> ref key0 key0 5 test.t1.f1 10 100.00 Using where +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 +3 DERIVED t11 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t11 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`v31`.`f1` AS `f1`,`v31`.`f11` AS `f11`,`v3`.`f1` AS `f1`,`v3`.`f11` AS `f11` from `test`.`t1` join `test`.`v3` `v31` join `test`.`v3` where ((`v31`.`f1` = `test`.`t1`.`f1`) and (`v3`.`f1` = `test`.`t1`.`f1`)) +flush status; +select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +f1 f11 f1 f11 f1 f11 +1 1 1 1 1 1 +2 2 2 2 2 2 +3 3 3 3 3 3 +5 5 5 5 5 5 +9 9 9 9 9 9 +7 7 7 7 7 7 +17 17 17 17 17 17 +13 13 13 13 13 13 +11 11 11 11 11 11 +15 15 15 15 15 15 +19 19 19 19 19 19 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 22 +Handler_read_next 22 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 60 +explain showing late materialization +flush status; +explain select * from t1 join v2 on f1=f2; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 +1 PRIMARY <derived2> ref key0 key0 5 test.t1.f1 2 Using where +2 DERIVED t2 ALL NULL NULL NULL NULL 11 Using temporary; Using filesort +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 0 +Handler_read_next 0 +Handler_read_prev 0 +Handler_read_rnd 0 +Handler_read_rnd_next 0 +flush status; +select * from t1 join v2 on f1=f2; +f1 f11 f2 f22 +1 1 1 1 +2 2 2 2 +3 3 3 3 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 11 +Handler_read_next 3 +Handler_read_prev 0 +Handler_read_rnd 11 +Handler_read_rnd_next 36 +explain extended select * from v1 join v4 on f1=f2; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 11 100.00 Using where +1 PRIMARY <derived2> ref key0 key0 5 test.t2.f2 2 100.00 Using where +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11`,`test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`v1` join `test`.`t2` where ((`v1`.`f1` = `test`.`t2`.`f2`) and (`test`.`t2`.`f2` in (2,3))) +select * from v1 join v4 on f1=f2; +f1 f11 f2 f22 +3 3 3 3 +2 2 2 2 +merged derived in merged derived +explain extended select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7)) +select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2) zz; +f1 f11 +3 3 +5 5 +materialized derived in merged derived +explain extended select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) +select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; +f1 f11 +3 3 +5 5 +merged derived in materialized derived +explain extended select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `zz`.`f1` AS `f1`,`zz`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where ((`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7)) group by `test`.`t1`.`f1`) `zz` +select * from (select * from +(select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; +f1 f11 +3 3 +5 5 +materialized derived in materialized derived +explain extended select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `zz`.`f1` AS `f1`,`zz`.`f11` AS `f11` from (select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) group by `tt`.`f1`) `zz` +select * from (select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; +f1 f11 +3 3 +5 5 +mat in merged derived join mat in merged derived +explain extended select * from +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z +on x.f1 = z.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE <derived3> ALL key0 NULL NULL NULL 11 100.00 Using where +1 SIMPLE <derived5> ref key0 key0 5 tt.f1 2 100.00 Using where +5 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11`,`tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` join (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where ((`tt`.`f1` = `tt`.`f1`) and (`tt`.`f1` > 2) and (`tt`.`f1` > 2)) +flush status; +select * from +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join +(select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z +on x.f1 = z.f1; +f1 f11 f1 f11 +3 3 3 3 +5 5 5 5 +show status like 'Handler_read%'; +Variable_name Value +Handler_read_first 0 +Handler_read_key 2 +Handler_read_next 2 +Handler_read_prev 0 +Handler_read_rnd 8 +Handler_read_rnd_next 39 +flush status; +merged in merged derived join merged in merged derived +explain extended select * from +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) z +on x.f1 = z.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using join buffer +Warnings: +Note 1003 select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11`,`test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` join `test`.`t1` where ((`test`.`t1`.`f1` = `test`.`t1`.`f1`) and (`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7) and ((`test`.`t1`.`f1` > 2) and (`test`.`t1`.`f1` < 7))) +select * from +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join +(select * from +(select * from t1 where f1 < 7 ) tt where f1 > 2 ) z +on x.f1 = z.f1; +f1 f11 f1 f11 +3 3 3 3 +5 5 5 5 +materialized in materialized derived join +materialized in materialized derived +explain extended select * from +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z +on x.f1 = z.f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL key0 NULL NULL NULL 11 100.00 +1 PRIMARY <derived4> ref key0 key0 5 x.f1 2 100.00 Using where +4 DERIVED <derived5> ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +5 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +2 DERIVED <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `x`.`f1` AS `f1`,`x`.`f11` AS `f11`,`z`.`f1` AS `f1`,`z`.`f11` AS `f11` from (select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) group by `tt`.`f1`) `x` join (select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`f11` AS `f11` from `test`.`t1` where (`test`.`t1`.`f1` < 7) group by `test`.`t1`.`f1`) `tt` where (`tt`.`f1` > 2) group by `tt`.`f1`) `z` where (`z`.`f1` = `x`.`f1`) +select * from +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join +(select * from +(select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z +on x.f1 = z.f1; +f1 f11 f1 f11 +3 3 3 3 +5 5 5 5 +merged view in materialized derived +explain extended +select * from (select * from v4 group by 1) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED t2 ALL NULL NULL NULL NULL 11 100.00 Using where; Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f2` AS `f2`,`tt`.`f22` AS `f22` from (select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` where (`test`.`t2`.`f2` in (2,3)) group by 1) `tt` +select * from (select * from v4 group by 1) tt; +f2 f22 +2 2 +3 3 +materialized view in merged derived +explain extended +select * from ( select * from v1 where f1 < 7) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE <derived3> ALL NULL NULL NULL NULL 11 100.00 Using where +3 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`v1` where (`v1`.`f1` < 7) +select * from ( select * from v1 where f1 < 7) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +merged view in a merged view in a merged derived +create view v6 as select * from v4 where f2 < 7; +explain extended select * from (select * from v6) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where +Warnings: +Note 1003 select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22` from `test`.`t2` where ((`test`.`t2`.`f2` < 7) and (`test`.`t2`.`f2` in (2,3))) +select * from (select * from v6) tt; +f2 f22 +3 3 +2 2 +materialized view in a merged view in a materialized derived +create view v7 as select * from v1; +explain extended select * from (select * from v7 group by 1) tt; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 100.00 +2 DERIVED <derived4> ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +4 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `tt`.`f1` AS `f1`,`tt`.`f11` AS `f11` from (select `v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`v1` group by 1) `tt` +select * from (select * from v7 group by 1) tt; +f1 f11 +1 1 +2 2 +3 3 +5 5 +7 7 +9 9 +11 11 +13 13 +15 15 +17 17 +19 19 +join of above two +explain extended select * from v6 join v7 on f2=f1; +id select_type table type possible_keys key key_len ref rows filtered Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 11 100.00 Using where +1 SIMPLE <derived5> ref key0 key0 5 test.t2.f2 2 100.00 Using where +5 DERIVED t1 ALL NULL NULL NULL NULL 11 100.00 Using temporary; Using filesort +Warnings: +Note 1003 select `test`.`t2`.`f2` AS `f2`,`test`.`t2`.`f22` AS `f22`,`v1`.`f1` AS `f1`,`v1`.`f11` AS `f11` from `test`.`t2` join `test`.`v1` where ((`v1`.`f1` = `test`.`t2`.`f2`) and (`test`.`t2`.`f2` < 7) and (`test`.`t2`.`f2` in (2,3))) +select * from v6 join v7 on f2=f1; +f2 f22 f1 f11 +3 3 3 3 +2 2 2 2 +test two keys +explain select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 11 +1 PRIMARY <derived2> ALL key0 NULL NULL NULL 11 Using where; Using join buffer +1 PRIMARY xx ALL NULL NULL NULL NULL 11 Using where; Using join buffer +2 DERIVED t2 ALL NULL NULL NULL NULL 11 Using temporary; Using filesort +select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; +f1 f11 f2 f22 f1 f11 +1 1 1 1 1 1 +2 2 2 2 2 2 +3 3 3 3 3 3 +TODO: Add test with 64 tables mergeable view to test fall back to +materialization on tables > MAX_TABLES merge +drop table t1,t2; +drop view v1,v2,v3,v4,v6,v7; === modified file 'mysql-test/r/explain.result' --- a/mysql-test/r/explain.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/explain.result 2010-04-29 21:10:39 +0000 @@ -102,7 +102,7 @@ INSERT INTO t2 VALUES (),(),(); EXPLAIN SELECT 1 FROM (SELECT 1 FROM t2,t1 WHERE b < c GROUP BY 1 LIMIT 1) AS d2; id select_type table type possible_keys key key_len ref rows Extra -X X X X X X X X X const row not found +X X X X X X X X X X X X X X X X X X X X X X X X X X X Range checked for each record (index map: 0xFFFFFFFFFF) DROP TABLE t2; @@ -114,7 +114,7 @@ INSERT INTO t2 VALUES (1),(2); EXPLAIN EXTENDED SELECT 1 FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: @@ -122,7 +122,7 @@ Note 1003 select 1 AS `1` from (select c EXPLAIN EXTENDED SELECT 1 FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: @@ -132,7 +132,7 @@ prepare s1 from FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1'; execute s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: @@ -142,14 +142,14 @@ prepare s1 from FROM (SELECT COUNT(DISTINCT t1.a) FROM t1,t2 GROUP BY t1.a) AS s1'; execute s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: Note 1003 select 1 AS `1` from (select count(distinct `test`.`t1`.`a`) AS `COUNT(DISTINCT t1.a)` from `test`.`t1` join `test`.`t2` group by `test`.`t1`.`a`) `s1` execute s1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 100.00 2 DERIVED t1 ALL NULL NULL NULL NULL 2 100.00 Using temporary; Using filesort 2 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using join buffer Warnings: === modified file 'mysql-test/r/func_str.result' --- a/mysql-test/r/func_str.result 2009-12-04 15:36:58 +0000 +++ b/mysql-test/r/func_str.result 2010-04-29 21:10:39 +0000 @@ -2549,14 +2549,12 @@ create table t1(f1 tinyint default null) insert into t1 values (-1),(null); explain select 1 as a from t1,(select decode(f1,f1) as b from t1) a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY t1 ALL NULL NULL NULL NULL 2 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 Using join buffer -2 DERIVED t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 Using join buffer explain select 1 as a from t1,(select encode(f1,f1) as b from t1) a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY t1 ALL NULL NULL NULL NULL 2 -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 Using join buffer -2 DERIVED t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 +1 SIMPLE t1 ALL NULL NULL NULL NULL 2 Using join buffer drop table t1; # # Bug#49141: Encode function is significantly slower in 5.1 compared to 5.0 === modified file 'mysql-test/r/index_merge_myisam.result' --- a/mysql-test/r/index_merge_myisam.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/index_merge_myisam.result 2010-04-29 21:10:39 +0000 @@ -285,8 +285,7 @@ id select_type table type possible_keys NULL UNION RESULT <union1,2> ALL NULL NULL NULL NULL NULL explain select * from (select * from t1 where key1 = 3 or key2 =3) as Z where key8 >5; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index_merge i1,i2 i1,i2 4,4 NULL 2 Using union(i1,i2); Using where; Using index +1 SIMPLE t1 ALL i1,i2,i8 NULL NULL NULL 1024 Using where create table t3 like t0; insert into t3 select * from t0; alter table t3 add key9 int not null, add index i9(key9); === modified file 'mysql-test/r/information_schema.result' --- a/mysql-test/r/information_schema.result 2010-03-15 11:51:23 +0000 +++ b/mysql-test/r/information_schema.result 2010-04-29 21:10:39 +0000 @@ -1285,8 +1285,7 @@ id select_type table type possible_keys 1 SIMPLE tables ALL NULL NULL NULL NULL NULL Open_frm_only; Scanned all databases; Using filesort explain select * from (select table_name from information_schema.tables) as a; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found -2 DERIVED tables ALL NULL NULL NULL NULL NULL Skip_open_table; Scanned all databases +1 SIMPLE tables ALL NULL NULL NULL NULL NULL Skip_open_table; Scanned all databases drop view v1; create table t1 (f1 int(11)); create table t2 (f1 int(11), f2 int(11)); === modified file 'mysql-test/r/innodb_mysql.result' --- a/mysql-test/r/innodb_mysql.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/innodb_mysql.result 2010-04-29 21:10:39 +0000 @@ -1731,8 +1731,8 @@ EXPLAIN SELECT 1 FROM (SELECT COUNT(DISTINCT c1) FROM t1 WHERE c2 IN (1, 1) AND c3 = 2 GROUP BY c2) x; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index c3,c2 c2 10 NULL 5 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +2 DERIVED t1 index_merge c3,c2 c3,c2 5,10 NULL 1 Using intersect(c3,c2); Using where; Using filesort DROP TABLE t1; CREATE TABLE t1 (c1 REAL, c2 REAL, c3 REAL, KEY (c3), KEY (c2, c3)) ENGINE=InnoDB; @@ -1745,8 +1745,8 @@ EXPLAIN SELECT 1 FROM (SELECT COUNT(DISTINCT c1) FROM t1 WHERE c2 IN (1, 1) AND c3 = 2 GROUP BY c2) x; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index c3,c2 c2 18 NULL 5 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +2 DERIVED t1 index_merge c3,c2 c3,c2 9,18 NULL 1 Using intersect(c3,c2); Using where; Using filesort DROP TABLE t1; CREATE TABLE t1 (c1 DECIMAL(12,2), c2 DECIMAL(12,2), c3 DECIMAL(12,2), KEY (c3), KEY (c2, c3)) @@ -1760,8 +1760,8 @@ EXPLAIN SELECT 1 FROM (SELECT COUNT(DISTINCT c1) FROM t1 WHERE c2 IN (1, 1) AND c3 = 2 GROUP BY c2) x; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 -2 DERIVED t1 index c3,c2 c2 14 NULL 5 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +2 DERIVED t1 index_merge c3,c2 c3,c2 7,14 NULL 1 Using intersect(c3,c2); Using where; Using filesort DROP TABLE t1; End of 5.1 tests drop table if exists t1, t2, t3; === modified file 'mysql-test/r/myisam_mrr.result' --- a/mysql-test/r/myisam_mrr.result 2010-03-11 21:43:31 +0000 +++ b/mysql-test/r/myisam_mrr.result 2010-04-29 21:10:39 +0000 @@ -347,7 +347,7 @@ GROUP BY t2.pk ); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL NULL Impossible WHERE -2 SUBQUERY t2 ALL int_key int_key 5 3 33.33 Using index condition; Using filesort +2 SUBQUERY t2 ALL int_key int_key 5 const 3 33.33 Using index condition; Using filesort Warnings: Note 1003 select min(`test`.`t1`.`pk`) AS `MIN(t1.pk)` from `test`.`t1` where 0 DROP TABLE t1, t2; === modified file 'mysql-test/r/ps.result' --- a/mysql-test/r/ps.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/ps.result 2010-04-29 21:10:39 +0000 @@ -156,7 +156,6 @@ prepare stmt1 from @stmt ; execute stmt1 ; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables -6 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table 5 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 4 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables @@ -164,7 +163,6 @@ id select_type table type possible_keys execute stmt1 ; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables -6 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table 5 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 4 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables @@ -172,7 +170,6 @@ id select_type table type possible_keys explain SELECT (SELECT SUM(c1 + c12 + 0.0) FROM t2 where (t1.c2 - 0e-3) = t2.c2 GROUP BY t1.c15 LIMIT 1) as scalar_s, exists (select 1.0e+0 from t2 where t2.c3 * 9.0000000000 = t1.c4) as exists_s, c5 * 4 in (select c6 + 0.3e+1 from t2) as in_s, (c7 - 4, c8 - 4) in (select c9 + 4.0, c10 + 40e-1 from t2) as in_row_s FROM t1, (select c25 x, c32 y from t2) tt WHERE x * 1 = c25; id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables -6 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table 5 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 4 DEPENDENT SUBQUERY t2 system NULL NULL NULL NULL 0 const row not found 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables === modified file 'mysql-test/r/strict.result' --- a/mysql-test/r/strict.result 2009-06-11 16:21:32 +0000 +++ b/mysql-test/r/strict.result 2010-04-29 21:10:39 +0000 @@ -1107,6 +1107,8 @@ Warnings: Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date +Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date +Error 1411 Incorrect datetime value: '2004.12.12 10:22:61' for function str_to_date drop table t1; create table t1 (col1 char(3), col2 integer); insert into t1 (col1) values (cast(1000 as char(3))); === modified file 'mysql-test/r/subselect.result' --- a/mysql-test/r/subselect.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect.result 2010-04-29 21:10:39 +0000 @@ -46,13 +46,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -201,11 +201,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -365,9 +364,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM @@ -1339,7 +1338,7 @@ a explain extended select * from t2 where t2.a in (select a from t1); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where (`test`.`t1`.`a` = `test`.`t2`.`a`) select * from t2 where t2.a in (select a from t1 where t1.b <> 30); @@ -1349,7 +1348,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) select * from t2 where t2.a in (select t1.a from t1,t3 where t1.b=t3.a); @@ -1360,7 +1359,7 @@ explain extended select * from t2 where id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index 1 PRIMARY t3 index a a 5 NULL 3 100.00 Using index -1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 116 100.61 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 11 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1` join `test`.`t3`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` = `test`.`t3`.`a`)) insert into t1 values (3,31); @@ -1376,7 +1375,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) drop table t0, t1, t2, t3; === modified file 'mysql-test/r/subselect3.result' --- a/mysql-test/r/subselect3.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3.result 2010-04-29 21:10:39 +0000 @@ -879,7 +879,7 @@ Level Code Message Note 1276 Field or reference 'test.t1.a' of SELECT #3 was resolved in SELECT #2 Note 1276 Field or reference 'test.t1.c' of SELECT #3 was resolved in SELECT #2 Error 1054 Unknown column 'c' in 'field list' -Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `t1`.`c`) AS `(SELECT COUNT(a) FROM +Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `test`.`t1`.`c`) AS `(SELECT COUNT(a) FROM (SELECT COUNT(b) FROM t1) AS x GROUP BY c )` from `test`.`t1` group by `test`.`t1`.`b`) `y` DROP TABLE t1; @@ -1105,9 +1105,8 @@ a set @@optimizer_switch=default; explain select * from (select a from t0) X where a in (select a from t1); id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 -1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(<derived2>) -2 DERIVED t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(t0) drop table t0, t1; create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); === modified file 'mysql-test/r/subselect3_jcl6.result' --- a/mysql-test/r/subselect3_jcl6.result 2010-03-29 14:04:35 +0000 +++ b/mysql-test/r/subselect3_jcl6.result 2010-04-29 21:10:39 +0000 @@ -883,7 +883,7 @@ Level Code Message Note 1276 Field or reference 'test.t1.a' of SELECT #3 was resolved in SELECT #2 Note 1276 Field or reference 'test.t1.c' of SELECT #3 was resolved in SELECT #2 Error 1054 Unknown column 'c' in 'field list' -Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `t1`.`c`) AS `(SELECT COUNT(a) FROM +Note 1003 select `c` AS `c` from (select (select count(`test`.`t1`.`a`) AS `COUNT(a)` from (select count(`test`.`t1`.`b`) AS `COUNT(b)` from `test`.`t1`) `x` group by `test`.`t1`.`c`) AS `(SELECT COUNT(a) FROM (SELECT COUNT(b) FROM t1) AS x GROUP BY c )` from `test`.`t1` group by `test`.`t1`.`b`) `y` DROP TABLE t1; @@ -1110,9 +1110,8 @@ a set @@optimizer_switch=default; explain select * from (select a from t0) X where a in (select a from t1); id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 11 -1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(<derived2>); Using join buffer -2 DERIVED t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t0 ALL NULL NULL NULL NULL 11 +1 PRIMARY t1 ALL NULL NULL NULL NULL 20 Using where; FirstMatch(t0); Using join buffer drop table t0, t1; create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); === modified file 'mysql-test/r/subselect_no_mat.result' --- a/mysql-test/r/subselect_no_mat.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_mat.result 2010-04-29 21:10:39 +0000 @@ -50,13 +50,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -205,11 +205,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -369,9 +368,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM @@ -1343,7 +1342,7 @@ a explain extended select * from t2 where t2.a in (select a from t1); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where (`test`.`t1`.`a` = `test`.`t2`.`a`) select * from t2 where t2.a in (select a from t1 where t1.b <> 30); @@ -1353,7 +1352,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) select * from t2 where t2.a in (select t1.a from t1,t3 where t1.b=t3.a); @@ -1364,7 +1363,7 @@ explain extended select * from t2 where id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index 1 PRIMARY t3 index a a 5 NULL 3 100.00 Using index -1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 116 100.61 Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 10 test.t2.a,test.t3.a 11 100.00 Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1` join `test`.`t3`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` = `test`.`t3`.`a`)) insert into t1 values (3,31); @@ -1380,7 +1379,7 @@ a explain extended select * from t2 where t2.a in (select a from t1 where t1.b <> 30); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t2 index a a 5 NULL 4 100.00 Using index -1 PRIMARY t1 ref a a 5 test.t2.a 101 100.00 Using where; Using index; FirstMatch(t2) +1 PRIMARY t1 ref a a 5 test.t2.a 100 100.00 Using where; Using index; FirstMatch(t2) Warnings: Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` semi join (`test`.`t1`) where ((`test`.`t1`.`a` = `test`.`t2`.`a`) and (`test`.`t1`.`b` <> 30)) drop table t0, t1, t2, t3; === modified file 'mysql-test/r/subselect_no_opts.result' --- a/mysql-test/r/subselect_no_opts.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_opts.result 2010-04-29 21:10:39 +0000 @@ -50,13 +50,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -205,11 +205,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -369,9 +368,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM === modified file 'mysql-test/r/subselect_no_semijoin.result' --- a/mysql-test/r/subselect_no_semijoin.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/subselect_no_semijoin.result 2010-04-29 21:10:39 +0000 @@ -50,13 +50,13 @@ SELECT (SELECT a) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) EXPLAIN EXTENDED SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 1 100.00 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 100.00 3 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DERIVED NULL NULL NULL NULL NULL NULL NULL NULL No tables used Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select `b`.`a` AS `a`) = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -205,11 +205,10 @@ select (select t3.a from t3 where a<8 or explain extended select (select t3.a from t3 where a<8 order by 1 desc limit 1), a from (select * from t2 where a>1) as tt; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 -3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where +1 PRIMARY t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,`test`.`t2`.`a` AS `a` from `test`.`t2` where (`test`.`t2`.`a` > 1) select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -369,9 +368,9 @@ INSERT INTO t8 (pseudo,email) VALUES ('2 EXPLAIN EXTENDED SELECT pseudo,(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce')) FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index -4 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +4 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 -3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index +3 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 Using index Warnings: Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM === modified file 'mysql-test/r/view.result' --- a/mysql-test/r/view.result 2010-03-20 12:01:47 +0000 +++ b/mysql-test/r/view.result 2010-04-29 21:10:39 +0000 @@ -117,7 +117,7 @@ c 12 explain extended select c from v5; id select_type table type possible_keys key key_len ref rows filtered Extra -1 PRIMARY <derived3> ALL NULL NULL NULL NULL 5 100.00 +1 SIMPLE <derived3> ALL NULL NULL NULL NULL 5 100.00 3 DERIVED t1 ALL NULL NULL NULL NULL 5 100.00 Warnings: Note 1003 select (`v2`.`c` + 1) AS `c` from `test`.`v2` @@ -237,7 +237,7 @@ a 3 explain select * from v1; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 3 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 6 2 DERIVED t1 ALL NULL NULL NULL NULL 6 Using temporary select * from t1; a @@ -302,7 +302,7 @@ a+1 4 explain select * from v1; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 2 DERIVED t1 ALL NULL NULL NULL NULL 4 Using filesort drop view v1; drop table t1; === modified file 'mysql-test/r/view_grant.result' --- a/mysql-test/r/view_grant.result 2009-10-16 11:12:21 +0000 +++ b/mysql-test/r/view_grant.result 2010-04-29 21:10:39 +0000 @@ -110,7 +110,7 @@ show create view mysqltest.v1; ERROR 42000: SHOW VIEW command denied to user 'mysqltest_1'@'localhost' for table 'v1' explain select c from mysqltest.v2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 2 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table show create view mysqltest.v2; ERROR 42000: SHOW VIEW command denied to user 'mysqltest_1'@'localhost' for table 'v2' @@ -131,7 +131,7 @@ View Create View character_set_client co v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `mysqltest`.`v1` AS select (`mysqltest`.`t1`.`a` + 1) AS `c`,(`mysqltest`.`t1`.`b` + 1) AS `d` from `mysqltest`.`t1` latin1 latin1_swedish_ci explain select c from mysqltest.v2; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 2 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table show create view mysqltest.v2; View Create View character_set_client collation_connection @@ -144,7 +144,7 @@ View Create View character_set_client co v3 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `mysqltest`.`v3` AS select (`mysqltest`.`t2`.`a` + 1) AS `c`,(`mysqltest`.`t2`.`b` + 1) AS `d` from `mysqltest`.`t2` latin1 latin1_swedish_ci explain select c from mysqltest.v4; id select_type table type possible_keys key key_len ref rows Extra -1 PRIMARY <derived2> system NULL NULL NULL NULL 0 const row not found +1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 2 DERIVED NULL NULL NULL NULL NULL NULL NULL no matching row in const table show create view mysqltest.v4; View Create View character_set_client collation_connection === added file 'mysql-test/t/derived_view.test' --- a/mysql-test/t/derived_view.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/derived_view.test 2010-04-29 21:10:39 +0000 @@ -0,0 +1,217 @@ +--disable_warnings +drop table if exists t1,t2; +drop view if exists v1,v2,v3,v4; +--enable_warnings +create table t1(f1 int, f11 int); +create table t2(f2 int, f22 int); +insert into t1 values(1,1),(2,2),(3,3),(5,5),(9,9),(7,7); +insert into t1 values(17,17),(13,13),(11,11),(15,15),(19,19); +insert into t2 values(1,1),(3,3),(2,2),(4,4),(8,8),(6,6); +insert into t2 values(12,12),(14,14),(10,10),(18,18),(16,16); + +--echo Tests: + +--echo for merged derived tables +--echo explain for simple derived +explain select * from (select * from t1) tt; +select * from (select * from t1) tt; +--echo explain for multitable derived +explain extended select * from (select * from t1 join t2 on f1=f2) tt; +select * from (select * from t1 join t2 on f1=f2) tt; +--echo explain for derived with where +explain extended + select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +--echo join of derived +explain extended + select * from (select * from t1 where f1 in (2,3)) tt join + (select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; +select * from (select * from t1 where f1 in (2,3)) tt join + (select * from t1 where f1 in (1,2)) aa on tt.f1=aa.f1; + +flush status; +explain extended + select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +show status like 'Handler_read%'; +flush status; +select * from (select * from t1 where f1 in (2,3)) tt where f11=2; +show status like 'Handler_read%'; + +--echo for merged views +create view v1 as select * from t1; +create view v2 as select * from t1 join t2 on f1=f2; +create view v3 as select * from t1 where f1 in (2,3); +create view v4 as select * from t2 where f2 in (2,3); +--echo explain for simple views +explain extended select * from v1; +select * from v1; +--echo explain for multitable views +explain extended select * from v2; +select * from v2; +--echo explain for views with where +explain extended select * from v3 where f11 in (1,3); +select * from v3 where f11 in (1,3); +--echo explain for joined views +explain extended + select * from v3 join v4 on f1=f2; +select * from v3 join v4 on f1=f2; + +flush status; +explain extended select * from v4 where f2 in (1,3); +show status like 'Handler_read%'; +flush status; +select * from v4 where f2 in (1,3); +show status like 'Handler_read%'; + +--echo for materialized derived tables +--echo explain for simple derived +explain extended select * from (select * from t1 group by f1) tt; +select * from (select * from t1 having f1=f1) tt; +--echo explain showing created indexes +explain extended + select * from t1 join (select * from t2 group by f2) tt on f1=f2; +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +--echo explain showing late materialization +flush status; +explain select * from t1 join (select * from t2 group by f2) tt on f1=f2; +show status like 'Handler_read%'; +flush status; +select * from t1 join (select * from t2 group by f2) tt on f1=f2; +show status like 'Handler_read%'; + +--echo for materialized views +drop view v1,v2,v3; +create view v1 as select * from t1 group by f1; +create view v2 as select * from t2 group by f2; +create view v3 as select t1.f1,t1.f11 from t1 join t1 as t11 where t1.f1=t11.f1 + having t1.f1<100; +--echo explain for simple derived +explain extended select * from v1; +select * from v1; +--echo explain showing created indexes +explain extended select * from t1 join v2 on f1=f2; +select * from t1 join v2 on f1=f2; +explain extended + select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +flush status; +select * from t1,v3 as v31,v3 where t1.f1=v31.f1 and t1.f1=v3.f1; +show status like 'Handler_read%'; +--echo explain showing late materialization +flush status; +explain select * from t1 join v2 on f1=f2; +show status like 'Handler_read%'; +flush status; +select * from t1 join v2 on f1=f2; +show status like 'Handler_read%'; + +explain extended select * from v1 join v4 on f1=f2; +select * from v1 join v4 on f1=f2; + +--echo merged derived in merged derived +explain extended select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2) zz; +select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2) zz; + +--echo materialized derived in merged derived +explain extended select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; +select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) zz; + +--echo merged derived in materialized derived +explain extended select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; +select * from (select * from + (select * from t1 where f1 < 7) tt where f1 > 2 group by f1) zz; + +--echo materialized derived in materialized derived +explain extended select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; +select * from (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) zz; + +--echo mat in merged derived join mat in merged derived +explain extended select * from + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z + on x.f1 = z.f1; + +flush status; +select * from + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) x +join + (select * from (select * from t1 where f1 < 7 group by f1) tt where f1 > 2) z + on x.f1 = z.f1; +show status like 'Handler_read%'; +flush status; + +--echo merged in merged derived join merged in merged derived +explain extended select * from + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) z + on x.f1 = z.f1; + +select * from + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) x +join + (select * from + (select * from t1 where f1 < 7 ) tt where f1 > 2 ) z + on x.f1 = z.f1; + +--echo materialized in materialized derived join +--echo materialized in materialized derived +explain extended select * from + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z + on x.f1 = z.f1; + +select * from + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) x +join + (select * from + (select * from t1 where f1 < 7 group by f1) tt where f1 > 2 group by f1) z + on x.f1 = z.f1; + +--echo merged view in materialized derived +explain extended +select * from (select * from v4 group by 1) tt; +select * from (select * from v4 group by 1) tt; + +--echo materialized view in merged derived +explain extended +select * from ( select * from v1 where f1 < 7) tt; +select * from ( select * from v1 where f1 < 7) tt; + +--echo merged view in a merged view in a merged derived +create view v6 as select * from v4 where f2 < 7; +explain extended select * from (select * from v6) tt; +select * from (select * from v6) tt; + +--echo materialized view in a merged view in a materialized derived +create view v7 as select * from v1; +explain extended select * from (select * from v7 group by 1) tt; +select * from (select * from v7 group by 1) tt; + +--echo join of above two +explain extended select * from v6 join v7 on f2=f1; +select * from v6 join v7 on f2=f1; + +--echo test two keys +explain select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; +select * from t1 join (select * from t2 group by f2) tt on t1.f1=tt.f2 join t1 xx on tt.f22=xx.f1; + + +--echo TODO: Add test with 64 tables mergeable view to test fall back to +--echo materialization on tables > MAX_TABLES merge +drop table t1,t2; +drop view v1,v2,v3,v4,v6,v7; === modified file 'sql/field.cc' --- a/sql/field.cc 2010-03-20 12:01:47 +0000 +++ b/sql/field.cc 2010-04-29 21:10:39 +0000 @@ -10458,3 +10458,27 @@ Field::set_datetime_warning(MYSQL_ERROR: field_name); } } + + +/* + @brief + Return possible keys for a field + + @details + Return bit map of keys over this field which can be used by the range + optimizer. For a field of a generic table such keys are all keys that starts + from this field. For a field of a materialized derived table/view such keys + are all keys in which this field takes a part. This is less restrictive as + keys for a materialized derived table/view are generated on the fly from + present fields, thus the case when a field for the beginning of a key is + absent is impossible. + + @return map of possible keys +*/ + +key_map Field::get_possible_keys() +{ + DBUG_ASSERT(table->pos_in_table_list); + return (table->pos_in_table_list->is_materialized_derived() ? + part_of_key : key_start); +} === modified file 'sql/field.h' --- a/sql/field.h 2010-03-15 11:51:23 +0000 +++ b/sql/field.h 2010-04-29 21:10:39 +0000 @@ -582,6 +582,9 @@ public: DBUG_ASSERT(0); return GEOM_GEOMETRY; } + + key_map get_possible_keys(); + /* Hash value */ virtual void hash(ulong *nr, ulong *nr2); friend bool reopen_table(THD *,struct st_table *,bool); === modified file 'sql/handler.cc' --- a/sql/handler.cc 2010-03-20 12:01:47 +0000 +++ b/sql/handler.cc 2010-04-29 21:10:39 +0000 @@ -2480,8 +2480,9 @@ int handler::update_auto_increment() void handler::column_bitmaps_signal() { DBUG_ENTER("column_bitmaps_signal"); - DBUG_PRINT("info", ("read_set: 0x%lx write_set: 0x%lx", (long) table->read_set, - (long) table->write_set)); + if (table) + DBUG_PRINT("info", ("read_set: 0x%lx write_set: 0x%lx", + (long) table->read_set, (long) table->write_set)); DBUG_VOID_RETURN; } === modified file 'sql/item.cc' --- a/sql/item.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item.cc 2010-04-29 21:10:39 +0000 @@ -711,6 +711,23 @@ bool Item_field::register_field_in_bitma return 0; } + +/* + Mark field in write_map + + NOTES + This is used by UPDATE to register underlying fields of used view fields. +*/ + +bool Item_field::register_field_in_write_map(uchar *arg) +{ + TABLE *table= (TABLE *) arg; + if (field->table == table || !table) + bitmap_set_bit(field->table->write_set, field->field_index); + return 0; +} + + bool Item::check_cols(uint c) { if (c != 1) @@ -2202,6 +2219,10 @@ table_map Item_field::used_tables() cons return (depended_from ? OUTER_REF_TABLE_BIT : field->table->map); } +table_map Item_field::all_used_tables() const +{ + return (depended_from ? OUTER_REF_TABLE_BIT : field->table->map); +} void Item_field::fix_after_pullout(st_select_lex *new_parent, Item **ref) { @@ -2454,7 +2475,7 @@ my_decimal *Item_float::val_decimal(my_d void Item_string::print(String *str, enum_query_type query_type) { - if (query_type == QT_ORDINARY && is_cs_specified()) + if (query_type != QT_IS && is_cs_specified()) { str->append('_'); str->append(collation.collation->csname); @@ -2462,7 +2483,7 @@ void Item_string::print(String *str, enu str->append('\''); - if (query_type == QT_ORDINARY || + if (query_type != QT_IS || my_charset_same(str_value.charset(), system_charset_info)) { str_value.print(str); @@ -3945,6 +3966,34 @@ resolve_ref_in_select_and_group(THD *thd } +/* + @brief + Whether a table belongs to an outer select. + + @param table table to check + @param select current select + + @details + Try to find select the table belongs to by ascending the derived tables chain. +*/ + +static +bool is_outer_table(TABLE_LIST *table, SELECT_LEX *select) +{ + DBUG_ASSERT(table->select_lex != select); + TABLE_LIST *tl; + + for (tl= select->master_unit()->derived; + tl && tl->is_merged_derived(); + select= tl->select_lex, tl= select->master_unit()->derived) + { + if (tl->select_lex == table->select_lex) + return FALSE; + } + return TRUE; +} + + /** Resolve the name of an outer select column reference. @@ -4382,7 +4431,8 @@ bool Item_field::fix_fields(THD *thd, It if (!outer_fixed && cached_table && cached_table->select_lex && context->select_lex && - cached_table->select_lex != context->select_lex) + cached_table->select_lex != context->select_lex && + is_outer_table(cached_table, context->select_lex)) { int ret; if ((ret= fix_outer_field(thd, &from_field, reference)) < 0) @@ -5786,8 +5836,9 @@ public: st_select_lex *sel; for (sel= current_select; sel; sel= sel->outer_select()) { + List_iterator<TABLE_LIST> li(sel->leaf_tables); TABLE_LIST *tbl; - for (tbl= sel->leaf_tables; tbl; tbl= tbl->next_leaf) + while ((tbl= li++)) { if (tbl->table == item->field->table) { @@ -7506,6 +7557,8 @@ Item_result Item_type_holder::result_typ enum_field_types Item_type_holder::get_real_type(Item *item) { + if (item->type() == REF_ITEM) + item= item->real_item(); switch(item->type()) { case FIELD_ITEM: === modified file 'sql/item.h' --- a/sql/item.h 2010-03-20 12:01:47 +0000 +++ b/sql/item.h 2010-04-29 21:10:39 +0000 @@ -778,6 +778,7 @@ public: class Field_enumerator) */ virtual table_map used_tables() const { return (table_map) 0L; } + virtual table_map all_used_tables() const { return used_tables(); } /* Return table map of tables that can't be NULL tables (tables that are used in a context where if they would contain a NULL row generated @@ -934,8 +935,12 @@ public: virtual bool reset_query_id_processor(uchar *query_id_arg) { return 0; } virtual bool is_expensive_processor(uchar *arg) { return 0; } virtual bool register_field_in_read_map(uchar *arg) { return 0; } + virtual bool register_field_in_write_map(uchar *arg) { return 0; } virtual bool enumerate_field_refs_processor(uchar *arg) { return 0; } virtual bool mark_as_eliminated_processor(uchar *arg) { return 0; } + virtual bool view_used_tables_processor(uchar *arg) { return 0; } + virtual bool eval_not_null_tables(uchar *opt_arg) { return 0; } + /* The next function differs from the previous one that a bitmap to be updated is passed as uchar *arg. @@ -1143,6 +1148,12 @@ public: { return Field::GEOM_GEOMETRY; }; String *check_well_formed_result(String *str, bool send_error= 0); bool eq_by_collation(Item *item, bool binary_cmp, CHARSET_INFO *cs); + table_map view_used_tables(TABLE_LIST *view) + { + view->view_used_tables= 0; + walk(&Item::view_used_tables_processor, 0, (uchar *) view); + return view->view_used_tables; + } }; @@ -1616,6 +1627,7 @@ public: int save_in_field(Field *field,bool no_conversions); void save_org_in_field(Field *field); table_map used_tables() const; + table_map all_used_tables() const; enum Item_result result_type () const { return field->result_type(); @@ -1645,6 +1657,7 @@ public: bool add_field_to_set_processor(uchar * arg); bool find_item_in_field_list_processor(uchar *arg); bool register_field_in_read_map(uchar *arg); + bool register_field_in_write_map(uchar *arg); bool register_field_in_bitmap(uchar *arg); bool check_partition_func_processor(uchar *int_arg) {return FALSE;} bool vcol_in_partition_func_processor(uchar *bool_arg); @@ -2515,11 +2528,14 @@ public: */ class Item_direct_view_ref :public Item_direct_ref { + TABLE_LIST *view; public: Item_direct_view_ref(Name_resolution_context *context_arg, Item **item, - const char *table_name_arg, - const char *field_name_arg) - :Item_direct_ref(context_arg, item, table_name_arg, field_name_arg) {} + const char *table_name_arg, + const char *field_name_arg, + TABLE_LIST *view_arg) + :Item_direct_ref(context_arg, item, table_name_arg, field_name_arg), + view(view_arg) {} /* Constructor need to process subselect with temporary tables (see Item) */ Item_direct_view_ref(THD *thd, Item_direct_ref *item) :Item_direct_ref(thd, item) {} @@ -2533,6 +2549,24 @@ public: return item; } virtual Ref_Type ref_type() { return VIEW_REF; } + table_map used_tables() const + { + return depended_from ? + OUTER_REF_TABLE_BIT : + (view->merged ? (*ref)->used_tables() : view->table->map); + } + bool walk(Item_processor processor, bool walk_subquery, uchar *arg) + { + return (*ref)->walk(processor, walk_subquery, arg) || + (this->*processor)(arg); + } + bool view_used_tables_processor(uchar *arg) + { + TABLE_LIST *view_arg= (TABLE_LIST *) arg; + if (view_arg == view) + view_arg->view_used_tables|= (*ref)->used_tables(); + return 0; + } }; @@ -2885,6 +2919,17 @@ public: value. */ +/* + Cached_item_XXX objects are not exactly caches. They do the following: + + Each Cached_item_XXX object has + - its source item + - saved value of the source item + - cmp() method that compares the saved value with the current value of the + source item, and if they were not equal saves item's value into the saved + value. +*/ + class Cached_item :public Sql_alloc { public: === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.cc 2010-04-29 21:10:39 +0000 @@ -2204,6 +2204,16 @@ bool Item_func_between::fix_fields(THD * thd->lex->current_select->between_count++; + + return 0; +} + + +bool Item_func_between::eval_not_null_tables(uchar *opt_arg) +{ + if (Item_func_opt_neg::eval_not_null_tables(NULL)) + return 1; + /* not_null_tables_cache == union(T1(e),T1(e1),T1(e2)) */ if (pred_level && !negated) return 0; @@ -2212,9 +2222,8 @@ bool Item_func_between::fix_fields(THD * not_null_tables_cache= (args[0]->not_null_tables() | (args[1]->not_null_tables() & args[2]->not_null_tables())); - return 0; -} +} void Item_func_between::fix_length_and_dec() @@ -2575,13 +2584,22 @@ Item_func_if::fix_fields(THD *thd, Item if (Item_func::fix_fields(thd, ref)) return 1; + return 0; +} + + +bool +Item_func_if::eval_not_null_tables(uchar *opt_arg) +{ + if (Item_func::eval_not_null_tables(NULL)) + return 1; + not_null_tables_cache= (args[1]->not_null_tables() & args[2]->not_null_tables()); return 0; } - void Item_func_if::fix_length_and_dec() { @@ -3761,11 +3779,22 @@ bool Item_func_in::nulls_in_row() bool Item_func_in::fix_fields(THD *thd, Item **ref) { - Item **arg, **arg_end; if (Item_func_opt_neg::fix_fields(thd, ref)) return 1; + return 0; +} + + +bool +Item_func_in::eval_not_null_tables(uchar *opt_arg) +{ + Item **arg, **arg_end; + + if (Item_func_opt_neg::eval_not_null_tables(NULL)) + return 1; + /* not_null_tables_cache == union(T1(e),union(T1(ei))) */ if (pred_level && negated) return 0; @@ -4186,7 +4215,6 @@ Item_cond::fix_fields(THD *thd, Item **r */ while ((item=li++)) { - table_map tmp_table_map; while (item->type() == Item::COND_ITEM && ((Item_cond*) item)->functype() == functype() && !((Item_cond*) item)->list.is_empty()) @@ -4204,15 +4232,9 @@ Item_cond::fix_fields(THD *thd, Item **r (item= *li.ref())->check_cols(1)) return TRUE; /* purecov: inspected */ used_tables_cache|= item->used_tables(); - if (item->const_item()) - and_tables_cache= (table_map) 0; - else - { - tmp_table_map= item->not_null_tables(); - not_null_tables_cache|= tmp_table_map; - and_tables_cache&= tmp_table_map; + if (!item->const_item()) const_item_cache= FALSE; - } + with_sum_func= with_sum_func || item->with_sum_func; with_subselect|= item->with_subselect; if (item->maybe_null) @@ -4226,6 +4248,27 @@ Item_cond::fix_fields(THD *thd, Item **r } +bool +Item_cond::eval_not_null_tables(uchar *opt_arg) +{ + Item *item; + List_iterator<Item> li(list); + while ((item=li++)) + { + table_map tmp_table_map; + if (item->const_item()) + and_tables_cache= (table_map) 0; + else + { + tmp_table_map= item->not_null_tables(); + not_null_tables_cache|= tmp_table_map; + and_tables_cache&= tmp_table_map; + } + } + return 0; +} + + void Item_cond::fix_after_pullout(st_select_lex *new_parent, Item **ref) { List_iterator<Item> li(list); === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-03-20 12:01:47 +0000 +++ b/sql/item_cmpfunc.h 2010-04-29 21:10:39 +0000 @@ -631,6 +631,7 @@ public: bool is_bool_func() { return 1; } CHARSET_INFO *compare_collation() { return cmp_collation.collation; } uint decimal_precision() const { return 1; } + bool eval_not_null_tables(uchar *opt_arg); }; @@ -730,6 +731,7 @@ public: void fix_length_and_dec(); uint decimal_precision() const; const char *func_name() const { return "if"; } + bool eval_not_null_tables(uchar *opt_arg); }; @@ -1256,6 +1258,7 @@ public: bool nulls_in_row(); bool is_bool_func() { return 1; } CHARSET_INFO *compare_collation() { return cmp_collation.collation; } + bool eval_not_null_tables(uchar *opt_arg); }; class cmp_item_row :public cmp_item @@ -1510,6 +1513,7 @@ public: bool subst_argument_checker(uchar **arg) { return TRUE; } Item *compile(Item_analyzer analyzer, uchar **arg_p, Item_transformer transformer, uchar *arg_t); + bool eval_not_null_tables(uchar *opt_arg); }; @@ -1774,6 +1778,19 @@ inline Item *and_conds(Item *a, Item *b) { if (!b) return a; if (!a) return b; + /* Try to minimize item tree by adding to already present AND functions. */ + if (a->type() == Item::COND_ITEM && + ((Item_cond*) a)->functype() == Item_func::COND_AND_FUNC) + { + ((Item_cond*)a)->add(b); + return a; + } + else if (b->type() == Item::COND_ITEM && + ((Item_cond*) b)->functype() == Item_func::COND_AND_FUNC) + { + ((Item_cond*)b)->add(a); + return b; + } return new Item_cond_and(a, b); } === modified file 'sql/item_func.cc' --- a/sql/item_func.cc 2010-03-20 12:01:47 +0000 +++ b/sql/item_func.cc 2010-04-29 21:10:39 +0000 @@ -192,7 +192,6 @@ Item_func::fix_fields(THD *thd, Item **r with_sum_func= with_sum_func || item->with_sum_func; used_tables_cache|= item->used_tables(); - not_null_tables_cache|= item->not_null_tables(); const_item_cache&= item->const_item(); with_subselect|= item->with_subselect; } @@ -206,6 +205,21 @@ Item_func::fix_fields(THD *thd, Item **r } +bool +Item_func::eval_not_null_tables(uchar *opt_arg) +{ + Item **arg,**arg_end; + if (arg_count) + { + for (arg=args, arg_end=args+arg_count; arg != arg_end ; arg++) + { + not_null_tables_cache|= (*arg)->not_null_tables(); + } + } + return FALSE; +} + + void Item_func::fix_after_pullout(st_select_lex *new_parent, Item **ref) { Item **arg,**arg_end; @@ -3895,6 +3909,20 @@ bool Item_func_set_user_var::fix_fields( entry->collation.set(args[0]->collation.collation, DERIVATION_IMPLICIT); collation.set(entry->collation.collation, DERIVATION_IMPLICIT); cached_result_type= args[0]->result_type(); + { + /* + When this function is used in a derived table/view force the derived + table to be materialized to preserve possible side-effect of setting a + user variable. + */ + SELECT_LEX_UNIT *unit= thd->lex->current_select->master_unit(); + TABLE_LIST *derived; + for (derived= unit->derived; + derived; + derived= derived->select_lex->master_unit()->derived) + derived->set_materialized_derived(); + } + return FALSE; } === modified file 'sql/item_func.h' --- a/sql/item_func.h 2010-03-20 12:01:47 +0000 +++ b/sql/item_func.h 2010-04-29 21:10:39 +0000 @@ -181,6 +181,7 @@ public: Item_transformer transformer, uchar *arg_t); void traverse_cond(Cond_traverser traverser, void * arg, traverse_order order); + bool eval_not_null_tables(uchar *opt_arg); // bool is_expensive_processor(uchar *arg); // virtual bool is_expensive() { return 0; } inline double fix_result(double value) @@ -1617,14 +1618,7 @@ public: void fix_length_and_dec() { decimals=0; max_length=1; maybe_null=1;} bool check_vcol_func_processor(uchar *int_arg) { -#if 0 - DBUG_ENTER("Item_func_is_free_lock::check_vcol_func_processor"); - DBUG_PRINT("info", - ("check_vcol_func_processor returns TRUE: unsupported function")); - DBUG_RETURN(TRUE); -#else return trace_unsupported_by_check_vcol_func_processor(func_name()); -#endif } }; === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.cc 2010-04-29 21:10:39 +0000 @@ -3158,10 +3158,13 @@ void subselect_uniquesubquery_engine::ex } -table_map subselect_engine::calc_const_tables(TABLE_LIST *table) +table_map subselect_engine::calc_const_tables(List<TABLE_LIST> &list) { table_map map= 0; - for (; table; table= table->next_leaf) + List_iterator<TABLE_LIST> ti(list); + TABLE_LIST *table; + //for (; table; table= table->next_leaf) + while ((table= ti++)) { TABLE *tbl= table->table; if (tbl && tbl->const_table) @@ -3173,14 +3176,13 @@ table_map subselect_engine::calc_const_t table_map subselect_single_select_engine::upper_select_const_tables() { - return calc_const_tables((TABLE_LIST *) select_lex->outer_select()-> - leaf_tables); + return calc_const_tables(select_lex->outer_select()->leaf_tables); } table_map subselect_union_engine::upper_select_const_tables() { - return calc_const_tables((TABLE_LIST *) unit->outer_select()->leaf_tables); + return calc_const_tables(unit->outer_select()->leaf_tables); } @@ -3711,7 +3713,7 @@ bool subselect_hash_sj_engine::init_perm if (((select_union*) result)->create_result_table( thd, tmp_columns, TRUE, tmp_create_options, - "materialized subselect", TRUE)) + "materialized subselect", TRUE, TRUE)) DBUG_RETURN(TRUE); tmp_table= ((select_union*) result)->table; === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-03-29 14:04:35 +0000 +++ b/sql/item_subselect.h 2010-04-29 21:10:39 +0000 @@ -531,6 +531,7 @@ public: virtual bool may_be_null() { return maybe_null; }; virtual table_map upper_select_const_tables()= 0; static table_map calc_const_tables(TABLE_LIST *); + static table_map calc_const_tables(List<TABLE_LIST> &list); virtual void print(String *str, enum_query_type query_type)= 0; virtual bool change_result(Item_subselect *si, select_result_interceptor *result)= 0; === modified file 'sql/mysql_priv.h' --- a/sql/mysql_priv.h 2010-03-20 12:01:47 +0000 +++ b/sql/mysql_priv.h 2010-04-29 21:10:39 +0000 @@ -62,12 +62,15 @@ class Parser_state; QT_ORDINARY -- ordinary SQL query. QT_IS -- SQL query to be shown in INFORMATION_SCHEMA (in utf8 and without - character set introducers). + character set introducers). + QT_VIEW_INTERNAL -- view internal representation (like QT_ORDINARY except + ORDER BY clause) */ enum enum_query_type { QT_ORDINARY, - QT_IS + QT_IS, + QT_VIEW_INTERNAL }; /* TODO convert all these three maps to Bitmap classes */ @@ -511,7 +514,6 @@ protected: #define OPTION_PROFILING (ULL(1) << 33) - /** Maximum length of time zone name that we support (Time zone name is char(64) in db). mysqlbinlog needs it. @@ -1276,11 +1278,9 @@ int mysql_explain_select(THD *thd, SELEC select_result *result); bool mysql_union(THD *thd, LEX *lex, select_result *result, SELECT_LEX_UNIT *unit, ulong setup_tables_done_option); -bool mysql_handle_derived(LEX *lex, bool (*processor)(THD *thd, - LEX *lex, - TABLE_LIST *table)); -bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *t); -bool mysql_derived_filling(THD *thd, LEX *lex, TABLE_LIST *t); +bool mysql_handle_derived(LEX *lex, uint phases); +bool mysql_handle_single_derived(LEX *lex, TABLE_LIST *derived, uint phases); +bool mysql_handle_list_of_derived(LEX *lex, TABLE_LIST *dt_list, uint phases); Field *create_tmp_field(THD *thd, TABLE *table,Item *item, Item::Type type, Item ***copy_func, Field **from_field, Field **def_field, @@ -1288,6 +1288,17 @@ Field *create_tmp_field(THD *thd, TABLE bool table_cant_handle_bit_fields, bool make_copy_field, uint convert_blob_length); +bool open_tmp_table(TABLE *table); +#if defined(WITH_MARIA_STORAGE_ENGINE) && defined(USE_MARIA_FOR_TMP_TABLES) +#define TMP_ENGINE_HTON maria_hton +#else +#define TMP_ENGINE_HTON myisam_hton +#endif +bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, + ENGINE_COLUMNDEF *start_recinfo, + ENGINE_COLUMNDEF **recinfo, + ulonglong options); + void sp_prepare_create_field(THD *thd, Create_field *sql_field); int prepare_create_field(Create_field *sql_field, uint *blob_columns, @@ -1600,17 +1611,21 @@ bool get_key_map_from_key_list(key_map * bool insert_fields(THD *thd, Name_resolution_context *context, const char *db_name, const char *table_name, List_iterator<Item> *it, bool any_privileges); +void make_leaves_list(List<TABLE_LIST> &list, TABLE_LIST *tables, + bool full_table_list, TABLE_LIST *boundary); bool setup_tables(THD *thd, Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, bool select_insert); + List<TABLE_LIST> &leaves, bool select_insert, + bool full_table_list); bool setup_tables_and_check_access(THD *thd, Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, + List<TABLE_LIST> &leaves, bool select_insert, ulong want_access_first, - ulong want_access); + ulong want_access, + bool full_table_list); int setup_wild(THD *thd, TABLE_LIST *tables, List<Item> &fields, List<Item> *sum_func_list, uint wild_num); bool setup_fields(THD *thd, Item** ref_pointer_array, @@ -1629,7 +1644,7 @@ inline bool setup_fields_with_no_wrap(TH thd->lex->select_lex.no_wrap_view_item= FALSE; return res; } -int setup_conds(THD *thd, TABLE_LIST *tables, TABLE_LIST *leaves, +int setup_conds(THD *thd, TABLE_LIST *tables, List<TABLE_LIST> &leaves, COND **conds); int setup_ftfuncs(SELECT_LEX* select); int init_ftfuncs(THD *thd, SELECT_LEX* select, bool no_order); @@ -1651,7 +1666,8 @@ inline int open_and_lock_tables(THD *thd /* simple open_and_lock_tables without derived handling for single table */ TABLE *open_n_lock_single_table(THD *thd, TABLE_LIST *table_l, thr_lock_type lock_type); -bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags); +bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags, + uint dt_phases); int lock_tables(THD *thd, TABLE_LIST *tables, uint counter, bool *need_reopen); int decide_logging_format(THD *thd, TABLE_LIST *tables); TABLE *open_temporary_table(THD *thd, const char *path, const char *db, @@ -1680,6 +1696,7 @@ void remove_db_from_cache(const char *db void flush_tables(); bool is_equal(const LEX_STRING *a, const LEX_STRING *b); char *make_default_log_name(char *buff,const char* log_ext); +void unfix_fields(List<Item> &items); #ifdef WITH_PARTITION_STORAGE_ENGINE uint fast_alter_partition_table(THD *thd, TABLE *table, @@ -2528,7 +2545,7 @@ Item * all_any_subquery_creator(Item *le inline void setup_table_map(TABLE *table, TABLE_LIST *table_list, uint tablenr) { table->used_fields= 0; - table->const_table= 0; + table_list->reset_const_table(); table->null_row= 0; table->status= STATUS_NO_RECORD; table->maybe_null= table_list->outer_join; @@ -2544,6 +2561,14 @@ inline void setup_table_map(TABLE *table table->force_index_order= table->force_index_group= 0; table->covering_keys= table->s->keys_for_keyread; table->merge_keys.clear_all(); + TABLE_LIST *orig= table_list->select_lex ? + table_list->select_lex->master_unit()->derived : 0; + if (!orig || !orig->is_merged_derived()) + { + /* Tables merged from derived were set up already.*/ + table->covering_keys= table->s->keys_for_keyread; + table->merge_keys.clear_all(); + } } === modified file 'sql/opt_range.cc' --- a/sql/opt_range.cc 2010-03-20 12:01:47 +0000 +++ b/sql/opt_range.cc 2010-04-29 21:10:39 +0000 @@ -7450,7 +7450,7 @@ ha_rows check_quick_select(PARAM *param, SEL_ARG_RANGE_SEQ seq; RANGE_SEQ_IF seq_if = {sel_arg_range_seq_init, sel_arg_range_seq_next, 0, 0}; handler *file= param->table->file; - ha_rows rows; + ha_rows rows= HA_POS_ERROR; uint keynr= param->real_keynr[idx]; DBUG_ENTER("check_quick_select"); @@ -7490,8 +7490,13 @@ ha_rows check_quick_select(PARAM *param, *mrr_flags |= HA_MRR_USE_DEFAULT_IMPL; *bufsize= param->thd->variables.mrr_buff_size; - rows= file->multi_range_read_info_const(keynr, &seq_if, (void*)&seq, 0, - bufsize, mrr_flags, cost); + /* + Skip materialized derived table/view result table from MRR check as + they aren't contain any data yet. + */ + if (param->table->pos_in_table_list->is_non_derived()) + rows= file->multi_range_read_info_const(keynr, &seq_if, (void*)&seq, 0, + bufsize, mrr_flags, cost); if (rows != HA_POS_ERROR) { param->table->quick_rows[keynr]=rows; === modified file 'sql/opt_subselect.cc' --- a/sql/opt_subselect.cc 2010-03-15 19:52:58 +0000 +++ b/sql/opt_subselect.cc 2010-04-29 21:10:39 +0000 @@ -154,9 +154,9 @@ int check_and_do_in_subquery_rewrites(JO !join->having && !select_lex->with_sum_func && // 4 thd->thd_marker.emb_on_expr_nest && // 5 select_lex->outer_select()->join && // 6 - select_lex->master_unit()->first_select()->leaf_tables && // 7 + select_lex->master_unit()->first_select()->leaf_tables.elements && // 7 in_subs->exec_method == Item_in_subselect::NOT_TRANSFORMED && // 8 - select_lex->outer_select()->leaf_tables && // 9 + select_lex->outer_select()->leaf_tables.elements && // 9 !((join->select_options | // 10 select_lex->outer_select()->join->select_options) // 10 & SELECT_STRAIGHT_JOIN)) // 10 @@ -212,9 +212,9 @@ int check_and_do_in_subquery_rewrites(JO if (optimizer_flag(thd, OPTIMIZER_SWITCH_MATERIALIZATION) && in_subs && // 1 !select_lex->is_part_of_union() && // 2 - select_lex->master_unit()->first_select()->leaf_tables && // 3 + select_lex->master_unit()->first_select()->leaf_tables.elements && // 3 thd->lex->sql_command == SQLCOM_SELECT && // * - select_lex->outer_select()->leaf_tables && // 3A + select_lex->outer_select()->leaf_tables.elements && // 3A subquery_types_allow_materialization(in_subs) && // psergey-todo: duplicated_subselect_card_check: where it's done? (in_subs->is_top_level_item() || @@ -391,11 +391,26 @@ bool convert_join_subqueries_to_semijoin Item_in_subselect **in_subq; Item_in_subselect **in_subq_end; THD *thd= join->thd; + TABLE_LIST *tbl; + List_iterator<TABLE_LIST> ti(join->select_lex->leaf_tables); DBUG_ENTER("convert_join_subqueries_to_semijoins"); if (join->sj_subselects.elements() == 0) DBUG_RETURN(FALSE); + for (in_subq= join->sj_subselects.front(), + in_subq_end= join->sj_subselects.back(); + in_subq != in_subq_end; + in_subq++) + { + SELECT_LEX *subq_sel= (*in_subq)->get_select_lex(); + if (subq_sel->handle_derived(thd->lex, DT_OPTIMIZE)) + DBUG_RETURN(1); + if (subq_sel->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + subq_sel->update_used_tables(); + } + /* First, convert child join's subqueries. We proceed bottom-up here */ for (in_subq= join->sj_subselects.front(), in_subq_end= join->sj_subselects.back(); @@ -422,11 +437,12 @@ bool convert_join_subqueries_to_semijoin // Temporary measure: disable semi-joins when they are together with outer // joins. - for (TABLE_LIST *tbl= join->select_lex->leaf_tables; tbl; tbl=tbl->next_leaf) + while ((tbl= ti++)) { TABLE_LIST *embedding= tbl->embedding; - if (tbl->on_expr || (tbl->embedding && !(embedding->sj_on_expr && - !embedding->embedding))) + if (tbl->on_expr || + (embedding && embedding->outer_join && + !(embedding->sj_on_expr && !embedding->embedding))) { in_subq= join->sj_subselects.front(); arena= thd->activate_stmt_arena_if_needed(&backup); @@ -737,7 +753,7 @@ static bool convert_subq_to_sj(JOIN *par st_select_lex *subq_lex= subq_pred->unit->first_select(); nested_join->join_list.empty(); List_iterator_fast<TABLE_LIST> li(subq_lex->top_join_list); - TABLE_LIST *tl, *last_leaf; + TABLE_LIST *tl; while ((tl= li++)) { tl->embedding= sj_nest; @@ -752,17 +768,15 @@ static bool convert_subq_to_sj(JOIN *par NOTE: We actually insert them at the front! That's because the order is reversed in this list. */ - for (tl= parent_lex->leaf_tables; tl->next_leaf; tl= tl->next_leaf) ; - tl->next_leaf= subq_lex->leaf_tables; - last_leaf= tl; + parent_lex->leaf_tables.concat(&subq_lex->leaf_tables); /* Same as above for next_local chain (a theory: a next_local chain always starts with ::leaf_tables because view's tables are inserted after the view) */ - for (tl= parent_lex->leaf_tables; tl->next_local; tl= tl->next_local) ; - tl->next_local= subq_lex->leaf_tables; + for (tl= parent_lex->leaf_tables.head(); tl->next_local; tl= tl->next_local) ; + tl->next_local= subq_lex->leaf_tables.head(); /* A theory: no need to re-connect the next_global chain */ @@ -776,7 +790,8 @@ static bool convert_subq_to_sj(JOIN *par /* n. Adjust the parent_join->tables counter */ uint table_no= parent_join->tables; /* n. Walk through child's tables and adjust table->map */ - for (tl= subq_lex->leaf_tables; tl; tl= tl->next_leaf, table_no++) + List_iterator_fast<TABLE_LIST> si(subq_lex->leaf_tables); + while ((tl= si++)) { tl->table->tablenr= table_no; tl->table->map= ((table_map)1) << table_no; @@ -786,6 +801,7 @@ static bool convert_subq_to_sj(JOIN *par emb && emb->select_lex == old_sl; emb= emb->embedding) emb->select_lex= parent_join->select_lex; + table_no++; } parent_join->tables += subq_lex->join->tables; @@ -872,7 +888,8 @@ static bool convert_subq_to_sj(JOIN *par { /* Inject into the WHERE */ parent_join->conds= and_items(parent_join->conds, sj_nest->sj_on_expr); - parent_join->conds->fix_fields(parent_join->thd, &parent_join->conds); + if (!parent_join->conds->fixed) + parent_join->conds->fix_fields(parent_join->thd, &parent_join->conds); parent_join->select_lex->where= parent_join->conds; } @@ -1424,6 +1441,7 @@ void advance_sj_state(JOIN *join, table_ TABLE_LIST *emb_sj_nest; POSITION *pos= join->positions + idx; remaining_tables &= ~new_join_tab->table->map; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; pos->prefix_cost.convert_from_cost(*current_read_time); pos->prefix_record_count= *current_record_count; @@ -1593,7 +1611,8 @@ void advance_sj_state(JOIN *join, table_ optimize_wo_join_buffering(join, pos->first_loosescan_table, idx, remaining_tables, TRUE, //first_alt - pos->first_loosescan_table + n_tables, + disable_jbuf ? join->tables : + pos->first_loosescan_table + n_tables, &reopt_rec_count, &reopt_cost, &sj_inner_fanout); /* @@ -1734,8 +1753,8 @@ void advance_sj_state(JOIN *join, table_ /* Need to re-run best-access-path as we prefix_rec_count has changed */ for (i= first_tab + mat_info->tables; i <= idx; i++) { - best_access_path(join, join->positions[i].table, rem_tables, i, FALSE, - prefix_rec_count, &curpos, &dummy); + best_access_path(join, join->positions[i].table, rem_tables, i, + disable_jbuf, prefix_rec_count, &curpos, &dummy); prefix_rec_count *= curpos.records_read; prefix_cost += curpos.read_time; } @@ -2031,6 +2050,7 @@ at_sjmat_pos(const JOIN *join, table_map void fix_semijoin_strategies_for_picked_join_order(JOIN *join) { uint table_count=join->tables; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; uint tablenr; table_map remaining_tables= 0; table_map handled_tabs= 0; @@ -2092,8 +2112,9 @@ void fix_semijoin_strategies_for_picked_ join->cur_sj_inner_tables= 0; for (i= first + sjm->tables; i <= tablenr; i++) { - best_access_path(join, join->best_positions[i].table, rem_tables, i, FALSE, - prefix_rec_count, join->best_positions + i, &dummy); + best_access_path(join, join->best_positions[i].table, rem_tables, i, + disable_jbuf, prefix_rec_count, + join->best_positions + i, &dummy); prefix_rec_count *= join->best_positions[i].records_read; rem_tables &= ~join->best_positions[i].table->table->map; } === modified file 'sql/opt_sum.cc' --- a/sql/opt_sum.cc 2010-01-04 17:54:42 +0000 +++ b/sql/opt_sum.cc 2010-04-29 21:10:39 +0000 @@ -74,10 +74,12 @@ static int maxmin_in_range(bool max_fl, # Multiplication of number of rows in all tables */ -static ulonglong get_exact_record_count(TABLE_LIST *tables) +static ulonglong get_exact_record_count(List<TABLE_LIST> &tables) { ulonglong count= 1; - for (TABLE_LIST *tl= tables; tl; tl= tl->next_leaf) + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(tables); + while ((tl= ti++)) { ha_rows tmp= tl->table->file->records(); if ((tmp == HA_POS_ERROR)) @@ -110,9 +112,11 @@ static ulonglong get_exact_record_count( HA_ERR_... if a deadlock or a lock wait timeout happens, for example */ -int opt_sum_query(TABLE_LIST *tables, List<Item> &all_fields,COND *conds) +int opt_sum_query(List<TABLE_LIST> &tables, List<Item> &all_fields,COND *conds) { List_iterator_fast<Item> it(all_fields); + List_iterator<TABLE_LIST> ti(tables); + TABLE_LIST *tl; int const_result= 1; bool recalc_const_item= 0; ulonglong count= 1; @@ -120,7 +124,7 @@ int opt_sum_query(TABLE_LIST *tables, Li table_map removed_tables= 0, outer_tables= 0, used_tables= 0; table_map where_tables= 0; Item *item; - int error; + int error= 0; if (conds) where_tables= conds->used_tables(); @@ -129,7 +133,7 @@ int opt_sum_query(TABLE_LIST *tables, Li Analyze outer join dependencies, and, if possible, compute the number of returned rows. */ - for (TABLE_LIST *tl= tables; tl; tl= tl->next_leaf) + while ((tl= ti++)) { TABLE_LIST *embedded; for (embedded= tl ; embedded; embedded= embedded->embedding) @@ -170,6 +174,14 @@ int opt_sum_query(TABLE_LIST *tables, Li is_exact_count= FALSE; count= 1; // ensure count != 0 } + else if (tl->is_materialized_derived()) + { + /* + Can't remove a derived table as it's number of rows is just an + estimate. + */ + return 0; + } else { error= tl->table->file->info(HA_STATUS_VARIABLE | HA_STATUS_NO_LOCK); === modified file 'sql/records.cc' --- a/sql/records.cc 2010-02-01 06:14:12 +0000 +++ b/sql/records.cc 2010-04-29 21:10:39 +0000 @@ -286,7 +286,8 @@ void end_read_record(READ_RECORD *info) if (info->table) { filesort_free_buffers(info->table,0); - (void) info->file->extra(HA_EXTRA_NO_CACHE); + if (info->table->created) + (void) info->file->extra(HA_EXTRA_NO_CACHE); if (info->read_record != rr_quick) // otherwise quick_range does it (void) info->file->ha_index_or_rnd_end(); info->table=0; === modified file 'sql/sp_head.cc' --- a/sql/sp_head.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sp_head.cc 2010-04-29 21:10:39 +0000 @@ -2821,6 +2821,9 @@ int sp_instr::exec_open_and_lock_tables( result= -1; else result= 0; + /* Prepare all derived tables/views to catch possible errors. */ + if (!result) + result= mysql_handle_derived(thd->lex, DT_PREPARE) ? -1 : 0; return result; } === modified file 'sql/sql_acl.cc' --- a/sql/sql_acl.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sql_acl.cc 2010-04-29 21:10:39 +0000 @@ -3003,7 +3003,8 @@ int mysql_table_grant(THD *thd, TABLE_LI class LEX_COLUMN *column; List_iterator <LEX_COLUMN> column_iter(columns); - if (open_and_lock_tables(thd, table_list)) + if (open_and_lock_tables(thd, table_list) || + mysql_handle_derived(thd->lex, DT_PREPARE)) DBUG_RETURN(TRUE); while ((column = column_iter++)) === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_base.cc 2010-04-29 21:10:39 +0000 @@ -2998,6 +2998,7 @@ TABLE *open_table(THD *thd, TABLE_LIST * table->fulltext_searched= 0; table->file->ft_handler= 0; table->reginfo.impossible_range= 0; + table->created= TRUE; /* Catch wrong handling of the auto_increment_field_not_null. */ DBUG_ASSERT(!table->auto_increment_field_not_null); table->auto_increment_field_not_null= FALSE; @@ -5044,9 +5045,10 @@ int open_and_lock_tables_derived(THD *th close_tables_for_reopen(thd, &tables); } if (derived && - (mysql_handle_derived(thd->lex, &mysql_derived_prepare) || - (thd->fill_derived_tables() && - mysql_handle_derived(thd->lex, &mysql_derived_filling)))) + (mysql_handle_derived(thd->lex, DT_INIT))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + if (thd->prepare_derived_at_open && derived && + (mysql_handle_derived(thd->lex, DT_PREPARE))) DBUG_RETURN(TRUE); /* purecov: inspected */ DBUG_RETURN(0); } @@ -5062,6 +5064,7 @@ int open_and_lock_tables_derived(THD *th flags - bitmap of flags to modify how the tables will be open: MYSQL_LOCK_IGNORE_FLUSH - open table even if someone has done a flush or namelock on it. + dt_phases - set of flags to pass to the mysql_handle_derived RETURN FALSE - ok @@ -5072,13 +5075,14 @@ int open_and_lock_tables_derived(THD *th data from the tables. */ -bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags) +bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags, + uint dt_phases) { uint counter; DBUG_ENTER("open_normal_and_derived_tables"); DBUG_ASSERT(!thd->fill_derived_tables()); if (open_tables(thd, &tables, &counter, flags) || - mysql_handle_derived(thd->lex, &mysql_derived_prepare)) + mysql_handle_derived(thd->lex, dt_phases)) DBUG_RETURN(TRUE); /* purecov: inspected */ DBUG_RETURN(0); } @@ -5714,9 +5718,7 @@ find_field_in_view(THD *thd, TABLE_LIST Field_iterator_view field_it; field_it.set(table_list); Query_arena *arena= 0, backup; - - DBUG_ASSERT(table_list->schema_table_reformed || - (ref != 0 && table_list->view != 0)); + for (; !field_it.end_of_fields(); field_it.next()) { if (!my_strcasecmp(system_charset_info, field_it.name(), name)) @@ -5735,6 +5737,8 @@ find_field_in_view(THD *thd, TABLE_LIST if (!item) DBUG_RETURN(0); + if (!ref) + DBUG_RETURN((Field*) view_ref_found); /* *ref != NULL means that *ref contains the item that we need to replace. If the item was aliased by the user, set the alias to @@ -6134,6 +6138,8 @@ find_field_in_table_ref(THD *thd, TABLE_ Field *field_to_set= NULL; if (fld == view_ref_found) { + if (!ref) + DBUG_RETURN(fld); Item *it= (*ref)->real_item(); if (it->type() == Item::FIELD_ITEM) field_to_set= ((Item_field*)it)->field; @@ -6141,6 +6147,8 @@ find_field_in_table_ref(THD *thd, TABLE_ { if (thd->mark_used_columns == MARK_COLUMNS_READ) it->walk(&Item::register_field_in_read_map, 1, (uchar *) 0); + else + it->walk(&Item::register_field_in_write_map, 1, (uchar *) 0); } } else @@ -6280,7 +6288,7 @@ find_field_in_tables(THD *thd, Item_iden find_field_in_table even in the case of information schema tables when table_ref->field_translation != NULL. */ - if (table_ref->table && !table_ref->view) + if (table_ref->table && !table_ref->is_merged_derived()) found= find_field_in_table(thd, table_ref->table, name, length, TRUE, &(item->cached_field_index)); else @@ -6298,7 +6306,8 @@ find_field_in_tables(THD *thd, Item_iden Only views fields should be marked as dependent, not an underlying fields. */ - if (!table_ref->belong_to_view) + if (!table_ref->belong_to_view && + !table_ref->belong_to_derived) { SELECT_LEX *current_sel= thd->lex->current_select; SELECT_LEX *last_select= table_ref->select_lex; @@ -6884,6 +6893,10 @@ mark_common_columns(THD *thd, TABLE_LIST */ if (nj_col_2 && (!using_fields ||is_using_column_1)) { + /* + Create non-fixed fully qualified field and let fix_fields to + resolve it. + */ Item *item_1= nj_col_1->create_item(thd); Item *item_2= nj_col_2->create_item(thd); Field *field_1= nj_col_1->field(); @@ -7548,27 +7561,36 @@ bool setup_fields(THD *thd, Item **ref_p make_leaves_list() list pointer to pointer on list first element tables table list + full_table_list whether to include tables from mergeable derived table/view. + we need them for checks for INSERT/UPDATE statements only. RETURN pointer on pointer to next_leaf of last element */ -TABLE_LIST **make_leaves_list(TABLE_LIST **list, TABLE_LIST *tables) +void make_leaves_list(List<TABLE_LIST> &list, TABLE_LIST *tables, + bool full_table_list, TABLE_LIST *boundary) + { for (TABLE_LIST *table= tables; table; table= table->next_local) { - if (table->merge_underlying_list) - { - DBUG_ASSERT(table->view && - table->effective_algorithm == VIEW_ALGORITHM_MERGE); - list= make_leaves_list(list, table->merge_underlying_list); + if (table == boundary) + full_table_list= !full_table_list; + if (full_table_list && table->is_merged_derived()) + { + SELECT_LEX *select_lex= table->get_single_select(); + /* + It's safe to use select_lex->leaf_tables because all derived + tables/views were already prepared and has their leaf_tables + set properly. + */ + make_leaves_list(list, select_lex->get_table_list(), + full_table_list, boundary); } else { - *list= table; - list= &table->next_leaf; + list.push_back(table); } } - return list; } /* @@ -7583,6 +7605,7 @@ TABLE_LIST **make_leaves_list(TABLE_LIST leaves List of join table leaves list (select_lex->leaf_tables) refresh It is onle refresh for subquery select_insert It is SELECT ... INSERT command + full_table_list a parameter to pass to the make_leaves_list function NOTE Check also that the 'used keys' and 'ignored keys' exists and set up the @@ -7601,9 +7624,13 @@ TABLE_LIST **make_leaves_list(TABLE_LIST bool setup_tables(THD *thd, Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, bool select_insert) + List<TABLE_LIST> &leaves, bool select_insert, + bool full_table_list) { uint tablenr= 0; + List_iterator<TABLE_LIST> ti(leaves); + TABLE_LIST *table_list; + DBUG_ENTER("setup_tables"); DBUG_ASSERT ((select_insert && !tables->next_name_resolution_table) || !tables || @@ -7615,40 +7642,56 @@ bool setup_tables(THD *thd, Name_resolut TABLE_LIST *first_select_table= (select_insert ? tables->next_local: 0); - if (!(*leaves)) - make_leaves_list(leaves, tables); - - TABLE_LIST *table_list; - for (table_list= *leaves; - table_list; - table_list= table_list->next_leaf, tablenr++) + SELECT_LEX *select_lex= thd->lex->current_select; + if (select_lex->first_cond_optimization) { - TABLE *table= table_list->table; - table->pos_in_table_list= table_list; - if (first_select_table && - table_list->top_table() == first_select_table) - { - /* new counting for SELECT of INSERT ... SELECT command */ - first_select_table= 0; - tablenr= 0; + leaves.empty(); + select_lex->leaf_tables_exec.empty(); + make_leaves_list(leaves, tables, full_table_list, first_select_table); + + while ((table_list= ti++)) + { + TABLE *table= table_list->table; + table->pos_in_table_list= table_list; + if (first_select_table && + table_list->top_table() == first_select_table) + { + /* new counting for SELECT of INSERT ... SELECT command */ + first_select_table= 0; + thd->lex->select_lex.insert_tables= tablenr; + tablenr= 0; + } + setup_table_map(table, table_list, tablenr); + if (table_list->process_index_hints(table)) + DBUG_RETURN(1); + tablenr++; } - setup_table_map(table, table_list, tablenr); - if (table_list->process_index_hints(table)) + if (tablenr > MAX_TABLES) + { + my_error(ER_TOO_MANY_TABLES,MYF(0),MAX_TABLES); DBUG_RETURN(1); + } } - if (tablenr > MAX_TABLES) - { - my_error(ER_TOO_MANY_TABLES,MYF(0),MAX_TABLES); - DBUG_RETURN(1); - } + else + { + List_iterator_fast <TABLE_LIST> ti(select_lex->leaf_tables_exec); + select_lex->leaf_tables.empty(); + while ((table_list= ti++)) + { + table_list->table->tablenr= table_list->tablenr_exec; + table_list->table->map= table_list->map_exec; + table_list->table->pos_in_table_list= table_list; + select_lex->leaf_tables.push_back(table_list); + } + } + for (table_list= tables; table_list; table_list= table_list->next_local) { if (table_list->merge_underlying_list) { - DBUG_ASSERT(table_list->view && - table_list->effective_algorithm == VIEW_ALGORITHM_MERGE); + DBUG_ASSERT(table_list->is_merged_derived()); Query_arena *arena= thd->stmt_arena, backup; bool res; if (arena->is_conventional()) @@ -7675,7 +7718,7 @@ bool setup_tables(THD *thd, Name_resolut prepare tables and check access for the view tables SYNOPSIS - setup_tables_and_check_view_access() + setup_tables_and_check_access() thd Thread handler context name resolution contest to setup table list there from_clause Top-level list of table references in the FROM clause @@ -7685,6 +7728,7 @@ bool setup_tables(THD *thd, Name_resolut refresh It is onle refresh for subquery select_insert It is SELECT ... INSERT command want_access what access is needed + full_table_list a parameter to pass to the make_leaves_list function NOTE a wrapper for check_tables that will also check the resulting @@ -7698,33 +7742,32 @@ bool setup_tables_and_check_access(THD * Name_resolution_context *context, List<TABLE_LIST> *from_clause, TABLE_LIST *tables, - TABLE_LIST **leaves, + List<TABLE_LIST> &leaves, bool select_insert, ulong want_access_first, - ulong want_access) + ulong want_access, + bool full_table_list) { - TABLE_LIST *leaves_tmp= NULL; bool first_table= true; if (setup_tables(thd, context, from_clause, tables, - &leaves_tmp, select_insert)) + leaves, select_insert, full_table_list)) return TRUE; - if (leaves) - *leaves= leaves_tmp; - - for (; leaves_tmp; leaves_tmp= leaves_tmp->next_leaf) + List_iterator<TABLE_LIST> ti(leaves); + TABLE_LIST *table_list; + while((table_list= ti++)) { - if (leaves_tmp->belong_to_view && + if (table_list->belong_to_view && check_single_table_access(thd, first_table ? want_access_first : - want_access, leaves_tmp, FALSE)) + want_access, table_list, FALSE)) { tables->hide_view_error(thd); return TRUE; } first_table= 0; } - return FALSE; + return FALSE; } @@ -7860,8 +7903,8 @@ insert_fields(THD *thd, Name_resolution_ information_schema table, or a nested table reference. See the comment for TABLE_LIST. */ - if (!((table && !tables->view && (table->grant.privilege & SELECT_ACL)) || - (tables->view && (tables->grant.privilege & SELECT_ACL))) && + if (!(table && tables->is_non_derived() && (table->grant.privilege & SELECT_ACL) || + (!tables->is_non_derived() && (tables->grant.privilege & SELECT_ACL))) && !any_privileges) { field_iterator.set(tables); @@ -7891,7 +7934,7 @@ insert_fields(THD *thd, Name_resolution_ if (!(item= field_iterator.create_item(thd))) DBUG_RETURN(TRUE); - DBUG_ASSERT(item->fixed); +// DBUG_ASSERT(item->fixed); /* cache the table for the Item_fields inserted by expanding stars */ if (item->type() == Item::FIELD_ITEM && tables->cacheable_table) ((Item_field *)item)->cached_table= tables; @@ -8021,13 +8064,14 @@ insert_fields(THD *thd, Name_resolution_ FALSE if all is OK */ -int setup_conds(THD *thd, TABLE_LIST *tables, TABLE_LIST *leaves, +int setup_conds(THD *thd, TABLE_LIST *tables, List<TABLE_LIST> &leaves, COND **conds) { SELECT_LEX *select_lex= thd->lex->current_select; Query_arena *arena= thd->stmt_arena, backup; TABLE_LIST *table= NULL; // For HP compilers TABLE_LIST *save_emb_on_expr_nest= thd->thd_marker.emb_on_expr_nest; + List_iterator<TABLE_LIST> ti(leaves); /* it_is_update set to TRUE when tables of primary SELECT_LEX (SELECT_LEX which belong to LEX, i.e. most up SELECT) will be updated by @@ -8039,9 +8083,15 @@ int setup_conds(THD *thd, TABLE_LIST *ta bool it_is_update= (select_lex == &thd->lex->select_lex) && thd->lex->which_check_option_applicable(); bool save_is_item_list_lookup= select_lex->is_item_list_lookup; - select_lex->is_item_list_lookup= 0; + TABLE_LIST *derived= select_lex->master_unit()->derived; DBUG_ENTER("setup_conds"); + /* Do not fix conditions for the derived tables that have been merged */ + if (derived && derived->merged) + DBUG_RETURN(0); + + select_lex->is_item_list_lookup= 0; + if (select_lex->conds_processed_with_permanent_arena || arena->is_conventional()) arena= 0; // For easier test @@ -8054,7 +8104,10 @@ int setup_conds(THD *thd, TABLE_LIST *ta for (table= tables; table; table= table->next_local) { - if (table->prepare_where(thd, conds, FALSE)) + if (select_lex == &thd->lex->select_lex && + select_lex->first_cond_optimization && + table->merged_for_insert && + table->prepare_where(thd, conds, FALSE)) goto err_no_arena; } @@ -8072,7 +8125,7 @@ int setup_conds(THD *thd, TABLE_LIST *ta Apply fix_fields() to all ON clauses at all levels of nesting, including the ones inside view definitions. */ - for (table= leaves; table; table= table->next_leaf) + while ((table= ti++)) { TABLE_LIST *embedded; /* The table at the current level of nesting. */ TABLE_LIST *embedding= table; /* The parent nested table reference. */ @@ -9283,6 +9336,27 @@ void close_performance_schema_table(THD thd->restore_backup_open_tables_state(backup); } + +/** + @brief + Remove 'fixed' flag from items in a list + + @param items list of items to un-fix + + @details + This function sets to 0 the 'fixed' flag for items in the 'items' list. + It's needed to force correct marking of views' fields for INSERT/UPDATE + statements. +*/ + +void unfix_fields(List<Item> &fields) +{ + List_iterator<Item> li(fields); + Item *item; + while ((item= li++)) + item->fixed= 0; +} + /** @} (end of group Data_Dictionary) */ === modified file 'sql/sql_bitmap.h' --- a/sql/sql_bitmap.h 2009-08-12 22:34:21 +0000 +++ b/sql/sql_bitmap.h 2010-04-29 21:10:39 +0000 @@ -91,6 +91,10 @@ public: DBUG_ASSERT(sizeof(buffer) >= 4); return (ulonglong) uint4korr(buffer); } + uint bits_set() + { + return bitmap_bits_set(&map); + } }; /* An iterator to quickly walk over bits in unlonglong bitmap. */ @@ -169,5 +173,16 @@ public: public: Iterator(Bitmap<64> &bmp) : Table_map_iterator(bmp.map) {} }; + uint bits_set() + { + //TODO: use my_count_bits() + uint res= 0, i= 0; + for (; i < 64 ; i++) + { + if (map & ((ulonglong)1<<i)) + res++; + } + return res; + } }; === modified file 'sql/sql_cache.cc' --- a/sql/sql_cache.cc 2010-01-29 10:42:31 +0000 +++ b/sql/sql_cache.cc 2010-04-29 21:10:39 +0000 @@ -3477,16 +3477,17 @@ Query_cache::process_and_count_tables(TH } else { - DBUG_PRINT("qcache", ("table: %s db: %s type: %u", - tables_used->table->s->table_name.str, - tables_used->table->s->db.str, - tables_used->table->s->db_type()->db_type)); if (tables_used->derived) { + DBUG_PRINT("qcache", ("table: %s", tables_used->alias)); table_count--; DBUG_PRINT("qcache", ("derived table skipped")); continue; } + DBUG_PRINT("qcache", ("table: %s db: %s type: %u", + tables_used->table->s->table_name.str, + tables_used->table->s->db.str, + tables_used->table->s->db_type()->db_type)); *tables_type|= tables_used->table->file->table_cache_type(); /* === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.cc 2010-04-29 21:10:39 +0000 @@ -771,6 +771,7 @@ THD::THD() thr_lock_owner_init(&main_lock_id, &lock_info); m_internal_handler= NULL; + prepare_derived_at_open= FALSE; } @@ -2946,7 +2947,8 @@ bool select_materialize_with_stats:: create_result_table(THD *thd_arg, List<Item> *column_types, bool is_union_distinct, ulonglong options, - const char *table_alias, bool bit_fields_as_long) + const char *table_alias, bool bit_fields_as_long, + bool create_table) { DBUG_ASSERT(table == 0); tmp_table_param.field_count= column_types->elements; === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_class.h 2010-04-29 21:10:39 +0000 @@ -1474,6 +1474,9 @@ public: */ TABLE_LIST *emb_on_expr_nest; } thd_marker; + + bool prepare_derived_at_open; + #ifndef MYSQL_CLIENT int binlog_setup_trx_data(); @@ -2787,6 +2790,11 @@ public: */ bool bit_fields_as_long; + /* + Whether to create or postpone actual creation of this temporary table. + TRUE <=> create_tmp_table will create only the TABLE structure. + */ + bool skip_create_table; TMP_TABLE_PARAM() :copy_field(0), group_parts(0), group_length(0), group_null_parts(0), convert_blob_length(0), @@ -2810,12 +2818,12 @@ public: class select_union :public select_result_interceptor { -protected: - TMP_TABLE_PARAM tmp_table_param; public: + TMP_TABLE_PARAM tmp_table_param; TABLE *table; + ha_rows records; - select_union() :table(0) { tmp_table_param.init(); } + select_union() :table(0), records(0) { tmp_table_param.init(); } int prepare(List<Item> &list, SELECT_LEX_UNIT *u); bool send_data(List<Item> &items); bool send_eof(); @@ -2823,7 +2831,9 @@ public: virtual bool create_result_table(THD *thd, List<Item> *column_types, bool is_distinct, ulonglong options, - const char *alias, bool bit_fields_as_long); + const char *alias, + bool bit_fields_as_long, + bool create_table); }; /* Base subselect interface class */ @@ -2885,9 +2895,11 @@ protected: public: select_materialize_with_stats() {} - virtual bool create_result_table(THD *thd, List<Item> *column_types, - bool is_distinct, ulonglong options, - const char *alias, bool bit_fields_as_long); + bool create_result_table(THD *thd, List<Item> *column_types, + bool is_distinct, ulonglong options, + const char *alias, + bool bit_fields_as_long, + bool create_table); bool init_result_table(ulonglong select_options); bool send_data(List<Item> &items); void cleanup() @@ -3175,7 +3187,7 @@ public: class multi_update :public select_result_interceptor { TABLE_LIST *all_tables; /* query/update command tables */ - TABLE_LIST *leaves; /* list of leves of join table tree */ + List<TABLE_LIST> *leaves; /* list of leves of join table tree */ TABLE_LIST *update_tables, *table_being_updated; TABLE **tmp_tables, *main_table, *table_to_update; TMP_TABLE_PARAM *tmp_table_param; @@ -3201,7 +3213,7 @@ class multi_update :public select_result bool error_handled; public: - multi_update(TABLE_LIST *ut, TABLE_LIST *leaves_list, + multi_update(TABLE_LIST *ut, List<TABLE_LIST> *leaves_list, List<Item> *fields, List<Item> *values, enum_duplicates handle_duplicates, bool ignore); ~multi_update(); === modified file 'sql/sql_cursor.cc' --- a/sql/sql_cursor.cc 2010-02-17 21:59:41 +0000 +++ b/sql/sql_cursor.cc 2010-04-29 21:10:39 +0000 @@ -715,8 +715,8 @@ bool Select_materialize::send_fields(Lis DBUG_ASSERT(table == 0); if (create_result_table(unit->thd, unit->get_unit_column_types(), FALSE, thd->options | TMP_TABLE_ALL_COLUMNS, "", - FALSE)) - return TRUE; + FALSE, TRUE)) + return TRUE; materialized_cursor= new (&table->mem_root) Materialized_cursor(result, table); === modified file 'sql/sql_delete.cc' --- a/sql/sql_delete.cc 2010-03-10 13:55:40 +0000 +++ b/sql/sql_delete.cc 2010-04-29 21:10:39 +0000 @@ -58,10 +58,18 @@ bool mysql_delete(THD *thd, TABLE_LIST * if (open_and_lock_tables(thd, table_list)) DBUG_RETURN(TRUE); - if (!(table= table_list->table)) + + if (mysql_handle_list_of_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT) || + mysql_handle_list_of_derived(thd->lex, table_list, DT_PREPARE)) + DBUG_RETURN(TRUE); + + if (!(table= table_list->table) || !table->created) { - my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), - table_list->view_db.str, table_list->view_name.str); + if (!table_list->updatable) + my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "DELETE"); + else + my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), + table_list->view_db.str, table_list->view_name.str); DBUG_RETURN(TRUE); } thd_proc_info(thd, "init"); @@ -70,6 +78,11 @@ bool mysql_delete(THD *thd, TABLE_LIST * if (mysql_prepare_delete(thd, table_list, &conds)) DBUG_RETURN(TRUE); + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } /* check ORDER BY even if it can be ignored */ if (order && order->elements) { @@ -481,8 +494,8 @@ int mysql_prepare_delete(THD *thd, TABLE if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, table_list, - &select_lex->leaf_tables, FALSE, - DELETE_ACL, SELECT_ACL) || + select_lex->leaf_tables, FALSE, + DELETE_ACL, SELECT_ACL, TRUE) || setup_conds(thd, table_list, select_lex->leaf_tables, conds) || setup_ftfuncs(select_lex)) DBUG_RETURN(TRUE); @@ -549,8 +562,8 @@ int mysql_multi_delete_prepare(THD *thd) if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, lex->query_tables, - &lex->select_lex.leaf_tables, FALSE, - DELETE_ACL, SELECT_ACL)) + lex->select_lex.leaf_tables, FALSE, + DELETE_ACL, SELECT_ACL, TRUE)) DBUG_RETURN(TRUE); @@ -564,16 +577,13 @@ int mysql_multi_delete_prepare(THD *thd) target_tbl; target_tbl= target_tbl->next_local) { + if (!(target_tbl->table= target_tbl->correspondent_table->table)) { - DBUG_ASSERT(target_tbl->correspondent_table->view && - target_tbl->correspondent_table->merge_underlying_list && - target_tbl->correspondent_table->merge_underlying_list-> - next_local); - my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), - target_tbl->correspondent_table->view_db.str, - target_tbl->correspondent_table->view_name.str); - DBUG_RETURN(TRUE); + my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0), + target_tbl->correspondent_table->view_db.str, + target_tbl->correspondent_table->view_name.str); + DBUG_RETURN(TRUE); } if (!target_tbl->correspondent_table->updatable || @@ -623,6 +633,12 @@ multi_delete::prepare(List<Item> &values unit= u; do_delete= 1; thd_proc_info(thd, "deleting from main table"); + SELECT_LEX *select_lex= u->first_select(); + if (select_lex->first_cond_optimization) + { + if (select_lex->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + } DBUG_RETURN(0); } === modified file 'sql/sql_derived.cc' --- a/sql/sql_derived.cc 2010-02-17 21:59:41 +0000 +++ b/sql/sql_derived.cc 2010-04-29 21:10:39 +0000 @@ -23,38 +23,79 @@ #include "mysql_priv.h" #include "sql_select.h" +typedef bool (*dt_processor)(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_init(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_optimize(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_merge(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_create(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_fill(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_reinit(THD *thd, LEX *lex, TABLE_LIST *derived); +bool mysql_derived_merge_for_insert(THD *thd, LEX *lex, TABLE_LIST *derived); + + +dt_processor processors[]= +{ + &mysql_derived_init, + &mysql_derived_prepare, + &mysql_derived_optimize, + &mysql_derived_merge, + &mysql_derived_merge_for_insert, + &mysql_derived_create, + &mysql_derived_fill, + &mysql_derived_reinit, +}; /* - Call given derived table processor (preparing or filling tables) + @brief + Run specified phases on all derived tables/views in given LEX. - SYNOPSIS - mysql_handle_derived() - lex LEX for this thread - processor procedure of derived table processing - - RETURN - FALSE OK - TRUE Error -*/ + @param lex LEX for this thread + @param phases phases to run derived tables/views through + @return FALSE OK + @return TRUE Error +*/ bool -mysql_handle_derived(LEX *lex, bool (*processor)(THD*, LEX*, TABLE_LIST*)) +mysql_handle_derived(LEX *lex, uint phases) { bool res= FALSE; - if (lex->derived_tables) + THD *thd= lex->thd; + if (!lex->derived_tables) + return FALSE; + + lex->thd->derived_tables_processing= TRUE; + + for (uint phase= 0; phase < DT_PHASES && !res; phase++) { - lex->thd->derived_tables_processing= TRUE; + uint phase_flag= DT_INIT << phase; + if (phase_flag > phases) + break; + if (!(phases & phase_flag)) + continue; + if (phase_flag >= DT_CREATE && !thd->fill_derived_tables()) + break; + for (SELECT_LEX *sl= lex->all_selects_list; - sl; + sl && !res; sl= sl->next_select_in_list()) { for (TABLE_LIST *cursor= sl->get_table_list(); - cursor; + cursor && !res; cursor= cursor->next_local) { - if ((res= (*processor)(lex->thd, lex, cursor))) - goto out; + uint8 allowed_phases= (cursor->is_merged_derived() ? DT_PHASES_MERGE : + DT_PHASES_MATERIALIZE); + /* + Skip derived tables to which the phase isn't applicable. + TODO: mark derived at the parse time, later set it's type + (merged or materialized) + */ + if ((phase_flag != DT_PREPARE && !(allowed_phases & phase_flag)) || + (cursor->merged_for_insert && phase_flag != DT_REINIT)) + continue; + res= (*processors[phase])(lex->thd, lex, cursor); } if (lex->describe) { @@ -67,30 +108,439 @@ mysql_handle_derived(LEX *lex, bool (*pr } } } -out: + lex->thd->derived_tables_processing= FALSE; + return res; +} + +/* + @brief + Run through phases for the given derived table/view. + + @param lex LEX for this thread + @param derived the derived table to handle + @param phase_map phases to process tables/views through + + @details + + This function process the derived table (view) 'derived' to performs all + actions that are to be done on the table at the phases specified by + phase_map. The processing is carried out starting from the actions + performed at the earlier phases (those having smaller ordinal numbers). + + @note + This function runs specified phases of the derived tables handling on the + given derived table/view. This function is used in the chain of calls: + SELECT_LEX::handle_derived -> + TABLE_LIST::handle_derived -> + mysql_handle_single_derived + This chain of calls implements the bottom-up handling of the derived tables: + i.e. most inner derived tables/views are handled first. This order is + required for the all phases except the merge and the create steps. + For the sake of code simplicity this order is kept for all phases. + + @return FALSE ok + @return TRUE error +*/ + +bool +mysql_handle_single_derived(LEX *lex, TABLE_LIST *derived, uint phases) +{ + bool res= FALSE; + THD *thd= lex->thd; + uint8 allowed_phases= (derived->is_merged_derived() ? DT_PHASES_MERGE : + DT_PHASES_MATERIALIZE); + if (!lex->derived_tables) + return FALSE; + + lex->thd->derived_tables_processing= TRUE; + + for (uint phase= 0; phase < DT_PHASES; phase++) + { + uint phase_flag= DT_INIT << phase; + if (phase_flag > phases) + break; + if (!(phases & phase_flag) || + derived->merged_for_insert && phase_flag != DT_REINIT) + continue; + /* Skip derived tables to which the phase isn't applicable. */ + if (phase_flag != DT_PREPARE && + !(allowed_phases & phase_flag)) + continue; + if (phase_flag >= DT_CREATE && !thd->fill_derived_tables()) + break; + + if ((res= (*processors[phase])(lex->thd, lex, derived))) + break; + } lex->thd->derived_tables_processing= FALSE; return res; } /** - @brief Create temporary table structure (but do not fill it). + @brief + Run specified phases for derived tables/views in the given list + + @param lex LEX for this thread + @param table_list list of derived tables/view to handle + @param phase_map phases to process tables/views through + + @details + This function runs phases specified by the 'phases_map' on derived + tables/views found in the 'dt_list' with help of the + TABLE_LIST::handle_derived function. + 'lex' is passed as an argument to the TABLE_LIST::handle_derived. - @param thd Thread handle - @param lex LEX for this thread - @param orig_table_list TABLE_LIST for the upper SELECT + @return FALSE ok + @return TRUE error +*/ + +bool +mysql_handle_list_of_derived(LEX *lex, TABLE_LIST *table_list, uint phases) +{ + for (TABLE_LIST *tl= table_list; tl; tl= tl->next_local) + { + if (tl->is_view_or_derived() && + tl->handle_derived(lex, phases)) + return TRUE; + } + return FALSE; +} - @details - This function is called before any command containing derived tables is - executed. Currently the function is used for derived tables, i.e. +/** + @brief + Merge a derived table/view into the embedding select - - Anonymous derived tables, or - - Named derived tables (aka views) with the @c TEMPTABLE algorithm. - - The table reference, contained in @c orig_table_list, is updated with the - fields of a new temporary table. + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + This function merges the given derived table / view into the parent select + construction. Any derived table/reference to view occurred in the FROM + clause of the embedding select is represented by a TABLE_LIST structure a + pointer to which is passed to the function as in the parameter 'derived'. + This structure contains the number/map, alias, a link to SELECT_LEX of the + derived table and other info. If the 'derived' table is used in a nested join + then additionally the structure contains a reference to the ON expression + for this join. + + The merge process results in elimination of the derived table (or the + reference to a view) such that: + - the FROM list of the derived table/view is wrapped into a nested join + after which the nest is added to the FROM list of the embedding select + - the WHERE condition of the derived table (view) is ANDed with the ON + condition attached to the table. + + @note + Tables are merged into the leaf_tables list, original derived table is removed + from this list also. SELECT_LEX::table_list list is left untouched. + Where expression is merged with derived table's on_expr and can be found after + the merge through the SELECT_LEX::table_list. + + Examples of the derived table/view merge: + + Schema: + Tables: t1(f1), t2(f2), t3(f3) + View v1: SELECT f1 FROM t1 WHERE f1 < 1 + + Example with a view: + Before merge: + + The query (Q1): SELECT f1,f2 FROM t2 LEFT JOIN v1 ON f1 = f2 + + (LEX of the main query) + | + (select_lex) + | + (FROM table list) + | + (join list)= t2, v1 + / \ + / (on_expr)= (f1 = f2) + | + (LEX of the v1 view) + | + (select_lex)= SELECT f1 FROM t1 WHERE f1 < 1 + + + After merge: + + The rewritten query Q1 (Q1'): + SELECT f1,f2 FROM t2 LEFT JOIN (t1) ON ((f1 = f2) and (f1 < 1)) + + (LEX of the main query) + | + (select_lex) + | + (FROM table list) + | + (join list)= t2, (t1) + \ + (on_expr)= (f1 = f2) and (f1 < 1) + + In this example table numbers are assigned as follows: + (outer select): t2 - 1, v1 - 2 + (inner select): t1 - 1 + After the merge table numbers will be: + (outer select): t2 - 1, t1 - 2 + + Example with a derived table: + The query Q2: + SELECT f1,f2 + FROM (SELECT f1 FROM t1, t3 WHERE f1=f3 and f1 < 1) tt, t2 + WHERE f1 = f2 + + Before merge: + (LEX of the main query) + | + (select_lex) + / \ + (FROM table list) (WHERE clause)= (f1 = f2) + | + (join list)= tt, t2 + / \ + / (on_expr)= (empty) + / + (select_lex)= SELECT f1 FROM t1, t3 WHERE f1 = f3 and f1 < 1 + + After merge: + + The rewritten query Q2 (Q2'): + SELECT f1,f2 + FROM (t1, t3) JOIN t2 ON (f1 = f3 and f1 < 1) + WHERE f1 = f2 + + (LEX of the main query) + | + (select_lex) + / \ + (FROM table list) (WHERE clause)= (f1 = f2) + | + (join list)= t2, (t1, t3) + \ + (on_expr)= (f1 = f3 and f1 < 1) + + In this example table numbers are assigned as follows: + (outer select): tt - 1, t2 - 2 + (inner select): t1 - 1, t3 - 2 + After the merge table numbers will be: + (outer select): t1 - 1, t2 - 2, t3 - 3 + + @return FALSE if derived table/view were successfully merged. + @return TRUE if an error occur. +*/ + +bool mysql_derived_merge(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + bool res= FALSE; + SELECT_LEX *dt_select= derived->get_single_select(); + table_map map; + uint tablenr; + SELECT_LEX *parent_lex= derived->select_lex; + Query_arena *arena, backup; + + if (derived->merged) + return FALSE; + + arena= thd->activate_stmt_arena_if_needed(&backup); // For easier test + derived->merged= TRUE; + /* + Check whether there is enough free bits in table map to merge subquery. + If not - materialize it. This check isn't cached so when there is a big + and small subqueries, and the bigger one can't be merged it wouldn't + block the smaller one. + */ + if (parent_lex->get_free_table_map(&map, &tablenr)) + { + /* There is no enough table bits, fall back to materialization. */ + derived->change_refs_to_fields(); + derived->set_materialized_derived(); + goto exit_merge; + } + + if (dt_select->leaf_tables.elements + tablenr > MAX_TABLES) + { + /* There is no enough table bits, fall back to materialization. */ + derived->change_refs_to_fields(); + derived->set_materialized_derived(); + goto exit_merge; + } + + if (dt_select->options & OPTION_SCHEMA_TABLE) + parent_lex->options |= OPTION_SCHEMA_TABLE; + + parent_lex->cond_count+= dt_select->cond_count; + + if (!derived->get_unit()->prepared) + { + dt_select->leaf_tables.empty(); + make_leaves_list(dt_select->leaf_tables, derived, TRUE, 0); + } + + if (!derived->merged_for_insert) + { derived->nested_join= (NESTED_JOIN*) thd->calloc(sizeof(NESTED_JOIN)); + if (!derived->nested_join) + { + res= TRUE; + goto exit_merge; + } + + /* Merge derived table's subquery in the parent select. */ + if (parent_lex->merge_subquery(derived, dt_select, tablenr, map)) + { + res= TRUE; + goto exit_merge; + } + + /* + exclude select lex so it doesn't show up in explain. + do this only for derived table as for views this is already done. + + From sql_view.cc + Add subqueries units to SELECT into which we merging current view. + unit(->next)* chain starts with subqueries that are used by this + view and continues with subqueries that are used by other views. + We must not add any subquery twice (otherwise we'll form a loop), + to do this we remember in end_unit the first subquery that has + been already added. + */ + derived->get_unit()->exclude_level(); + if (parent_lex->join) + parent_lex->join->tables+= dt_select->join->tables - 1; + } + if (derived->get_unit()->prepared) + { + Item *expr= derived->on_expr; + expr= and_conds(expr, dt_select->join ? dt_select->join->conds : 0); + if (expr && (derived->prep_on_expr || expr != derived->on_expr)) + { + derived->on_expr= expr; + derived->prep_on_expr= expr->copy_andor_structure(thd); + } + if (derived->on_expr && + ((!derived->on_expr->fixed && + derived->on_expr->fix_fields(thd, &derived->on_expr)) || + derived->on_expr->check_cols(1))) + { + res= TRUE; /* purecov: inspected */ + goto exit_merge; + } + // Update used tables cache according to new table map + if (derived->on_expr) + derived->on_expr->update_used_tables(); + } + +exit_merge: + if (arena) + thd->restore_active_arena(arena, &backup); + return res; +} + + +/** + @brief + Merge a view for the embedding INSERT/UPDATE/DELETE + + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + This function substitutes the derived table for the first table from + the query of the derived table thus making it a correct target table for the + INSERT/UPDATE/DELETE statements. As this operation is correct only for + single table views only, for multi table views this function does nothing. + The derived parameter isn't checked to be a view as derived tables aren't + allowed for INSERT/UPDATE/DELETE statements. + + @return FALSE if derived table/view were successfully merged. + @return TRUE if an error occur. +*/ + +bool mysql_derived_merge_for_insert(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + SELECT_LEX *dt_select= derived->get_single_select(); + + if (derived->merged_for_insert) + return FALSE; + if (!derived->is_multitable()) + { + TABLE_LIST *tl=((TABLE_LIST*)dt_select->table_list.first); + TABLE *table= tl->table; + /* preserve old map & tablenr. */ + if (!derived->merged_for_insert && derived->table) + table->set_table_map(derived->table->map, derived->table->tablenr); + + derived->table= table; + derived->schema_table= + ((TABLE_LIST*)dt_select->table_list.first)->schema_table; + derived->select_lex->leaf_tables.push_back(tl); + } + else + { + if (mysql_derived_merge(thd, lex, derived)) + return TRUE; + } + derived->merged_for_insert= TRUE; + + return FALSE; +} + + +/* + @brief + Initialize a derived table/view + + @param thd Thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @detail + Fill info about derived table/view without preparing an + underlying select. Such as: create a field translation for views, mark it as + a multitable if it is and so on. + + @return + false OK + true Error +*/ + + +bool mysql_derived_init(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + SELECT_LEX_UNIT *unit= derived->get_unit(); + DBUG_ENTER("mysql_derived_init"); + + // Skip already prepared views/DT + if (!unit || unit->prepared) + DBUG_RETURN(FALSE); + + DBUG_RETURN(derived->init_derived(thd, TRUE)); +} + + +/* + @brief + Create temporary table structure (but do not fill it) + + @param thd Thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @detail + Prepare underlying select for a derived table/view. To properly resolve + names in the embedding query the TABLE structure is created. Actual table + is created later by the mysql_derived_create function. + + This function is called before any command containing derived table + is executed. All types of derived tables are handled by this function: + - Anonymous derived tables, or + - Named derived tables (aka views). + The table reference, contained in @c derived, is updated with the + fields of a new temporary table. Derived tables are stored in @c thd->derived_tables and closed by close_thread_tables(). @@ -114,202 +564,359 @@ out: the state of privilege checking (GRANT_INFO struct) is copied as-is to the temporary table. - This function implements a signature called "derived table processor", and - is passed as a function pointer to mysql_handle_derived(). + Only the TABLE structure is created here, actual table is created by the + mysql_derived_create function. @note This function sets @c SELECT_ACL for @c TEMPTABLE views as well as anonymous derived tables, but this is ok since later access checking will distinguish between them. - @see mysql_handle_derived(), mysql_derived_filling(), GRANT_INFO + @see mysql_handle_derived(), mysql_derived_fill(), GRANT_INFO @return false OK true Error */ -bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *orig_table_list) +bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *derived) { - SELECT_LEX_UNIT *unit= orig_table_list->derived; - ulonglong create_options; + SELECT_LEX_UNIT *unit= derived->get_unit(); DBUG_ENTER("mysql_derived_prepare"); bool res= FALSE; - if (unit) - { - SELECT_LEX *first_select= unit->first_select(); - TABLE *table= 0; - select_union *derived_result; - /* prevent name resolving out of derived table */ - for (SELECT_LEX *sl= first_select; sl; sl= sl->next_select()) - sl->context.outer_context= 0; + // Skip already prepared views/DT + if (!unit || unit->prepared || derived->merged_for_insert) + DBUG_RETURN(FALSE); - if (!(derived_result= new select_union)) - DBUG_RETURN(TRUE); // out of memory + /* It's a target view for an INSERT, create field translation only. */ + if (derived->skip_prepare_derived && !derived->is_multitable()) + { + res= derived->create_field_translation(thd); + DBUG_RETURN(res); + } - // st_select_lex_unit::prepare correctly work for single select - if ((res= unit->prepare(thd, derived_result, 0))) - goto exit; + Query_arena *arena= thd->stmt_arena, backup; + if (arena->is_conventional()) + arena= 0; // For easier test + else + thd->set_n_backup_active_arena(arena, &backup); - if ((res= check_duplicate_names(unit->types, 0))) - goto exit; + SELECT_LEX *first_select= unit->first_select(); - create_options= (first_select->options | thd->options | - TMP_TABLE_ALL_COLUMNS); - /* - Temp table is created so that it hounours if UNION without ALL is to be - processed + /* prevent name resolving out of derived table */ + for (SELECT_LEX *sl= first_select; sl; sl= sl->next_select()) + { + sl->context.outer_context= 0; + // Prepare underlying views/DT first. + sl->handle_derived(lex, DT_PREPARE); + } - As 'distinct' parameter we always pass FALSE (0), because underlying - query will control distinct condition by itself. Correct test of - distinct underlying query will be is_union && - !unit->union_distinct->next_select() (i.e. it is union and last distinct - SELECT is last SELECT of UNION). - */ - if ((res= derived_result->create_result_table(thd, &unit->types, FALSE, - create_options, - orig_table_list->alias, - FALSE))) - goto exit; + unit->derived= derived; + + if (!(derived->derived_result= new select_union)) + DBUG_RETURN(TRUE); // out of memory - table= derived_result->table; + // st_select_lex_unit::prepare correctly work for single select + if ((res= unit->prepare(thd, derived->derived_result, 0))) + goto exit; + + if ((res= check_duplicate_names(unit->types, 0))) + goto exit; + + /* + Check whether we can merge this derived table into main select. + Depending on the result field translation will or will not + be created. + */ + if (derived->init_derived(thd, FALSE)) + goto exit; + + /* + Temp table is created so that it hounours if UNION without ALL is to be + processed + + As 'distinct' parameter we always pass FALSE (0), because underlying + query will control distinct condition by itself. Correct test of + distinct underlying query will be is_union && + !unit->union_distinct->next_select() (i.e. it is union and last distinct + SELECT is last SELECT of UNION). + */ + if (derived->derived_result->create_result_table(thd, &unit->types, FALSE, + (first_select->options | + thd->options | + TMP_TABLE_ALL_COLUMNS), + derived->alias, + FALSE, FALSE)) + goto exit; + + derived->table= derived->derived_result->table; + if (derived->is_derived() && derived->is_merged_derived()) + first_select->mark_as_belong_to_derived(derived); exit: - /* Hide "Unknown column" or "Unknown function" error */ - if (orig_table_list->view) - { - if (thd->is_error() && + /* Hide "Unknown column" or "Unknown function" error */ + if (derived->view) + { + if (thd->is_error() && (thd->main_da.sql_errno() == ER_BAD_FIELD_ERROR || - thd->main_da.sql_errno() == ER_FUNC_INEXISTENT_NAME_COLLISION || - thd->main_da.sql_errno() == ER_SP_DOES_NOT_EXIST)) - { - thd->clear_error(); - my_error(ER_VIEW_INVALID, MYF(0), orig_table_list->db, - orig_table_list->table_name); - } - } - - /* - if it is preparation PS only or commands that need only VIEW structure - then we do not need real data and we can skip execution (and parameters - is not defined, too) - */ - if (res) + thd->main_da.sql_errno() == ER_FUNC_INEXISTENT_NAME_COLLISION || + thd->main_da.sql_errno() == ER_SP_DOES_NOT_EXIST)) { - if (table) - free_tmp_table(thd, table); - delete derived_result; + thd->clear_error(); + my_error(ER_VIEW_INVALID, MYF(0), derived->db, + derived->table_name); } + } + + /* + if it is preparation PS only or commands that need only VIEW structure + then we do not need real data and we can skip execution (and parameters + is not defined, too) + */ + if (res) + { + if (derived->table) + free_tmp_table(thd, derived->table); + delete derived->derived_result; + } + else + { + TABLE *table= derived->table; + table->derived_select_number= first_select->select_number; + table->s->tmp_table= NON_TRANSACTIONAL_TMP_TABLE; +#ifndef NO_EMBEDDED_ACCESS_CHECKS + if (derived->referencing_view) + table->grant= derived->grant; else { - if (!thd->fill_derived_tables()) - { - delete derived_result; - derived_result= NULL; - } - orig_table_list->derived_result= derived_result; - orig_table_list->table= table; - orig_table_list->table_name= table->s->table_name.str; - orig_table_list->table_name_length= table->s->table_name.length; - table->derived_select_number= first_select->select_number; - table->s->tmp_table= NON_TRANSACTIONAL_TMP_TABLE; -#ifndef NO_EMBEDDED_ACCESS_CHECKS - if (orig_table_list->referencing_view) - table->grant= orig_table_list->grant; - else - table->grant.privilege= SELECT_ACL; -#endif - orig_table_list->db= (char *)""; - orig_table_list->db_length= 0; - // Force read of table stats in the optimizer - table->file->info(HA_STATUS_VARIABLE); - /* Add new temporary table to list of open derived tables */ - table->next= thd->derived_tables; - thd->derived_tables= table; + table->grant.privilege= SELECT_ACL; + if (derived->is_derived()) + derived->grant.privilege= SELECT_ACL; } +#endif + /* Add new temporary table to list of open derived tables */ + table->next= thd->derived_tables; + thd->derived_tables= table; } - else if (orig_table_list->merge_underlying_list) - orig_table_list->set_underlying_merge(); + if (arena) + thd->restore_active_arena(arena, &backup); DBUG_RETURN(res); } -/* - fill derived table +/** + @brief + Runs optimize phase for a derived table/view. - SYNOPSIS - mysql_derived_filling() - thd Thread handle - lex LEX for this thread - unit node that contains all SELECT's for derived tables - orig_table_list TABLE_LIST for the upper SELECT - - IMPLEMENTATION - Derived table is resolved with temporary table. It is created based on the - queries defined. After temporary table is filled, if this is not EXPLAIN, - then the entire unit / node is deleted. unit is deleted if UNION is used - for derived table and node is deleted is it is a simple SELECT. - If you use this function, make sure it's not called at prepare. - Due to evaluation of LIMIT clause it can not be used at prepared stage. - - RETURN - FALSE OK - TRUE Error + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + Runs optimize phase for given 'derived' derived table/view. + If optimizer finds out that it's of the type "SELECT a_constant" then this + functions also materializes it. + + @return FALSE ok. + @return TRUE if an error occur. */ -bool mysql_derived_filling(THD *thd, LEX *lex, TABLE_LIST *orig_table_list) +bool mysql_derived_optimize(THD *thd, LEX *lex, TABLE_LIST *derived) { - TABLE *table= orig_table_list->table; - SELECT_LEX_UNIT *unit= orig_table_list->derived; + SELECT_LEX_UNIT *unit= derived->get_unit(); + SELECT_LEX *first_select= unit->first_select(); + SELECT_LEX *save_current_select= lex->current_select; + bool res= FALSE; - /*check that table creation pass without problem and it is derived table */ - if (table && unit) + if (unit->optimized && !unit->uncacheable && !unit->describe) + return FALSE; + lex->current_select= first_select; + + if (unit->is_union()) + { + // optimize union without execution + res= unit->optimize(); + } + else if (unit->derived) { - SELECT_LEX *first_select= unit->first_select(); - select_union *derived_result= orig_table_list->derived_result; - SELECT_LEX *save_current_select= lex->current_select; - if (unit->is_union()) + if (!derived->is_merged_derived()) { - // execute union without clean up - res= unit->exec(); + unit->optimized= TRUE; + if ((res= first_select->join->optimize())) + goto err; } - else + } + /* + Materialize derived tables/views of the "SELECT a_constant" type. + Such tables should be materialized at the optimization phase for + correct constant evaluation. + */ + if (!res && derived->fill_me && !derived->merged_for_insert) + { + if (derived->is_merged_derived()) { - unit->set_limit(first_select); - if (unit->select_limit_cnt == HA_POS_ERROR) - first_select->options&= ~OPTION_FOUND_ROWS; - - lex->current_select= first_select; - res= mysql_select(thd, &first_select->ref_pointer_array, - (TABLE_LIST*) first_select->table_list.first, - first_select->with_wild, - first_select->item_list, first_select->where, - (first_select->order_list.elements+ - first_select->group_list.elements), - (ORDER *) first_select->order_list.first, - (ORDER *) first_select->group_list.first, - first_select->having, (ORDER*) NULL, - (first_select->options | thd->options | - SELECT_NO_UNLOCK), - derived_result, unit, first_select); + derived->change_refs_to_fields(); + derived->set_materialized_derived(); } + if ((res= mysql_derived_create(thd, lex, derived))) + goto err; + if ((res= mysql_derived_fill(thd, lex, derived))) + goto err; + } +err: + lex->current_select= save_current_select; + return res; +} - if (!res) - { - /* - Here we entirely fix both TABLE_LIST and list of SELECT's as - there were no derived tables - */ - if (derived_result->flush()) - res= TRUE; - if (!lex->describe) - unit->cleanup(); - } - else - unit->cleanup(); - lex->current_select= save_current_select; +/** + @brief + Actually create result table for a materialized derived table/view. + + @param thd thread handle + @param lex LEX of the embedding query. + @param derived reference to the derived table. + + @details + This function actually creates the result table for given 'derived' + table/view, but it doesn't fill it. + 'thd' and 'lex' parameters are not used by this function. + + @return FALSE ok. + @return TRUE if an error occur. +*/ + +bool mysql_derived_create(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + TABLE *table= derived->table; + SELECT_LEX_UNIT *unit= derived->get_unit(); + + if (table->created) + return FALSE; + select_union *result= (select_union*)unit->result; + if (table->s->db_type() == TMP_ENGINE_HTON) + { + if (create_internal_tmp_table(table, result->tmp_table_param.keyinfo, + result->tmp_table_param.start_recinfo, + &result->tmp_table_param.recinfo, + (unit->first_select()->options | + thd->options | TMP_TABLE_ALL_COLUMNS))) + return(TRUE); } + if (open_tmp_table(table)) + return TRUE; + table->file->extra(HA_EXTRA_WRITE_CACHE); + table->file->extra(HA_EXTRA_IGNORE_DUP_KEY); + return FALSE; +} + + +/* + @brief + Execute subquery of a materialized derived table/view and fill the result + table. + + @param thd Thread handle + @param lex LEX for this thread + @param derived reference to the derived table. + + @details + Execute subquery of given 'derived' table/view and fill the result + table. After result table is filled, if this is not the EXPLAIN statement, + the entire unit / node is deleted. unit is deleted if UNION is used + for derived table and node is deleted is it is a simple SELECT. + 'lex' is unused and 'thd' is passed as an argument to an underlying function. + + @note + If you use this function, make sure it's not called at prepare. + Due to evaluation of LIMIT clause it can not be used at prepared stage. + + @return FALSE OK + @return TRUE Error +*/ + +bool mysql_derived_fill(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + TABLE *table= derived->table; + SELECT_LEX_UNIT *unit= derived->get_unit(); + bool res= FALSE; + + if (unit->executed && !unit->uncacheable && !unit->describe) + return FALSE; + /*check that table creation passed without problems. */ + DBUG_ASSERT(table && table->created); + SELECT_LEX *first_select= unit->first_select(); + select_union *derived_result= derived->derived_result; + SELECT_LEX *save_current_select= lex->current_select; + if (unit->is_union()) + { + // execute union without clean up + res= unit->exec(); + } + else + { + unit->set_limit(first_select); + if (unit->select_limit_cnt == HA_POS_ERROR) + first_select->options&= ~OPTION_FOUND_ROWS; + + lex->current_select= first_select; + res= mysql_select(thd, &first_select->ref_pointer_array, + (TABLE_LIST*) first_select->table_list.first, + first_select->with_wild, + first_select->item_list, first_select->where, + (first_select->order_list.elements+ + first_select->group_list.elements), + (ORDER *) first_select->order_list.first, + (ORDER *) first_select->group_list.first, + first_select->having, (ORDER*) NULL, + (first_select->options | thd->options | + SELECT_NO_UNLOCK), + derived_result, unit, first_select); + } + + if (!res) + { + if (derived_result->flush()) + res= TRUE; + unit->executed= TRUE; + } + if (res || !lex->describe) + unit->cleanup(); + lex->current_select= save_current_select; + return res; } + + +/** + @brief + Re-initialize given derived table/view for the next execution. + + @param thd thread handle + @param lex LEX for this thread + @param derived reference to the derived table. + + @details + Re-initialize given 'derived' table/view for the next execution. + All underlying views/derived tables are recursively reinitialized prior + to re-initialization of given derived table. + 'thd' and 'lex' are passed as arguments to called functions. + + @return FALSE OK + @return TRUE Error +*/ + +bool mysql_derived_reinit(THD *thd, LEX *lex, TABLE_LIST *derived) +{ + st_select_lex_unit *unit= derived->get_unit(); + + if (derived->table) + derived->merged_for_insert= FALSE; + unit->unclean(); + unit->types.empty(); + /* for derived tables & PS (which can't be reset by Item_subquery) */ + unit->reinit_exec_mechanism(); + unit->set_thd(thd); + return FALSE; +} === modified file 'sql/sql_help.cc' --- a/sql/sql_help.cc 2009-10-19 17:14:48 +0000 +++ b/sql/sql_help.cc 2010-04-29 21:10:39 +0000 @@ -628,7 +628,7 @@ bool mysqld_help(THD *thd, const char *m Protocol *protocol= thd->protocol; SQL_SELECT *select; st_find_field used_fields[array_elements(init_used_fields)]; - TABLE_LIST *leaves= 0; + List<TABLE_LIST> leaves; TABLE_LIST tables[4]; List<String> topics_list, categories_list, subcategories_list; String name, description, example; @@ -667,7 +667,7 @@ bool mysqld_help(THD *thd, const char *m thd->lex->select_lex.context.first_name_resolution_table= &tables[0]; if (setup_tables(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, - tables, &leaves, FALSE)) + tables, leaves, FALSE, FALSE)) goto error; memcpy((char*) used_fields, (char*) init_used_fields, sizeof(used_fields)); if (init_fields(thd, tables, used_fields, array_elements(used_fields))) === modified file 'sql/sql_insert.cc' --- a/sql/sql_insert.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sql_insert.cc 2010-04-29 21:10:39 +0000 @@ -124,7 +124,7 @@ bool check_view_single_update(List<Item> { it.init(*values); while ((item= it++)) - tables|= item->used_tables(); + tables|= item->view_used_tables(view); } /* Convert to real table bits */ @@ -140,6 +140,11 @@ bool check_view_single_update(List<Item> if (view->check_single_table(&tbl, tables, view) || tbl == 0) goto error; + /* + A buffer for the insert values was allocated for the merged view. + Use it. + */ + //tbl->table->insert_values= view->table->insert_values; view->table= tbl->table; *map= tables; @@ -243,6 +248,10 @@ static int check_insert_fields(THD *thd, */ table_list->next_local= 0; context->resolve_in_table_list_only(table_list); + /* 'Unfix' fields to allow correct marking by the setup_fields function. */ + if (table_list->is_view()) + unfix_fields(fields); + res= setup_fields(thd, 0, fields, MARK_COLUMNS_WRITE, 0, 0); /* Restore the current context. */ @@ -252,7 +261,7 @@ static int check_insert_fields(THD *thd, if (res) return -1; - if (table_list->effective_algorithm == VIEW_ALGORITHM_MERGE) + if (table_list->is_view() && table_list->is_merged_derived()) { if (check_view_single_update(fields, fields_and_values_from_different_maps ? @@ -341,7 +350,8 @@ static int check_update_fields(THD *thd, if (setup_fields(thd, 0, update_fields, MARK_COLUMNS_WRITE, 0, 0)) return -1; - if (insert_table_list->effective_algorithm == VIEW_ALGORITHM_MERGE && + if (insert_table_list->is_view() && + insert_table_list->is_merged_derived() && check_view_single_update(update_fields, &update_values, insert_table_list, map)) return -1; @@ -641,6 +651,12 @@ bool mysql_insert(THD *thd,TABLE_LIST *t table_list->table_name); DBUG_RETURN(TRUE); } + /* + mark the table_list as a target for insert, to skip the DT/view prepare phase + for correct access rights checks + TODO: remove this hack + */ + table_list->skip_prepare_derived= TRUE; if (table_list->lock_type == TL_WRITE_DELAYED) { @@ -652,6 +668,7 @@ bool mysql_insert(THD *thd,TABLE_LIST *t if (open_and_lock_tables(thd, table_list)) DBUG_RETURN(TRUE); } + lock_type= table_list->lock_type; thd_proc_info(thd, "init"); @@ -1010,6 +1027,12 @@ bool mysql_insert(THD *thd,TABLE_LIST *t ::my_ok(thd, (ulong) thd->row_count_func, id, buff); } thd->abort_on_warning= 0; + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } + DBUG_RETURN(FALSE); abort: @@ -1138,6 +1161,11 @@ static bool mysql_prepare_insert_check_t bool insert_into_view= (table_list->view != 0); DBUG_ENTER("mysql_prepare_insert_check_table"); + if (!table_list->updatable) + { + my_error(ER_NON_INSERTABLE_TABLE, MYF(0), table_list->alias, "INSERT"); + DBUG_RETURN(TRUE); + } /* first table in list is the one we'll INSERT into, requires INSERT_ACL. all others require SELECT_ACL only. the ACL requirement below is for @@ -1148,14 +1176,16 @@ static bool mysql_prepare_insert_check_t if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, table_list, - &thd->lex->select_lex.leaf_tables, - select_insert, INSERT_ACL, SELECT_ACL)) + thd->lex->select_lex.leaf_tables, + select_insert, INSERT_ACL, SELECT_ACL, + TRUE)) DBUG_RETURN(TRUE); if (insert_into_view && !fields.elements) { thd->lex->empty_field_list_on_rset= 1; - if (!table_list->table) + if (table_list->is_multitable() && !table_list->table || + !table_list->table->created) { my_error(ER_VIEW_NO_INSERT_FIELD_LIST, MYF(0), table_list->view_db.str, table_list->view_name.str); @@ -1246,6 +1276,10 @@ bool mysql_prepare_insert(THD *thd, TABL /* INSERT should have a SELECT or VALUES clause */ DBUG_ASSERT (!select_insert || !values); + if (table_list->handle_derived(thd->lex, DT_MERGE_FOR_INSERT)) + DBUG_RETURN(TRUE); + if (mysql_handle_list_of_derived(thd->lex, table_list, DT_PREPARE)) + DBUG_RETURN(TRUE); /* For subqueries in VALUES() we should not see the table in which we are inserting (for INSERT ... SELECT this is done by changing table_list, @@ -2913,9 +2947,9 @@ bool mysql_insert_select_prepare(THD *th { LEX *lex= thd->lex; SELECT_LEX *select_lex= &lex->select_lex; - TABLE_LIST *first_select_leaf_table; DBUG_ENTER("mysql_insert_select_prepare"); + /* Statement-based replication of INSERT ... SELECT ... LIMIT is not safe as order of rows is not defined, so in mixed mode we go to row-based. @@ -2941,21 +2975,37 @@ bool mysql_insert_select_prepare(THD *th &select_lex->where, TRUE, FALSE, FALSE)) DBUG_RETURN(TRUE); + DBUG_ASSERT(select_lex->leaf_tables.elements != 0); + List_iterator<TABLE_LIST> ti(select_lex->leaf_tables); + TABLE_LIST *table; + uint insert_tables; + + if (select_lex->first_cond_optimization) + { + /* Back up leaf_tables list. */ + Query_arena *arena= thd->stmt_arena, backup; + arena= thd->activate_stmt_arena_if_needed(&backup); // For easier test + + insert_tables= select_lex->insert_tables; + while ((table= ti++) && insert_tables--) + { + select_lex->leaf_tables_exec.push_back(table); + table->tablenr_exec= table->table->tablenr; + table->map_exec= table->table->map; + } + if (arena) + thd->restore_active_arena(arena, &backup); + } + ti.rewind(); /* exclude first table from leaf tables list, because it belong to INSERT */ - DBUG_ASSERT(select_lex->leaf_tables != 0); - lex->leaf_tables_insert= select_lex->leaf_tables; /* skip all leaf tables belonged to view where we are insert */ - for (first_select_leaf_table= select_lex->leaf_tables->next_leaf; - first_select_leaf_table && - first_select_leaf_table->belong_to_view && - first_select_leaf_table->belong_to_view == - lex->leaf_tables_insert->belong_to_view; - first_select_leaf_table= first_select_leaf_table->next_leaf) - {} - select_lex->leaf_tables= first_select_leaf_table; + insert_tables= select_lex->insert_tables; + while ((table= ti++) && insert_tables--) + ti.remove(); + DBUG_RETURN(FALSE); } @@ -3169,7 +3219,7 @@ void select_insert::cleanup() select_insert::~select_insert() { DBUG_ENTER("~select_insert"); - if (table) + if (table && table->created) { table->next_number_field=0; table->auto_increment_field_not_null= FALSE; === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.cc 2010-04-29 21:10:39 +0000 @@ -23,6 +23,7 @@ #include <hash.h> #include "sp.h" #include "sp_head.h" +#include "sql_select.h" /* We are using pointer to this variable for distinguishing between assignment @@ -317,7 +318,6 @@ void lex_start(THD *thd) lex->derived_tables= 0; lex->lock_option= TL_READ; lex->safe_to_cache_query= 1; - lex->leaf_tables_insert= 0; lex->parsing_options.reset(); lex->empty_field_list_on_rset= 0; lex->select_lex.select_number= 1; @@ -1590,6 +1590,7 @@ void st_select_lex_unit::init_query() item_list.empty(); describe= 0; found_rows_for_union= 0; + derived= 0; } void st_select_lex::init_query() @@ -1598,7 +1599,8 @@ void st_select_lex::init_query() table_list.empty(); top_join_list.empty(); join_list= &top_join_list; - embedding= leaf_tables= 0; + embedding= 0; + leaf_tables.empty(); item_list.empty(); join= 0; having= prep_having= where= prep_where= 0; @@ -2060,9 +2062,27 @@ void st_select_lex::print_order(String * { if (order->counter_used) { - char buffer[20]; - size_t length= my_snprintf(buffer, 20, "%d", order->counter); - str->append(buffer, (uint) length); + if (query_type != QT_VIEW_INTERNAL) + { + char buffer[20]; + size_t length= my_snprintf(buffer, 20, "%d", order->counter); + str->append(buffer, (uint) length); + } + else + { + /* replace numeric reference with expression */ + if (order->item[0]->type() == Item::INT_ITEM && + order->item[0]->basic_const_item()) + { + char buffer[20]; + size_t length= my_snprintf(buffer, 20, "%d", order->counter); + str->append(buffer, (uint) length); + /* make it expression instead of integer constant */ + str->append(STRING_WITH_LEN("+0")); + } + else + (*order->item)->print(str, query_type); + } } else (*order->item)->print(str, query_type); @@ -2264,22 +2284,6 @@ bool st_lex::can_be_merged() /* find non VIEW subqueries/unions */ bool selects_allow_merge= select_lex.next_select() == 0; - if (selects_allow_merge) - { - for (SELECT_LEX_UNIT *tmp_unit= select_lex.first_inner_unit(); - tmp_unit; - tmp_unit= tmp_unit->next_unit()) - { - if (tmp_unit->first_select()->parent_lex == this && - (tmp_unit->item == 0 || - (tmp_unit->item->place() != IN_WHERE && - tmp_unit->item->place() != IN_ON))) - { - selects_allow_merge= 0; - break; - } - } - } return (selects_allow_merge && select_lex.group_list.elements == 0 && @@ -2909,7 +2913,11 @@ static void fix_prepare_info_in_table_li tbl->prep_on_expr= tbl->on_expr; tbl->on_expr= tbl->on_expr->copy_andor_structure(thd); } - fix_prepare_info_in_table_list(thd, tbl->merge_underlying_list); + if (tbl->is_view_or_derived() && tbl->is_merged_derived()) + { + SELECT_LEX *sel= tbl->get_single_select(); + fix_prepare_info_in_table_list(thd, sel->get_table_list()); + } } } @@ -3024,6 +3032,384 @@ bool st_select_lex::add_index_hint (THD str, length)); } + +/** + @brief Process all derived tables/views of the SELECT. + + @param lex LEX of this thread + @param phase phases to run derived tables/views through + + @details + This function runs specified 'phases' on all tables from the + table_list of this select. + + @return FALSE ok. + @return TRUE an error occur. +*/ + +bool st_select_lex::handle_derived(struct st_lex *lex, uint phases) +{ + for (TABLE_LIST *cursor= (TABLE_LIST*) table_list.first; + cursor; + cursor= cursor->next_local) + { + if (cursor->is_view_or_derived() && cursor->handle_derived(lex, phases)) + return TRUE; + } + return FALSE; +} + + +/** + @brief + Returns first unoccupied table map and table number + + @param map [out] return found map + @param tablenr [out] return found tablenr + + @details + Returns first unoccupied table map and table number in this select. + Map and table are returned in *'map' and *'tablenr' accordingly. + + @retrun TRUE no free table map/table number + @return FALSE found free table map/table number +*/ + +bool st_select_lex::get_free_table_map(table_map *map, uint *tablenr) +{ + *map= 0; + *tablenr= 0; + TABLE_LIST *tl; + if (!join) + { + (*map)= 1<<1; + (*tablenr)++; + return FALSE; + } + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (tl->table->map > *map) + *map= tl->table->map; + if (tl->table->tablenr > *tablenr) + *tablenr= tl->table->tablenr; + } + (*map)<<= 1; + (*tablenr)++; + if (*tablenr >= MAX_TABLES) + return TRUE; + return FALSE; +} + + +/** + @brief + Append given table to the leaf_tables list. + + @param link Offset to which list in table structure to use + @param table Table to append + + @details + Append given 'table' to the leaf_tables list using the 'link' offset. + If the 'table' is linked with other tables through next_leaf/next_local + chains then whole list will be appended. +*/ + +void st_select_lex::append_table_to_list(TABLE_LIST *TABLE_LIST::*link, + TABLE_LIST *table) +{ + TABLE_LIST *tl; + for (tl= leaf_tables.head(); tl->*link; tl= tl->*link); + tl->*link= table; +} + +/* + @brief + Remove given table from the leaf_tables list. + + @param link Offset to which list in table structure to use + @param table Table to remove + + @details + Remove 'table' from the leaf_tables list using the 'link' offset. +*/ + +void st_select_lex::remove_table_from_list(TABLE_LIST *table) +{ + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (tl == table) + { + ti.remove(); + break; + } + } +} + + +/** + @brief + Assigns new table maps to tables in the leaf_tables list + + @param derived Derived table to take initial table map from + @param map table map to begin with + @param tablenr table number to begin with + @param parent_lex new parent select_lex + + @details + Assign new table maps/table numbers to all tables in the leaf_tables list. + 'map'/'tablenr' are used for the first table and shifted to left/ + increased for each consequent table in the leaf_tables list. + If the 'derived' table is given then it's table map/number is used for the + first table in the list and 'map'/'tablenr' are used for the second and + all consequent tables. + The 'parent_lex' is set as the new parent select_lex for all tables in the + list. +*/ + +void st_select_lex::remap_tables(TABLE_LIST *derived, table_map map, + uint tablenr, SELECT_LEX *parent_lex) +{ + bool first_table= TRUE; + TABLE_LIST *tl; + table_map first_map; + uint first_tablenr; + + if (derived && derived->table) + { + first_map= derived->table->map; + first_tablenr= derived->table->tablenr; + } + else + { + first_map= map; + map<<= 1; + first_tablenr= tablenr++; + } + /* + Assign table bit/table number. + To the first table of the subselect the table bit/tablenr of the + derived table is assigned. The rest of tables are getting bits + sequentially, starting from the provided table map/tablenr. + */ + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (first_table) + { + first_table= FALSE; + tl->table->set_table_map(first_map, first_tablenr); + } + else + { + tl->table->set_table_map(map, tablenr); + tablenr++; + map<<= 1; + } + SELECT_LEX *old_sl= tl->select_lex; + tl->select_lex= parent_lex; + for(TABLE_LIST *emb= tl->embedding; + emb && emb->select_lex == old_sl; + emb= emb->embedding) + emb->select_lex= parent_lex; + } +} + +/** + @brief + Merge a subquery into this select. + + @param derived derived table of the subquery to be merged + @param subq_select select_lex of the subquery + @param map table map for assigning to merged tables from subquery + @param table_no table number for assigning to merged tables from subquery + + @details + This function merges a subquery into its parent select. In short the + merge operation appends the subquery FROM table list to the parent's + FROM table list. In more details: + .) the top_join_list of the subquery is wrapped into a join_nest + and attached to 'derived' + .) subquery's leaf_tables list is merged with the leaf_tables + list of this select_lex + .) the table maps and table numbers of the tables merged from + the subquery are adjusted to reflect their new binding to + this select + + @return TRUE an error occur + @return FALSE ok +*/ + +bool SELECT_LEX::merge_subquery(TABLE_LIST *derived, SELECT_LEX *subq_select, + uint table_no, table_map map) +{ + derived->wrap_into_nested_join(subq_select->top_join_list); + /* Reconnect the next_leaf chain. */ + leaf_tables.concat(&subq_select->leaf_tables); + + ftfunc_list->concat(subq_select->ftfunc_list); + if (join) + { + Item_in_subselect **in_subq; + Item_in_subselect **in_subq_end; + for (in_subq= subq_select->join->sj_subselects.front(), + in_subq_end= subq_select->join->sj_subselects.back(); + in_subq != in_subq_end; + in_subq++) + { + join->sj_subselects.append(join->thd->mem_root, *in_subq); + (*in_subq)->emb_on_expr_nest= derived; + } + } + /* + Remove merged table from chain. + When merge_subquery is called at a subquery-to-semijoin transformation + the derived isn't in the leaf_tables list, so in this case the call of + remove_table_from_list does not cause any actions. + */ + remove_table_from_list(derived); + + /* Walk through child's tables and adjust table map, tablenr, + * parent_lex */ + subq_select->remap_tables(derived, map, table_no, this); + return FALSE; +} + + +/** + @brief + Mark tables from the leaf_tables list as belong to a derived table. + + @param derived tables will be marked as belonging to this derived + + @details + Run through the leaf_list and mark all tables as belonging to the 'derived'. +*/ + +void SELECT_LEX::mark_as_belong_to_derived(TABLE_LIST *derived) +{ + /* Mark tables as belonging to this DT */ + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + tl->skip_temporary= 1; + tl->belong_to_derived= derived; + } +} + + +/** + @brief + Update used_tables cache for this select + + @details + This function updates used_tables cache of ON expressions of all tables + in the leaf_tables list and of the conds expression (if any). +*/ + +void SELECT_LEX::update_used_tables() +{ + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(leaf_tables); + while ((tl= ti++)) + { + if (tl->on_expr) + { + tl->on_expr->update_used_tables(); + tl->on_expr->walk(&Item::eval_not_null_tables, 0, NULL); + } + TABLE_LIST *embedding= tl->embedding; + while (embedding) + { + if (embedding->on_expr && + embedding->nested_join->join_list.head() == tl) + { + embedding->on_expr->update_used_tables(); + embedding->on_expr->walk(&Item::eval_not_null_tables, 0, NULL); + } + tl= embedding; + embedding= tl->embedding; + } + } + if (join->conds) + { + join->conds->update_used_tables(); + join->conds->walk(&Item::eval_not_null_tables, 0, NULL); + } +} + +/** + @brief + Increase estimated number of records for a derived table/view + + @param records number of records to increase estimate by + + @details + This function increases estimated number of records by the 'records' + for the derived table to which this select belongs to. +*/ + +void SELECT_LEX::increase_derived_records(uint records) +{ + SELECT_LEX_UNIT *unit= master_unit(); + DBUG_ASSERT(unit->derived); + + select_union *result= (select_union*)unit->result; + result->records+= records; +} + + +/** + @brief + Mark select's derived table as a const one. + + @param empty Whether select has an empty result set + + @details + Mark derived table/view of this select as a constant one (to + materialize it at the optimization phase) unless this select belongs to a + union. Estimated number of rows is incremented if this select has non empty + result set. +*/ + +void SELECT_LEX::mark_const_derived(bool empty) +{ + TABLE_LIST *derived= master_unit()->derived; + if (!join->thd->lex->describe && derived) + { + if (!empty) + increase_derived_records(1); + if (!master_unit()->is_union() && !derived->is_merged_derived()) + derived->fill_me= TRUE; + } +} + +bool st_select_lex::save_leaf_tables(THD *thd) +{ + Query_arena *arena= thd->stmt_arena, backup; + if (arena->is_conventional()) + arena= 0; + else + thd->set_n_backup_active_arena(arena, &backup); + + List_iterator_fast<TABLE_LIST> li(leaf_tables); + TABLE_LIST *table; + while ((table= li++)) + { + if (leaf_tables_exec.push_back(table)) + return 1; + table->tablenr_exec= table->table->tablenr; + table->map_exec= table->table->map; + } + if (arena) + thd->restore_active_arena(arena, &backup); + + return 0; +} + /** A routine used by the parser to decide whether we are specifying a full partitioning or if only partitions to add or to split. === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_lex.h 2010-04-29 21:10:39 +0000 @@ -469,6 +469,11 @@ public: friend bool mysql_new_select(struct st_lex *lex, bool move_down); friend bool mysql_make_view(THD *thd, File_parser *parser, TABLE_LIST *table, uint flags); + friend bool mysql_derived_prepare(THD *thd, st_lex *lex, + TABLE_LIST *orig_table_list); + friend bool mysql_derived_merge(THD *thd, st_lex *lex, + TABLE_LIST *orig_table_list); + friend bool TABLE_LIST::init_derived(THD *thd, bool init_view); private: void fast_exclude(); }; @@ -487,13 +492,12 @@ class st_select_lex_unit: public st_sele protected: TABLE_LIST result_table_list; select_union *union_result; - TABLE *table; /* temporary table using for appending UNION results */ - - select_result *result; ulonglong found_rows_for_union; bool saved_error; public: + TABLE *table; /* temporary table using for appending UNION results */ + select_result *result; bool prepared, // prepare phase already performed for UNION (unit) optimized, // optimize phase already performed for UNION (unit) executed, // already executed @@ -520,6 +524,11 @@ public: ha_rows select_limit_cnt, offset_limit_cnt; /* not NULL if unit used in subselect, point to subselect item */ Item_subselect *item; + /* + TABLE_LIST representing this union in the embedding select. Used for + derived tables/views handling. + */ + TABLE_LIST *derived; /* thread handler */ THD *thd; /* @@ -549,6 +558,7 @@ public: /* UNION methods */ bool prepare(THD *thd, select_result *result, ulong additional_options); + bool optimize(); bool exec(); bool cleanup(); inline void unclean() { cleaned= 0; } @@ -610,8 +620,15 @@ public: Beginning of the list of leaves in a FROM clause, where the leaves inlcude all base tables including view tables. The tables are connected by TABLE_LIST::next_leaf, so leaf_tables points to the left-most leaf. - */ - TABLE_LIST *leaf_tables; + + List of all base tables local to a subquery including all view + tables. Unlike 'next_local', this in this list views are *not* + leaves. Created in setup_tables() -> make_leaves_list(). + */ + List<TABLE_LIST> leaf_tables; + List<TABLE_LIST> leaf_tables_exec; + uint insert_tables; + const char *type; /* type of select for EXPLAIN */ SQL_LIST order_list; /* ORDER clause */ @@ -832,6 +849,28 @@ public: void clear_index_hints(void) { index_hints= NULL; } bool is_part_of_union() { return master_unit()->is_union(); } + bool handle_derived(struct st_lex *lex, uint phases); + void append_table_to_list(TABLE_LIST *TABLE_LIST::*link, TABLE_LIST *table); + bool get_free_table_map(table_map *map, uint *tablenr); + void remove_table_from_list(TABLE_LIST *table); + void remap_tables(TABLE_LIST *derived, table_map map, + uint tablenr, st_select_lex *parent_lex); + bool merge_subquery(TABLE_LIST *derived, st_select_lex *subq_lex, + uint tablenr, table_map map); + inline bool is_mergeable() + { + return (next_select() == 0 && group_list.elements == 0 && + having == 0 && with_sum_func == 0 && + table_list.elements >= 1 && !(options & SELECT_DISTINCT) && + select_limit == 0); + } + void mark_as_belong_to_derived(TABLE_LIST *derived); + void increase_derived_records(uint records); + void update_used_tables(); + void mark_const_derived(bool empty); + + bool save_leaf_tables(THD *thd); + private: /* current index hint kind. used in filling up index_hints */ enum index_hint_type current_index_hint_type; @@ -1556,8 +1595,6 @@ typedef struct st_lex : public Query_tab CHARSET_INFO *charset; bool text_string_is_7bit; - /* store original leaf_tables for INSERT SELECT and PS/SP */ - TABLE_LIST *leaf_tables_insert; /** SELECT of CREATE VIEW statement */ LEX_STRING create_view_select; @@ -1673,7 +1710,7 @@ typedef struct st_lex : public Query_tab DERIVED_SUBQUERY and DERIVED_VIEW). */ uint8 derived_tables; - uint8 create_view_algorithm; + uint16 create_view_algorithm; uint8 create_view_check; bool drop_if_exists, drop_temporary, local_file, one_shot_set; bool autocommit; === modified file 'sql/sql_list.h' --- a/sql/sql_list.h 2009-09-15 10:46:35 +0000 +++ b/sql/sql_list.h 2010-04-29 21:10:39 +0000 @@ -168,6 +168,11 @@ public: { if (!list->is_empty()) { + if (is_empty()) + { + *this= *list; + return; + } *last= list->first; last= list->last; elements+= list->elements; @@ -188,11 +193,13 @@ public: list_node *node= first; list_node *list_first= list->first; elements=0; - while (node && node != list_first) + while (node != list_first) { prev= &node->next; node= node->next; elements++; + if (node == &end_of_list) + return; } *prev= *last; last= prev; === modified file 'sql/sql_load.cc' --- a/sql/sql_load.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_load.cc 2010-04-29 21:10:39 +0000 @@ -164,12 +164,15 @@ int mysql_load(THD *thd,sql_exchange *ex if (open_and_lock_tables(thd, table_list)) DBUG_RETURN(TRUE); + if (mysql_handle_single_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT) || + mysql_handle_single_derived(thd->lex, table_list, DT_PREPARE)) + DBUG_RETURN(TRUE); if (setup_tables_and_check_access(thd, &thd->lex->select_lex.context, &thd->lex->select_lex.top_join_list, table_list, - &thd->lex->select_lex.leaf_tables, FALSE, + thd->lex->select_lex.leaf_tables, FALSE, INSERT_ACL | UPDATE_ACL, - INSERT_ACL | UPDATE_ACL)) + INSERT_ACL | UPDATE_ACL, FALSE)) DBUG_RETURN(-1); if (!table_list->table || // do not suport join view !table_list->updatable || // and derived tables === modified file 'sql/sql_olap.cc' --- a/sql/sql_olap.cc 2007-05-10 09:59:39 +0000 +++ b/sql/sql_olap.cc 2010-04-29 21:10:39 +0000 @@ -154,7 +154,7 @@ int handle_olaps(LEX *lex, SELECT_LEX *s if (setup_tables(lex->thd, &select_lex->context, &select_lex->top_join_list, (TABLE_LIST *)select_lex->table_list.first - &select_lex->leaf_tables, FALSE) || + FALSE, FALSE) || setup_fields(lex->thd, 0, select_lex->item_list, MARK_COLUMNS_READ, &all_fields,1) || setup_fields(lex->thd, 0, item_list_copy, MARK_COLUMNS_READ, === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_parse.cc 2010-04-29 21:10:39 +0000 @@ -458,7 +458,7 @@ static void handle_bootstrap_impl(THD *t thd->init_for_queries(); while (fgets(buff, thd->net.max_packet, file)) { - char *query; + char *query, *res; /* strlen() can't be deleted because fgets() doesn't return length */ ulong length= (ulong) strlen(buff); while (buff[length-1] != '\n' && !feof(file)) @@ -2769,6 +2769,9 @@ mysql_execute_command(THD *thd) } } } + if (mysql_handle_single_derived(thd->lex, create_table, + DT_MERGE_FOR_INSERT)) + DBUG_RETURN(1); /* select_create is currently not re-execution friendly and @@ -3300,6 +3303,10 @@ end_with_restore_list: if (!(res= open_and_lock_tables(thd, all_tables))) { + /* + Only the INSERT table should be merged. Other will be handled by + select. + */ /* Skip first table, which is the table we are inserting in */ TABLE_LIST *second_table= first_table->next_local; select_lex->table_list.first= (uchar*) second_table; @@ -3418,6 +3425,9 @@ end_with_restore_list: thd_proc_info(thd, "init"); if ((res= open_and_lock_tables(thd, all_tables))) break; + if (mysql_handle_list_of_derived(lex, all_tables, DT_MERGE_FOR_INSERT) || + mysql_handle_list_of_derived(lex, all_tables, DT_PREPARE)) + DBUG_RETURN(1); if ((res= mysql_multi_delete_prepare(thd))) goto error; @@ -5183,6 +5193,8 @@ bool check_single_table_access(THD *thd, /* Show only 1 table for check_grant */ if (!(all_tables->belong_to_view && (thd->lex->sql_command == SQLCOM_SHOW_FIELDS)) && + !(all_tables->is_view() && + all_tables->is_merged_derived()) && check_grant(thd, privilege, all_tables, 0, 1, no_errors)) goto deny; === modified file 'sql/sql_prepare.cc' --- a/sql/sql_prepare.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_prepare.cc 2010-04-29 21:10:39 +0000 @@ -1133,7 +1133,9 @@ static bool mysql_test_insert(Prepared_s If we would use locks, then we have to ensure we are not using TL_WRITE_DELAYED as having two such locks can cause table corruption. */ - if (open_normal_and_derived_tables(thd, table_list, 0)) + if (open_normal_and_derived_tables(thd, table_list, 0, + DT_INIT | DT_PREPARE | DT_CREATE) || + mysql_handle_single_derived(thd->lex, table_list, DT_MERGE_FOR_INSERT)) goto error; if ((values= its++)) @@ -1217,7 +1219,10 @@ static int mysql_test_update(Prepared_st open_tables(thd, &table_list, &table_count, 0)) goto error; - if (table_list->multitable_view) + if (mysql_handle_derived(thd->lex, DT_INIT)) + goto error; + + if (table_list->is_multitable()) { DBUG_ASSERT(table_list->view != 0); DBUG_PRINT("info", ("Switch to multi-update")); @@ -1231,7 +1236,7 @@ static int mysql_test_update(Prepared_st thd->fill_derived_tables() is false here for sure (because it is preparation of PS, so we even do not check it). */ - if (mysql_handle_derived(thd->lex, &mysql_derived_prepare)) + if (mysql_handle_derived(thd->lex, DT_PREPARE)) goto error; #ifndef NO_EMBEDDED_ACCESS_CHECKS @@ -1291,7 +1296,8 @@ static bool mysql_test_delete(Prepared_s DBUG_ENTER("mysql_test_delete"); if (delete_precheck(thd, table_list) || - open_normal_and_derived_tables(thd, table_list, 0)) + open_normal_and_derived_tables(thd, table_list, 0, + DT_PREPARE | DT_CREATE)) goto error; if (!table_list->table) @@ -1349,7 +1355,8 @@ static int mysql_test_select(Prepared_st goto error; } - if (open_normal_and_derived_tables(thd, tables, 0)) + if (open_normal_and_derived_tables(thd, tables, 0, + DT_PREPARE | DT_CREATE)) goto error; thd->used_tables= 0; // Updated by setup_fields @@ -1410,7 +1417,8 @@ static bool mysql_test_do_fields(Prepare if (tables && check_table_access(thd, SELECT_ACL, tables, UINT_MAX, FALSE)) DBUG_RETURN(TRUE); - if (open_normal_and_derived_tables(thd, tables, 0)) + if (open_normal_and_derived_tables(thd, tables, 0, + DT_PREPARE | DT_CREATE)) DBUG_RETURN(TRUE); DBUG_RETURN(setup_fields(thd, 0, *values, MARK_COLUMNS_NONE, 0, 0)); } @@ -1440,7 +1448,8 @@ static bool mysql_test_set_fields(Prepar if ((tables && check_table_access(thd, SELECT_ACL, tables, UINT_MAX, FALSE)) || - open_normal_and_derived_tables(thd, tables, 0)) + open_normal_and_derived_tables(thd, tables, 0, + DT_PREPARE | DT_CREATE)) goto error; while ((var= it++)) @@ -1477,7 +1486,7 @@ static bool mysql_test_call_fields(Prepa if ((tables && check_table_access(thd, SELECT_ACL, tables, UINT_MAX, FALSE)) || - open_normal_and_derived_tables(thd, tables, 0)) + open_normal_and_derived_tables(thd, tables, 0, DT_PREPARE)) goto err; while ((item= it++)) @@ -1560,7 +1569,8 @@ select_like_stmt_test_with_open(Prepared prepared EXPLAIN yet so derived tables will clean up after themself. */ - if (open_normal_and_derived_tables(stmt->thd, tables, 0)) + if (open_normal_and_derived_tables(stmt->thd, tables, 0, + DT_PREPARE | DT_CREATE)) DBUG_RETURN(TRUE); DBUG_RETURN(select_like_stmt_test(stmt, specific_prepare, @@ -1605,7 +1615,8 @@ static bool mysql_test_create_table(Prep create_table->skip_temporary= true; } - if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0)) + if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0, + DT_PREPARE | DT_CREATE)) DBUG_RETURN(TRUE); if (!(lex->create_info.options & HA_LEX_CREATE_TMP_TABLE)) @@ -1623,7 +1634,8 @@ static bool mysql_test_create_table(Prep we validate metadata of all CREATE TABLE statements, which keeps metadata validation code simple. */ - if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0)) + if (open_normal_and_derived_tables(stmt->thd, lex->query_tables, 0, + DT_PREPARE)) DBUG_RETURN(TRUE); } @@ -1658,7 +1670,7 @@ static bool mysql_test_create_view(Prepa if (create_view_precheck(thd, tables, view, lex->create_view_mode)) goto err; - if (open_normal_and_derived_tables(thd, tables, 0)) + if (open_normal_and_derived_tables(thd, tables, 0, DT_PREPARE)) goto err; lex->view_prepare_mode= 1; @@ -2349,6 +2361,7 @@ void reinit_stmt_before_use(THD *thd, LE /* Fix ORDER list */ for (order= (ORDER *)sl->order_list.first; order; order= order->next) order->item= &order->item_ptr; + sl->handle_derived(lex, DT_REINIT); /* clear the no_error flag for INSERT/UPDATE IGNORE */ sl->no_error= FALSE; @@ -2392,9 +2405,6 @@ void reinit_stmt_before_use(THD *thd, LE } lex->current_select= &lex->select_lex; - /* restore original list used in INSERT ... SELECT */ - if (lex->leaf_tables_insert) - lex->select_lex.leaf_tables= lex->leaf_tables_insert; if (lex->result) { === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-03-29 20:09:40 +0000 +++ b/sql/sql_select.cc 2010-04-29 21:10:39 +0000 @@ -47,8 +47,8 @@ const char *join_type_str[]={ "UNKNOWN", struct st_sargable_param; static void optimize_keyuse(JOIN *join, DYNAMIC_ARRAY *keyuse_array); -static bool make_join_statistics(JOIN *join, TABLE_LIST *leaves, COND *conds, - DYNAMIC_ARRAY *keyuse); +static bool make_join_statistics(JOIN *join, List<TABLE_LIST> &leaves, + COND *conds, DYNAMIC_ARRAY *keyuse); static bool update_ref_and_keys(THD *thd, DYNAMIC_ARRAY *keyuse, JOIN_TAB *join_tab, uint tables, COND *conds, @@ -99,7 +99,8 @@ static void update_depend_map(JOIN *join static void update_depend_map(JOIN *join, ORDER *order); static ORDER *remove_const(JOIN *join,ORDER *first_order,COND *cond, bool change_list, bool *simple_order); -static int return_zero_rows(JOIN *join, select_result *res,TABLE_LIST *tables, +static int return_zero_rows(JOIN *join, select_result *res, + List<TABLE_LIST> &tables, List<Item> &fields, bool send_row, ulonglong select_options, const char *info, Item *having); @@ -210,7 +211,7 @@ static ORDER *create_distinct_group(THD List<Item> &all_fields, bool *all_order_by_fields_used); static bool test_if_subpart(ORDER *a,ORDER *b); -static TABLE *get_sort_by_table(ORDER *a,ORDER *b,TABLE_LIST *tables); +static TABLE *get_sort_by_table(ORDER *a,ORDER *b,List<TABLE_LIST> &tables); static void calc_group_buffer(JOIN *join,ORDER *group); static bool make_group_fields(JOIN *main_join, JOIN *curr_join); static bool alloc_group_fields(JOIN *join,ORDER *group); @@ -237,6 +238,7 @@ static void add_group_and_distinct_keys( void get_partial_join_cost(JOIN *join, uint idx, double *read_time_arg, double *record_count_arg); static uint make_join_orderinfo(JOIN *join); +static bool generate_derived_keys(List<TABLE_LIST> &tables); static int join_read_record_no_init(JOIN_TAB *tab); @@ -405,7 +407,7 @@ fix_inner_refs(THD *thd, List<Item> &all */ inline int setup_without_group(THD *thd, Item **ref_pointer_array, TABLE_LIST *tables, - TABLE_LIST *leaves, + List<TABLE_LIST> &leaves, List<Item> &fields, List<Item> &all_fields, COND **conds, @@ -483,28 +485,26 @@ JOIN::prepare(Item ***rref_pointer_array join_list= &select_lex->top_join_list; union_part= unit_arg->is_union(); + if (select_lex->handle_derived(thd->lex, DT_PREPARE)) + DBUG_RETURN(1); + thd->lex->current_select->is_item_list_lookup= 1; /* If we have already executed SELECT, then it have not sense to prevent its table from update (see unique_table()) + Affects only materialized derived tables. */ - if (thd->derived_tables_processing) - select_lex->exclude_from_table_unique_test= TRUE; - /* Check that all tables, fields, conds and order are ok */ - - if (!(select_options & OPTION_SETUP_TABLES_DONE) && - setup_tables_and_check_access(thd, &select_lex->context, join_list, - tables_list, &select_lex->leaf_tables, - FALSE, SELECT_ACL, SELECT_ACL)) + if (!(select_options & OPTION_SETUP_TABLES_DONE)) + { + if (setup_tables_and_check_access(thd, &select_lex->context, join_list, + tables_list, select_lex->leaf_tables, + FALSE, SELECT_ACL, SELECT_ACL, FALSE)) DBUG_RETURN(-1); - - TABLE_LIST *table_ptr; - for (table_ptr= select_lex->leaf_tables; - table_ptr; - table_ptr= table_ptr->next_leaf) - tables++; + } + tables= select_lex->leaf_tables.elements; + if (setup_wild(thd, tables_list, fields_list, &all_fields, wild_num) || select_lex->setup_ref_array(thd, og_num) || setup_fields(thd, (*rref_pointer_array), fields_list, MARK_COLUMNS_READ, @@ -605,10 +605,6 @@ JOIN::prepare(Item ***rref_pointer_array } } - if (setup_ftfuncs(select_lex)) /* should be after having->fix_fields */ - DBUG_RETURN(-1); - - /* Check if there are references to un-aggregated columns when computing aggregate functions with implicit grouping (there is no GROUP BY). @@ -720,13 +716,49 @@ JOIN::optimize() if (optimized) DBUG_RETURN(0); optimized= 1; - thd_proc_info(thd, "optimizing"); + + /* Run optimize phase for all derived tables/views used in this SELECT. */ + if (select_lex->handle_derived(thd->lex, DT_OPTIMIZE)) + DBUG_RETURN(1); + + if (select_lex->first_cond_optimization) + { + //Do it only for the first execution + /* Merge all mergeable derived tables/views in this SELECT. */ + if (select_lex->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + tables= select_lex->leaf_tables.elements; + select_lex->update_used_tables(); + + /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ + if (convert_join_subqueries_to_semijoins(this)) + DBUG_RETURN(1); /* purecov: inspected */ + /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ + select_lex->update_used_tables(); + + /* Save this info for the next executions */ + if (select_lex->save_leaf_tables(thd)) + DBUG_RETURN(1); + } + + tables= select_lex->leaf_tables.elements; - /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ - if (convert_join_subqueries_to_semijoins(this)) - DBUG_RETURN(1); /* purecov: inspected */ - /* dump_TABLE_LIST_graph(select_lex, select_lex->leaf_tables); */ +#if 0 + if (thd->lex->describe) + { + /* + Force join->join_tmp creation, because we will use this JOIN + twice for EXPLAIN and we have to have unchanged join for EXPLAINing + */ + select_lex->uncacheable|= UNCACHEABLE_EXPLAIN; + select_lex->master_unit()->uncacheable|= UNCACHEABLE_EXPLAIN; + } +#else +#endif + + if (setup_ftfuncs(select_lex)) /* should be after having->fix_fields */ + DBUG_RETURN(-1); row_limit= ((select_distinct || order || group_list) ? HA_POS_ERROR : unit->select_limit_cnt); @@ -760,7 +792,8 @@ JOIN::optimize() } } #endif - SELECT_LEX *sel= thd->lex->current_select; + + SELECT_LEX *sel= select_lex; if (sel->first_cond_optimization) { /* @@ -785,7 +818,7 @@ JOIN::optimize() if (arena) thd->restore_active_arena(arena, &backup); } - + conds= optimize_cond(this, conds, join_list, &cond_value); if (thd->is_error()) { @@ -823,7 +856,8 @@ JOIN::optimize() #ifdef WITH_PARTITION_STORAGE_ENGINE { TABLE_LIST *tbl; - for (tbl= select_lex->leaf_tables; tbl; tbl= tbl->next_leaf) + List_iterator_fast<TABLE_LIST> li(select_lex->leaf_tables); + while ((tbl= li++)) { /* If tbl->embedding!=NULL that means that this table is in the inner @@ -930,6 +964,8 @@ JOIN::optimize() DBUG_RETURN(1); } + drop_unused_derived_keys(); + if (rollup.state != ROLLUP::STATE_NONE) { if (rollup_process_const_fields()) @@ -1030,6 +1066,7 @@ JOIN::optimize() { zero_result_cause= "Impossible WHERE noticed after reading const tables"; + select_lex->mark_const_derived(zero_result_cause); goto setup_subq_exit; } @@ -1348,7 +1385,7 @@ JOIN::optimize() if (select_options & SELECT_DESCRIBE) { error= 0; - DBUG_RETURN(0); + goto derived_exit; } having= 0; @@ -1497,6 +1534,9 @@ setup_subq_exit: if (setup_subquery_materialization()) DBUG_RETURN(1); error= 0; + +derived_exit: + select_lex->mark_const_derived(zero_result_cause); DBUG_RETURN(0); } @@ -1733,6 +1773,11 @@ JOIN::exec() !tables ? "No tables used" : NullS); DBUG_VOID_RETURN; } + else + { + /* it's a const select, materialize it. */ + select_lex->mark_const_derived(zero_result_cause); + } JOIN *curr_join= this; List<Item> *curr_all_fields= &all_fields; @@ -2232,6 +2277,7 @@ JOIN::destroy() } tmp_join->tmp_join= 0; tmp_table_param.cleanup(); + tmp_join->tmp_table_param.copy_field= 0; DBUG_RETURN(tmp_join->destroy()); } cond_equal= 0; @@ -2512,12 +2558,11 @@ typedef struct st_sargable_param */ static bool -make_join_statistics(JOIN *join, TABLE_LIST *tables_arg, COND *conds, - DYNAMIC_ARRAY *keyuse_array) +make_join_statistics(JOIN *join, List<TABLE_LIST> &tables_list, + COND *conds, DYNAMIC_ARRAY *keyuse_array) { - int error; + int error= 0; TABLE *table; - TABLE_LIST *tables= tables_arg; uint i,table_count,const_count,key; table_map found_const_table_map, all_table_map, found_ref, refs; key_map const_ref, eq_part; @@ -2528,6 +2573,8 @@ make_join_statistics(JOIN *join, TABLE_L table_map no_rows_const_tables= 0; SARGABLE_PARAM *sargables= 0; JOIN_TAB *stat_vector[MAX_TABLES+1]; + List_iterator<TABLE_LIST> ti(tables_list); + TABLE_LIST *tables; DBUG_ENTER("make_join_statistics"); table_count=join->tables; @@ -2543,9 +2590,7 @@ make_join_statistics(JOIN *join, TABLE_L found_const_table_map= all_table_map=0; const_count=0; - for (s= stat, i= 0; - tables; - s++, tables= tables->next_leaf, i++) + for (s= stat, i= 0; (tables= ti++); s++, i++) { TABLE_LIST *embedding= tables->embedding; stat_vector[i]=s; @@ -2555,7 +2600,7 @@ make_join_statistics(JOIN *join, TABLE_L s->needed_reg.init(); table_vector[i]=s->table=table=tables->table; table->pos_in_table_list= tables; - error= table->file->info(HA_STATUS_VARIABLE | HA_STATUS_NO_LOCK); + error= tables->fetch_number_of_rows(); if (error) { table->file->print_error(error, MYF(0)); @@ -2626,6 +2671,7 @@ make_join_statistics(JOIN *join, TABLE_L no_rows_const_tables |= table->map; } } + stat_vector[i]=0; join->outer_join=outer_join; @@ -2641,6 +2687,8 @@ make_join_statistics(JOIN *join, TABLE_L */ for (i= 0, s= stat ; i < table_count ; i++, s++) { + if (!s->dependent) + continue; for (uint j= 0 ; j < table_count ; j++) { table= stat[j].table; @@ -2874,7 +2922,7 @@ make_join_statistics(JOIN *join, TABLE_L } /* Approximate found rows and time to read them */ s->found_records=s->records=s->table->file->stats.records; - s->read_time=(ha_rows) s->table->file->scan_time(); + s->scan_time(); /* Set a max range of how many seeks we can expect when using keys @@ -2959,17 +3007,31 @@ make_join_statistics(JOIN *join, TABLE_L if (optimize_semijoin_nests(join, all_table_map)) DBUG_RETURN(TRUE); /* purecov: inspected */ - /* Find an optimal join order of the non-constant tables. */ - if (join->const_tables != join->tables) - { - if (choose_plan(join, all_table_map & ~join->const_table_map)) - goto error; - } - else - { - memcpy((uchar*) join->best_positions,(uchar*) join->positions, - sizeof(POSITION)*join->const_tables); - join->best_read=1.0; + { + ha_rows records= 1; + SELECT_LEX_UNIT *unit= join->select_lex->master_unit(); + + /* Find an optimal join order of the non-constant tables. */ + if (join->const_tables != join->tables) + { + if (choose_plan(join, all_table_map & ~join->const_table_map)) + goto error; + /* + Calculate estimated number of rows for materialized derived + table/view. + */ + for (i= 0; i < join->tables ; i++) + records*= join->best_positions[i].records_read ? + (ha_rows)join->best_positions[i].records_read : 1; + } + else + { + memcpy((uchar*) join->best_positions,(uchar*) join->positions, + sizeof(POSITION)*join->const_tables); + join->best_read=1.0; + } + if (unit->derived && unit->derived->is_materialized_derived()) + join->select_lex->increase_derived_records(records); } /* Generate an execution plan from the found optimal join order. */ DBUG_RETURN(join->thd->killed || get_best_combination(join)); @@ -2981,8 +3043,12 @@ error: may not be assigned yet by this function (which is building join_tab). Dangling TABLE::reginfo.join_tab may cause part_of_refkey to choke. */ - for (tables= tables_arg; tables; tables= tables->next_leaf) - tables->table->reginfo.join_tab= NULL; + { + TABLE_LIST *table; + List_iterator<TABLE_LIST> ti(tables_list); + while ((table= ti++)) + table->table->reginfo.join_tab= NULL; + } DBUG_RETURN (1); } @@ -3246,6 +3312,10 @@ add_key_field(KEY_FIELD **key_fields,uin table_map usable_tables, SARGABLE_PARAM **sargables) { uint exists_optimize= 0; + if (field->table->pos_in_table_list->is_materialized_derived() && + !field->table->created) + field->table->pos_in_table_list->update_derived_keys(field, value, + num_values); if (!(field->flags & PART_KEY_FLAG)) { // Don't remove column IS NULL on a LEFT JOIN table @@ -3277,7 +3347,7 @@ add_key_field(KEY_FIELD **key_fields,uin else { JOIN_TAB *stat=field->table->reginfo.join_tab; - key_map possible_keys=field->key_start; + key_map possible_keys=field->get_possible_keys(); possible_keys.intersect(field->table->keys_in_use_for_query); stat[0].keys.merge(possible_keys); // Add possible keys @@ -3965,19 +4035,21 @@ update_ref_and_keys(THD *thd, DYNAMIC_AR if (my_init_dynamic_array(keyuse,sizeof(KEYUSE),20,64)) return TRUE; + if (cond) { + KEY_FIELD *saved_field= field; add_key_fields(join_tab->join, &end, &and_level, cond, normal_tables, sargables); for (; field != end ; field++) { - if (add_key_part(keyuse,field)) - return TRUE; + /* Mark that we can optimize LEFT JOIN */ if (field->val->type() == Item::NULL_ITEM && !field->field->real_maybe_null()) field->field->table->reginfo.not_exists_optimize=1; } + field= saved_field; } for (i=0 ; i < tables ; i++) { @@ -4008,6 +4080,9 @@ update_ref_and_keys(THD *thd, DYNAMIC_AR } } + /* Generate keys descriptions for derived tables */ + generate_derived_keys(select_lex->leaf_tables); + /* fill keyuse with found key parts */ for ( ; field != end ; field++) { @@ -4107,7 +4182,7 @@ static void optimize_keyuse(JOIN *join, ~OUTER_REF_TABLE_BIT))) { uint tablenr; - for (tablenr=0 ; ! (map & 1) ; map>>=1, tablenr++) ; + tablenr= my_count_bits(map); if (map == 1) // Only one table { TABLE *tmp_table=join->all_tables[tablenr]; @@ -4714,7 +4789,7 @@ best_access_path(JOIN *join, else { /* Estimate cost of reading table. */ - tmp= s->table->file->scan_time(); + tmp= s->scan_time(); if ((s->table->map & join->outer_join) || disable_jbuf) // Can't use join cache { /* @@ -5065,6 +5140,7 @@ optimize_straight_join(JOIN *join, table { JOIN_TAB *s; uint idx= join->const_tables; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; double record_count= 1.0; double read_time= 0.0; POSITION loose_scan_pos; @@ -5072,7 +5148,7 @@ optimize_straight_join(JOIN *join, table for (JOIN_TAB **pos= join->best_ref + idx ; (s= *pos) ; pos++) { /* Find the best access method from 's' to the current partial plan */ - best_access_path(join, s, join_tables, idx, FALSE, record_count, + best_access_path(join, s, join_tables, idx, disable_jbuf, record_count, join->positions + idx, &loose_scan_pos); /* compute the cost of the new plan extended with 's' */ @@ -5452,6 +5528,7 @@ best_extension_by_limited_search(JOIN JOIN_TAB *s; double best_record_count= DBL_MAX; double best_read_time= DBL_MAX; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; DBUG_EXECUTE("opt", print_plan(join, idx, record_count, read_time, read_time, "part_plan");); @@ -5473,8 +5550,8 @@ best_extension_by_limited_search(JOIN /* Find the best access method from 's' to the current partial plan */ POSITION loose_scan_pos; - best_access_path(join, s, remaining_tables, idx, FALSE, record_count, - join->positions + idx, &loose_scan_pos); + best_access_path(join, s, remaining_tables, idx, disable_jbuf, + record_count, join->positions + idx, &loose_scan_pos); /* Compute the cost of extending the plan with 's' */ @@ -5618,6 +5695,7 @@ find_best(JOIN *join,table_map rest_tabl JOIN_TAB *s; double best_record_count=DBL_MAX,best_read_time=DBL_MAX; + bool disable_jbuf= join->thd->variables.join_cache_level == 0; for (JOIN_TAB **pos=join->best_ref+idx ; (s=*pos) ; pos++) { table_map real_table_bit=s->table->map; @@ -5626,7 +5704,7 @@ find_best(JOIN *join,table_map rest_tabl { double records, best; POSITION loose_scan_pos; - best_access_path(join, s, rest_tables, idx, FALSE, record_count, + best_access_path(join, s, rest_tables, idx, disable_jbuf, record_count, join->positions + idx, &loose_scan_pos); records= join->positions[idx].records_read; best= join->positions[idx].read_time; @@ -5999,8 +6077,7 @@ static bool create_ref_for_key(JOIN *joi if (keyuse->null_rejecting) j->ref.null_rejecting |= 1 << i; keyuse_uses_no_tables= keyuse_uses_no_tables && !keyuse->used_tables; - if (!keyuse->used_tables && - !(join->select_options & SELECT_DESCRIBE)) + if (!keyuse->used_tables && !thd->lex->describe) { // Compare against constant store_key_item tmp(thd, keyinfo->key_part[i].field, key_buff + maybe_null, @@ -6411,7 +6488,7 @@ make_outerjoin_info(JOIN *join) for ( ; embedding ; embedding= embedding->embedding) { /* Ignore sj-nests: */ - if (!embedding->on_expr) + if (!(embedding->on_expr && embedding->outer_join)) continue; NESTED_JOIN *nested_join= embedding->nested_join; if (!nested_join->counter) @@ -6902,6 +6979,63 @@ make_join_select(JOIN *join,SQL_SELECT * } +/** + @brief + Add keys to derived tables'/views' result tables in a list + + @param tables list of tables to generate keys for + + @details + This function generates keys for all derived tables/views in the 'tables' + list with help of the TABLE_LIST:generate_keys function. + + @note currently this function can't fail because errors from the + TABLE_LIST:generate_keys function is ignored as they aren't critical to the + query execution. + + @return FALSE all keys were successfully added. +*/ + +static +bool generate_derived_keys(List<TABLE_LIST> &tables) +{ + TABLE_LIST *table; + List_iterator<TABLE_LIST> ti(tables); + while ((table= ti++)) + { + /* Process tables that aren't materialized yet. */ + if (table->is_materialized_derived() && !table->table->created) + table->generate_keys(); + } + return FALSE; +} + + +/* + @brief + Drops unused keys for each materialized derived table/view + + @details + For materialized derived tables only ref access can be used, it employs + only one index, thus we don't need the rest. For each materialized derived + table/view call TABLE::use_index to save one index chosen by the optimizer + and free others. No key is chosen then all keys will be dropped. +*/ + +void JOIN::drop_unused_derived_keys() +{ + for (uint i= const_tables ; i < tables ; i++) + { + JOIN_TAB *tab=join_tab+i; + TABLE *table=tab->table; + if (!table->pos_in_table_list->is_materialized_derived() || + table->max_keys <= 1) + continue; + table->use_index(tab->ref.key); + tab->ref.key= 0; + } +} + /* Determine {after which table we'll produce ordered set} @@ -7695,6 +7829,30 @@ void JOIN_TAB::cleanup() /** + Initialize the join_tab before reading. + Currently only derived table/view materialization is done here. +*/ + +bool JOIN_TAB::preread_init() +{ + TABLE_LIST *derived= table->pos_in_table_list; + if (!derived->is_materialized_derived()) + { + preread_init_done= TRUE; + return FALSE; + } + + /* Materialize derived table/view. */ + if (!derived->get_unit()->executed && + mysql_handle_single_derived(join->thd->lex, + derived, DT_CREATE | DT_FILL)) + return TRUE; + preread_init_done= TRUE; + return FALSE; +} + + +/** Partially cleanup JOIN after it has executed: close index or rnd read (table cursors), free quick selects. @@ -8118,7 +8276,7 @@ remove_const(JOIN *join,ORDER *first_ord static int -return_zero_rows(JOIN *join, select_result *result,TABLE_LIST *tables, +return_zero_rows(JOIN *join, select_result *result, List<TABLE_LIST> &tables, List<Item> &fields, bool send_row, ulonglong select_options, const char *info, Item *having) { @@ -8134,7 +8292,9 @@ return_zero_rows(JOIN *join, select_resu if (send_row) { - for (TABLE_LIST *table= tables; table; table= table->next_leaf) + List_iterator<TABLE_LIST> ti(tables); + TABLE_LIST *table; + while ((table= ti++)) mark_as_null_row(table->table); // All fields are NULL if (having && having->val_int() == 0) send_row=0; @@ -9685,9 +9845,8 @@ simplify_joins(JOIN *join, List<TABLE_LI { conds= and_conds(conds, table->on_expr); conds->top_level_item(); - /* conds is always a new item as both cond and on_expr existed */ - DBUG_ASSERT(!conds->fixed); - conds->fix_fields(join->thd, &conds); + if (!conds->fixed) + conds->fix_fields(join->thd, &conds); } else conds= table->on_expr; @@ -10707,13 +10866,29 @@ Field *create_tmp_field(THD *thd, TABLE If item have to be able to store NULLs but underlaid field can't do it, create_tmp_field_from_field() can't be used for tmp field creation. */ - if (field->maybe_null && !field->field->maybe_null()) + if ((field->maybe_null || + (orig_item && orig_item->maybe_null)) && /* for outer joined views/dt*/ + !field->field->maybe_null()) { + bool save_maybe_null; + /* + The item the ref points to may have maybe_null flag set while + the ref doesn't have it. This may happen for outer fields + when the outer query decided at some point after name resolution phase + that this field might be null. Take this into account here. + */ + if (orig_item) + { + save_maybe_null= item->maybe_null; + item->maybe_null= orig_item->maybe_null; + } result= create_tmp_field_from_item(thd, item, table, NULL, modify_item, convert_blob_length); *from_field= field->field; if (result && modify_item) field->result_field= result; + if (orig_item) + item->maybe_null= save_maybe_null; } else if (table_cant_handle_bit_fields && field->field->type() == MYSQL_TYPE_BIT) @@ -11397,7 +11572,7 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA share->uniques= test(using_unique_constraint); table->key_info= table->s->key_info= keyinfo; keyinfo->key_part=key_part_info; - keyinfo->flags=HA_NOSAME; + keyinfo->flags=HA_NOSAME | HA_BINARY_PACK_KEY | HA_PACK_KEY; keyinfo->usable_key_parts=keyinfo->key_parts= param->group_parts; keyinfo->key_length=0; keyinfo->rec_per_key=0; @@ -11483,7 +11658,7 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA bzero((void*) key_part_info, keyinfo->key_parts * sizeof(KEY_PART_INFO)); table->key_info= table->s->key_info= keyinfo; keyinfo->key_part=key_part_info; - keyinfo->flags=HA_NOSAME | HA_NULL_ARE_EQUAL; + keyinfo->flags=HA_NOSAME | HA_NULL_ARE_EQUAL | HA_BINARY_PACK_KEY | HA_PACK_KEY; keyinfo->key_length= 0; // Will compute the sum of the parts below. keyinfo->name= (char*) "distinct_key"; keyinfo->algorithm= HA_KEY_ALG_UNDEF; @@ -11551,15 +11726,17 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA if (thd->is_fatal_error) // If end of memory goto err; /* purecov: inspected */ share->db_record_offset= 1; - if (share->db_type() == TMP_ENGINE_HTON) + if (!param->skip_create_table) { - if (create_internal_tmp_table(table, param->keyinfo, param->start_recinfo, - &param->recinfo, select_options)) + if (share->db_type() == TMP_ENGINE_HTON) + { + if (create_internal_tmp_table(table, param->keyinfo, param->start_recinfo, + &param->recinfo, select_options)) + goto err; + } + if (open_tmp_table(table)) goto err; } - if (open_tmp_table(table)) - goto err; - thd->mem_root= mem_root_save; DBUG_RETURN(table); @@ -11714,6 +11891,7 @@ bool open_tmp_table(TABLE *table) return(1); } (void) table->file->extra(HA_EXTRA_QUICK); /* Faster */ + table->created= TRUE; return(0); } @@ -12022,6 +12200,7 @@ static bool create_internal_tmp_table(TA } status_var_increment(table->in_use->status_var.created_tmp_disk_tables); share->db_record_offset= 1; + table->created= TRUE; DBUG_RETURN(0); err: DBUG_RETURN(1); @@ -12177,7 +12356,7 @@ free_tmp_table(THD *thd, TABLE *entry) save_proc_info=thd->proc_info; thd_proc_info(thd, "removing tmp table"); - if (entry->file) + if (entry->file && entry->created) { if (entry->db_stat) entry->file->ha_drop_table(entry->s->table_name.str); @@ -12777,6 +12956,9 @@ sub_select(JOIN *join,JOIN_TAB *join_tab do_sj_reset(join_tab->flush_weedout_table); } + if (!join_tab->preread_init_done && join_tab->preread_init()) + DBUG_RETURN(NESTED_LOOP_ERROR); + if (join->resume_nested_loop) { /* If not the last table, plunge down the nested loop */ @@ -13135,13 +13317,21 @@ static int join_read_const_table(JOIN_TAB *tab, POSITION *pos) { int error; + TABLE_LIST *tbl; DBUG_ENTER("join_read_const_table"); TABLE *table=tab->table; table->const_table=1; table->null_row=0; table->status=STATUS_NO_RECORD; - if (tab->type == JT_SYSTEM) + if (tab->table->pos_in_table_list->is_materialized_derived() && + !tab->table->pos_in_table_list->fill_me) + { + //TODO: don't get here at all + /* Skip materialized derived tables/views. */ + DBUG_RETURN(0); + } + else if (tab->type == JT_SYSTEM) { if ((error=join_read_system(tab))) { // Info for DESCRIBE @@ -13203,26 +13393,27 @@ join_read_const_table(JOIN_TAB *tab, POS if (!table->null_row) table->maybe_null=0; - /* Check appearance of new constant items in Item_equal objects */ - JOIN *join= tab->join; - if (join->conds) - update_const_equal_items(join->conds, tab); - TABLE_LIST *tbl; - for (tbl= join->select_lex->leaf_tables; tbl; tbl= tbl->next_leaf) { - TABLE_LIST *embedded; - TABLE_LIST *embedding= tbl; - do + JOIN *join= tab->join; + List_iterator<TABLE_LIST> ti(join->select_lex->leaf_tables); + /* Check appearance of new constant items in Item_equal objects */ + if (join->conds) + update_const_equal_items(join->conds, tab); + while ((tbl= ti++)) { - embedded= embedding; - if (embedded->on_expr) - update_const_equal_items(embedded->on_expr, tab); - embedding= embedded->embedding; + TABLE_LIST *embedded; + TABLE_LIST *embedding= tbl; + do + { + embedded= embedding; + if (embedded->on_expr) + update_const_equal_items(embedded->on_expr, tab); + embedding= embedded->embedding; + } + while (embedding && + embedding->nested_join->join_list.head() == embedded); } - while (embedding && - embedding->nested_join->join_list.head() == embedded); } - DBUG_RETURN(0); } @@ -13576,6 +13767,9 @@ int join_init_read_record(JOIN_TAB *tab) { if (tab->select && tab->select->quick && tab->select->quick->reset()) return 1; + if (!tab->preread_init_done && tab->preread_init()) + return 1; + init_read_record(&tab->read_record, tab->join->thd, tab->table, tab->select,1,1, FALSE); return (*tab->read_record.read_record)(&tab->read_record); @@ -15500,6 +15694,8 @@ create_sort_index(THD *thd, JOIN *join, get_schema_tables_result(join, PROCESSED_BY_CREATE_SORT_INDEX)) goto err; + if (!tab->preread_init_done && tab->preread_init()) + goto err; if (table->s->tmp_table) table->file->info(HA_STATUS_VARIABLE); // Get record count table->sort.found_records=filesort(thd, table,join->sortorder, length, @@ -16053,7 +16249,7 @@ find_order_in_list(THD *thd, Item **ref_ order->in_field_list= 1; order->counter= count; order->counter_used= 1; - return FALSE; + return FALSE; } /* Lookup the current GROUP/ORDER field in the SELECT clause. */ select_item= find_item_in_list(order_item, fields, &counter, @@ -16494,8 +16690,10 @@ test_if_subpart(ORDER *a,ORDER *b) */ static TABLE * -get_sort_by_table(ORDER *a,ORDER *b,TABLE_LIST *tables) +get_sort_by_table(ORDER *a,ORDER *b, List<TABLE_LIST> &tables) { + TABLE_LIST *table; + List_iterator<TABLE_LIST> ti(tables); table_map map= (table_map) 0; DBUG_ENTER("get_sort_by_table"); @@ -16513,11 +16711,11 @@ get_sort_by_table(ORDER *a,ORDER *b,TABL if (!map || (map & (RAND_TABLE_BIT | OUTER_REF_TABLE_BIT))) DBUG_RETURN(0); - for (; !(map & tables->table->map); tables= tables->next_leaf) ; - if (map != tables->table->map) + while ((table= ti++) && !(map & table->table->map)); + if (map != table->table->map) DBUG_RETURN(0); // More than one table - DBUG_PRINT("exit",("sort by table: %d",tables->table->tablenr)); - DBUG_RETURN(tables->table); + DBUG_PRINT("exit",("sort by table: %d",table->table->tablenr)); + DBUG_RETURN(table->table); } @@ -17886,7 +18084,8 @@ static void select_describe(JOIN *join, if (result->send_data(item_list)) join->error= 1; } - else + else if (!join->select_lex->master_unit()->derived || + join->select_lex->master_unit()->derived->is_materialized_derived()) { table_map used_tables=0; @@ -18194,6 +18393,7 @@ static void select_describe(JOIN *join, if (examined_rows) f= (float) (100.0 * join->best_positions[i].records_read / examined_rows); + set_if_smaller(f, 100.0); item_list.push_back(new Item_float(f, 2)); } } @@ -18456,11 +18656,32 @@ bool mysql_explain_union(THD *thd, SELEC sl; sl= sl->next_select()) { + bool is_primary= FALSE; + if (sl->next_select()) + is_primary= TRUE; + + if (!is_primary && sl->first_inner_unit()) + { + /* + If there is at least one materialized derived|view then it's a PRIMARY select. + Otherwise, all derived tables/views were merged and this select is a SIMPLE one. + */ + for (SELECT_LEX_UNIT *un= sl->first_inner_unit(); + un; + un= un->next_unit()) + { + if ((!un->derived || + un->derived->is_materialized_derived())) + { + is_primary= TRUE; + break; + } + } + } // drop UNCACHEABLE_EXPLAIN, because it is for internal usage only uint8 uncacheable= (sl->uncacheable & ~UNCACHEABLE_EXPLAIN); sl->type= (((&thd->lex->select_lex)==sl)? - (sl->first_inner_unit() || sl->next_select() ? - "PRIMARY" : "SIMPLE"): + (is_primary ? "PRIMARY" : "SIMPLE"): ((sl == first)? ((sl->linkage == DERIVED_TABLE_TYPE) ? "DERIVED": === modified file 'sql/sql_select.h' --- a/sql/sql_select.h 2010-03-20 12:01:47 +0000 +++ b/sql/sql_select.h 2010-04-29 21:10:39 +0000 @@ -293,6 +293,8 @@ typedef struct st_join_table { */ uint sj_strategy; + bool preread_init_done; + void cleanup(); inline bool is_using_loose_index_scan() { @@ -364,6 +366,22 @@ typedef struct st_join_table { select->cond= new_cond; return tmp_select_cond; } + double scan_time() + { + double res; + if (table->created) + { + res= table->file->scan_time(); + read_time=(ha_rows) res; + } + else + { + read_time= found_records ? found_records: 10;// TODO:fix this stub + res= (double)read_time; + } + return res; + } + bool preread_init(); } JOIN_TAB; @@ -1551,6 +1569,7 @@ public: bool union_part; ///< this subselect is part of union bool optimized; ///< flag to avoid double optimization in EXPLAIN + Array<Item_in_subselect> sj_subselects; /* Temporary tables used to weed-out semi-join duplicates */ @@ -1700,6 +1719,7 @@ public: { return (table_map(1) << tables) - 1; } + void drop_unused_derived_keys(); /* Return the table for which an index scan can be used to satisfy the sort order needed by the ORDER BY/(implicit) GROUP BY clause @@ -1744,7 +1764,7 @@ Field* create_tmp_field_from_field(THD * /* functions from opt_sum.cc */ bool simple_pred(Item_func *func_item, Item **args, bool *inv_order); -int opt_sum_query(TABLE_LIST *tables, List<Item> &all_fields,COND *conds); +int opt_sum_query(List<TABLE_LIST> &tables, List<Item> &all_fields,COND *conds); /* from sql_delete.cc, used by opt_range.cc */ extern "C" int refpos_order_cmp(void* arg, const void *a,const void *b); === modified file 'sql/sql_show.cc' --- a/sql/sql_show.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_show.cc 2010-04-29 21:10:39 +0000 @@ -719,7 +719,8 @@ mysqld_show_create(THD *thd, TABLE_LIST { Show_create_error_handler view_error_suppressor(thd, table_list); thd->push_internal_handler(&view_error_suppressor); - bool error= open_normal_and_derived_tables(thd, table_list, 0); + bool error= open_normal_and_derived_tables(thd, table_list, 0, + DT_PREPARE | DT_CREATE); thd->pop_internal_handler(); if (error && (thd->killed || thd->main_da.is_error())) DBUG_RETURN(TRUE); @@ -894,7 +895,8 @@ mysqld_list_fields(THD *thd, TABLE_LIST DBUG_ENTER("mysqld_list_fields"); DBUG_PRINT("enter",("table: %s",table_list->table_name)); - if (open_normal_and_derived_tables(thd, table_list, 0)) + if (open_normal_and_derived_tables(thd, table_list, 0, + DT_PREPARE | DT_CREATE)) DBUG_VOID_RETURN; table= table_list->table; @@ -1680,7 +1682,7 @@ view_store_options(THD *thd, TABLE_LIST static void append_algorithm(TABLE_LIST *table, String *buff) { buff->append(STRING_WITH_LEN("ALGORITHM=")); - switch ((int8)table->algorithm) { + switch ((int16)table->algorithm) { case VIEW_ALGORITHM_UNDEFINED: buff->append(STRING_WITH_LEN("UNDEFINED ")); break; @@ -3360,8 +3362,9 @@ fill_schema_show_cols_or_idxs(THD *thd, SQLCOM_SHOW_FIELDS is used because it satisfies 'only_view_structure()' */ lex->sql_command= SQLCOM_SHOW_FIELDS; - res= open_normal_and_derived_tables(thd, show_table_list, - MYSQL_LOCK_IGNORE_FLUSH); + res= (open_normal_and_derived_tables(thd, show_table_list, + MYSQL_LOCK_IGNORE_FLUSH, + DT_PREPARE | DT_CREATE)); lex->sql_command= save_sql_command; /* get_all_tables() returns 1 on failure and 0 on success thus @@ -3792,8 +3795,9 @@ int get_all_tables(THD *thd, TABLE_LIST lex->sql_command= SQLCOM_SHOW_FIELDS; show_table_list->i_s_requested_object= schema_table->i_s_requested_object; - res= open_normal_and_derived_tables(thd, show_table_list, - MYSQL_LOCK_IGNORE_FLUSH); + res= (open_normal_and_derived_tables(thd, show_table_list, + MYSQL_LOCK_IGNORE_FLUSH, + DT_PREPARE | DT_CREATE)); lex->sql_command= save_sql_command; /* XXX: show_table_list has a flag i_is_requested, === modified file 'sql/sql_table.cc' --- a/sql/sql_table.cc 2010-03-15 11:51:23 +0000 +++ b/sql/sql_table.cc 2010-04-29 21:10:39 +0000 @@ -4623,8 +4623,13 @@ static bool mysql_admin_table(THD* thd, thd->no_warnings_for_error= no_warnings_for_error; if (view_operator_func == NULL) table->required_type=FRMTYPE_TABLE; - + if (lex->sql_command == SQLCOM_CHECK || + lex->sql_command == SQLCOM_REPAIR || + lex->sql_command == SQLCOM_ANALYZE || + lex->sql_command == SQLCOM_OPTIMIZE) + thd->prepare_derived_at_open= TRUE; open_and_lock_tables(thd, table); + thd->prepare_derived_at_open= FALSE; thd->no_warnings_for_error= 0; table->next_global= save_next_global; table->next_local= save_next_local; @@ -4722,7 +4727,7 @@ static bool mysql_admin_table(THD* thd, else /* Default failure code is corrupt table */ result_code= HA_ADMIN_CORRUPT; - goto send_result; + goto send_result; } if (table->view) === modified file 'sql/sql_union.cc' --- a/sql/sql_union.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_union.cc 2010-04-29 21:10:39 +0000 @@ -105,6 +105,7 @@ bool select_union::flush() options create options table_alias name of the temporary table bit_fields_as_long convert bit fields to ulonglong + create_table whether to physically create result table DESCRIPTION Create a temporary table that is used to store the result of a UNION, @@ -119,19 +120,24 @@ bool select_union::create_result_table(THD *thd_arg, List<Item> *column_types, bool is_union_distinct, ulonglong options, const char *alias, - bool bit_fields_as_long) + bool bit_fields_as_long, bool create_table) { DBUG_ASSERT(table == 0); tmp_table_param.init(); tmp_table_param.field_count= column_types->elements; tmp_table_param.bit_fields_as_long= bit_fields_as_long; + tmp_table_param.skip_create_table= !create_table; + if (! (table= create_tmp_table(thd_arg, &tmp_table_param, *column_types, (ORDER*) 0, is_union_distinct, 1, options, HA_POS_ERROR, (char*) alias))) return TRUE; - table->file->extra(HA_EXTRA_WRITE_CACHE); - table->file->extra(HA_EXTRA_IGNORE_DUP_KEY); + if (create_table) + { + table->file->extra(HA_EXTRA_WRITE_CACHE); + table->file->extra(HA_EXTRA_IGNORE_DUP_KEY); + } return FALSE; } @@ -269,6 +275,7 @@ bool st_select_lex_unit::prepare(THD *th (is_union_select ? (ORDER*) 0 : (ORDER*) thd_arg->lex->proc_list.first), sl, this); + /* There are no * in the statement anymore (for PS) */ sl->with_wild= 0; last_procedure= join->procedure; @@ -331,6 +338,8 @@ bool st_select_lex_unit::prepare(THD *th List_iterator_fast<Item> tp(types); Item *type; ulonglong create_options; + uint save_tablenr; + table_map save_map; while ((type= tp++)) { @@ -383,12 +392,22 @@ bool st_select_lex_unit::prepare(THD *th create_options= create_options | TMP_TABLE_FORCE_MYISAM; if (union_result->create_result_table(thd, &types, test(union_distinct), - create_options, "", FALSE)) + create_options, "", FALSE, TRUE)) goto err; + if (!lex_select_save->first_cond_optimization) + { + save_tablenr= result_table_list.tablenr_exec; + save_map= result_table_list.map_exec; + } bzero((char*) &result_table_list, sizeof(result_table_list)); result_table_list.db= (char*) ""; result_table_list.table_name= result_table_list.alias= (char*) "union"; result_table_list.table= table= union_result->table; + if (!lex_select_save->first_cond_optimization) + { + result_table_list.tablenr_exec= save_tablenr; + result_table_list.map_exec= save_map; + } thd_arg->lex->current_select= lex_select_save; if (!item_list.elements) @@ -453,18 +472,21 @@ err: } -bool st_select_lex_unit::exec() +/** + Run optimization phase. + + @return FALSE unit successfully passed optimization phase. + @return TRUE an error occur. +*/ +bool st_select_lex_unit::optimize() { SELECT_LEX *lex_select_save= thd->lex->current_select; SELECT_LEX *select_cursor=first_select(); - ulonglong add_rows=0; - ha_rows examined_rows= 0; - DBUG_ENTER("st_select_lex_unit::exec"); + DBUG_ENTER("st_select_lex_unit::optimize"); - if (executed && !uncacheable && !describe) + if (optimized && !uncacheable && !describe) DBUG_RETURN(FALSE); - executed= 1; - + if (uncacheable || !item || !item->assigned() || describe) { if (item) @@ -485,7 +507,6 @@ bool st_select_lex_unit::exec() } for (SELECT_LEX *sl= select_cursor; sl; sl= sl->next_select()) { - ha_rows records_at_start= 0; thd->lex->current_select= sl; if (optimized) @@ -512,6 +533,66 @@ bool st_select_lex_unit::exec() sl->join->select_options= (select_limit_cnt == HA_POS_ERROR || sl->braces) ? sl->options & ~OPTION_FOUND_ROWS : sl->options | found_rows_for_union; + + saved_error= sl->join->optimize(); + } + + if (saved_error) + { + thd->lex->current_select= lex_select_save; + DBUG_RETURN(saved_error); + } + } + } + optimized= 1; + + thd->lex->current_select= lex_select_save; + DBUG_RETURN(saved_error); +} + + +bool st_select_lex_unit::exec() +{ + SELECT_LEX *lex_select_save= thd->lex->current_select; + SELECT_LEX *select_cursor=first_select(); + ulonglong add_rows=0; + ha_rows examined_rows= 0; + DBUG_ENTER("st_select_lex_unit::exec"); + + if (executed && !uncacheable && !describe) + DBUG_RETURN(FALSE); + executed= 1; + + saved_error= optimize(); + + if (uncacheable || !item || !item->assigned() || describe) + { + for (SELECT_LEX *sl= select_cursor; sl; sl= sl->next_select()) + { + ha_rows records_at_start= 0; + thd->lex->current_select= sl; + + { + set_limit(sl); + if (sl == global_parameters || describe) + { + offset_limit_cnt= 0; + /* + We can't use LIMIT at this stage if we are using ORDER BY for the + whole query + */ + if (sl->order_list.first || describe) + select_limit_cnt= HA_POS_ERROR; + } + + /* + When using braces, SQL_CALC_FOUND_ROWS affects the whole query: + we don't calculate found_rows() per union part. + Otherwise, SQL_CALC_FOUND_ROWS should be done on all sub parts. + */ + sl->join->select_options= + (select_limit_cnt == HA_POS_ERROR || sl->braces) ? + sl->options & ~OPTION_FOUND_ROWS : sl->options | found_rows_for_union; saved_error= sl->join->optimize(); } if (!saved_error) @@ -564,7 +645,6 @@ bool st_select_lex_unit::exec() } } } - optimized= 1; /* Send result to 'result' */ saved_error= TRUE; === modified file 'sql/sql_update.cc' --- a/sql/sql_update.cc 2010-03-20 12:01:47 +0000 +++ b/sql/sql_update.cc 2010-04-29 21:10:39 +0000 @@ -212,7 +212,11 @@ int mysql_update(THD *thd, if (open_tables(thd, &table_list, &table_count, 0)) DBUG_RETURN(1); - if (table_list->multitable_view) + //Prepare views so they are handled correctly. + if (mysql_handle_derived(thd->lex, DT_INIT)) + DBUG_RETURN(1); + + if (table_list->is_multitable()) { DBUG_ASSERT(table_list->view != 0); DBUG_PRINT("info", ("Switch to multi-update")); @@ -227,15 +231,19 @@ int mysql_update(THD *thd, DBUG_RETURN(1); close_tables_for_reopen(thd, &table_list); } - - if (mysql_handle_derived(thd->lex, &mysql_derived_prepare) || - (thd->fill_derived_tables() && - mysql_handle_derived(thd->lex, &mysql_derived_filling))) + if (table_list->handle_derived(thd->lex, DT_MERGE_FOR_INSERT)) + DBUG_RETURN(1); + if (table_list->handle_derived(thd->lex, DT_PREPARE)) DBUG_RETURN(1); thd_proc_info(thd, "init"); table= table_list->table; + if (!table_list->updatable) + { + my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "UPDATE"); + DBUG_RETURN(1); + } /* Calculate "table->covering_keys" based on the WHERE */ table->covering_keys= table->s->keys_in_use; table->quick_keys.clear_all(); @@ -254,13 +262,17 @@ int mysql_update(THD *thd, table_list->grant.want_privilege= table->grant.want_privilege= want_privilege; table_list->register_want_access(want_privilege); #endif + /* 'Unfix' fields to allow correct marking by the setup_fields function. */ + if (table_list->is_view()) + unfix_fields(fields); + if (setup_fields_with_no_wrap(thd, 0, fields, MARK_COLUMNS_WRITE, 0, 0)) DBUG_RETURN(1); /* purecov: inspected */ if (table_list->view && check_fields(thd, fields)) { DBUG_RETURN(1); } - if (!table_list->updatable || check_key_in_view(thd, table_list)) + if (check_key_in_view(thd, table_list)) { my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "UPDATE"); DBUG_RETURN(1); @@ -839,6 +851,11 @@ int mysql_update(THD *thd, } thd->count_cuted_fields= CHECK_FIELD_IGNORE; /* calc cuted fields */ thd->abort_on_warning= 0; + if (thd->lex->current_select->first_cond_optimization) + { + thd->lex->current_select->save_leaf_tables(thd); + thd->lex->current_select->first_cond_optimization= 0; + } DBUG_RETURN((error >= 0 || thd->is_error()) ? 1 : 0); err: @@ -903,8 +920,8 @@ bool mysql_prepare_update(THD *thd, TABL if (setup_tables_and_check_access(thd, &select_lex->context, &select_lex->top_join_list, table_list, - &select_lex->leaf_tables, - FALSE, UPDATE_ACL, SELECT_ACL) || + select_lex->leaf_tables, + FALSE, UPDATE_ACL, SELECT_ACL, TRUE) || setup_conds(thd, table_list, select_lex->leaf_tables, conds) || select_lex->setup_ref_array(thd, order_num) || setup_order(thd, select_lex->ref_pointer_array, @@ -941,8 +958,8 @@ static table_map get_table_map(List<Item Item_field *item; table_map map= 0; - while ((item= (Item_field *) item_it++)) - map|= item->used_tables(); + while ((item= (Item_field *) item_it++)) + map|= item->all_used_tables(); DBUG_PRINT("info", ("table_map: 0x%08lx", (long) map)); return map; } @@ -964,7 +981,7 @@ int mysql_multi_update_prepare(THD *thd) { LEX *lex= thd->lex; TABLE_LIST *table_list= lex->query_tables; - TABLE_LIST *tl, *leaves; + TABLE_LIST *tl; List<Item> *fields= &lex->select_lex.item_list; table_map tables_for_update; bool update_view= 0; @@ -987,19 +1004,24 @@ reopen_tables: /* open tables and create derived ones, but do not lock and fill them */ if (((original_multiupdate || need_reopen) && open_tables(thd, &table_list, &table_count, 0)) || - mysql_handle_derived(lex, &mysql_derived_prepare)) + mysql_handle_derived(lex, DT_INIT)) DBUG_RETURN(TRUE); /* setup_tables() need for VIEWs. JOIN::prepare() will call setup_tables() second time, but this call will do nothing (there are check for second call in setup_tables()). */ + //We need to merge for insert prior to prepare. + if (mysql_handle_list_of_derived(lex, table_list, DT_MERGE_FOR_INSERT)) + DBUG_RETURN(1); + if (mysql_handle_list_of_derived(lex, table_list, DT_PREPARE)) + DBUG_RETURN(1); if (setup_tables_and_check_access(thd, &lex->select_lex.context, &lex->select_lex.top_join_list, table_list, - &lex->select_lex.leaf_tables, FALSE, - UPDATE_ACL, SELECT_ACL)) + lex->select_lex.leaf_tables, FALSE, + UPDATE_ACL, SELECT_ACL, TRUE)) DBUG_RETURN(TRUE); if (setup_fields_with_no_wrap(thd, 0, *fields, MARK_COLUMNS_WRITE, 0, 0)) @@ -1024,8 +1046,8 @@ reopen_tables: /* Setup timestamp handling and locking mode */ - leaves= lex->select_lex.leaf_tables; - for (tl= leaves; tl; tl= tl->next_leaf) + List_iterator<TABLE_LIST> ti(lex->select_lex.leaf_tables); + while ((tl= ti++)) { TABLE *table= tl->table; /* Only set timestamp column if this is not modified */ @@ -1067,7 +1089,7 @@ reopen_tables: for (tl= table_list; tl; tl= tl->next_local) { /* Check access privileges for table */ - if (!tl->derived) + if (!tl->is_derived()) { uint want_privilege= tl->updating ? UPDATE_ACL : SELECT_ACL; if (check_access(thd, want_privilege, @@ -1081,7 +1103,7 @@ reopen_tables: /* check single table update for view compound from several tables */ for (tl= table_list; tl; tl= tl->next_local) { - if (tl->effective_algorithm == VIEW_ALGORITHM_MERGE) + if (tl->is_merged_derived()) { TABLE_LIST *for_update= 0; if (tl->check_single_table(&for_update, tables_for_update, tl)) @@ -1136,6 +1158,8 @@ reopen_tables: */ unit->unclean(); } + // Reset 'prepared' flags for all derived tables/views + mysql_handle_list_of_derived(thd->lex, table_list, DT_REINIT); /* Also we need to cleanup Natural_join_column::table_field items. @@ -1158,7 +1182,8 @@ reopen_tables: */ lex->select_lex.exclude_from_table_unique_test= TRUE; /* We only need SELECT privilege for columns in the values list */ - for (tl= leaves; tl; tl= tl->next_leaf) + ti.rewind(); + while ((tl= ti++)) { TABLE *table= tl->table; TABLE_LIST *tlist; @@ -1187,10 +1212,6 @@ reopen_tables: */ lex->select_lex.exclude_from_table_unique_test= FALSE; - if (thd->fill_derived_tables() && - mysql_handle_derived(lex, &mysql_derived_filling)) - DBUG_RETURN(TRUE); - DBUG_RETURN (FALSE); } @@ -1213,7 +1234,7 @@ bool mysql_multi_update(THD *thd, DBUG_ENTER("mysql_multi_update"); if (!(result= new multi_update(table_list, - thd->lex->select_lex.leaf_tables, + &thd->lex->select_lex.leaf_tables, fields, values, handle_duplicates, ignore))) DBUG_RETURN(TRUE); @@ -1247,7 +1268,7 @@ bool mysql_multi_update(THD *thd, multi_update::multi_update(TABLE_LIST *table_list, - TABLE_LIST *leaves_list, + List<TABLE_LIST> *leaves_list, List<Item> *field_list, List<Item> *value_list, enum enum_duplicates handle_duplicates_arg, bool ignore_arg) @@ -1265,6 +1286,7 @@ multi_update::multi_update(TABLE_LIST *t int multi_update::prepare(List<Item> &not_used_values, SELECT_LEX_UNIT *lex_unit) + { TABLE_LIST *table_ref; SQL_LIST update; @@ -1274,12 +1296,20 @@ int multi_update::prepare(List<Item> &no List_iterator_fast<Item> value_it(*values); uint i, max_fields; uint leaf_table_count= 0; + List_iterator<TABLE_LIST> ti(*leaves); DBUG_ENTER("multi_update::prepare"); thd->count_cuted_fields= CHECK_FIELD_WARN; thd->cuted_fields=0L; thd_proc_info(thd, "updating main table"); + SELECT_LEX *select_lex= lex_unit->first_select(); + if (select_lex->first_cond_optimization) + { + if (select_lex->handle_derived(thd->lex, DT_MERGE)) + DBUG_RETURN(TRUE); + } + tables_to_update= get_table_map(fields); if (!tables_to_update) @@ -1293,7 +1323,7 @@ int multi_update::prepare(List<Item> &no TABLE::tmp_set by pointing TABLE::read_set to it and then restore it after setup_fields(). */ - for (table_ref= leaves; table_ref; table_ref= table_ref->next_leaf) + while ((table_ref= ti++)) { TABLE *table= table_ref->table; if (tables_to_update & table->map) @@ -1311,7 +1341,8 @@ int multi_update::prepare(List<Item> &no int error= setup_fields(thd, 0, *values, MARK_COLUMNS_READ, 0, 0); - for (table_ref= leaves; table_ref; table_ref= table_ref->next_leaf) + ti.rewind(); + while ((table_ref= ti++)) { TABLE *table= table_ref->table; if (tables_to_update & table->map) @@ -1331,7 +1362,8 @@ int multi_update::prepare(List<Item> &no */ update.empty(); - for (table_ref= leaves; table_ref; table_ref= table_ref->next_leaf) + ti.rewind(); + while ((table_ref= ti++)) { /* TODO: add support of view of join support */ TABLE *table=table_ref->table; @@ -1557,9 +1589,9 @@ loop_end: { table_map unupdated_tables= table_ref->check_option->used_tables() & ~first_table_for_update->map; - for (TABLE_LIST *tbl_ref =leaves; - unupdated_tables && tbl_ref; - tbl_ref= tbl_ref->next_leaf) + List_iterator<TABLE_LIST> ti(*leaves); + TABLE_LIST *tbl_ref; + while ((tbl_ref= ti++) && unupdated_tables) { if (unupdated_tables & tbl_ref->table->map) unupdated_tables&= ~tbl_ref->table->map; === modified file 'sql/sql_view.cc' --- a/sql/sql_view.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_view.cc 2010-04-29 21:10:39 +0000 @@ -219,7 +219,7 @@ fill_defined_view_parts (THD *thd, TABLE view->definer.user= decoy.definer.user; lex->definer= &view->definer; } - if (lex->create_view_algorithm == VIEW_ALGORITHM_UNDEFINED) + if (lex->create_view_algorithm == DTYPE_ALGORITHM_UNDEFINED) lex->create_view_algorithm= (uint8) decoy.algorithm; if (lex->create_view_suid == VIEW_SUID_DEFAULT) lex->create_view_suid= decoy.view_suid ? @@ -814,7 +814,7 @@ static int mysql_register_view(THD *thd, ulong sql_mode= thd->variables.sql_mode & MODE_ANSI_QUOTES; thd->variables.sql_mode&= ~MODE_ANSI_QUOTES; - lex->unit.print(&view_query, QT_ORDINARY); + lex->unit.print(&view_query, QT_VIEW_INTERNAL); lex->unit.print(&is_query, QT_IS); thd->variables.sql_mode|= sql_mode; @@ -847,7 +847,7 @@ static int mysql_register_view(THD *thd, { push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_VIEW_MERGE, ER(ER_WARN_VIEW_MERGE)); - lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; + lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; } view->algorithm= lex->create_view_algorithm; view->definer.user= lex->definer->user; @@ -1415,7 +1415,7 @@ bool mysql_make_view(THD *thd, File_pars List_iterator_fast<TABLE_LIST> ti(view_select->top_join_list); - table->effective_algorithm= VIEW_ALGORITHM_MERGE; + table->derived_type= VIEW_ALGORITHM_MERGE; DBUG_PRINT("info", ("algorithm: MERGE")); table->updatable= (table->updatable_view != 0); table->effective_with_check= @@ -1429,67 +1429,10 @@ bool mysql_make_view(THD *thd, File_pars /* prepare view context */ lex->select_lex.context.resolve_in_table_list_only(view_main_select_tables); lex->select_lex.context.outer_context= 0; - lex->select_lex.context.select_lex= table->select_lex; lex->select_lex.select_n_having_items+= table->select_lex->select_n_having_items; - /* - Tables of the main select of the view should be marked as belonging - to the same select as original view (again we can use LEX::select_lex - for this purprose because we don't support MERGE algorithm for views - with unions). - */ - for (tbl= lex->select_lex.get_table_list(); tbl; tbl= tbl->next_local) - tbl->select_lex= table->select_lex; - - { - if (view_main_select_tables->next_local) - { - table->multitable_view= TRUE; - if (table->belong_to_view) - table->belong_to_view->multitable_view= TRUE; - } - /* make nested join structure for view tables */ - NESTED_JOIN *nested_join; - if (!(nested_join= table->nested_join= - (NESTED_JOIN *) thd->calloc(sizeof(NESTED_JOIN)))) - goto err; - nested_join->join_list= view_select->top_join_list; - - /* re-nest tables of VIEW */ - ti.rewind(); - while ((tbl= ti++)) - { - tbl->join_list= &nested_join->join_list; - tbl->embedding= table; - } - } - - /* Store WHERE clause for post-processing in setup_underlying */ table->where= view_select->where; - /* - Add subqueries units to SELECT into which we merging current view. - unit(->next)* chain starts with subqueries that are used by this - view and continues with subqueries that are used by other views. - We must not add any subquery twice (otherwise we'll form a loop), - to do this we remember in end_unit the first subquery that has - been already added. - - NOTE: we do not support UNION here, so we take only one select - */ - SELECT_LEX_NODE *end_unit= table->select_lex->slave; - SELECT_LEX_UNIT *next_unit; - for (SELECT_LEX_UNIT *unit= lex->select_lex.first_inner_unit(); - unit; - unit= next_unit) - { - if (unit == end_unit) - break; - SELECT_LEX_NODE *save_slave= unit->slave; - next_unit= unit->next_unit(); - unit->include_down(table->select_lex); - unit->slave= save_slave; // fix include_down initialisation - } /* We can safely ignore the VIEW's ORDER BY if we merge into union @@ -1506,23 +1449,22 @@ bool mysql_make_view(THD *thd, File_pars goto ok; } - table->effective_algorithm= VIEW_ALGORITHM_TMPTABLE; + table->derived_type= VIEW_ALGORITHM_TMPTABLE; DBUG_PRINT("info", ("algorithm: TEMPORARY TABLE")); view_select->linkage= DERIVED_TABLE_TYPE; table->updatable= 0; table->effective_with_check= VIEW_CHECK_NONE; old_lex->subqueries= TRUE; - /* SELECT tree link */ - lex->unit.include_down(table->select_lex); - lex->unit.slave= view_select; // fix include_down initialisation - table->derived= &lex->unit; } else goto err; ok: + /* SELECT tree link */ + lex->unit.include_down(table->select_lex); + lex->unit.slave= view_select; // fix include_down initialisation /* global SELECT list linking */ end= view_select; // primary SELECT_LEX is always last end->link_next= old_lex->all_selects_list; === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2010-03-15 11:51:23 +0000 +++ b/sql/sql_yacc.yy 2010-04-29 21:10:39 +0000 @@ -1920,7 +1920,7 @@ create: | CREATE { Lex->create_view_mode= VIEW_CREATE_NEW; - Lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; + Lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; Lex->create_view_suid= TRUE; } view_or_trigger_or_sp_or_event @@ -5858,7 +5858,7 @@ alter: my_error(ER_SP_BADSTATEMENT, MYF(0), "ALTER VIEW"); MYSQL_YYABORT; } - lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; + lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; lex->create_view_mode= VIEW_ALTER; } view_tail @@ -13369,7 +13369,7 @@ view_replace: view_algorithm: ALGORITHM_SYM EQ UNDEFINED_SYM - { Lex->create_view_algorithm= VIEW_ALGORITHM_UNDEFINED; } + { Lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; } | ALGORITHM_SYM EQ MERGE_SYM { Lex->create_view_algorithm= VIEW_ALGORITHM_MERGE; } | ALGORITHM_SYM EQ TEMPTABLE_SYM === modified file 'sql/table.cc' --- a/sql/table.cc 2010-03-20 12:01:47 +0000 +++ b/sql/table.cc 2010-04-29 21:10:39 +0000 @@ -20,7 +20,7 @@ #include "sql_trigger.h" #include <m_ctype.h> #include "my_md5.h" - +#include "my_bit.h" /* INFORMATION_SCHEMA name */ LEX_STRING INFORMATION_SCHEMA_NAME= {C_STRING_WITH_LEN("information_schema")}; @@ -3442,129 +3442,118 @@ void TABLE_LIST::calc_md5(char *buffer) /** - @brief Set underlying table for table place holder of view. - - @details - - Replace all views that only use one table with the table itself. This - allows us to treat the view as a simple table and even update it (it is a - kind of optimization). + @brief + Create field translation for mergeable derived table/view. - @note + @param thd Thread handle - This optimization is potentially dangerous as it makes views - masquerade as base tables: Views don't have the pointer TABLE_LIST::table - set to non-@c NULL. + @details + Create field translation for mergeable derived table/view. - We may have the case where a view accesses tables not normally accessible - in the current Security_context (only in the definer's - Security_context). According to the table's GRANT_INFO (TABLE::grant), - access is fulfilled, but this is implicitly meant in the definer's security - context. Hence we must never look at only a TABLE's GRANT_INFO without - looking at the one of the referring TABLE_LIST. + @return FALSE ok. + @return TRUE an error occur. */ -void TABLE_LIST::set_underlying_merge() +bool TABLE_LIST::create_field_translation(THD *thd) { - TABLE_LIST *tbl; + Item *item; + Field_translator *transl; + SELECT_LEX *select= get_single_select(); + List_iterator_fast<Item> it(select->item_list); + uint field_count= 0; + Query_arena *arena= thd->stmt_arena, backup; + bool res= FALSE; - if ((tbl= merge_underlying_list)) + used_items.empty(); + + if (field_translation) { - /* This is a view. Process all tables of view */ - DBUG_ASSERT(view && effective_algorithm == VIEW_ALGORITHM_MERGE); - do + /* + Update items in the field translation aftet view have been prepared. + It's needed because some items in the select list, like IN subselects, + might be substituted for optimized ones. + */ + if (is_view() && get_unit()->prepared && !field_translation_updated) { - if (tbl->merge_underlying_list) // This is a view + while ((item= it++)) { - DBUG_ASSERT(tbl->view && - tbl->effective_algorithm == VIEW_ALGORITHM_MERGE); - /* - This is the only case where set_ancestor is called on an object - that may not be a view (in which case ancestor is 0) - */ - tbl->merge_underlying_list->set_underlying_merge(); + field_translation[field_count++].item= item; } - } while ((tbl= tbl->next_local)); - - if (!multitable_view) - { - table= merge_underlying_list->table; - schema_table= merge_underlying_list->schema_table; + field_translation_updated= TRUE; } + + return FALSE; + } + + if (arena->is_conventional()) + arena= 0; // For easier test + else + thd->set_n_backup_active_arena(arena, &backup); + + /* Create view fields translation table */ + + if (!(transl= + (Field_translator*)(thd->stmt_arena-> + alloc(select->item_list.elements * + sizeof(Field_translator))))) + { + res= TRUE; + goto exit; } + + while ((item= it++)) + { + transl[field_count].name= item->name; + transl[field_count++].item= item; + } + field_translation= transl; + field_translation_end= transl + field_count; + +exit: + if (arena) + thd->restore_active_arena(arena, &backup); + + return res; } -/* - setup fields of placeholder of merged VIEW +/** + @brief + Create field translation for mergeable derived table/view. - SYNOPSIS - TABLE_LIST::setup_underlying() - thd - thread handler + @param thd Thread handle - DESCRIPTION - It is: - - preparing translation table for view columns - If there are underlying view(s) procedure first will be called for them. + @details + Create field translation for mergeable derived table/view. - RETURN - FALSE - OK - TRUE - error + @return FALSE ok. + @return TRUE an error occur. */ bool TABLE_LIST::setup_underlying(THD *thd) { DBUG_ENTER("TABLE_LIST::setup_underlying"); - if (!field_translation && merge_underlying_list) + if (!view || (!field_translation && merge_underlying_list)) { - Field_translator *transl; - SELECT_LEX *select= &view->select_lex; - Item *item; - TABLE_LIST *tbl; + SELECT_LEX *select= get_single_select(); List_iterator_fast<Item> it(select->item_list); - uint field_count= 0; + TABLE_LIST *tbl; - if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*) &field_count)) + if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*) &tbl)) { DBUG_RETURN(TRUE); } - - for (tbl= merge_underlying_list; tbl; tbl= tbl->next_local) - { - if (tbl->merge_underlying_list && - tbl->setup_underlying(thd)) - { - DBUG_RETURN(TRUE); - } - } - - /* Create view fields translation table */ - - if (!(transl= - (Field_translator*)(thd->stmt_arena-> - alloc(select->item_list.elements * - sizeof(Field_translator))))) - { + if (create_field_translation(thd)) DBUG_RETURN(TRUE); - } - - while ((item= it++)) - { - transl[field_count].name= item->name; - transl[field_count++].item= item; - } - field_translation= transl; - field_translation_end= transl + field_count; - /* TODO: use hash for big number of fields */ /* full text function moving to current select */ - if (view->select_lex.ftfunc_list->elements) + if (select->ftfunc_list->elements) { Item_func_match *ifm; SELECT_LEX *current_select= thd->lex->current_select; List_iterator_fast<Item_func_match> - li(*(view->select_lex.ftfunc_list)); + li(*(select_lex->ftfunc_list)); while ((ifm= li++)) current_select->ftfunc_list->push_front(ifm); } @@ -3574,7 +3563,7 @@ bool TABLE_LIST::setup_underlying(THD *t /* - Prepare where expression of view + Prepare where expression of derived table/view SYNOPSIS TABLE_LIST::prep_where() @@ -3598,7 +3587,8 @@ bool TABLE_LIST::prep_where(THD *thd, It for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) { - if (tbl->view && tbl->prep_where(thd, conds, no_where_clause)) + if (tbl->is_view_or_derived() && + tbl->prep_where(thd, conds, no_where_clause)) { DBUG_RETURN(TRUE); } @@ -3606,6 +3596,8 @@ bool TABLE_LIST::prep_where(THD *thd, It if (where) { + if (where->fixed) + where->update_used_tables(); if (!where->fixed && where->fix_fields(thd, &where)) { DBUG_RETURN(TRUE); @@ -3638,7 +3630,13 @@ bool TABLE_LIST::prep_where(THD *thd, It } } if (tbl == 0) + { + if (*conds && !(*conds)->fixed) + (*conds)->fix_fields(thd, conds); *conds= and_conds(*conds, where->copy_andor_structure(thd)); + if (*conds && !(*conds)->fixed) + (*conds)->fix_fields(thd, conds); + } if (arena) thd->restore_active_arena(arena, &backup); where_processed= TRUE; @@ -3677,10 +3675,11 @@ merge_on_conds(THD *thd, TABLE_LIST *tab DBUG_PRINT("info", ("alias: %s", table->alias)); if (table->on_expr) cond= table->on_expr->copy_andor_structure(thd); - if (!table->nested_join) + if (!table->view) DBUG_RETURN(cond); - List_iterator<TABLE_LIST> li(table->nested_join->join_list); - while (TABLE_LIST *tbl= li++) + for (TABLE_LIST *tbl= (TABLE_LIST*)table->view->select_lex.table_list.first; + tbl; + tbl= tbl->next_local) { if (tbl->view && !is_cascaded) continue; @@ -3720,7 +3719,7 @@ bool TABLE_LIST::prep_check_option(THD * { DBUG_ENTER("TABLE_LIST::prep_check_option"); bool is_cascaded= check_opt_type == VIEW_CHECK_CASCADED; - + TABLE_LIST *merge_underlying_list= view->select_lex.get_table_list(); for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) { /* see comment of check_opt_type parameter */ @@ -3833,10 +3832,14 @@ void TABLE_LIST::hide_view_error(THD *th TABLE_LIST *TABLE_LIST::find_underlying_table(TABLE *table_to_find) { /* is this real table and table which we are looking for? */ - if (table == table_to_find && merge_underlying_list == 0) + if (table == table_to_find && view == 0) return this; + if (!view) + return 0; - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + for (TABLE_LIST *tbl= view->select_lex.get_table_list(); + tbl; + tbl= tbl->next_local) { TABLE_LIST *result; if ((result= tbl->find_underlying_table(table_to_find))) @@ -3918,7 +3921,12 @@ bool TABLE_LIST::check_single_table(TABL table_map map, TABLE_LIST *view_arg) { - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + if (!select_lex) + return FALSE; + DBUG_ASSERT(is_merged_derived()); + for (TABLE_LIST *tbl= get_single_select()->get_table_list(); + tbl; + tbl= tbl->next_local) { if (tbl->table) { @@ -3960,8 +3968,10 @@ bool TABLE_LIST::set_insert_values(MEM_R } else { - DBUG_ASSERT(view && merge_underlying_list); - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + DBUG_ASSERT(is_view_or_derived() && is_merged_derived()); + for (TABLE_LIST *tbl= (TABLE_LIST*)view->select_lex.table_list.first; + tbl; + tbl= tbl->next_local) if (tbl->set_insert_values(mem_root)) return TRUE; } @@ -3987,7 +3997,7 @@ bool TABLE_LIST::set_insert_values(MEM_R */ bool TABLE_LIST::is_leaf_for_name_resolution() { - return (view || is_natural_join || is_join_columns_complete || + return (is_merged_derived() || is_natural_join || is_join_columns_complete || !nested_join); } @@ -4125,7 +4135,11 @@ void TABLE_LIST::register_want_access(ul if (table) table->grant.want_privilege= want_access; } - for (TABLE_LIST *tbl= merge_underlying_list; tbl; tbl= tbl->next_local) + if (!view) + return; + for (TABLE_LIST *tbl= view->select_lex.get_table_list(); + tbl; + tbl= tbl->next_local) tbl->register_want_access(want_access); } @@ -4358,14 +4372,23 @@ const char *Natural_join_column::db_name DBUG_ASSERT(!strcmp(table_ref->db, table_ref->table->s->db.str) || (table_ref->schema_table && - table_ref->table->s->db.str[0] == 0)); + table_ref->table->s->db.str[0] == 0) || + table_ref->is_materialized_derived()); return table_ref->db; } GRANT_INFO *Natural_join_column::grant() { - if (view_field) +/* if (view_field) + return &(table_ref->grant); + return &(table_ref->table->grant);*/ + /* + Have to check algorithm because merged derived also has + field_translation. + */ +//if (table_ref->effective_algorithm == DTYPE_ALGORITHM_MERGE) + if (table_ref->is_merged_derived()) return &(table_ref->grant); return &(table_ref->table->grant); } @@ -4448,7 +4471,15 @@ Item *create_view_field(THD *thd, TABLE_ } Item *item= new Item_direct_view_ref(&view->view->select_lex.context, field_ref, view->alias, - name); + name, view); + /* + Force creation of nullable item for the result tmp table for outer joined + views/derived tables. + */ + if (view->outer_join) + item->maybe_null= TRUE; + /* Save item in case we will need to fall back to materialization. */ + view->used_items.push_back(item); DBUG_RETURN(item); } @@ -4502,8 +4533,7 @@ void Field_iterator_table_ref::set_field /* This is a merge view, so use field_translation. */ else if (table_ref->field_translation) { - DBUG_ASSERT(table_ref->view && - table_ref->effective_algorithm == VIEW_ALGORITHM_MERGE); + DBUG_ASSERT(table_ref->is_merged_derived()); field_it= &view_field_it; DBUG_PRINT("info", ("field_it for '%s' is Field_iterator_view", table_ref->alias)); @@ -5096,6 +5126,142 @@ void st_table::mark_virtual_columns_for_ file->column_bitmaps_signal(); } + +/** + @brief + Allocate space for keys + + @param key_count number of keys to allocate. + + @details + Allocate space enough to fit 'key_count' keys for this table. + + @return FALSE space was successfully allocated. + @return TRUE an error occur. +*/ + +bool TABLE::alloc_keys(uint key_count) +{ + DBUG_ASSERT(!s->keys); + key_info= s->key_info= (KEY*) my_malloc(sizeof(KEY)*key_count, MYF(0)); + max_keys= key_count; + return !(key_info); +} + + +/** + @brief Adds one key to a temporary table. + + @param key_parts bitmap of fields that take a part in the key. + @param key_name name of the key + + @details + Creates a key for this table from fields which corresponds the bits set to 1 + in the 'key_parts' bitmap. The 'key_name' name is given to the newly created + key. + + @return <0 an error occur. + @return >=0 number of newly added key. +*/ + +int TABLE::add_tmp_key(ulonglong key_parts, char *key_name) +{ + DBUG_ASSERT(!created && s->keys< max_keys); + + KEY* keyinfo; + Field **reg_field; + uint i; + bool key_start= TRUE; + uint key_part_count= my_count_bits(key_parts); + KEY_PART_INFO* key_part_info= + (KEY_PART_INFO*) my_malloc(sizeof(KEY_PART_INFO)* key_part_count, MYF(0)); + if (!key_part_info) + return -1; + keyinfo= key_info + s->keys; + keyinfo->key_part=key_part_info; + keyinfo->usable_key_parts=keyinfo->key_parts= key_part_count; + keyinfo->key_length=0; + keyinfo->algorithm= HA_KEY_ALG_UNDEF; + keyinfo->name= key_name; + keyinfo->flags= HA_GENERATED_KEY; + keyinfo->rec_per_key= (ulong*)my_malloc(sizeof(ulong)*key_part_count, MYF(0)); + if (!keyinfo->rec_per_key) + return -1; + bzero(keyinfo->rec_per_key, sizeof(ulong)*key_part_count); + for (i= 0, reg_field=field ; + *reg_field; + i++, reg_field++) + { + if (!(key_parts & (1 << i))) + continue; + if (key_start) + (*reg_field)->key_start.set_bit(s->keys); + key_start= FALSE; + (*reg_field)->part_of_key.set_bit(s->keys); + (*reg_field)->flags|= PART_KEY_FLAG; + key_part_info->null_bit= (*reg_field)->null_bit; + key_part_info->null_offset= (uint) ((*reg_field)->null_ptr - + (uchar*) record[0]); + key_part_info->field= *reg_field; + key_part_info->offset= (*reg_field)->offset(record[0]); + key_part_info->length= (uint16) (*reg_field)->pack_length(); + keyinfo->key_length+= key_part_info->length; + /* TODO: + The below method of computing the key format length of the + key part is a copy/paste from opt_range.cc, and table.cc. + This should be factored out, e.g. as a method of Field. + In addition it is not clear if any of the Field::*_length + methods is supposed to compute the same length. If so, it + might be reused. + */ + key_part_info->store_length= key_part_info->length; + + if ((*reg_field)->real_maybe_null()) + key_part_info->store_length+= HA_KEY_NULL_LENGTH; + if ((*reg_field)->type() == MYSQL_TYPE_BLOB || + (*reg_field)->real_type() == MYSQL_TYPE_VARCHAR) + key_part_info->store_length+= HA_KEY_BLOB_LENGTH; + + key_part_info->type= (uint8) (*reg_field)->key_type(); + key_part_info->key_type = + ((ha_base_keytype) key_part_info->type == HA_KEYTYPE_TEXT || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT1 || + (ha_base_keytype) key_part_info->type == HA_KEYTYPE_VARTEXT2) ? + 0 : FIELDFLAG_BINARY; + key_part_info++; + } + set_if_bigger(s->max_key_length, keyinfo->key_length); + return ++s->keys - 1; +} + + +/* + @brief + Drop all indexes except specified one. + + @param key_to_save the key to save + + @details + Drop all indexes on this table except 'key_to_save'. The saved key becomes + key #0. Memory occupied by key parts of dropped keys are freed. + If the 'key_to_save' is negative then all keys are freed. +*/ + +void TABLE::use_index(int key_to_save) +{ + uint i= 1; + DBUG_ASSERT(!created && key_to_save < (int)s->keys); + if (key_to_save >= 0) + /* Save the given key. */ + memcpy(key_info, key_info + key_to_save, sizeof(KEY)); + else + /* Drop all keys; */ + i= 0; + + s->keys= (key_to_save < 0) ? 0 : 1; +} + + /** @brief Check if this is part of a MERGE table with attached children. @@ -5412,6 +5578,431 @@ int update_virtual_fields(TABLE *table, DBUG_RETURN(0); } +/* + @brief Reset const_table flag + + @detail + Reset const_table flag for this table. If this table is a merged derived + table/view the flag is recursively reseted for all tables of the underlying + select. +*/ + +void TABLE_LIST::reset_const_table() +{ + table->const_table= 0; + if (is_merged_derived()) + { + SELECT_LEX *select_lex= get_unit()->first_select(); + TABLE_LIST *tl; + List_iterator<TABLE_LIST> ti(select_lex->leaf_tables); + while ((tl= ti++)) + tl->reset_const_table(); + } +} + + +/* + @brief Run derived tables/view handling phases on underlying select_lex. + + @param lex LEX for this thread + @param phases derived tables/views handling phases to run + (set of DT_XXX constants) + @details + This function runs this derived table through specified 'phases'. + Underlying tables of this select are handled prior to this derived. + 'lex' is passed as an argument to called functions. + + @return TRUE on error + @return FALSE ok +*/ + +bool TABLE_LIST::handle_derived(struct st_lex *lex, uint phases) +{ + SELECT_LEX_UNIT *unit= get_unit(); + if (unit) + { + for (SELECT_LEX *sl= unit->first_select(); sl; sl= sl->next_select()) + if (sl->handle_derived(lex, phases)) + return TRUE; + return mysql_handle_single_derived(lex, this, phases); + } + return FALSE; +} + + +/** + @brief + Return unit of this derived table/view + + @return reference to a unit if it's a derived table/view. + @return 0 when it's not a derived table/view. +*/ + +inline st_select_lex_unit *TABLE_LIST::get_unit() +{ + return (view ? &view->unit : derived); +} + + +/** + @brief + Return select_lex of this derived table/view + + @return select_lex of this derived table/view. + @return 0 when it's not a derived table. +*/ + +inline st_select_lex *TABLE_LIST::get_single_select() +{ + SELECT_LEX_UNIT *unit= get_unit(); + return (unit ? unit->first_select() : 0); +} + + +/** + @brief + Attach a join table list as a nested join to this TABLE_LIST. + + @param join_list join table list to attach + + @details + This function wraps 'join_list' into a nested_join of this table, thus + turning it to a nested join leaf. +*/ + +void TABLE_LIST::wrap_into_nested_join(List<TABLE_LIST> &join_list) +{ + TABLE_LIST *tl; + /* + Walk through derived table top list and set 'embedding' to point to + the nesting table. + */ + nested_join->join_list.empty(); + List_iterator_fast<TABLE_LIST> li(join_list); + nested_join->join_list= join_list; + while ((tl= li++)) + { + tl->embedding= this; + tl->join_list= &nested_join->join_list; + } +} + + +/** + @brief + Initialize this derived table/view + + @param thd Thread handle + + @details + This function makes initial preparations of this derived table/view for + further processing: + if it's a derived table this function marks it either as mergeable or + materializable + creates temporary table for name resolution purposes + creates field translation for mergeable derived table/view + + @return TRUE an error occur + @return FALSE ok +*/ + +bool TABLE_LIST::init_derived(THD *thd, bool init_view) +{ + SELECT_LEX *first_select= get_single_select(); + SELECT_LEX_UNIT *unit= get_unit(); + + if (!unit) + return FALSE; + /* + Check whether we can merge this derived table into main select. + Depending on the result field translation will or will not + be created. + */ + TABLE_LIST *first_table= (TABLE_LIST *) first_select->table_list.first; + if (first_select->table_list.elements > 1 || + first_table && first_table->is_multitable()) + set_multitable(); + + unit->derived= this; + if (init_view && !view) + { + /* This is all what we can do for a derived table for now. */ + set_derived(); + } + + if (!is_view()) + { + /* A subquery might be forced to be materialized due to a side-effect. */ + if (!is_materialized_derived() && first_select->is_mergeable()) + set_merged_derived(); + else + set_materialized_derived(); + } + /* + Derived tables/view are materialized prior to UPDATE, thus we can skip + them from table uniqueness check + */ + if (is_materialized_derived()) + { + SELECT_LEX *sl; + for (sl= first_select ;sl ; sl= sl->next_select()) + sl->exclude_from_table_unique_test= TRUE; + } + /* + Create field translation for mergeable derived tables/views. + For derived tables field translation can be created only after + unit is prepared so all '*' are get unrolled. + */ + if (is_merged_derived()) + { + if (is_view() || unit->prepared) + create_field_translation(thd); + } + + return FALSE; +} + + +/** + @brief + Retrieve number of rows in the table + + @details + Retrieve number of rows in the table referred by this TABLE_LIST and + store it in the table's stats.records variable. If this TABLE_LIST refers + to a materialized derived table/view then the estimated number of rows of + the derived table/view is used instead. + + @return 0 ok + @return non zero error +*/ + +int TABLE_LIST::fetch_number_of_rows() +{ + int error= 0; + if (is_materialized_derived() && !fill_me) + + { + table->file->stats.records= ((select_union*)derived->result)->records; + set_if_bigger(table->file->stats.records, 2); + } + else + error= table->file->info(HA_STATUS_VARIABLE | HA_STATUS_NO_LOCK); + return error; +} + +/* + Procedure of keys generation for result tables of materialized derived + tables/views. + + A key is generated for each equi-join pair derived table-another table. + Each generated key consists of fields of derived table used in equi-join. + Example: + + SELECT * FROM (SELECT * FROM t1 GROUP BY 1) tt JOIN + t1 ON tt.f1=t1.f3 and tt.f2.=t1.f4; + In this case for the derived table tt one key will be generated. It will + consist of two parts f1 and f2. + Example: + + SELECT * FROM (SELECT * FROM t1 GROUP BY 1) tt JOIN + t1 ON tt.f1=t1.f3 JOIN + t2 ON tt.f2=t2.f4; + In this case for the derived table tt two keys will be generated. + One key over f1 field, and another key over f2 field. + Currently optimizer may choose to use only one such key, thus the second + one will be dropped after range optimizer is finished. + See also JOIN::drop_unused_derived_keys function. + Example: + + SELECT * FROM (SELECT * FROM t1 GROUP BY 1) tt JOIN + t1 ON tt.f1=a_function(t1.f3); + In this case for the derived table tt one key will be generated. It will + consist of one field - f1. + + Implementation is split in two steps: + gathering information on all used fields of derived tables/view and + store it in lists of possible keys, one per a derived table/view. + add keys to result tables of derived tables/view using info from above + lists. + + The above procedure is implemented in 4 functions: + TABLE_LIST::update_derived_keys + Create/extend list of possible keys for one derived + table/view based on given field/used tables info. + (Step one) + generate_derived_keys This function is called at the moment when all + possible info on keys is gathered and it's safe to + add keys. Walk over list of derived tables/views and + calls to TABLE_LIST::generate_keys to actually + generate keys. (Step two) + TABLE_LIST::generate_keys + Walks over list of possible keys for this derived + table/view to add keys to the result table. + Calls to TABLE::add_tmp_index to actually add + keys. (Step two) + TABLE::add_tmp_index Creates one index description according to given + bitmap of used fields. (Step two) + There is also the fifth function called TABLE::use_index. It saves used + key and frees others. It is called when the optimizer has chosen which key + it will use, thus we don't need other keys anymore. +*/ + + +/* + @brief + Update derived table's list of possible keys + + @param field derived table's field to take part in a key + @param values array of values + @param num_values number of elements in the array values + + @details + This function creates/extends a list of possible keys for this derived + table/view. For each table used by a value from the 'values' array the + corresponding possible key is extended to include the 'field'. + If there is no such possible key then it is created. field's + key_start/part_of_key bitmaps are updated accordingly. + + @return TRUE new possible key can't be allocated. + @return FALSE list of possible keys successfully updated. +*/ + +bool TABLE_LIST::update_derived_keys(Field *field, Item **values, + uint num_values) +{ + DERIVED_KEY_MAP *entry= 0; + List_iterator<DERIVED_KEY_MAP> ki(derived_keymap_list); + uint i; + + /* Allow all keys to be used. */ + if (!derived_keymap_list.elements) + { + table->keys_in_use_for_query.set_all(); + table->s->uniques= 0; + derived_keymap_list.empty(); + } + + for (i= 0; i < num_values; i++) + { + uint tbl; + table_map tables= values[i]->used_tables() & ~OUTER_REF_TABLE_BIT; + for (tbl= 1; tables >= tbl; tbl<<= 1) + { + uint key= 0; + if (! (tables & tbl)) + continue; + ki.rewind(); + while ((entry= ki++)) + { + key++; + if (entry->referenced_by & tbl) + break; + } + if (!entry) + { + key++; + entry= (DERIVED_KEY_MAP*)my_malloc(sizeof(DERIVED_KEY_MAP), MYF(0)); + if (!entry) + return TRUE; + entry->referenced_by|= tbl; + entry->used_fields.clear_all(); + derived_keymap_list.push_back(entry); + field->key_start.set_bit(key); + table->max_keys++; + } + field->part_of_key.set_bit(key - 1); + field->flags|= PART_KEY_FLAG; + entry->used_fields.set_bit(field->field_index); + } + } + return FALSE; +} + + +/** + @brief + Generate keys for a materialized derived table/view + + @details + This function adds keys to the result table by walking over the list of + possible keys for this derived table/view and calling to the + TABLE::add_tmp_index to actually add keys. A name "key" with a sequential + number is given to each key to ease debugging. + + @return TRUE an error occur. + @return FALSE all keys were successfully added. +*/ + +bool TABLE_LIST::generate_keys() +{ + List_iterator<DERIVED_KEY_MAP> it(derived_keymap_list); + DERIVED_KEY_MAP *entry; + uint key= 0; + char buf[NAME_CHAR_LEN]; + DBUG_ASSERT(is_materialized_derived()); + + if (!derived_keymap_list.elements) + return FALSE; + + table->alloc_keys(table->max_keys); + while ((entry= it++)) + { + table->s->key_parts+= entry->used_fields.bits_set(); + sprintf(buf, "key%i", key++); + if (table->add_tmp_key(entry->used_fields.to_ulonglong(), + table->in_use->strdup(buf)) < 0) + return TRUE; + } + return FALSE; +} + + +/* + @brief + Change references to underlying items of a merged derived table/view + for fields in derived table's result table. + + @return FALSE ok + @return TRUE Out of memory +*/ +bool TABLE_LIST::change_refs_to_fields() +{ + List_iterator<Item> li(used_items); + Item_direct_ref *ref; + Field_iterator_view field_it; + THD *thd= table->in_use; + DBUG_ASSERT(is_merged_derived()); + + if (!used_items.elements) + return FALSE; + + materialized_items= (Item**)thd->calloc(sizeof(void*) * table->s->fields); + + while ((ref= (Item_direct_ref*)li++)) + { + uint idx; + Item *orig_item= *ref->ref; + field_it.set(this); + for (idx= 0; !field_it.end_of_fields(); field_it.next(), idx++) + { + if (field_it.item() == orig_item) + break; + } + DBUG_ASSERT(!field_it.end_of_fields()); + if (!materialized_items[idx]) + { + materialized_items[idx]= new Item_field(table->field[idx]); + if (!materialized_items[idx]) + return TRUE; + } + ref->ref= materialized_items + idx; + } + + return FALSE; +} + + /***************************************************************************** ** Instansiate templates *****************************************************************************/ === modified file 'sql/table.h' --- a/sql/table.h 2010-03-20 12:01:47 +0000 +++ b/sql/table.h 2010-04-29 21:10:39 +0000 @@ -858,6 +858,7 @@ struct st_table { my_bool insert_or_update; /* Can be used by the handler */ my_bool alias_name_used; /* true if table_name is alias */ my_bool get_fields_in_item_tree; /* Signal to fix_field */ + my_bool created; /* For tmp tables. TRUE <=> tmp table was actually created.*/ /* If MERGE children attached to parent. See top comment in ha_myisammrg.cc */ my_bool children_attached; @@ -870,6 +871,7 @@ struct st_table { bool no_partitions_used; /* If true, all partitions have been pruned away */ #endif + uint max_keys; /* Size of allocated key_info array. */ bool fill_item_list(List<Item> *item_list) const; void reset_item_list(List<Item> *item_list) const; void clear_column_bitmaps(void); @@ -913,6 +915,14 @@ struct st_table { */ inline bool needs_reopen_or_name_lock() { return s->version != refresh_version; } + bool alloc_keys(uint key_count); + int add_tmp_key(ulonglong key_parts, char *key_name); + void use_index(int key_to_save); + void set_table_map(table_map map_arg, uint tablenr_arg) + { + map= map_arg; + tablenr= tablenr_arg; + } bool is_children_attached(void); }; @@ -1045,13 +1055,52 @@ typedef struct st_schema_table } ST_SCHEMA_TABLE; +/* + Types of derived tables. The ending part is a bitmap of phases that are + applicable to a derived table of the type. + * / +#define VIEW_ALGORITHM_UNDEFINED 0 +#define VIEW_ALGORITHM_MERGE 1 + DT_COMMON + DT_MERGE +#define DERIVED_ALGORITHM_MERGE 2 + DT_COMMON + DT_MERGE +#define VIEW_ALGORITHM_TMPTABLE 3 + DT_COMMON + DT_MATERIALIZE +#define DERIVED_ALGORITHM_MATERIALIZE 4 + DT_COMMON + DT_MATERIALIZE +*/ +#define DTYPE_ALGORITHM_UNDEFINED 0 +#define DTYPE_VIEW 1 +#define DTYPE_TABLE 2 +#define DTYPE_MERGE 4 +#define DTYPE_MATERIALIZE 8 +#define DTYPE_MULTITABLE 16 +#define DTYPE_MASK 19 + +/* + Phases of derived tables/views handling, see sql_derived.cc + Values are used as parts of a bitmap attached to derived table types. +*/ +#define DT_INIT 1 +#define DT_PREPARE 2 +#define DT_OPTIMIZE 4 +#define DT_MERGE 8 +#define DT_MERGE_FOR_INSERT 16 +#define DT_CREATE 32 +#define DT_FILL 64 +#define DT_REINIT 128 +#define DT_PHASES 8 +/* Phases that are applicable to all derived tables. */ +#define DT_COMMON (DT_INIT + DT_PREPARE + DT_REINIT + DT_OPTIMIZE) +/* Phases that are applicable only to materialized derived tables. */ +#define DT_MATERIALIZE (DT_CREATE + DT_FILL) + +#define DT_PHASES_MERGE (DT_COMMON | DT_MERGE | DT_MERGE_FOR_INSERT) +#define DT_PHASES_MATERIALIZE (DT_COMMON | DT_MATERIALIZE) + +#define VIEW_ALGORITHM_UNDEFINED 0 +#define VIEW_ALGORITHM_MERGE (DTYPE_VIEW | DTYPE_MERGE) +#define VIEW_ALGORITHM_TMPTABLE (DTYPE_VIEW + DTYPE_MATERIALIZE ) + #define JOIN_TYPE_LEFT 1 #define JOIN_TYPE_RIGHT 2 -#define VIEW_ALGORITHM_UNDEFINED 0 -#define VIEW_ALGORITHM_TMPTABLE 1 -#define VIEW_ALGORITHM_MERGE 2 - #define VIEW_SUID_INVOKER 0 #define VIEW_SUID_DEFINER 1 #define VIEW_SUID_DEFAULT 2 @@ -1141,6 +1190,7 @@ class Item_in_subselect; also (TABLE_LIST::field_translation != NULL) - tmptable (TABLE_LIST::effective_algorithm == VIEW_ALGORITHM_TMPTABLE) also (TABLE_LIST::field_translation == NULL) + 2.5) TODO: Add derived tables description here 3) nested table reference (TABLE_LIST::nested_join != NULL) - table sequence - e.g. (t1, t2, t3) TODO: how to distinguish from a JOIN? @@ -1152,7 +1202,22 @@ class Item_in_subselect; (TABLE_LIST::join_using_fields != NULL) */ +/* + This structure is used to keep info about possible key for the result table + of a derived table/view. + The 'referenced_by' is the table map of the table to which this possible + key corresponds. + The 'used_field' is a map of fields of which this key consists of. + See also the comment for the TABLE_LIST::update_derived_keys function. +*/ +struct st_derived_table_key_map { + table_map referenced_by; + key_map used_fields; +}; +typedef st_derived_table_key_map DERIVED_KEY_MAP; + class Index_hint; +struct st_lex; struct TABLE_LIST { TABLE_LIST() {} /* Remove gcc warning */ @@ -1246,6 +1311,8 @@ struct TABLE_LIST filling procedure */ select_union *derived_result; + /* Stub used for materialized derived tables. */ + table_map map; /* ID bit of table (1,2,4,8,16...) */ /* Reference from aux_tables to local list entry of main select of multi-delete statement: @@ -1290,6 +1357,7 @@ struct TABLE_LIST Field_translator *field_translation; /* array of VIEW fields */ /* pointer to element after last one in translation table above */ Field_translator *field_translation_end; + bool field_translation_updated; /* List (based on next_local) of underlying tables of this view. I.e. it does not include the tables of subqueries used in the view. Is set only @@ -1304,11 +1372,18 @@ struct TABLE_LIST List<TABLE_LIST> *view_tables; /* most upper view this table belongs to */ TABLE_LIST *belong_to_view; + /* A derived table this table belongs to */ + TABLE_LIST *belong_to_derived; /* The view directly referencing this table (non-zero only for merged underlying tables of a view). */ TABLE_LIST *referencing_view; + + table_map view_used_tables; + table_map map_exec; + uint tablenr_exec; + /* Ptr to parent MERGE table list item. See top comment in ha_myisammrg.cc */ TABLE_LIST *parent_l; /* @@ -1321,13 +1396,7 @@ struct TABLE_LIST SQL SECURITY DEFINER) */ Security_context *view_sctx; - /* - List of all base tables local to a subquery including all view - tables. Unlike 'next_local', this in this list views are *not* - leaves. Created in setup_tables() -> make_leaves_list(). - */ bool allowed_show; - TABLE_LIST *next_leaf; Item *where; /* VIEW WHERE clause condition */ Item *check_option; /* WITH CHECK OPTION condition */ LEX_STRING select_stmt; /* text of (CREATE/SELECT) statement */ @@ -1363,7 +1432,7 @@ struct TABLE_LIST - VIEW_ALGORITHM_MERGE @to do Replace with an enum */ - uint8 effective_algorithm; + uint8 derived_type; GRANT_INFO grant; /* data need by some engines in query cache*/ ulonglong engine_data; @@ -1390,7 +1459,6 @@ struct TABLE_LIST bool skip_temporary; /* this table shouldn't be temporary */ /* TRUE if this merged view contain auto_increment field */ bool contain_auto_increment; - bool multitable_view; /* TRUE iff this is multitable view */ bool compact_view_format; /* Use compact format for SHOW CREATE VIEW */ /* view where processed */ bool where_processed; @@ -1414,6 +1482,17 @@ struct TABLE_LIST bool internal_tmp_table; bool deleting; /* going to delete this table */ + /* TRUE <=> derived table should be filled right after optimization. */ + bool fill_me; + /* TRUE <=> view/DT is merged. */ + bool merged; + bool merged_for_insert; + /* TRUE <=> don't prepare this derived table/view as it should be merged.*/ + bool skip_prepare_derived; + + List<Item> used_items; + Item **materialized_items; + /* View creation context. */ View_creation_ctx *view_creation_ctx; @@ -1451,9 +1530,12 @@ struct TABLE_LIST bool has_table_lookup_value; uint table_open_method; enum enum_schema_table_state schema_table_state; + + List<DERIVED_KEY_MAP> derived_keymap_list; + void calc_md5(char *buffer); - void set_underlying_merge(); int view_check_option(THD *thd, bool ignore_failure); + bool create_field_translation(THD *thd); bool setup_underlying(THD *thd); void cleanup_items(); bool placeholder() @@ -1483,7 +1565,7 @@ struct TABLE_LIST inline bool prepare_where(THD *thd, Item **conds, bool no_where_clause) { - if (effective_algorithm == VIEW_ALGORITHM_MERGE) + if (!view || is_merged_derived()) return prep_where(thd, conds, no_where_clause); return FALSE; } @@ -1549,6 +1631,60 @@ struct TABLE_LIST m_table_ref_version= s->get_table_ref_version(); } + /* Set of functions returning/setting state of a derived table/view. */ + inline bool is_non_derived() + { + return (!derived_type); + } + inline bool is_view_or_derived() + { + return (derived_type); + } + inline bool is_view() + { + return (derived_type & DTYPE_VIEW); + } + inline bool is_derived() + { + return (derived_type & DTYPE_TABLE); + } + inline void set_view() + { + derived_type= DTYPE_VIEW; + } + inline void set_derived() + { + derived_type= DTYPE_TABLE; + } + inline bool is_merged_derived() + { + return (derived_type & DTYPE_MERGE); + } + inline void set_merged_derived() + { + derived_type= ((derived_type & DTYPE_MASK) | + DTYPE_TABLE | DTYPE_MERGE); + } + inline bool is_materialized_derived() + { + return (derived_type & DTYPE_MATERIALIZE); + } + inline void set_materialized_derived() + { + derived_type= ((derived_type & DTYPE_MASK) | + DTYPE_TABLE | DTYPE_MATERIALIZE); + } + inline bool is_multitable() + { + return (derived_type & DTYPE_MULTITABLE); + } + inline void set_multitable() + { + derived_type|= DTYPE_MULTITABLE; + } + void reset_const_table(); + bool handle_derived(struct st_lex *lex, uint phases); + /** @brief True if this TABLE_LIST represents an anonymous derived table, i.e. the result of a subquery. @@ -1568,6 +1704,14 @@ struct TABLE_LIST respectively. */ char *get_table_name() { return view != NULL ? view_name.str : table_name; } + st_select_lex_unit *get_unit(); + st_select_lex *get_single_select(); + void wrap_into_nested_join(List<TABLE_LIST> &join_list); + bool init_derived(THD *thd, bool init_view); + int fetch_number_of_rows(); + bool update_derived_keys(Field *field, Item **values, uint num_values); + bool generate_keys(); + bool change_refs_to_fields(); private: bool prep_check_option(THD *thd, uint8 check_opt_type);

1 0

[Maria-developers] ORDER BY bug for TEXT in federatedx
by Kristian Nielsen 29 Apr '10

29 Apr '10

Hi Patrick Please check this mysql bug: http://bugs.mysql.com/bug.php?id=32426 This bug is also in FederatedX (I noticed when I merged the test case from MySQL 5.1.46 :-) I created a MariaDB bug: https://bugs.launchpad.net/maria/+bug/571200 Can you look into this? For now I commented out the test in federated.test, but we need it fixed properly of course. Antony: I Cc:'ed you, in case you can help, as it seems you have hacked on federatedx also? - Kristian.

3 3

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2833)
by knielsen＠knielsen-hq.org 29 Apr '10

29 Apr '10

#At lp:maria 2833 knielsen(a)knielsen-hq.org 2010-04-29 Fix buffer overflow in COM_FIELD_LIST. Fix missing bounds check in string conversion. Bump version number for security fix release. modified: configure.in sql/sql_base.cc sql/sql_parse.cc strings/ctype-utf8.c === modified file 'configure.in' --- a/configure.in 2010-03-04 08:03:07 +0000 +++ b/configure.in 2010-04-29 07:57:25 +0000 @@ -7,7 +7,7 @@ AC_PREREQ(2.59) # Remember to also update version.c in ndb. # When changing major version number please also check switch statement # in mysqlbinlog::check_master_version(). -AC_INIT([MariaDB Server], [5.1.44-MariaDB], [], [mysql]) +AC_INIT([MariaDB Server], [5.1.44a-MariaDB], [], [mysql]) AC_CONFIG_SRCDIR([sql/mysqld.cc]) AC_CANONICAL_SYSTEM # USTAR format gives us the possibility to store longer path names in === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_base.cc 2010-04-29 07:57:25 +0000 @@ -233,8 +233,12 @@ static void check_unused(void) uint create_table_def_key(THD *thd, char *key, TABLE_LIST *table_list, bool tmp_table) { - uint key_length= (uint) (strmov(strmov(key, table_list->db)+1, - table_list->table_name)-key)+1; + char *db_end= strnmov(key, table_list->db, MAX_DBKEY_LENGTH - 2); + *db_end++= '\0'; + char *table_end= strnmov(db_end, table_list->table_name, + key + MAX_DBKEY_LENGTH - 1 - db_end); + *table_end++= '\0'; + uint key_length= (uint) (table_end-key); if (tmp_table) { int4store(key + key_length, thd->server_id); === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-03-04 08:03:07 +0000 +++ b/sql/sql_parse.cc 2010-04-29 07:57:25 +0000 @@ -1304,10 +1304,12 @@ bool dispatch_command(enum enum_server_c break; #else { - char *fields, *packet_end= packet + packet_length, *arg_end; + char *fields, *packet_end= packet + packet_length, *wildcard; /* Locked closure of all tables */ TABLE_LIST table_list; - LEX_STRING conv_name; + char db_buff[NAME_LEN+1]; + uint32 db_length; + uint dummy_errors; /* used as fields initializator */ lex_start(thd); @@ -1319,11 +1321,22 @@ bool dispatch_command(enum enum_server_c /* We have name + wildcard in packet, separated by endzero */ - arg_end= strend(packet); - thd->convert_string(&conv_name, system_charset_info, - packet, (uint) (arg_end - packet), thd->charset()); - table_list.alias= table_list.table_name= conv_name.str; - packet= arg_end + 1; + wildcard= strend(packet); + db_length= wildcard - packet; + wildcard++; + uint query_length= (uint) (packet_end - wildcard); // Don't count end \0 + if (db_length > NAME_LEN || query_length > NAME_LEN) + { + my_message(ER_UNKNOWN_COM_ERROR, ER(ER_UNKNOWN_COM_ERROR), MYF(0)); + break; + } + db_length= copy_and_convert(db_buff, sizeof(db_buff)-1, + system_charset_info, packet, db_length, + thd->charset(), &dummy_errors); + db_buff[db_length]= '\0'; + table_list.alias= table_list.table_name= db_buff; + if (!(fields= (char *) thd->memdup(wildcard, query_length + 1))) + break; if (is_schema_db(table_list.db, table_list.db_length)) { @@ -1332,9 +1345,6 @@ bool dispatch_command(enum enum_server_c table_list.schema_table= schema_table; } - uint query_length= (uint) (packet_end - packet); // Don't count end \0 - if (!(fields= (char *) thd->memdup(packet, query_length + 1))) - break; thd->set_query(fields, query_length); general_log_print(thd, command, "%s %s", table_list.table_name, fields); if (lower_case_table_names) === modified file 'strings/ctype-utf8.c' --- a/strings/ctype-utf8.c 2009-10-15 21:38:29 +0000 +++ b/strings/ctype-utf8.c 2010-04-29 07:57:25 +0000 @@ -4116,6 +4116,10 @@ my_wc_mb_filename(CHARSET_INFO *cs __att { int code; char hex[]= "0123456789abcdef"; + + if (s >= e) + return MY_CS_TOOSMALL; + if (wc < 128 && filename_safe_char[wc]) { *s= (uchar) wc;

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2851)
by knielsen＠knielsen-hq.org 29 Apr '10

29 Apr '10

#At lp:maria 2851 knielsen(a)knielsen-hq.org 2010-04-29 (Hopefully) better fix for Windows warning on redefined TAILQ_EMPTY; the previous attempt broke build on Debian4. modified: extra/libevent/event-internal.h === modified file 'extra/libevent/event-internal.h' --- a/extra/libevent/event-internal.h 2010-04-09 10:39:27 +0000 +++ b/extra/libevent/event-internal.h 2010-04-29 07:41:35 +0000 @@ -69,14 +69,17 @@ struct event_base { }; /* Internal use only: Functions that might be missing from <sys/queue.h> */ -#ifndef HAVE_TAILQFOREACH /* These following macros are copied from BSD sys/queue.h Copyright (c) 1991, 1993, The Regents of the University of California. All rights reserved. */ +#ifndef TAILQ_EMPTY +#define TAILQ_EMPTY(head) ((head)->tqh_first == NULL) #define TAILQ_FIRST(head) ((head)->tqh_first) #define TAILQ_END(head) NULL #define TAILQ_NEXT(elm, field) ((elm)->field.tqe_next) +#endif /* TAILQ_EMPTY */ +#ifndef HAVE_TAILQFOREACH #define TAILQ_FOREACH(var, head, field) \ for((var) = TAILQ_FIRST(head); \ (var) != TAILQ_END(head); \

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2850: Fix missing bounds check in string conversion.
by noreply＠launchpad.net 29 Apr '10

29 Apr '10

------------------------------------------------------------ revno: 2850 committer: knielsen(a)knielsen-hq.org branch nick: tmp timestamp: Thu 2010-04-29 09:29:04 +0200 message: Fix missing bounds check in string conversion. Bump version number for security fix release. modified: configure.in strings/ctype-utf8.c -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2849: Fix buffer overflow in COM_FIELD_LIST.
by noreply＠launchpad.net 29 Apr '10

29 Apr '10

------------------------------------------------------------ revno: 2849 committer: Kristian Nielsen <knielsen@odin> branch nick: work-5.1-security-fix timestamp: Wed 2010-04-28 07:48:03 +0200 message: Fix buffer overflow in COM_FIELD_LIST. modified: sql/sql_base.cc sql/sql_parse.cc -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2850)
by knielsen＠knielsen-hq.org 29 Apr '10

29 Apr '10

#At lp:maria 2850 knielsen(a)knielsen-hq.org 2010-04-29 Fix missing bounds check in string conversion. Bump version number for security fix release. modified: configure.in strings/ctype-utf8.c === modified file 'configure.in' --- a/configure.in 2010-03-04 08:03:07 +0000 +++ b/configure.in 2010-04-29 07:29:04 +0000 @@ -7,7 +7,7 @@ AC_PREREQ(2.59) # Remember to also update version.c in ndb. # When changing major version number please also check switch statement # in mysqlbinlog::check_master_version(). -AC_INIT([MariaDB Server], [5.1.44-MariaDB], [], [mysql]) +AC_INIT([MariaDB Server], [5.1.44a-MariaDB], [], [mysql]) AC_CONFIG_SRCDIR([sql/mysqld.cc]) AC_CANONICAL_SYSTEM # USTAR format gives us the possibility to store longer path names in === modified file 'strings/ctype-utf8.c' --- a/strings/ctype-utf8.c 2010-03-30 12:36:49 +0000 +++ b/strings/ctype-utf8.c 2010-04-29 07:29:04 +0000 @@ -4116,6 +4116,10 @@ my_wc_mb_filename(CHARSET_INFO *cs __att { int code; char hex[]= "0123456789abcdef"; + + if (s >= e) + return MY_CS_TOOSMALL; + if (wc < 128 && filename_safe_char[wc]) { *s= (uchar) wc;

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2850)
by knielsen＠knielsen-hq.org 29 Apr '10

29 Apr '10

#At lp:maria 2850 knielsen(a)knielsen-hq.org 2010-04-29 Fix missing bounds check in string conversion. modified: strings/ctype-utf8.c === modified file 'strings/ctype-utf8.c' --- a/strings/ctype-utf8.c 2010-03-30 12:36:49 +0000 +++ b/strings/ctype-utf8.c 2010-04-29 07:09:50 +0000 @@ -4116,6 +4116,10 @@ my_wc_mb_filename(CHARSET_INFO *cs __att { int code; char hex[]= "0123456789abcdef"; + + if (s >= e) + return MY_CS_TOOSMALL; + if (wc < 128 && filename_safe_char[wc]) { *s= (uchar) wc;

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2855)
by knielsen＠knielsen-hq.org 28 Apr '10

28 Apr '10

#At lp:maria 2855 knielsen(a)knielsen-hq.org 2010-04-28 After-merge fixes for MySQL 5.1.46 merge into MariaDB: result file update and compiler warning removals. modified: mysql-test/mysql-test-run.pl mysql-test/suite/funcs_1/r/is_columns_is.result mysql-test/suite/funcs_1/r/is_tables_is.result mysql-test/suite/pbxt/r/default.result mysql-test/suite/pbxt/r/mysqlshow.result mysql-test/suite/pbxt/r/subselect.result mysql-test/suite/pbxt/r/type_timestamp.result sql/sql_acl.cc storage/myisammrg/ha_myisammrg.cc storage/xtradb/btr/btr0cur.c storage/xtradb/buf/buf0buf.c storage/xtradb/fsp/fsp0fsp.c storage/xtradb/handler/ha_innodb.cc storage/xtradb/include/fsp0types.h storage/xtradb/mtr/mtr0log.c === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-04-28 12:52:24 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-04-28 19:29:45 +0000 @@ -1460,7 +1460,8 @@ sub command_line_setup { push(@valgrind_args, @default_valgrind_args) unless @valgrind_args; - # Don't add --quiet; you will loose the summary reports. + # Make valgrind run in quiet mode so it only print errors + push(@valgrind_args, "--quiet" ); mtr_report("Running valgrind with options \"", join(" ", @valgrind_args), "\""); @@ -5591,66 +5592,6 @@ sub valgrind_arguments { } } -# -# Search server logs for valgrind reports printed at mysqld termination -# - -sub valgrind_exit_reports() { - foreach my $log_file (keys %mysqld_logs) - { - my @culprits= (); - my $valgrind_rep= ""; - my $found_report= 0; - my $err_in_report= 0; - - my $LOGF = IO::File->new($log_file) - or mtr_error("Could not open file '$log_file' for reading: $!"); - - while ( my $line = <$LOGF> ) - { - if ($line =~ /^CURRENT_TEST: (.+)$/) - { - my $testname= $1; - # If we have a report, report it if needed and start new list of tests - if ($found_report) - { - if ($err_in_report) - { - mtr_print ("Valgrind report from $log_file after tests:\n", - @culprits); - mtr_print_line(); - print ("$valgrind_rep\n"); - $err_in_report= 0; - } - # Make ready to collect new report - @culprits= (); - $found_report= 0; - $valgrind_rep= ""; - } - push (@culprits, $testname); - next; - } - # This line marks the start of a valgrind report - $found_report= 1 if $line =~ /ERROR SUMMARY:/; - - if ($found_report) { - $line=~ s/^==\d+== //; - $valgrind_rep .= $line; - $err_in_report= 1 if $line =~ /ERROR SUMMARY: [1-9]/; - $err_in_report= 1 if $line =~ /definitely lost: [1-9]/; - $err_in_report= 1 if $line =~ /possibly lost: [1-9]/; - } - } - - $LOGF= undef; - - if ($err_in_report) { - mtr_print ("Valgrind report from $log_file after tests:\n", @culprits); - mtr_print_line(); - print ("$valgrind_rep\n"); - } - } -} # # Usage === modified file 'mysql-test/suite/funcs_1/r/is_columns_is.result' --- a/mysql-test/suite/funcs_1/r/is_columns_is.result 2010-01-16 05:12:57 +0000 +++ b/mysql-test/suite/funcs_1/r/is_columns_is.result 2010-04-28 19:29:45 +0000 @@ -164,12 +164,13 @@ NULL information_schema INNODB_CMP_RESET NULL information_schema INNODB_CMP_RESET page_size 1 0 NO int NULL NULL 10 0 NULL NULL int(5) select NULL information_schema INNODB_CMP_RESET uncompress_ops 5 0 NO int NULL NULL 10 0 NULL NULL int(11) select NULL information_schema INNODB_CMP_RESET uncompress_time 6 0 NO int NULL NULL 10 0 NULL NULL int(11) select -NULL information_schema INNODB_INDEX_STATS fields 3 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_INDEX_STATS index_name 2 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select -NULL information_schema INNODB_INDEX_STATS index_size 5 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_INDEX_STATS leaf_pages 6 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_INDEX_STATS row_per_keys 4 NO varchar 256 768 NULL NULL utf8 utf8_general_ci varchar(256) select -NULL information_schema INNODB_INDEX_STATS table_name 1 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_INDEX_STATS fields 4 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_INDEX_STATS index_name 3 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_INDEX_STATS index_size 6 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_INDEX_STATS leaf_pages 7 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_INDEX_STATS row_per_keys 5 NO varchar 256 768 NULL NULL utf8 utf8_general_ci varchar(256) select +NULL information_schema INNODB_INDEX_STATS table_name 2 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_INDEX_STATS table_schema 1 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select NULL information_schema INNODB_LOCKS lock_data 10 NULL YES varchar 8192 24576 NULL NULL utf8 utf8_general_ci varchar(8192) select NULL information_schema INNODB_LOCKS lock_id 1 NO varchar 81 243 NULL NULL utf8 utf8_general_ci varchar(81) select NULL information_schema INNODB_LOCKS lock_index 6 NULL YES varchar 1024 3072 NULL NULL utf8 utf8_general_ci varchar(1024) select @@ -190,11 +191,27 @@ NULL information_schema INNODB_RSEG page NULL information_schema INNODB_RSEG rseg_id 1 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_RSEG space_id 2 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_RSEG zip_size 3 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_TABLE_STATS clust_size 3 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_TABLE_STATS modified 5 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_TABLE_STATS other_size 4 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_TABLE_STATS rows 2 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_TABLE_STATS table_name 1 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_SYS_INDEXES ID 2 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_INDEXES NAME 3 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_SYS_INDEXES N_FIELDS 4 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_INDEXES PAGE_NO 7 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_INDEXES SPACE 6 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_INDEXES TABLE_ID 1 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_INDEXES TYPE 5 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_TABLES CLUSTER_NAME 7 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_SYS_TABLES ID 2 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_TABLES MIX_ID 5 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_TABLES MIX_LEN 6 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_TABLES NAME 1 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_SYS_TABLES N_COLS 3 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_TABLES SPACE 8 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_SYS_TABLES TYPE 4 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_TABLE_STATS clust_size 4 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_TABLE_STATS modified 6 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_TABLE_STATS other_size 5 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_TABLE_STATS rows 3 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_TABLE_STATS table_name 2 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select +NULL information_schema INNODB_TABLE_STATS table_schema 1 NO varchar 192 576 NULL NULL utf8 utf8_general_ci varchar(192) select NULL information_schema INNODB_TRX trx_id 1 NO varchar 18 54 NULL NULL utf8 utf8_general_ci varchar(18) select NULL information_schema INNODB_TRX trx_mysql_thread_id 7 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_TRX trx_query 8 NULL YES varchar 1024 3072 NULL NULL utf8 utf8_general_ci varchar(1024) select @@ -619,6 +636,7 @@ NULL information_schema INNODB_CMP_RESET NULL information_schema INNODB_CMP_RESET compress_time int NULL NULL NULL NULL int(11) NULL information_schema INNODB_CMP_RESET uncompress_ops int NULL NULL NULL NULL int(11) NULL information_schema INNODB_CMP_RESET uncompress_time int NULL NULL NULL NULL int(11) +3.0000 information_schema INNODB_INDEX_STATS table_schema varchar 192 576 utf8 utf8_general_ci varchar(192) 3.0000 information_schema INNODB_INDEX_STATS table_name varchar 192 576 utf8 utf8_general_ci varchar(192) 3.0000 information_schema INNODB_INDEX_STATS index_name varchar 192 576 utf8 utf8_general_ci varchar(192) NULL information_schema INNODB_INDEX_STATS fields bigint NULL NULL NULL NULL bigint(21) unsigned @@ -645,6 +663,22 @@ NULL information_schema INNODB_RSEG zip_ NULL information_schema INNODB_RSEG page_no bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_RSEG max_size bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_RSEG curr_size bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_INDEXES TABLE_ID bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_INDEXES ID bigint NULL NULL NULL NULL bigint(21) unsigned +3.0000 information_schema INNODB_SYS_INDEXES NAME varchar 192 576 utf8 utf8_general_ci varchar(192) +NULL information_schema INNODB_SYS_INDEXES N_FIELDS bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_INDEXES TYPE bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_INDEXES SPACE bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_INDEXES PAGE_NO bigint NULL NULL NULL NULL bigint(21) unsigned +3.0000 information_schema INNODB_SYS_TABLES NAME varchar 192 576 utf8 utf8_general_ci varchar(192) +NULL information_schema INNODB_SYS_TABLES ID bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_TABLES N_COLS bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_TABLES TYPE bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_TABLES MIX_ID bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_SYS_TABLES MIX_LEN bigint NULL NULL NULL NULL bigint(21) unsigned +3.0000 information_schema INNODB_SYS_TABLES CLUSTER_NAME varchar 192 576 utf8 utf8_general_ci varchar(192) +NULL information_schema INNODB_SYS_TABLES SPACE bigint NULL NULL NULL NULL bigint(21) unsigned +3.0000 information_schema INNODB_TABLE_STATS table_schema varchar 192 576 utf8 utf8_general_ci varchar(192) 3.0000 information_schema INNODB_TABLE_STATS table_name varchar 192 576 utf8 utf8_general_ci varchar(192) NULL information_schema INNODB_TABLE_STATS rows bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_TABLE_STATS clust_size bigint NULL NULL NULL NULL bigint(21) unsigned === modified file 'mysql-test/suite/funcs_1/r/is_tables_is.result' --- a/mysql-test/suite/funcs_1/r/is_tables_is.result 2010-01-16 05:12:57 +0000 +++ b/mysql-test/suite/funcs_1/r/is_tables_is.result 2010-04-28 19:29:45 +0000 @@ -498,6 +498,52 @@ user_comment Separator ----------------------------------------------------- TABLE_CATALOG NULL TABLE_SCHEMA information_schema +TABLE_NAME INNODB_SYS_INDEXES +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema +TABLE_NAME INNODB_SYS_TABLES +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema TABLE_NAME INNODB_TABLE_STATS TABLE_TYPE SYSTEM VIEW ENGINE MEMORY @@ -1504,6 +1550,52 @@ user_comment Separator ----------------------------------------------------- TABLE_CATALOG NULL TABLE_SCHEMA information_schema +TABLE_NAME INNODB_SYS_INDEXES +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema +TABLE_NAME INNODB_SYS_TABLES +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema TABLE_NAME INNODB_TABLE_STATS TABLE_TYPE SYSTEM VIEW ENGINE MEMORY === modified file 'mysql-test/suite/pbxt/r/default.result' --- a/mysql-test/suite/pbxt/r/default.result 2009-04-02 10:03:14 +0000 +++ b/mysql-test/suite/pbxt/r/default.result 2010-04-28 19:29:45 +0000 @@ -180,7 +180,6 @@ insert into bug20691 values (2, 3, 5, '0 insert into bug20691 values (DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, 4); Warnings: Warning 1364 Field 'a' doesn't have a default value -Warning 1364 Field 'b' doesn't have a default value Warning 1364 Field 'c' doesn't have a default value Warning 1364 Field 'd' doesn't have a default value Warning 1364 Field 'e' doesn't have a default value @@ -193,6 +192,6 @@ a b c d e f g h i x two large 00:00:05 0007-01-01 11 13 17 0019-01-01 00:00:00 23 1 small 00:00:00 0000-00-00 0 0000-00-00 00:00:00 0 2 two large 00:00:05 0007-01-01 11 13 17 0019-01-01 00:00:00 23 3 - 00:00:00 0000-00-00 0 0000-00-00 00:00:00 0 4 + small 00:00:00 0000-00-00 0 0000-00-00 00:00:00 0 4 drop table bug20691; End of 5.0 tests. === modified file 'mysql-test/suite/pbxt/r/mysqlshow.result' --- a/mysql-test/suite/pbxt/r/mysqlshow.result 2010-01-16 05:12:57 +0000 +++ b/mysql-test/suite/pbxt/r/mysqlshow.result 2010-04-28 19:29:45 +0000 @@ -115,12 +115,14 @@ Database: information_schema | INNODB_BUFFER_POOL_PAGES_INDEX | | XTRADB_ADMIN_COMMAND | | INNODB_TRX | -| INNODB_CMP_RESET | +| INNODB_SYS_TABLES | | INNODB_LOCK_WAITS | | INNODB_CMPMEM_RESET | | INNODB_LOCKS | | INNODB_CMPMEM | | INNODB_TABLE_STATS | +| INNODB_SYS_INDEXES | +| INNODB_CMP_RESET | | INNODB_BUFFER_POOL_PAGES_BLOB | | INNODB_INDEX_STATS | +---------------------------------------+ @@ -164,12 +166,14 @@ Database: INFORMATION_SCHEMA | INNODB_BUFFER_POOL_PAGES_INDEX | | XTRADB_ADMIN_COMMAND | | INNODB_TRX | -| INNODB_CMP_RESET | +| INNODB_SYS_TABLES | | INNODB_LOCK_WAITS | | INNODB_CMPMEM_RESET | | INNODB_LOCKS | | INNODB_CMPMEM | | INNODB_TABLE_STATS | +| INNODB_SYS_INDEXES | +| INNODB_CMP_RESET | | INNODB_BUFFER_POOL_PAGES_BLOB | | INNODB_INDEX_STATS | +---------------------------------------+ === modified file 'mysql-test/suite/pbxt/r/subselect.result' --- a/mysql-test/suite/pbxt/r/subselect.result 2009-11-24 10:19:08 +0000 +++ b/mysql-test/suite/pbxt/r/subselect.result 2010-04-28 19:29:45 +0000 @@ -32,7 +32,7 @@ id select_type table type possible_keys NULL UNION RESULT <union3,4> ALL NULL NULL NULL NULL NULL NULL Warnings: Note 1249 Select 2 was reduced during optimization -Note 1003 select (select 0 AS `0` union select 0 AS `0`) AS `(SELECT (SELECT 0 UNION SELECT 0))` +Note 1003 select (select 0 union select 0) AS `(SELECT (SELECT 0 UNION SELECT 0))` SELECT (SELECT 1 FROM (SELECT 1) as b HAVING a=1) as a; ERROR 42S22: Reference 'a' not supported (forward reference in item list) SELECT (SELECT 1 FROM (SELECT 1) as b HAVING b=1) as a,(SELECT 1 FROM (SELECT 1) as c HAVING a=1) as b; @@ -50,7 +50,7 @@ id select_type table type possible_keys Warnings: Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 Note 1276 Field or reference 'b.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1' AS `a`) = 1) +Note 1003 select 1 AS `1` from (select 1 AS `a`) `b` having ((select '1') = 1) SELECT 1 FROM (SELECT 1 as a) as b HAVING (SELECT a)=1; 1 1 @@ -187,7 +187,7 @@ id select_type table type possible_keys 4 SUBQUERY t2 ALL NULL NULL NULL NULL 2 100.00 NULL UNION RESULT <union1,3> ALL NULL NULL NULL NULL NULL NULL Warnings: -Note 1003 (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`b` = (select `test`.`t3`.`a` AS `a` from `test`.`t3` order by 1 desc limit 1))) union (select `test`.`t4`.`a` AS `a`,`test`.`t4`.`b` AS `b` from `test`.`t4` where (`test`.`t4`.`b` = (select (max(`test`.`t2`.`a`) * 4) AS `max(t2.a)*4` from `test`.`t2`)) order by `a`) +Note 1003 (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`b` = (select `test`.`t3`.`a` from `test`.`t3` order by 1 desc limit 1))) union (select `test`.`t4`.`a` AS `a`,`test`.`t4`.`b` AS `b` from `test`.`t4` where (`test`.`t4`.`b` = (select (max(`test`.`t2`.`a`) * 4) from `test`.`t2`)) order by `a`) select (select a from t3 where a<t2.a*4 order by 1 desc limit 1), a from t2; (select a from t3 where a<t2.a*4 order by 1 desc limit 1) a 3 1 @@ -203,7 +203,7 @@ id select_type table type possible_keys 3 DERIVED t2 ALL NULL NULL NULL NULL 2 100.00 Using where 2 SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using filesort Warnings: -Note 1003 select (select `test`.`t3`.`a` AS `a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` +Note 1003 select (select `test`.`t3`.`a` from `test`.`t3` where (`test`.`t3`.`a` < 8) order by 1 desc limit 1) AS `(select t3.a from t3 where a<8 order by 1 desc limit 1)`,'2' AS `a` from (select `test`.`t2`.`a` AS `a`,`test`.`t2`.`b` AS `b` from `test`.`t2` where (`test`.`t2`.`a` > 1)) `tt` select * from t1 where t1.a=(select t2.a from t2 where t2.b=(select max(a) from t3) order by 1 desc limit 1); a 2 @@ -224,7 +224,7 @@ id select_type table type possible_keys 3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where Warnings: Note 1276 Field or reference 'test.t4.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select `test`.`t4`.`b` AS `b`,(select avg((`test`.`t2`.`a` + (select min(`test`.`t3`.`a`) AS `min(t3.a)` from `test`.`t3` where (`test`.`t3`.`a` >= `test`.`t4`.`a`)))) AS `avg(t2.a+(select min(t3.a) from t3 where t3.a >= t4.a))` from `test`.`t2`) AS `(select avg(t2.a+(select min(t3.a) from t3 where t3.a >= t4.a)) from t2)` from `test`.`t4` +Note 1003 select `test`.`t4`.`b` AS `b`,(select avg((`test`.`t2`.`a` + (select min(`test`.`t3`.`a`) from `test`.`t3` where (`test`.`t3`.`a` >= `test`.`t4`.`a`)))) from `test`.`t2`) AS `(select avg(t2.a+(select min(t3.a) from t3 where t3.a >= t4.a)) from t2)` from `test`.`t4` select * from t3 where exists (select * from t2 where t2.b=t3.a); a 7 @@ -314,7 +314,7 @@ NULL UNION RESULT <union2,3> ALL NULL NU Warnings: Note 1276 Field or reference 'test.t2.a' of SELECT #2 was resolved in SELECT #1 Note 1276 Field or reference 'test.t2.a' of SELECT #3 was resolved in SELECT #1 -Note 1003 select (select `test`.`t1`.`a` AS `a` from `test`.`t1` where (`test`.`t1`.`a` = `test`.`t2`.`a`) union select `test`.`t5`.`a` AS `a` from `test`.`t5` where (`test`.`t5`.`a` = `test`.`t2`.`a`)) AS `(select a from t1 where t1.a=t2.a union select a from t5 where t5.a=t2.a)`,`test`.`t2`.`a` AS `a` from `test`.`t2` +Note 1003 select (select `test`.`t1`.`a` from `test`.`t1` where (`test`.`t1`.`a` = `test`.`t2`.`a`) union select `test`.`t5`.`a` from `test`.`t5` where (`test`.`t5`.`a` = `test`.`t2`.`a`)) AS `(select a from t1 where t1.a=t2.a union select a from t5 where t5.a=t2.a)`,`test`.`t2`.`a` AS `a` from `test`.`t2` select (select a from t1 where t1.a=t2.a union all select a from t5 where t5.a=t2.a), a from t2; ERROR 21000: Subquery returns more than 1 row create table t6 (patient_uq int, clinic_uq int, index i1 (clinic_uq)); @@ -332,7 +332,7 @@ id select_type table type possible_keys 2 DEPENDENT SUBQUERY t7 eq_ref PRIMARY PRIMARY 4 test.t6.clinic_uq 1 100.00 Using index Warnings: Note 1276 Field or reference 'test.t6.clinic_uq' of SELECT #2 was resolved in SELECT #1 -Note 1003 select `test`.`t6`.`patient_uq` AS `patient_uq`,`test`.`t6`.`clinic_uq` AS `clinic_uq` from `test`.`t6` where exists(select 1 AS `Not_used` from `test`.`t7` where (`test`.`t7`.`uq` = `test`.`t6`.`clinic_uq`)) +Note 1003 select `test`.`t6`.`patient_uq` AS `patient_uq`,`test`.`t6`.`clinic_uq` AS `clinic_uq` from `test`.`t6` where exists(select 1 from `test`.`t7` where (`test`.`t7`.`uq` = `test`.`t6`.`clinic_uq`)) select * from t1 where a= (select a from t2,t4 where t2.b=t4.b); ERROR 23000: Column 'a' in field list is ambiguous drop table t1,t2,t3; @@ -367,7 +367,7 @@ id select_type table type possible_keys 2 SUBQUERY t8 const PRIMARY PRIMARY 37 const 1 100.00 3 SUBQUERY t8 const PRIMARY PRIMARY 37 1 100.00 Using index Warnings: -Note 1003 select 'joce' AS `pseudo`,(select 'test' AS `email` from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 +Note 1003 select 'joce' AS `pseudo`,(select 'test' from `test`.`t8` where 1) AS `(SELECT email FROM t8 WHERE pseudo=(SELECT pseudo FROM t8 WHERE pseudo='joce'))` from `test`.`t8` where 1 SELECT pseudo FROM t8 WHERE pseudo=(SELECT pseudo,email FROM t8 WHERE pseudo='joce'); ERROR 21000: Operand should contain 1 column(s) @@ -399,7 +399,7 @@ id select_type table type possible_keys 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 SUBQUERY t1 index NULL PRIMARY 43 NULL 2 100.00 Using where; Using index Warnings: -Note 1003 select (select distinct `test`.`t1`.`date` AS `date` from `test`.`t1` where (`test`.`t1`.`date` = '2002-08-03')) AS `(SELECT DISTINCT date FROM t1 WHERE date='2002-08-03')` +Note 1003 select (select distinct `test`.`t1`.`date` from `test`.`t1` where (`test`.`t1`.`date` = '2002-08-03')) AS `(SELECT DISTINCT date FROM t1 WHERE date='2002-08-03')` SELECT DISTINCT date FROM t1 WHERE date='2002-08-03'; date 2002-08-03 @@ -743,7 +743,7 @@ id select_type table type possible_keys 3 DEPENDENT UNION NULL NULL NULL NULL NULL NULL NULL NULL No tables used NULL UNION RESULT <union2,3> ALL NULL NULL NULL NULL NULL NULL Warnings: -Note 1003 select `test`.`t2`.`id` AS `id` from `test`.`t2` where <in_optimizer>(`test`.`t2`.`id`,<exists>(select 1 AS `1` having (<cache>(`test`.`t2`.`id`) = <ref_null_helper>(1)) union select 3 AS `3` having (<cache>(`test`.`t2`.`id`) = <ref_null_helper>(3)))) +Note 1003 select `test`.`t2`.`id` AS `id` from `test`.`t2` where <in_optimizer>(`test`.`t2`.`id`,<exists>(select 1 having (<cache>(`test`.`t2`.`id`) = <ref_null_helper>(1)) union select 3 having (<cache>(`test`.`t2`.`id`) = <ref_null_helper>(3)))) SELECT * FROM t2 WHERE id IN (SELECT 5 UNION SELECT 3); id SELECT * FROM t2 WHERE id IN (SELECT 5 UNION SELECT 2); @@ -906,7 +906,7 @@ id select_type table type possible_keys 2 DEPENDENT SUBQUERY t2 ref_or_null a a 5 func 2 100.00 Using where; Using index 2 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 3 100.00 Using where; Using join buffer Warnings: -Note 1003 select `test`.`t1`.`a` AS `a`,<in_optimizer>(`test`.`t1`.`a`,<exists>(select 1 AS `Not_used` from `test`.`t2` join `test`.`t3` where ((`test`.`t3`.`a` = `test`.`t2`.`a`) and ((<cache>(`test`.`t1`.`a`) = `test`.`t2`.`a`) or isnull(`test`.`t2`.`a`))) having <is_not_null_test>(`test`.`t2`.`a`))) AS `t1.a in (select t2.a from t2,t3 where t3.a=t2.a)` from `test`.`t1` +Note 1003 select `test`.`t1`.`a` AS `a`,<in_optimizer>(`test`.`t1`.`a`,<exists>(select 1 from `test`.`t2` join `test`.`t3` where ((`test`.`t3`.`a` = `test`.`t2`.`a`) and ((<cache>(`test`.`t1`.`a`) = `test`.`t2`.`a`) or isnull(`test`.`t2`.`a`))) having <is_not_null_test>(`test`.`t2`.`a`))) AS `t1.a in (select t2.a from t2,t3 where t3.a=t2.a)` from `test`.`t1` drop table t1,t2,t3; create table t1 (a float); select 10.5 IN (SELECT * from t1 LIMIT 1); @@ -1018,19 +1018,19 @@ id select_type table type possible_keys 1 PRIMARY t1 ALL NULL NULL NULL NULL 0 0.00 2 UNCACHEABLE SUBQUERY t1 ALL NULL NULL NULL NULL 0 0.00 Warnings: -Note 1003 select (select rand() AS `RAND()` from `test`.`t1`) AS `(SELECT RAND() FROM t1)` from `test`.`t1` +Note 1003 select (select rand() from `test`.`t1`) AS `(SELECT RAND() FROM t1)` from `test`.`t1` EXPLAIN EXTENDED SELECT (SELECT ENCRYPT('test') FROM t1) FROM t1; id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t1 ALL NULL NULL NULL NULL 0 0.00 2 UNCACHEABLE SUBQUERY t1 ALL NULL NULL NULL NULL 0 0.00 Warnings: -Note 1003 select (select encrypt('test') AS `ENCRYPT('test')` from `test`.`t1`) AS `(SELECT ENCRYPT('test') FROM t1)` from `test`.`t1` +Note 1003 select (select encrypt('test') from `test`.`t1`) AS `(SELECT ENCRYPT('test') FROM t1)` from `test`.`t1` EXPLAIN EXTENDED SELECT (SELECT BENCHMARK(1,1) FROM t1) FROM t1; id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t1 ALL NULL NULL NULL NULL 0 0.00 2 UNCACHEABLE SUBQUERY t1 ALL NULL NULL NULL NULL 0 0.00 Warnings: -Note 1003 select (select benchmark(1,1) AS `BENCHMARK(1,1)` from `test`.`t1`) AS `(SELECT BENCHMARK(1,1) FROM t1)` from `test`.`t1` +Note 1003 select (select benchmark(1,1) from `test`.`t1`) AS `(SELECT BENCHMARK(1,1) FROM t1)` from `test`.`t1` drop table t1; CREATE TABLE `t1` ( `mot` varchar(30) character set latin1 NOT NULL default '', @@ -1125,7 +1125,7 @@ id select_type table type possible_keys 2 UNCACHEABLE SUBQUERY t1 ALL NULL NULL NULL NULL 3 100.00 3 UNCACHEABLE SUBQUERY t1 ALL NULL NULL NULL NULL 3 100.00 Warnings: -Note 1003 select `test`.`t1`.`a` AS `a`,(select (select rand() AS `rand()` from `test`.`t1` limit 1) AS `(select rand() from t1 limit 1)` from `test`.`t1` limit 1) AS `(select (select rand() from t1 limit 1) from t1 limit 1)` from `test`.`t1` +Note 1003 select `test`.`t1`.`a` AS `a`,(select (select rand() from `test`.`t1` limit 1) from `test`.`t1` limit 1) AS `(select (select rand() from t1 limit 1) from t1 limit 1)` from `test`.`t1` drop table t1; select t1.Continent, t2.Name, t2.Population from t1 LEFT JOIN t2 ON t1.Code = t2.Country where t2.Population IN (select max(t2.Population) AS Population from t2, t1 where t2.Country = t1.Code group by Continent); ERROR 42S02: Table 'test.t1' doesn't exist @@ -1179,7 +1179,7 @@ id select_type table type possible_keys 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL Impossible WHERE Warnings: -Note 1003 select <in_optimizer>(0,<exists>(select 1 AS `Not_used` from `test`.`t1` `a` where 0)) AS `0 IN (SELECT 1 FROM t1 a)` +Note 1003 select <in_optimizer>(0,<exists>(select 1 from `test`.`t1` `a` where 0)) AS `0 IN (SELECT 1 FROM t1 a)` INSERT INTO t1 (pseudo) VALUES ('test1'); SELECT 0 IN (SELECT 1 FROM t1 a); 0 IN (SELECT 1 FROM t1 a) @@ -1189,7 +1189,7 @@ id select_type table type possible_keys 1 PRIMARY NULL NULL NULL NULL NULL NULL NULL NULL No tables used 2 DEPENDENT SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL Impossible WHERE Warnings: -Note 1003 select <in_optimizer>(0,<exists>(select 1 AS `Not_used` from `test`.`t1` `a` where 0)) AS `0 IN (SELECT 1 FROM t1 a)` +Note 1003 select <in_optimizer>(0,<exists>(select 1 from `test`.`t1` `a` where 0)) AS `0 IN (SELECT 1 FROM t1 a)` drop table t1; CREATE TABLE `t1` ( `i` int(11) NOT NULL default '0', @@ -1234,7 +1234,7 @@ id select_type table type possible_keys 1 PRIMARY t1 ref salary salary 5 const 1 100.00 Using where 2 SUBQUERY NULL NULL NULL NULL NULL NULL NULL NULL Select tables optimized away Warnings: -Note 1003 select `test`.`t1`.`id` AS `id` from `test`.`t1` where (`test`.`t1`.`salary` = (select max(`test`.`t1`.`salary`) AS `MAX(salary)` from `test`.`t1`)) +Note 1003 select `test`.`t1`.`id` AS `id` from `test`.`t1` where (`test`.`t1`.`salary` = (select max(`test`.`t1`.`salary`) from `test`.`t1`)) drop table t1; CREATE TABLE t1 ( ID int(10) unsigned NOT NULL auto_increment, @@ -1317,7 +1317,7 @@ id select_type table type possible_keys 2 DEPENDENT SUBQUERY t1 eq_ref PRIMARY PRIMARY 4 func 1 100.00 2 DEPENDENT SUBQUERY t3 eq_ref PRIMARY PRIMARY 4 test.t1.b 1 100.00 Using index Warnings: -Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` where <in_optimizer>(`test`.`t2`.`a`,<exists>(select 1 AS `Not_used` from `test`.`t1` join `test`.`t3` where ((`test`.`t3`.`a` = `test`.`t1`.`b`) and (<cache>(`test`.`t2`.`a`) = `test`.`t1`.`a`)))) +Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` where <in_optimizer>(`test`.`t2`.`a`,<exists>(select 1 from `test`.`t1` join `test`.`t3` where ((`test`.`t3`.`a` = `test`.`t1`.`b`) and (<cache>(`test`.`t2`.`a`) = `test`.`t1`.`a`)))) drop table t1, t2, t3; create table t1 (a int, b int, index a (a,b)); create table t2 (a int, index a (a)); @@ -1356,7 +1356,7 @@ id select_type table type possible_keys 2 DEPENDENT SUBQUERY t1 ref a a 5 func 1 100.00 Using where; Using index 2 DEPENDENT SUBQUERY t3 ref a a 5 test.t1.b 1 100.00 Using where; Using index Warnings: -Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` where <in_optimizer>(`test`.`t2`.`a`,<exists>(select 1 AS `Not_used` from `test`.`t1` join `test`.`t3` where ((`test`.`t3`.`a` = `test`.`t1`.`b`) and (<cache>(`test`.`t2`.`a`) = `test`.`t1`.`a`)))) +Note 1003 select `test`.`t2`.`a` AS `a` from `test`.`t2` where <in_optimizer>(`test`.`t2`.`a`,<exists>(select 1 from `test`.`t1` join `test`.`t3` where ((`test`.`t3`.`a` = `test`.`t1`.`b`) and (<cache>(`test`.`t2`.`a`) = `test`.`t1`.`a`)))) insert into t1 values (3,31); select * from t2 where t2.a in (select a from t1 where t1.b <> 30); a @@ -1515,7 +1515,7 @@ id select_type table type possible_keys 1 PRIMARY t3 ALL NULL NULL NULL NULL 3 100.00 Using where 2 SUBQUERY t2 ALL NULL NULL NULL NULL 0 0.00 Using temporary; Using filesort Warnings: -Note 1003 select `test`.`t3`.`a` AS `a` from `test`.`t3` where <not>((`test`.`t3`.`a` < <max>(select `test`.`t2`.`b` AS `b` from `test`.`t2` group by 1))) +Note 1003 select `test`.`t3`.`a` AS `a` from `test`.`t3` where <not>((`test`.`t3`.`a` < <max>(select `test`.`t2`.`b` from `test`.`t2` group by 1))) select * from t3 where a >= some (select b from t2 group by 1); a explain extended select * from t3 where a >= some (select b from t2 group by 1); @@ -1523,7 +1523,7 @@ id select_type table type possible_keys 1 PRIMARY t3 ALL NULL NULL NULL NULL 3 100.00 Using where 2 SUBQUERY t2 ALL NULL NULL NULL NULL 0 0.00 Using temporary; Using filesort Warnings: -Note 1003 select `test`.`t3`.`a` AS `a` from `test`.`t3` where <nop>((`test`.`t3`.`a` >= <min>(select `test`.`t2`.`b` AS `b` from `test`.`t2` group by 1))) +Note 1003 select `test`.`t3`.`a` AS `a` from `test`.`t3` where <nop>((`test`.`t3`.`a` >= <min>(select `test`.`t2`.`b` from `test`.`t2` group by 1))) select * from t3 where NULL >= any (select b from t2); a explain extended select * from t3 where NULL >= any (select b from t2); @@ -1566,7 +1566,7 @@ id select_type table type possible_keys 1 PRIMARY t3 ALL NULL NULL NULL NULL 3 100.00 Using where 2 SUBQUERY t2 ALL NULL NULL NULL NULL 4 100.00 Using temporary; Using filesort Warnings: -Note 1003 select `test`.`t3`.`a` AS `a` from `test`.`t3` where <not>((`test`.`t3`.`a` <= <max>(select max(`test`.`t2`.`b`) AS `max(b)` from `test`.`t2` group by `test`.`t2`.`a`))) +Note 1003 select `test`.`t3`.`a` AS `a` from `test`.`t3` where <not>((`test`.`t3`.`a` <= <max>(select max(`test`.`t2`.`b`) from `test`.`t2` group by `test`.`t2`.`a`))) drop table t2, t3; CREATE TABLE `t1` ( `id` mediumint(9) NOT NULL auto_increment, `taskid` bigint(20) NOT NULL default '0', `dbid` int(11) NOT NULL default '0', `create_date` datetime NOT NULL default '0000-00-00 00:00:00', `last_update` datetime NOT NULL default '0000-00-00 00:00:00', PRIMARY KEY (`id`)) ENGINE=MyISAM CHARSET=latin1 AUTO_INCREMENT=3 ; INSERT INTO `t1` (`id`, `taskid`, `dbid`, `create_date`,`last_update`) VALUES (1, 1, 15, '2003-09-29 10:31:36', '2003-09-29 10:31:36'), (2, 1, 21, now(), now()); @@ -1743,7 +1743,7 @@ id select_type table type possible_keys 2 DEPENDENT SUBQUERY t1 eq_ref PRIMARY PRIMARY 4 test.tt.id 1 100.00 Using where; Using index Warnings: Note 1276 Field or reference 'test.tt.id' of SELECT #2 was resolved in SELECT #1 -Note 1003 select `test`.`tt`.`id` AS `id`,`test`.`tt`.`text` AS `text` from `test`.`t1` `tt` where (not(exists(select `test`.`t1`.`id` AS `id` from `test`.`t1` where ((`test`.`t1`.`id` < 8) and (`test`.`t1`.`id` = `test`.`tt`.`id`)) having (`test`.`t1`.`id` is not null)))) +Note 1003 select `test`.`tt`.`id` AS `id`,`test`.`tt`.`text` AS `text` from `test`.`t1` `tt` where (not(exists(select `test`.`t1`.`id` from `test`.`t1` where ((`test`.`t1`.`id` < 8) and (`test`.`t1`.`id` = `test`.`tt`.`id`)) having (`test`.`t1`.`id` is not null)))) insert into t1 (id, text) values (1000, 'text1000'), (1001, 'text1001'); create table t2 (id int not null, text varchar(20) not null default '', primary key (id)); insert into t2 (id, text) values (1, 'text1'), (2, 'text2'), (3, 'text3'), (4, 'text4'), (5, 'text5'), (6, 'text6'), (7, 'text7'), (8, 'text8'), (9, 'text9'), (10, 'text10'), (11, 'text1'), (12, 'text2'), (13, 'text3'), (14, 'text4'), (15, 'text5'), (16, 'text6'), (17, 'text7'), (18, 'text8'), (19, 'text9'), (20, 'text10'),(21, 'text1'), (22, 'text2'), (23, 'text3'), (24, 'text4'), (25, 'text5'), (26, 'text6'), (27, 'text7'), (28, 'text8'), (29, 'text9'), (30, 'text10'), (31, 'text1'), (32, 'text2'), (33, 'text3'), (34, 'text4'), (35, 'text5'), (36, 'text6'), (37, 'text7'), (38, 'text8'), (39, 'text9'), (40, 'text10'), (41, 'text1'), (42, 'text2'), (43, 'text3'), (44, 'text4'), (45, 'text5'), (46, 'text6'), (47, 'text7'), (48, 'text8'), (49, 'text9'), (50, 'text10'); @@ -2279,7 +2279,7 @@ id select_type table type possible_keys 2 DEPENDENT SUBQUERY t1 ALL NULL NULL NULL NULL 2 100.00 Using where Warnings: Note 1276 Field or reference 'test.up.a' of SELECT #2 was resolved in SELECT #1 -Note 1003 select `test`.`up`.`a` AS `a`,`test`.`up`.`b` AS `b` from `test`.`t1` `up` where exists(select 1 AS `Not_used` from `test`.`t1` where (`test`.`t1`.`a` = `test`.`up`.`a`)) +Note 1003 select `test`.`up`.`a` AS `a`,`test`.`up`.`b` AS `b` from `test`.`t1` `up` where exists(select 1 from `test`.`t1` where (`test`.`t1`.`a` = `test`.`up`.`a`)) drop table t1; CREATE TABLE t1 (t1_a int); INSERT INTO t1 VALUES (1); @@ -2820,19 +2820,19 @@ id select_type table type possible_keys 1 PRIMARY t1 ALL NULL NULL NULL NULL 8 100.00 2 DEPENDENT SUBQUERY t2 ALL NULL NULL NULL NULL 9 100.00 Using where Warnings: -Note 1003 select `test`.`t1`.`one` AS `one`,`test`.`t1`.`two` AS `two`,<in_optimizer>((`test`.`t1`.`one`,`test`.`t1`.`two`),<exists>(select `test`.`t2`.`one` AS `one`,`test`.`t2`.`two` AS `two` from `test`.`t2` where ((`test`.`t2`.`flag` = '0') and trigcond(((<cache>(`test`.`t1`.`one`) = `test`.`t2`.`one`) or isnull(`test`.`t2`.`one`))) and trigcond(((<cache>(`test`.`t1`.`two`) = `test`.`t2`.`two`) or isnull(`test`.`t2`.`two`)))) having (trigcond(<is_not_null_test>(`test`.`t2`.`one`)) and trigcond(<is_not_null_test>(`test`.`t2`.`two`))))) AS `test` from `test`.`t1` +Note 1003 select `test`.`t1`.`one` AS `one`,`test`.`t1`.`two` AS `two`,<in_optimizer>((`test`.`t1`.`one`,`test`.`t1`.`two`),<exists>(select `test`.`t2`.`one`,`test`.`t2`.`two` from `test`.`t2` where ((`test`.`t2`.`flag` = '0') and trigcond(((<cache>(`test`.`t1`.`one`) = `test`.`t2`.`one`) or isnull(`test`.`t2`.`one`))) and trigcond(((<cache>(`test`.`t1`.`two`) = `test`.`t2`.`two`) or isnull(`test`.`t2`.`two`)))) having (trigcond(<is_not_null_test>(`test`.`t2`.`one`)) and trigcond(<is_not_null_test>(`test`.`t2`.`two`))))) AS `test` from `test`.`t1` explain extended SELECT one,two from t1 where ROW(one,two) IN (SELECT one,two FROM t2 WHERE flag = 'N'); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t1 ALL NULL NULL NULL NULL 8 100.00 Using where 2 DEPENDENT SUBQUERY t2 ALL NULL NULL NULL NULL 9 100.00 Using where Warnings: -Note 1003 select `test`.`t1`.`one` AS `one`,`test`.`t1`.`two` AS `two` from `test`.`t1` where <in_optimizer>((`test`.`t1`.`one`,`test`.`t1`.`two`),<exists>(select `test`.`t2`.`one` AS `one`,`test`.`t2`.`two` AS `two` from `test`.`t2` where ((`test`.`t2`.`flag` = 'N') and (<cache>(`test`.`t1`.`one`) = `test`.`t2`.`one`) and (<cache>(`test`.`t1`.`two`) = `test`.`t2`.`two`)))) +Note 1003 select `test`.`t1`.`one` AS `one`,`test`.`t1`.`two` AS `two` from `test`.`t1` where <in_optimizer>((`test`.`t1`.`one`,`test`.`t1`.`two`),<exists>(select `test`.`t2`.`one`,`test`.`t2`.`two` from `test`.`t2` where ((`test`.`t2`.`flag` = 'N') and (<cache>(`test`.`t1`.`one`) = `test`.`t2`.`one`) and (<cache>(`test`.`t1`.`two`) = `test`.`t2`.`two`)))) explain extended SELECT one,two,ROW(one,two) IN (SELECT one,two FROM t2 WHERE flag = '0' group by one,two) as 'test' from t1; id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY t1 ALL NULL NULL NULL NULL 8 100.00 2 DEPENDENT SUBQUERY t2 ALL NULL NULL NULL NULL 9 100.00 Using where; Using temporary; Using filesort Warnings: -Note 1003 select `test`.`t1`.`one` AS `one`,`test`.`t1`.`two` AS `two`,<in_optimizer>((`test`.`t1`.`one`,`test`.`t1`.`two`),<exists>(select `test`.`t2`.`one` AS `one`,`test`.`t2`.`two` AS `two` from `test`.`t2` where (`test`.`t2`.`flag` = '0') group by `test`.`t2`.`one`,`test`.`t2`.`two` having (trigcond(((<cache>(`test`.`t1`.`one`) = `test`.`t2`.`one`) or isnull(`test`.`t2`.`one`))) and trigcond(((<cache>(`test`.`t1`.`two`) = `test`.`t2`.`two`) or isnull(`test`.`t2`.`two`))) and trigcond(<is_not_null_test>(`test`.`t2`.`one`)) and trigcond(<is_not_null_test>(`test`.`t2`.`two`))))) AS `test` from `test`.`t1` +Note 1003 select `test`.`t1`.`one` AS `one`,`test`.`t1`.`two` AS `two`,<in_optimizer>((`test`.`t1`.`one`,`test`.`t1`.`two`),<exists>(select `test`.`t2`.`one`,`test`.`t2`.`two` from `test`.`t2` where (`test`.`t2`.`flag` = '0') group by `test`.`t2`.`one`,`test`.`t2`.`two` having (trigcond(((<cache>(`test`.`t1`.`one`) = `test`.`t2`.`one`) or isnull(`test`.`t2`.`one`))) and trigcond(((<cache>(`test`.`t1`.`two`) = `test`.`t2`.`two`) or isnull(`test`.`t2`.`two`))) and trigcond(<is_not_null_test>(`test`.`t2`.`one`)) and trigcond(<is_not_null_test>(`test`.`t2`.`two`))))) AS `test` from `test`.`t1` DROP TABLE t1,t2; CREATE TABLE t1 (a char(5), b char(5)); INSERT INTO t1 VALUES (NULL,'aaa'), ('aaa','aaa'); @@ -4275,7 +4275,7 @@ id select_type table type possible_keys 2 DEPENDENT SUBQUERY t2 ALL NULL NULL NULL NULL 2 100.00 Using where Warnings: Note 1276 Field or reference 'test.t1.a' of SELECT #2 was resolved in SELECT #1 -Note 1003 select 2 AS `2` from `test`.`t1` where exists(select 1 AS `1` from `test`.`t2` where (`test`.`t1`.`a` = `test`.`t2`.`a`)) +Note 1003 select 2 AS `2` from `test`.`t1` where exists(select 1 from `test`.`t2` where (`test`.`t1`.`a` = `test`.`t2`.`a`)) EXPLAIN EXTENDED SELECT 2 FROM t1 WHERE EXISTS ((SELECT 1 FROM t2 WHERE t1.a=t2.a) UNION (SELECT 1 FROM t2 WHERE t1.a = t2.a)); === modified file 'mysql-test/suite/pbxt/r/type_timestamp.result' --- a/mysql-test/suite/pbxt/r/type_timestamp.result 2009-08-17 15:57:58 +0000 +++ b/mysql-test/suite/pbxt/r/type_timestamp.result 2010-04-28 19:29:45 +0000 @@ -101,13 +101,13 @@ create table t1 (t2 timestamp(2), t4 tim t8 timestamp(8), t10 timestamp(10), t12 timestamp(12), t14 timestamp(14)); Warnings: -Warning 1287 The syntax 'TIMESTAMP(2)' is deprecated and will be removed in MySQL 6.0. Please use 'TIMESTAMP' instead -Warning 1287 The syntax 'TIMESTAMP(4)' is deprecated and will be removed in MySQL 6.0. Please use 'TIMESTAMP' instead -Warning 1287 The syntax 'TIMESTAMP(6)' is deprecated and will be removed in MySQL 6.0. Please use 'TIMESTAMP' instead -Warning 1287 The syntax 'TIMESTAMP(8)' is deprecated and will be removed in MySQL 6.0. Please use 'TIMESTAMP' instead -Warning 1287 The syntax 'TIMESTAMP(10)' is deprecated and will be removed in MySQL 6.0. Please use 'TIMESTAMP' instead -Warning 1287 The syntax 'TIMESTAMP(12)' is deprecated and will be removed in MySQL 6.0. Please use 'TIMESTAMP' instead -Warning 1287 The syntax 'TIMESTAMP(14)' is deprecated and will be removed in MySQL 6.0. Please use 'TIMESTAMP' instead +Warning 1287 'TIMESTAMP(2)' is deprecated and will be removed in a future release. Please use 'TIMESTAMP' instead +Warning 1287 'TIMESTAMP(4)' is deprecated and will be removed in a future release. Please use 'TIMESTAMP' instead +Warning 1287 'TIMESTAMP(6)' is deprecated and will be removed in a future release. Please use 'TIMESTAMP' instead +Warning 1287 'TIMESTAMP(8)' is deprecated and will be removed in a future release. Please use 'TIMESTAMP' instead +Warning 1287 'TIMESTAMP(10)' is deprecated and will be removed in a future release. Please use 'TIMESTAMP' instead +Warning 1287 'TIMESTAMP(12)' is deprecated and will be removed in a future release. Please use 'TIMESTAMP' instead +Warning 1287 'TIMESTAMP(14)' is deprecated and will be removed in a future release. Please use 'TIMESTAMP' instead insert t1 values (0,0,0,0,0,0,0), ("1997-12-31 23:47:59", "1997-12-31 23:47:59", "1997-12-31 23:47:59", "1997-12-31 23:47:59", "1997-12-31 23:47:59", "1997-12-31 23:47:59", === modified file 'sql/sql_acl.cc' --- a/sql/sql_acl.cc 2010-04-28 12:52:24 +0000 +++ b/sql/sql_acl.cc 2010-04-28 19:29:45 +0000 @@ -4442,8 +4442,10 @@ bool check_routine_level_acl(THD *thd, c ulong get_table_grant(THD *thd, TABLE_LIST *table) { ulong privilege; +#ifndef EMBEDDED_LIBRARY Security_context *sctx= thd->security_ctx; const char *db = table->db ? table->db : thd->db; +#endif GRANT_TABLE *grant_table; rw_rdlock(&LOCK_grant); === modified file 'storage/myisammrg/ha_myisammrg.cc' --- a/storage/myisammrg/ha_myisammrg.cc 2010-04-28 12:52:24 +0000 +++ b/storage/myisammrg/ha_myisammrg.cc 2010-04-28 19:29:45 +0000 @@ -295,8 +295,9 @@ static int myisammrg_parent_open_callbac } } - DBUG_PRINT("myrg", ("open: '%.*s'.'%.*s'", child_l->db_length, child_l->db, - child_l->table_name_length, child_l->table_name)); + DBUG_PRINT("myrg", ("open: '%.*s'.'%.*s'", (int)(child_l->db_length), + child_l->db, (int)(child_l->table_name_length), + child_l->table_name)); /* Convert to lowercase if required. */ if (lower_case_table_names && child_l->table_name_length) === modified file 'storage/xtradb/btr/btr0cur.c' --- a/storage/xtradb/btr/btr0cur.c 2010-04-28 14:35:00 +0000 +++ b/storage/xtradb/btr/btr0cur.c 2010-04-28 19:29:45 +0000 @@ -3417,7 +3417,7 @@ btr_estimate_n_pages_not_null( ibool diverged_lot; ulint divergence_level; ulint n_pages; - ulint i,j; + ulint i; mtr_t mtr; mem_heap_t* heap; === modified file 'storage/xtradb/buf/buf0buf.c' --- a/storage/xtradb/buf/buf0buf.c 2010-04-28 14:35:00 +0000 +++ b/storage/xtradb/buf/buf0buf.c 2010-04-28 19:29:45 +0000 @@ -3506,7 +3506,7 @@ corrupt: if (srv_pass_corrupt_table && bpage->space > 0 && bpage->space < SRV_LOG_SPACE_FIRST_ID) { fprintf(stderr, - "InnoDB: space %lu will be treated as corrupt.\n", + "InnoDB: space %u will be treated as corrupt.\n", bpage->space); fil_space_set_corrupt(bpage->space); if (trx && trx->dict_operation_lock_mode == 0) { === modified file 'storage/xtradb/fsp/fsp0fsp.c' --- a/storage/xtradb/fsp/fsp0fsp.c 2010-03-22 20:42:52 +0000 +++ b/storage/xtradb/fsp/fsp0fsp.c 2010-04-28 19:29:45 +0000 @@ -658,10 +658,6 @@ xdes_calc_descriptor_page( ulint offset) /*!< in: page offset */ { #ifndef DOXYGEN /* Doxygen gets confused of these */ -//# if UNIV_PAGE_SIZE <= XDES_ARR_OFFSET \ -// + (UNIV_PAGE_SIZE / FSP_EXTENT_SIZE) * XDES_SIZE -//# error -//# endif # if PAGE_ZIP_MIN_SIZE <= XDES_ARR_OFFSET \ + (PAGE_ZIP_MIN_SIZE / FSP_EXTENT_SIZE) * XDES_SIZE # error === modified file 'storage/xtradb/handler/ha_innodb.cc' --- a/storage/xtradb/handler/ha_innodb.cc 2010-04-28 14:35:00 +0000 +++ b/storage/xtradb/handler/ha_innodb.cc 2010-04-28 19:29:45 +0000 @@ -2042,7 +2042,7 @@ innobase_init( fprintf(stderr, "InnoDB: Warning: innodb_page_size has been changed from default value 16384. (###EXPERIMENTAL### operation)\n"); for (n_shift = 12; n_shift <= UNIV_PAGE_SIZE_SHIFT_MAX; n_shift++) { - if (innobase_page_size == (1 << n_shift)) { + if (innobase_page_size == (1u << n_shift)) { srv_page_size_shift = n_shift; srv_page_size = (1 << srv_page_size_shift); fprintf(stderr, === modified file 'storage/xtradb/include/fsp0types.h' --- a/storage/xtradb/include/fsp0types.h 2009-09-23 00:06:02 +0000 +++ b/storage/xtradb/include/fsp0types.h 2010-04-28 19:29:45 +0000 @@ -42,7 +42,7 @@ fseg_alloc_free_page) */ /* @} */ /** File space extent size (one megabyte) in pages */ -#define FSP_EXTENT_SIZE (1 << (20 - UNIV_PAGE_SIZE_SHIFT)) +#define FSP_EXTENT_SIZE (1u << (20 - UNIV_PAGE_SIZE_SHIFT)) /** On a page of any file segment, data may be put starting from this offset */ === modified file 'storage/xtradb/mtr/mtr0log.c' --- a/storage/xtradb/mtr/mtr0log.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/mtr/mtr0log.c 2010-04-28 19:29:45 +0000 @@ -408,7 +408,7 @@ mlog_parse_string( ptr += 2; if (UNIV_UNLIKELY(offset >= UNIV_PAGE_SIZE) - || UNIV_UNLIKELY(len + offset) > UNIV_PAGE_SIZE) { + || UNIV_UNLIKELY(len + offset > UNIV_PAGE_SIZE)) { recv_sys->found_corrupt_log = TRUE; return(NULL);

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2854)
by knielsen＠knielsen-hq.org 28 Apr '10

28 Apr '10

#At lp:maria 2854 knielsen(a)knielsen-hq.org 2010-04-28 [merge] Merge XtraDB 10 into MariaDB. modified: mysql-test/r/information_schema.result mysql-test/r/information_schema_all_engines.result storage/xtradb/btr/btr0btr.c storage/xtradb/btr/btr0cur.c storage/xtradb/btr/btr0pcur.c storage/xtradb/btr/btr0sea.c storage/xtradb/buf/buf0buddy.c storage/xtradb/buf/buf0buf.c storage/xtradb/buf/buf0flu.c storage/xtradb/buf/buf0rea.c storage/xtradb/build/debian/control storage/xtradb/build/debian/patches/60_percona_support.dpatch storage/xtradb/build/debian/rules storage/xtradb/build/percona-sql.spec storage/xtradb/dict/dict0dict.c storage/xtradb/dict/dict0mem.c storage/xtradb/fil/fil0fil.c storage/xtradb/fsp/fsp0fsp.c storage/xtradb/handler/ha_innodb.cc storage/xtradb/handler/ha_innodb.h storage/xtradb/handler/i_s.cc storage/xtradb/handler/i_s.h storage/xtradb/handler/innodb_patch_info.h storage/xtradb/include/btr0btr.ic storage/xtradb/include/buf0buddy.h storage/xtradb/include/buf0buf.h storage/xtradb/include/buf0buf.ic storage/xtradb/include/buf0types.h storage/xtradb/include/dict0dict.h storage/xtradb/include/dict0mem.h storage/xtradb/include/fil0fil.h storage/xtradb/include/fut0fut.ic storage/xtradb/include/page0cur.h storage/xtradb/include/page0types.h storage/xtradb/include/srv0srv.h storage/xtradb/include/trx0sys.h storage/xtradb/include/univ.i storage/xtradb/include/ut0rnd.h storage/xtradb/include/ut0rnd.ic storage/xtradb/lock/lock0lock.c storage/xtradb/log/log0log.c storage/xtradb/log/log0recv.c storage/xtradb/page/page0cur.c storage/xtradb/page/page0zip.c storage/xtradb/row/row0ins.c storage/xtradb/row/row0merge.c storage/xtradb/row/row0sel.c storage/xtradb/srv/srv0srv.c storage/xtradb/srv/srv0start.c === modified file 'mysql-test/r/information_schema.result' --- a/mysql-test/r/information_schema.result 2010-03-10 09:12:23 +0000 +++ b/mysql-test/r/information_schema.result 2010-04-28 14:35:00 +0000 @@ -65,6 +65,8 @@ INNODB_INDEX_STATS INNODB_LOCKS INNODB_LOCK_WAITS INNODB_RSEG +INNODB_SYS_INDEXES +INNODB_SYS_TABLES INNODB_TABLE_STATS INNODB_TRX KEY_COLUMN_USAGE === modified file 'mysql-test/r/information_schema_all_engines.result' --- a/mysql-test/r/information_schema_all_engines.result 2010-01-15 15:58:25 +0000 +++ b/mysql-test/r/information_schema_all_engines.result 2010-04-28 14:35:00 +0000 @@ -37,12 +37,14 @@ XTRADB_ENHANCEMENTS INNODB_BUFFER_POOL_PAGES_INDEX XTRADB_ADMIN_COMMAND INNODB_TRX -INNODB_CMP_RESET +INNODB_SYS_TABLES INNODB_LOCK_WAITS INNODB_CMPMEM_RESET INNODB_LOCKS INNODB_CMPMEM INNODB_TABLE_STATS +INNODB_SYS_INDEXES +INNODB_CMP_RESET INNODB_BUFFER_POOL_PAGES_BLOB INNODB_INDEX_STATS SELECT t.table_name, c1.column_name @@ -96,14 +98,16 @@ XTRADB_ENHANCEMENTS name INNODB_BUFFER_POOL_PAGES_INDEX schema_name XTRADB_ADMIN_COMMAND result_message INNODB_TRX trx_id -INNODB_CMP_RESET page_size +INNODB_SYS_TABLES NAME INNODB_LOCK_WAITS requesting_trx_id INNODB_CMPMEM_RESET page_size INNODB_LOCKS lock_id INNODB_CMPMEM page_size -INNODB_TABLE_STATS table_name +INNODB_TABLE_STATS table_schema +INNODB_SYS_INDEXES TABLE_ID +INNODB_CMP_RESET page_size INNODB_BUFFER_POOL_PAGES_BLOB space_id -INNODB_INDEX_STATS table_name +INNODB_INDEX_STATS table_schema SELECT t.table_name, c1.column_name FROM information_schema.tables t INNER JOIN @@ -155,14 +159,16 @@ XTRADB_ENHANCEMENTS name INNODB_BUFFER_POOL_PAGES_INDEX schema_name XTRADB_ADMIN_COMMAND result_message INNODB_TRX trx_id -INNODB_CMP_RESET page_size +INNODB_SYS_TABLES NAME INNODB_LOCK_WAITS requesting_trx_id INNODB_CMPMEM_RESET page_size INNODB_LOCKS lock_id INNODB_CMPMEM page_size -INNODB_TABLE_STATS table_name +INNODB_TABLE_STATS table_schema +INNODB_SYS_INDEXES TABLE_ID +INNODB_CMP_RESET page_size INNODB_BUFFER_POOL_PAGES_BLOB space_id -INNODB_INDEX_STATS table_name +INNODB_INDEX_STATS table_schema select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= (select cast(table_name as char) from information_schema.tables order by table_name limit 1) limit 1; @@ -205,6 +211,8 @@ INNODB_INDEX_STATS information_schema.IN INNODB_LOCKS information_schema.INNODB_LOCKS 1 INNODB_LOCK_WAITS information_schema.INNODB_LOCK_WAITS 1 INNODB_RSEG information_schema.INNODB_RSEG 1 +INNODB_SYS_INDEXES information_schema.INNODB_SYS_INDEXES 1 +INNODB_SYS_TABLES information_schema.INNODB_SYS_TABLES 1 INNODB_TABLE_STATS information_schema.INNODB_TABLE_STATS 1 INNODB_TRX information_schema.INNODB_TRX 1 KEY_COLUMN_USAGE information_schema.KEY_COLUMN_USAGE 1 @@ -267,12 +275,14 @@ Database: information_schema | INNODB_BUFFER_POOL_PAGES_INDEX | | XTRADB_ADMIN_COMMAND | | INNODB_TRX | -| INNODB_CMP_RESET | +| INNODB_SYS_TABLES | | INNODB_LOCK_WAITS | | INNODB_CMPMEM_RESET | | INNODB_LOCKS | | INNODB_CMPMEM | | INNODB_TABLE_STATS | +| INNODB_SYS_INDEXES | +| INNODB_CMP_RESET | | INNODB_BUFFER_POOL_PAGES_BLOB | | INNODB_INDEX_STATS | +---------------------------------------+ @@ -316,12 +326,14 @@ Database: INFORMATION_SCHEMA | INNODB_BUFFER_POOL_PAGES_INDEX | | XTRADB_ADMIN_COMMAND | | INNODB_TRX | -| INNODB_CMP_RESET | +| INNODB_SYS_TABLES | | INNODB_LOCK_WAITS | | INNODB_CMPMEM_RESET | | INNODB_LOCKS | | INNODB_CMPMEM | | INNODB_TABLE_STATS | +| INNODB_SYS_INDEXES | +| INNODB_CMP_RESET | | INNODB_BUFFER_POOL_PAGES_BLOB | | INNODB_INDEX_STATS | +---------------------------------------+ @@ -333,5 +345,5 @@ Wildcard: inf_rmation_schema +--------------------+ SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; table_schema count(*) -information_schema 44 +information_schema 46 mysql 22 === modified file 'storage/xtradb/btr/btr0btr.c' --- a/storage/xtradb/btr/btr0btr.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/btr/btr0btr.c 2010-03-22 20:42:52 +0000 @@ -137,6 +137,12 @@ btr_root_block_get( root_page_no = dict_index_get_page(index); block = btr_block_get(space, zip_size, root_page_no, RW_X_LATCH, mtr); + + if (srv_pass_corrupt_table && !block) { + return(0); + } + ut_a(block); + ut_a((ibool)!!page_is_comp(buf_block_get_frame(block)) == dict_table_is_comp(index->table)); #ifdef UNIV_BTR_DEBUG @@ -422,6 +428,12 @@ btr_get_size( root = btr_root_get(index, &mtr); + if (srv_pass_corrupt_table && !root) { + mtr_commit(&mtr); + return(0); + } + ut_a(root); + if (flag == BTR_N_LEAF_PAGES) { seg_header = root + PAGE_HEADER + PAGE_BTR_SEG_LEAF; @@ -862,6 +874,13 @@ leaf_loop: mtr_start(&mtr); root = btr_page_get(space, zip_size, root_page_no, RW_X_LATCH, &mtr); + + if (srv_pass_corrupt_table && !root) { + mtr_commit(&mtr); + return; + } + ut_a(root); + #ifdef UNIV_BTR_DEBUG ut_a(btr_root_fseg_validate(FIL_PAGE_DATA + PAGE_BTR_SEG_LEAF + root, space)); @@ -884,6 +903,12 @@ top_loop: mtr_start(&mtr); root = btr_page_get(space, zip_size, root_page_no, RW_X_LATCH, &mtr); + + if (srv_pass_corrupt_table && !root) { + mtr_commit(&mtr); + return; + } + ut_a(root); #ifdef UNIV_BTR_DEBUG ut_a(btr_root_fseg_validate(FIL_PAGE_DATA + PAGE_BTR_SEG_TOP + root, space)); @@ -917,6 +942,11 @@ btr_free_root( block = btr_block_get(space, zip_size, root_page_no, RW_X_LATCH, mtr); + if (srv_pass_corrupt_table && !block) { + return; + } + ut_a(block); + btr_search_drop_page_hash_index(block); header = buf_block_get_frame(block) + PAGE_HEADER + PAGE_BTR_SEG_TOP; === modified file 'storage/xtradb/btr/btr0cur.c' --- a/storage/xtradb/btr/btr0cur.c 2010-03-10 10:32:14 +0000 +++ b/storage/xtradb/btr/btr0cur.c 2010-04-28 14:35:00 +0000 @@ -227,6 +227,11 @@ btr_cur_latch_leaves( case BTR_MODIFY_LEAF: mode = latch_mode == BTR_SEARCH_LEAF ? RW_S_LATCH : RW_X_LATCH; get_block = btr_block_get(space, zip_size, page_no, mode, mtr); + + if (srv_pass_corrupt_table && !get_block) { + return; + } + ut_a(get_block); #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(get_block->frame) == page_is_comp(page)); #endif /* UNIV_BTR_DEBUG */ @@ -240,6 +245,11 @@ btr_cur_latch_leaves( get_block = btr_block_get(space, zip_size, left_page_no, RW_X_LATCH, mtr); + + if (srv_pass_corrupt_table && !get_block) { + return; + } + ut_a(get_block); #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(get_block->frame) == page_is_comp(page)); @@ -251,6 +261,11 @@ btr_cur_latch_leaves( get_block = btr_block_get(space, zip_size, page_no, RW_X_LATCH, mtr); + + if (srv_pass_corrupt_table && !get_block) { + return; + } + ut_a(get_block); #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(get_block->frame) == page_is_comp(page)); #endif /* UNIV_BTR_DEBUG */ @@ -262,6 +277,11 @@ btr_cur_latch_leaves( get_block = btr_block_get(space, zip_size, right_page_no, RW_X_LATCH, mtr); + + if (srv_pass_corrupt_table && !get_block) { + return; + } + ut_a(get_block); #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(get_block->frame) == page_is_comp(page)); @@ -283,6 +303,11 @@ btr_cur_latch_leaves( get_block = btr_block_get(space, zip_size, left_page_no, mode, mtr); cursor->left_block = get_block; + + if (srv_pass_corrupt_table && !get_block) { + return; + } + ut_a(get_block); #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(get_block->frame) == page_is_comp(page)); @@ -293,6 +318,11 @@ btr_cur_latch_leaves( } get_block = btr_block_get(space, zip_size, page_no, mode, mtr); + + if (srv_pass_corrupt_table && !get_block) { + return; + } + ut_a(get_block); #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(get_block->frame) == page_is_comp(page)); #endif /* UNIV_BTR_DEBUG */ @@ -522,6 +552,16 @@ retry_page_get: rw_latch, guess, buf_mode, __FILE__, __LINE__, mtr); if (block == NULL) { + if (srv_pass_corrupt_table && buf_mode != BUF_GET_IF_IN_POOL) { + page_cursor->block = 0; + page_cursor->rec = 0; + if (estimate) { + cursor->path_arr->nth_rec = ULINT_UNDEFINED; + } + break; + } + ut_a(buf_mode == BUF_GET_IF_IN_POOL); + /* This must be a search to perform an insert; try insert to the insert buffer */ @@ -549,6 +589,16 @@ retry_page_get: page = buf_block_get_frame(block); + if (srv_pass_corrupt_table && !page) { + page_cursor->block = 0; + page_cursor->rec = 0; + if (estimate) { + cursor->path_arr->nth_rec = ULINT_UNDEFINED; + } + break; + } + ut_a(page); + block->check_index_page_at_flush = TRUE; if (rw_latch != RW_NO_LATCH) { @@ -730,6 +780,17 @@ btr_cur_open_at_index_side( RW_NO_LATCH, NULL, BUF_GET, __FILE__, __LINE__, mtr); page = buf_block_get_frame(block); + + if (srv_pass_corrupt_table && !page) { + page_cursor->block = 0; + page_cursor->rec = 0; + if (estimate) { + cursor->path_arr->nth_rec = ULINT_UNDEFINED; + } + break; + } + ut_a(page); + ut_ad(0 == ut_dulint_cmp(index->id, btr_page_get_index_id(page))); @@ -849,6 +910,14 @@ btr_cur_open_at_rnd_pos( RW_NO_LATCH, NULL, BUF_GET, __FILE__, __LINE__, mtr); page = buf_block_get_frame(block); + + if (srv_pass_corrupt_table && !page) { + page_cursor->block = 0; + page_cursor->rec = 0; + break; + } + ut_a(page); + ut_ad(0 == ut_dulint_cmp(index->id, btr_page_get_index_id(page))); @@ -886,6 +955,108 @@ btr_cur_open_at_rnd_pos( } } +/**********************************************************************//** +Positions a cursor at a randomly chosen position within a B-tree +after the given path +@return TRUE if the position is at the first page, and cursor must point + the first record for used by the caller.*/ +UNIV_INTERN +ibool +btr_cur_open_at_rnd_pos_after_path( +/*====================*/ + dict_index_t* index, /*!< in: index */ + ulint latch_mode, /*!< in: BTR_SEARCH_LEAF, ... */ + btr_path_t* first_rec_path, + btr_cur_t* cursor, /*!< in/out: B-tree cursor */ + mtr_t* mtr) /*!< in: mtr */ +{ + page_cur_t* page_cursor; + btr_path_t* slot; + ibool is_first_rec = TRUE; + ulint page_no; + ulint space; + ulint zip_size; + ulint height; + rec_t* node_ptr; + mem_heap_t* heap = NULL; + ulint offsets_[REC_OFFS_NORMAL_SIZE]; + ulint* offsets = offsets_; + rec_offs_init(offsets_); + + if (latch_mode == BTR_MODIFY_TREE) { + mtr_x_lock(dict_index_get_lock(index), mtr); + } else { + mtr_s_lock(dict_index_get_lock(index), mtr); + } + + page_cursor = btr_cur_get_page_cur(cursor); + cursor->index = index; + + space = dict_index_get_space(index); + zip_size = dict_table_zip_size(index->table); + page_no = dict_index_get_page(index); + + height = ULINT_UNDEFINED; + slot = first_rec_path; + + for (;;) { + buf_block_t* block; + page_t* page; + + block = buf_page_get_gen(space, zip_size, page_no, + RW_NO_LATCH, NULL, BUF_GET, + __FILE__, __LINE__, mtr); + page = buf_block_get_frame(block); + ut_ad(0 == ut_dulint_cmp(index->id, + btr_page_get_index_id(page))); + + if (height == ULINT_UNDEFINED) { + /* We are in the root node */ + + height = btr_page_get_level(page, mtr); + } + + if (height == 0) { + btr_cur_latch_leaves(page, space, zip_size, page_no, + latch_mode, cursor, mtr); + } + + if (is_first_rec && slot->nth_rec != ULINT_UNDEFINED) { + if (height == 0) { + /* must open the first rec */ + page_cur_open_on_nth_user_rec(block, page_cursor, slot->nth_rec); + } else { + is_first_rec = page_cur_open_on_rnd_user_rec_after_nth(block, + page_cursor, slot->nth_rec); + } + } else { + is_first_rec = FALSE; + page_cur_open_on_rnd_user_rec(block, page_cursor); + } + + if (height == 0) { + break; + } + + ut_ad(height > 0); + + height--; + slot++; + + node_ptr = page_cur_get_rec(page_cursor); + offsets = rec_get_offsets(node_ptr, cursor->index, offsets, + ULINT_UNDEFINED, &heap); + /* Go to the child node */ + page_no = btr_node_ptr_get_child_page_no(node_ptr, offsets); + } + + if (UNIV_LIKELY_NULL(heap)) { + mem_heap_free(heap); + } + + return (is_first_rec); +} + /*==================== B-TREE INSERT =========================*/ /*************************************************************//** @@ -1064,6 +1235,12 @@ btr_cur_optimistic_insert( *big_rec = NULL; block = btr_cur_get_block(cursor); + + if (srv_pass_corrupt_table && !block) { + return(DB_CORRUPTION); + } + ut_a(block); + page = buf_block_get_frame(block); index = cursor->index; zip_size = buf_block_get_zip_size(block); @@ -2810,6 +2987,11 @@ btr_cur_optimistic_delete( block = btr_cur_get_block(cursor); + if (srv_pass_corrupt_table && !block) { + return(DB_CORRUPTION); + } + ut_a(block); + ut_ad(page_is_leaf(buf_block_get_frame(block))); rec = btr_cur_get_rec(cursor); @@ -3216,6 +3398,154 @@ btr_estimate_n_rows_in_range( } /*******************************************************************//** +Estimates the number of pages which have not null value of the key of n_cols. +@return estimated number of pages */ +UNIV_INTERN +ulint +btr_estimate_n_pages_not_null( +/*=========================*/ + dict_index_t* index, /*!< in: index */ + ulint n_cols, /*!< in: The cols should be not null */ + btr_path_t* path1) /*!< in: path1[BTR_PATH_ARRAY_N_SLOTS] */ +{ + dtuple_t* tuple1; + btr_path_t path2[BTR_PATH_ARRAY_N_SLOTS]; + btr_cur_t cursor; + btr_path_t* slot1; + btr_path_t* slot2; + ibool diverged; + ibool diverged_lot; + ulint divergence_level; + ulint n_pages; + ulint i,j; + mtr_t mtr; + mem_heap_t* heap; + + heap = mem_heap_create(n_cols * sizeof(dfield_t) + + sizeof(dtuple_t)); + + /* make tuple1 (NULL,NULL,,,) from n_cols */ + tuple1 = dtuple_create(heap, n_cols); + dict_index_copy_types(tuple1, index, n_cols); + + for (i = 0; i < n_cols; i++) { + dfield_set_null(dtuple_get_nth_field(tuple1, i)); + } + + mtr_start(&mtr); + + cursor.path_arr = path1; + + btr_cur_search_to_nth_level(index, 0, tuple1, PAGE_CUR_G, + BTR_SEARCH_LEAF | BTR_ESTIMATE, + &cursor, 0, &mtr); + + mtr_commit(&mtr); + + + + mtr_start(&mtr); + + cursor.path_arr = path2; + + btr_cur_open_at_index_side(FALSE, index, + BTR_SEARCH_LEAF | BTR_ESTIMATE, + &cursor, &mtr); + + mtr_commit(&mtr); + + mem_heap_free(heap); + + /* We have the path information for the range in path1 and path2 */ + + n_pages = 1; + diverged = FALSE; /* This becomes true when the path is not + the same any more */ + diverged_lot = FALSE; /* This becomes true when the paths are + not the same or adjacent any more */ + divergence_level = 1000000; /* This is the level where paths diverged + a lot */ + for (i = 0; ; i++) { + ut_ad(i < BTR_PATH_ARRAY_N_SLOTS); + + slot1 = path1 + i; + slot2 = path2 + i; + + if ((slot1 + 1)->nth_rec == ULINT_UNDEFINED + || (slot2 + 1)->nth_rec == ULINT_UNDEFINED) { + + if (i > divergence_level + 1) { + /* In trees whose height is > 1 our algorithm + tends to underestimate: multiply the estimate + by 2: */ + + n_pages = n_pages * 2; + } + + /* Do not estimate the number of rows in the range + to over 1 / 2 of the estimated rows in the whole + table */ + + if (n_pages > index->stat_n_leaf_pages / 2) { + n_pages = index->stat_n_leaf_pages / 2; + + /* If there are just 0 or 1 rows in the table, + then we estimate all rows are in the range */ + + if (n_pages == 0) { + n_pages = index->stat_n_leaf_pages; + } + } + + return(n_pages); + } + + if (!diverged && slot1->nth_rec != slot2->nth_rec) { + + diverged = TRUE; + + if (slot1->nth_rec < slot2->nth_rec) { + n_pages = slot2->nth_rec - slot1->nth_rec; + + if (n_pages > 1) { + diverged_lot = TRUE; + divergence_level = i; + } + } else { + /* Maybe the tree has changed between + searches */ + + return(10); + } + + } else if (diverged && !diverged_lot) { + + if (slot1->nth_rec < slot1->n_recs + || slot2->nth_rec > 1) { + + diverged_lot = TRUE; + divergence_level = i; + + n_pages = 0; + + if (slot1->nth_rec < slot1->n_recs) { + n_pages += slot1->n_recs + - slot1->nth_rec; + } + + if (slot2->nth_rec > 1) { + n_pages += slot2->nth_rec - 1; + } + } + } else if (diverged_lot) { + + n_pages = (n_pages * (slot1->n_recs + slot2->n_recs)) + / 2; + } + } +} + +/*******************************************************************//** Estimates the number of different key values in a given index, for each n-column prefix of the index where n <= dict_index_get_n_unique(index). The estimates are stored in the array index->stat_n_diff_key_vals. */ @@ -3231,9 +3561,7 @@ btr_estimate_number_of_different_key_val ulint n_cols; ulint matched_fields; ulint matched_bytes; - ib_int64_t n_recs = 0; ib_int64_t* n_diff; - ib_int64_t* n_not_nulls= 0; ullint n_sample_pages; /* number of pages to sample */ ulint not_empty_flag = 0; ulint total_external_size = 0; @@ -3247,22 +3575,37 @@ btr_estimate_number_of_different_key_val ulint* offsets_rec = offsets_rec_; ulint* offsets_next_rec= offsets_next_rec_; ulint stats_method = srv_stats_method; + btr_path_t first_rec_path[BTR_PATH_ARRAY_N_SLOTS]; + ulint effective_pages; /* effective leaf pages */ rec_offs_init(offsets_rec_); rec_offs_init(offsets_next_rec_); n_cols = dict_index_get_n_unique(index); - n_diff = mem_zalloc((n_cols + 1) * sizeof(ib_int64_t)); - if (stats_method == SRV_STATS_METHOD_IGNORE_NULLS) { - n_not_nulls = mem_zalloc((n_cols + 1) * sizeof(ib_int64_t)); + /* estimate effective pages and path for the first effective record */ + /* TODO: make it work also for n_cols > 1. */ + effective_pages = btr_estimate_n_pages_not_null(index, 1 /*k*/, first_rec_path); + + if (!effective_pages) { + for (j = 0; j <= n_cols; j++) { + index->stat_n_diff_key_vals[j] = (ib_int64_t)index->stat_n_leaf_pages; + } + return; + } else if (effective_pages > index->stat_n_leaf_pages) { + effective_pages = index->stat_n_leaf_pages; + } + } else { + effective_pages = index->stat_n_leaf_pages; } + n_diff = mem_zalloc((n_cols + 1) * sizeof(ib_int64_t)); + /* It makes no sense to test more pages than are contained in the index, thus we lower the number if it is too high */ - if (srv_stats_sample_pages > index->stat_index_size) { - if (index->stat_index_size > 0) { - n_sample_pages = index->stat_index_size; + if (srv_stats_sample_pages > effective_pages) { + if (effective_pages > 0) { + n_sample_pages = effective_pages; } else { n_sample_pages = 1; } @@ -3274,9 +3617,15 @@ btr_estimate_number_of_different_key_val for (i = 0; i < n_sample_pages; i++) { rec_t* supremum; + ibool is_first_page = TRUE; mtr_start(&mtr); + if (stats_method == SRV_STATS_METHOD_IGNORE_NULLS) { + is_first_page = btr_cur_open_at_rnd_pos_after_path(index, BTR_SEARCH_LEAF, + first_rec_path, &cursor, &mtr); + } else { btr_cur_open_at_rnd_pos(index, BTR_SEARCH_LEAF, &cursor, &mtr); + } /* Count the number of different key values for each prefix of the key on this index page. If the prefix does not determine @@ -3286,8 +3635,19 @@ btr_estimate_number_of_different_key_val page = btr_cur_get_page(&cursor); + if (srv_pass_corrupt_table && !page) { + break; + } + ut_a(page); + supremum = page_get_supremum_rec(page); + if (stats_method == SRV_STATS_METHOD_IGNORE_NULLS && is_first_page) { + /* the cursor should be the first record of the page. */ + /* Counting should be started from here. */ + rec = btr_cur_get_rec(&cursor); + } else { rec = page_rec_get_next(page_get_infimum_rec(page)); + } if (rec != supremum) { not_empty_flag = 1; @@ -3297,19 +3657,6 @@ btr_estimate_number_of_different_key_val while (rec != supremum) { rec_t* next_rec; - /* count recs */ - if (stats_method == SRV_STATS_METHOD_IGNORE_NULLS) { - n_recs++; - for (j = 0; j <= n_cols; j++) { - ulint f_len; - rec_get_nth_field(rec, offsets_rec, - j, &f_len); - if (f_len == UNIV_SQL_NULL) - break; - - n_not_nulls[j]++; - } - } next_rec = page_rec_get_next(rec); if (next_rec == supremum) { break; @@ -3324,7 +3671,10 @@ btr_estimate_number_of_different_key_val cmp_rec_rec_with_match(rec, next_rec, offsets_rec, offsets_next_rec, index, &matched_fields, - &matched_bytes, srv_stats_method); + &matched_bytes, + (stats_method==SRV_STATS_METHOD_NULLS_NOT_EQUAL) ? + SRV_STATS_METHOD_NULLS_NOT_EQUAL : + SRV_STATS_METHOD_NULLS_EQUAL); for (j = matched_fields + 1; j <= n_cols; j++) { /* We add one if this index record has @@ -3385,7 +3735,7 @@ btr_estimate_number_of_different_key_val for (j = 0; j <= n_cols; j++) { index->stat_n_diff_key_vals[j] = ((n_diff[j] - * (ib_int64_t)index->stat_n_leaf_pages + * (ib_int64_t)effective_pages + n_sample_pages - 1 + total_external_size + not_empty_flag) @@ -3400,7 +3750,7 @@ btr_estimate_number_of_different_key_val different key values, or even more. Let us try to approximate that: */ - add_on = index->stat_n_leaf_pages + add_on = effective_pages / (10 * (n_sample_pages + total_external_size)); @@ -3410,20 +3760,18 @@ btr_estimate_number_of_different_key_val index->stat_n_diff_key_vals[j] += add_on; - /* revision for 'nulls_ignored' */ if (stats_method == SRV_STATS_METHOD_IGNORE_NULLS) { - if (!n_not_nulls[j]) - n_not_nulls[j] = 1; + /* index->stat_n_diff_key_vals[k] is used for calc rec_per_key, + as "stats.records / index->stat_n_diff_key_vals[x]". + So it should be adjusted to the value which is based on whole of the index. */ index->stat_n_diff_key_vals[j] = - index->stat_n_diff_key_vals[j] * n_recs - / n_not_nulls[j]; + index->stat_n_diff_key_vals[j] * (ib_int64_t)index->stat_n_leaf_pages + / (ib_int64_t)effective_pages; } } mem_free(n_diff); - if (stats_method == SRV_STATS_METHOD_IGNORE_NULLS) { - mem_free(n_not_nulls); - } + if (UNIV_LIKELY_NULL(heap)) { mem_heap_free(heap); } === modified file 'storage/xtradb/btr/btr0pcur.c' --- a/storage/xtradb/btr/btr0pcur.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/btr/btr0pcur.c 2010-03-22 20:42:52 +0000 @@ -32,7 +32,7 @@ Created 2/23/1996 Heikki Tuuri #include "ut0byte.h" #include "rem0cmp.h" #include "trx0trx.h" - +#include "srv0srv.h" /**************************************************************//** Allocates memory for a persistent cursor object and initializes the cursor. @return own: persistent cursor */ @@ -102,6 +102,12 @@ btr_pcur_store_position( ut_ad(cursor->latch_mode != BTR_NO_LATCHES); block = btr_pcur_get_block(cursor); + + if (srv_pass_corrupt_table && !block) { + return; + } + ut_a(block); + index = btr_cur_get_index(btr_pcur_get_btr_cur(cursor)); page_cursor = btr_pcur_get_page_cur(cursor); @@ -413,6 +419,15 @@ btr_pcur_move_to_next_page( next_block = btr_block_get(space, zip_size, next_page_no, cursor->latch_mode, mtr); next_page = buf_block_get_frame(next_block); + + if (srv_pass_corrupt_table && !next_page) { + btr_leaf_page_release(btr_pcur_get_block(cursor), + cursor->latch_mode, mtr); + btr_pcur_get_page_cur(cursor)->block = 0; + btr_pcur_get_page_cur(cursor)->rec = 0; + return; + } + ut_a(next_page); #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(next_page) == page_is_comp(page)); ut_a(btr_page_get_prev(next_page, mtr) === modified file 'storage/xtradb/btr/btr0sea.c' --- a/storage/xtradb/btr/btr0sea.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/btr/btr0sea.c 2010-03-22 20:42:52 +0000 @@ -42,7 +42,7 @@ Created 2/17/1996 Heikki Tuuri #include "btr0pcur.h" #include "btr0btr.h" #include "ha0ha.h" - +#include "srv0srv.h" /** Flag: has the search system been enabled? Protected by btr_search_latch and btr_search_enabled_mutex. */ UNIV_INTERN char btr_search_enabled = TRUE; @@ -585,6 +585,11 @@ btr_search_info_update_slow( block = btr_cur_get_block(cursor); + if (srv_pass_corrupt_table && !block) { + return; + } + ut_a(block); + /* NOTE that the following two function calls do NOT protect info or block->n_fields etc. with any semaphore, to save CPU time! We cannot assume the fields are consistent when we return from === modified file 'storage/xtradb/buf/buf0buddy.c' --- a/storage/xtradb/buf/buf0buddy.c 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/buf/buf0buddy.c 2010-04-28 14:35:00 +0000 @@ -43,7 +43,7 @@ static ulint buf_buddy_n_frames; #endif /* UNIV_DEBUG */ /** Statistics of the buddy system, indexed by block size. Protected by buf_pool_mutex. */ -UNIV_INTERN buf_buddy_stat_t buf_buddy_stat[BUF_BUDDY_SIZES + 1]; +UNIV_INTERN buf_buddy_stat_t buf_buddy_stat[BUF_BUDDY_SIZES_MAX + 1]; /**********************************************************************//** Get the offset of the buddy of a compressed page frame. === modified file 'storage/xtradb/buf/buf0buf.c' --- a/storage/xtradb/buf/buf0buf.c 2010-01-28 11:35:10 +0000 +++ b/storage/xtradb/buf/buf0buf.c 2010-04-28 14:35:00 +0000 @@ -52,6 +52,7 @@ Created 11/5/1995 Heikki Tuuri #include "log0recv.h" #include "page0zip.h" #include "trx0trx.h" +#include "srv0start.h" /* prototypes for new functions added to ha_innodb.cc */ trx_t* innobase_get_trx(); @@ -348,6 +349,27 @@ buf_calc_page_new_checksum( return(checksum); } +UNIV_INTERN +ulint +buf_calc_page_new_checksum_32( +/*==========================*/ + const byte* page) /*!< in: buffer page */ +{ + ulint checksum; + + checksum = ut_fold_binary(page + FIL_PAGE_OFFSET, + FIL_PAGE_FILE_FLUSH_LSN - FIL_PAGE_OFFSET) + + ut_fold_binary(page + FIL_PAGE_DATA, + FIL_PAGE_DATA_ALIGN_32 - FIL_PAGE_DATA) + + ut_fold_binary_32(page + FIL_PAGE_DATA_ALIGN_32, + UNIV_PAGE_SIZE - FIL_PAGE_DATA_ALIGN_32 + - FIL_PAGE_END_LSN_OLD_CHKSUM); + + checksum = checksum & 0xFFFFFFFFUL; + + return(checksum); +} + /********************************************************************//** In versions < 4.0.14 and < 4.1.1 there was a bug that the checksum only looked at the first few bytes of the page. This calculates that old @@ -462,13 +484,25 @@ buf_page_is_corrupted( /* InnoDB versions < 4.0.14 and < 4.1.1 stored the space id (always equal to 0), to FIL_PAGE_SPACE_OR_CHKSUM */ - if (checksum_field != 0 + if (!srv_fast_checksum + && checksum_field != 0 && checksum_field != BUF_NO_CHECKSUM_MAGIC && checksum_field != buf_calc_page_new_checksum(read_buf)) { return(TRUE); } + + if (srv_fast_checksum + && checksum_field != 0 + && checksum_field != BUF_NO_CHECKSUM_MAGIC + && checksum_field + != buf_calc_page_new_checksum_32(read_buf) + && checksum_field + != buf_calc_page_new_checksum(read_buf)) { + + return(TRUE); + } } return(FALSE); @@ -488,6 +522,7 @@ buf_page_print( dict_index_t* index; #endif /* !UNIV_HOTBACKUP */ ulint checksum; + ulint checksum_32; ulint old_checksum; ulint size = zip_size; @@ -574,12 +609,14 @@ buf_page_print( checksum = srv_use_checksums ? buf_calc_page_new_checksum(read_buf) : BUF_NO_CHECKSUM_MAGIC; + checksum_32 = srv_use_checksums + ? buf_calc_page_new_checksum_32(read_buf) : BUF_NO_CHECKSUM_MAGIC; old_checksum = srv_use_checksums ? buf_calc_page_old_checksum(read_buf) : BUF_NO_CHECKSUM_MAGIC; ut_print_timestamp(stderr); fprintf(stderr, - " InnoDB: Page checksum %lu, prior-to-4.0.14-form" + " InnoDB: Page checksum %lu (32bit_calc: %lu), prior-to-4.0.14-form" " checksum %lu\n" "InnoDB: stored checksum %lu, prior-to-4.0.14-form" " stored checksum %lu\n" @@ -588,7 +625,7 @@ buf_page_print( "InnoDB: Page number (if stored to page already) %lu,\n" "InnoDB: space id (if created with >= MySQL-4.1.1" " and stored already) %lu\n", - (ulong) checksum, (ulong) old_checksum, + (ulong) checksum, (ulong) checksum_32, (ulong) old_checksum, (ulong) mach_read_from_4(read_buf + FIL_PAGE_SPACE_OR_CHKSUM), (ulong) mach_read_from_4(read_buf + UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM), @@ -1809,6 +1846,14 @@ err_exit: return(NULL); } + if (srv_pass_corrupt_table) { + if (bpage->is_corrupt) { + rw_lock_s_unlock(&page_hash_latch); + return(NULL); + } + } + ut_a(!(bpage->is_corrupt)); + block_mutex = buf_page_get_mutex_enter(bpage); rw_lock_s_unlock(&page_hash_latch); @@ -2246,6 +2291,14 @@ loop2: return(NULL); } + if (srv_pass_corrupt_table) { + if (block->page.is_corrupt) { + mutex_exit(block_mutex); + return(NULL); + } + } + ut_a(!(block->page.is_corrupt)); + switch (buf_block_get_state(block)) { buf_page_t* bpage; ibool success; @@ -2874,6 +2927,7 @@ buf_page_init_low( bpage->newest_modification = 0; bpage->oldest_modification = 0; HASH_INVALIDATE(bpage, hash); + bpage->is_corrupt = FALSE; #ifdef UNIV_DEBUG_FILE_ACCESSES bpage->file_page_was_freed = FALSE; #endif /* UNIV_DEBUG_FILE_ACCESSES */ @@ -3329,7 +3383,8 @@ UNIV_INTERN void buf_page_io_complete( /*=================*/ - buf_page_t* bpage) /*!< in: pointer to the block in question */ + buf_page_t* bpage, /*!< in: pointer to the block in question */ + trx_t* trx) { enum buf_io_fix io_type; const ibool uncompressed = (buf_page_get_state(bpage) @@ -3406,6 +3461,7 @@ buf_page_io_complete( (ulong) bpage->offset); } + if (!srv_pass_corrupt_table || !bpage->is_corrupt) { /* From version 3.23.38 up we store the page checksum to the 4 first bytes of the page end lsn field */ @@ -3447,6 +3503,19 @@ corrupt: REFMAN "forcing-recovery.html\n" "InnoDB: about forcing recovery.\n", stderr); + if (srv_pass_corrupt_table && bpage->space > 0 + && bpage->space < SRV_LOG_SPACE_FIRST_ID) { + fprintf(stderr, + "InnoDB: space %lu will be treated as corrupt.\n", + bpage->space); + fil_space_set_corrupt(bpage->space); + if (trx && trx->dict_operation_lock_mode == 0) { + dict_table_set_corrupt_by_space(bpage->space, TRUE); + } else { + dict_table_set_corrupt_by_space(bpage->space, FALSE); + } + bpage->is_corrupt = TRUE; + } else if (srv_force_recovery < SRV_FORCE_IGNORE_CORRUPT) { fputs("InnoDB: Ending processing because of" " a corrupt database page.\n", @@ -3454,6 +3523,7 @@ corrupt: exit(1); } } + } /**/ if (recv_recovery_is_on()) { /* Pages must be uncompressed for crash recovery. */ @@ -3463,8 +3533,11 @@ corrupt: if (uncompressed && !recv_no_ibuf_operations) { ibuf_merge_or_delete_for_page( + /* Delete possible entries, if bpage is_corrupt */ + (srv_pass_corrupt_table && bpage->is_corrupt) ? NULL : (buf_block_t*) bpage, bpage->space, bpage->offset, buf_page_get_zip_size(bpage), + (srv_pass_corrupt_table && bpage->is_corrupt) ? FALSE : TRUE); } } === modified file 'storage/xtradb/buf/buf0flu.c' --- a/storage/xtradb/buf/buf0flu.c 2010-04-28 13:53:04 +0000 +++ b/storage/xtradb/buf/buf0flu.c 2010-04-28 14:35:00 +0000 @@ -711,7 +711,9 @@ buf_flush_init_for_writing( mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, srv_use_checksums - ? buf_calc_page_new_checksum(page) + ? (!srv_fast_checksum + ? buf_calc_page_new_checksum(page) + : buf_calc_page_new_checksum_32(page)) : BUF_NO_CHECKSUM_MAGIC); /* We overwrite the first 4 bytes of the end lsn field to store === modified file 'storage/xtradb/buf/buf0rea.c' --- a/storage/xtradb/buf/buf0rea.c 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/buf/buf0rea.c 2010-04-28 14:35:00 +0000 @@ -187,12 +187,19 @@ not_to_recover: sync, space, 0, offset, 0, UNIV_PAGE_SIZE, ((buf_block_t*) bpage)->frame, bpage, trx); } + + if (srv_pass_corrupt_table) { + if (*err != DB_SUCCESS) { + bpage->is_corrupt = TRUE; + } + } else { ut_a(*err == DB_SUCCESS); + } if (sync) { /* The i/o is already completed when we arrive from fil_read */ - buf_page_io_complete(bpage); + buf_page_io_complete(bpage, trx); } return(1); === modified file 'storage/xtradb/build/debian/control' --- a/storage/xtradb/build/debian/control 2010-02-22 19:53:53 +0000 +++ b/storage/xtradb/build/debian/control 2010-04-06 08:23:16 +0000 @@ -47,9 +47,9 @@ Package: percona-xtradb-common Section: database Architecture: all Depends: ${shlibs:Depends}, ${misc:Depends} -Conflicts: mysql-common-4.1, mysql-common-5.0, mysql-common-5.1 -Provides: mysql-common-4.1, percona-xtradb-common -Replaces: mysql-common-4.1, mysql-common-5.0, mysql-common-5.1 +Conflicts: mysql-common-4.1, mysql-common-5.0, mysql-common-5.1, mysql-common +Provides: mysql-common +Replaces: mysql-common-4.1, mysql-common-5.0, mysql-common-5.1, mysql-common Description: Percona SQL database common files (e.g. /etc/mysql/my.cnf) Percona SQL is a fast, stable and true multi-user, multi-threaded SQL database server. SQL (Structured Query Language) is the most popular database query === modified file 'storage/xtradb/build/debian/patches/60_percona_support.dpatch' --- a/storage/xtradb/build/debian/patches/60_percona_support.dpatch 2010-01-26 18:02:46 +0000 +++ b/storage/xtradb/build/debian/patches/60_percona_support.dpatch 2010-03-25 12:25:33 +0000 @@ -4,12 +4,13 @@ --- a/scripts/mysql_install_db.sh 2009-08-08 09:20:07.000000000 +0000 +++ b/scripts/mysql_install_db.sh 2009-08-08 09:29:23.000000000 +0000 -@@ -471,7 +471,7 @@ +@@ -469,6 +469,9 @@ + echo echo "Please report any problems with the $scriptdir/mysqlbug script!" echo - echo "The latest information about MySQL is available at http://www.mysql.com/" -- echo "Support MySQL by buying support/licenses from http://shop.mysql.com/" + echo "For commercial support please contact Percona at http://www.percona.com/contacts.html" - echo ++ echo ++ fi - + + exit 0 === modified file 'storage/xtradb/build/debian/rules' --- a/storage/xtradb/build/debian/rules 2010-02-18 23:51:40 +0000 +++ b/storage/xtradb/build/debian/rules 2010-03-29 15:30:32 +0000 @@ -20,7 +20,7 @@ DEB_NOEPOCH_VERSION ?= $(shell echo $(DE DEB_UPSTREAM_VERSION ?= $(shell echo $(DEB_NOEPOCH_VERSION) | sed 's/-[^-]*$$//') DEB_UPSTREAM_VERSION_MAJOR_MINOR := $(shell echo $(DEB_UPSTREAM_VERSION) | sed -r -n 's/^([0-9]+\.[0-9]+).*/\1/p') -DISTRIBUTION = $(shell echo "Percona SQL Server (GPL), XtraDB $(BB_PERCONA_VERSION), Revision $(BB_PERCONA_REVISION)") +DISTRIBUTION = $(shell echo "Percona SQL Server (GPL), XtraDB 10") MAKE_J = -j$(shell if [ -f /proc/cpuinfo ] ; then grep -c processor.* /proc/cpuinfo ; else echo 1 ; fi) ifeq (${MAKE_J}, -j0) === modified file 'storage/xtradb/build/percona-sql.spec' --- a/storage/xtradb/build/percona-sql.spec 2010-02-18 18:47:04 +0000 +++ b/storage/xtradb/build/percona-sql.spec 2010-04-09 05:18:19 +0000 @@ -13,10 +13,13 @@ # pluginversion - Version of InnoDB plugin taken as the basis, e.g. 1.0.3 # redhatversion - 5 or 4 # xtradbversion - The XtraDB release, eg. 6 -# gotrevision - bzr revision of the sources the package is built of %define mysql_vendor Percona, Inc -%{!?redhatversion:%define redhatversion 5} +%define redhatversion %(cat /etc/redhat-release | awk '{ print $3}' | awk -F. '{ print $1}') +%define community 1 +%define mysqlversion 5.1.45 +%define pluginversion 1.0.6 +%define xtradbversion 10 %define distribution rhel%{redhatversion} %define release %{xtradbversion}.%{distribution} @@ -106,8 +109,8 @@ %define server_suffix -51 %define package_suffix -51 -%define ndbug_comment Percona SQL Server (GPL), XtraDB %{xtradbversion}, Revision %{gotrevision} -%define debug_comment Percona SQL Server - Debug (GPL), XtraDB %{xtradbversion}, Revision %{gotrevision} +%define ndbug_comment Percona SQL Server (GPL), XtraDB %{xtradbversion} +%define debug_comment Percona SQL Server - Debug (GPL), XtraDB %{xtradbversion} %define commercial 0 %define YASSL_BUILD 1 %define EMBEDDED_BUILD 0 @@ -143,6 +146,7 @@ Patch04: microsec_process.patch Patch05: userstat.patch Patch06: optimizer_fix.patch Patch07: mysql-test_for_xtradb.diff +Patch08: show_temp_51.patch %define perconaxtradbplugin percona-xtradb-%{pluginversion}-%{xtradbversion}.tar.gz @@ -162,8 +166,8 @@ Source: %{src_dir}.tar.gz URL: http://www.percona.com/ Packager: %{mysql_vendor} MySQL Development Team <mysql-dev(a)percona.com> Vendor: %{mysql_vendor} -Provides: msqlormysql MySQL-server mysql Percona-XtraDB-server -BuildRequires: gperf perl readline-devel gcc-c++ ncurses-devel zlib-devel libtool automake autoconf time ccache +Provides: msqlormysql MySQL-server Percona-XtraDB-server +BuildRequires: gperf perl readline-devel gcc-c++ ncurses-devel zlib-devel libtool automake autoconf time ccache bison # Think about what you use here since the first step is to # run a rm -rf @@ -187,7 +191,7 @@ For more information visist our web site Summary: %{ndbug_comment} for Red Hat Enterprise Linux %{redhatversion} Group: Applications/Databases Requires: chkconfig coreutils shadow-utils grep procps -Provides: msqlormysql mysql-server mysql MySQL MySQL-server Percona-XtraDB-server +Provides: msqlormysql mysql-server MySQL-server Percona-XtraDB-server Obsoletes: MySQL mysql mysql-server MySQL-server MySQL-server-community MySQL-server-percona %description -n Percona-XtraDB-server%{package_suffix} @@ -214,7 +218,7 @@ package "Percona-XtraDB-client%{package_ Summary: Percona-XtraDB - Client Group: Applications/Databases Obsoletes: mysql-client MySQL-client MySQL-client-community MySQL-client-percona -Provides: mysql-client MySQL-client Percona-XtraDB-client +Provides: mysql-client MySQL-client Percona-XtraDB-client mysql MySQL %description -n Percona-XtraDB-client%{package_suffix} This package contains the standard Percona-XtraDB clients and administration tools. @@ -308,6 +312,7 @@ judgment as a high-performance consultin %patch05 -p1 %patch06 -p1 %patch07 -p1 +%patch08 -p1 if [ "%{redhatversion}" = "5" ] ; then tar xfz $RPM_SOURCE_DIR/%{perconaxtradbplugin} -C storage/innobase --strip-components=1 @@ -1018,6 +1023,10 @@ fi # merging BK trees) ############################################################################## %changelog +* Mon Mar 22 2010 Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> + +XtraDB Release 10 + * Thu Feb 11 2010 Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> Package name changed to Percona-XtraDB === modified file 'storage/xtradb/dict/dict0dict.c' --- a/storage/xtradb/dict/dict0dict.c 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/dict/dict0dict.c 2010-04-28 14:35:00 +0000 @@ -54,6 +54,7 @@ UNIV_INTERN dict_index_t* dict_ind_compa #include "row0merge.h" #include "m_ctype.h" /* my_isspace() */ #include "ha_prototypes.h" /* innobase_strcasecmp() */ +#include "srv0start.h" /* SRV_LOG_SPACE_FIRST_ID */ #include <ctype.h> @@ -658,7 +659,7 @@ dict_table_get( mutex_exit(&(dict_sys->mutex)); if (table != NULL) { - if (!table->stat_initialized) { + if (!table->stat_initialized && !table->is_corrupt) { /* If table->ibd_file_missing == TRUE, this will print an error message and return without doing anything. */ @@ -1181,7 +1182,7 @@ retry: + dict_sys->size) > srv_dict_size_limit ) { prev_table = UT_LIST_GET_PREV(table_LRU, table); - if (table == self || table->n_mysql_handles_opened) + if (table == self || table->n_mysql_handles_opened || table->is_corrupt) goto next_loop; cached_foreign_tables = 0; @@ -4219,6 +4220,11 @@ dict_update_statistics_low( } while (index) { + if (table->is_corrupt) { + ut_a(srv_pass_corrupt_table); + return; + } + size = btr_get_size(index, BTR_TOTAL_SIZE); index->stat_index_size = size; @@ -4920,4 +4926,42 @@ dict_close(void) mem_free(dict_sys); dict_sys = NULL; } + +/************************************************************************* +set is_corrupt flag by space_id*/ + +void +dict_table_set_corrupt_by_space( +/*============================*/ + ulint space_id, + ibool need_mutex) +{ + dict_table_t* table; + ibool found = FALSE; + + ut_a(space_id != 0 && space_id < SRV_LOG_SPACE_FIRST_ID); + + if (need_mutex) + mutex_enter(&(dict_sys->mutex)); + + table = UT_LIST_GET_FIRST(dict_sys->table_LRU); + + while (table) { + if (table->space == space_id) { + table->is_corrupt = TRUE; + found = TRUE; + } + + table = UT_LIST_GET_NEXT(table_LRU, table); + } + + if (need_mutex) + mutex_exit(&(dict_sys->mutex)); + + if (!found) { + fprintf(stderr, "InnoDB: space to be marked as " + "crashed was not found for id %lu.\n", + (ulong) space_id); + } +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/xtradb/dict/dict0mem.c' --- a/storage/xtradb/dict/dict0mem.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/dict/dict0mem.c 2010-03-22 20:42:52 +0000 @@ -85,6 +85,8 @@ dict_mem_table_create( /* The number of transactions that are either waiting on the AUTOINC lock or have been granted the lock. */ table->n_waiting_or_granted_auto_inc_locks = 0; + + table->is_corrupt = FALSE; #endif /* !UNIV_HOTBACKUP */ ut_d(table->magic_n = DICT_TABLE_MAGIC_N); === modified file 'storage/xtradb/fil/fil0fil.c' --- a/storage/xtradb/fil/fil0fil.c 2010-03-28 18:10:00 +0000 +++ b/storage/xtradb/fil/fil0fil.c 2010-04-28 14:35:00 +0000 @@ -222,6 +222,7 @@ struct fil_space_struct { file we have written to */ ibool is_in_unflushed_spaces; /*!< TRUE if this space is currently in unflushed_spaces */ + ibool is_corrupt; UT_LIST_NODE_T(fil_space_t) space_list; /*!< list of all spaces */ ulint magic_n;/*!< FIL_SPACE_MAGIC_N */ @@ -1222,6 +1223,8 @@ try_again: ut_fold_string(name), space); space->is_in_unflushed_spaces = FALSE; + space->is_corrupt = FALSE; + UT_LIST_ADD_LAST(space_list, fil_system->space_list, space); mutex_exit(&fil_system->mutex); @@ -3044,7 +3047,9 @@ fil_open_single_table_tablespace( mach_write_ull(page + FIL_PAGE_FILE_FLUSH_LSN, current_lsn); mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, srv_use_checksums - ? buf_calc_page_new_checksum(page) + ? (!srv_fast_checksum + ? buf_calc_page_new_checksum(page) + : buf_calc_page_new_checksum_32(page)) : BUF_NO_CHECKSUM_MAGIC); mach_write_to_4(page + UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM, srv_use_checksums @@ -3156,7 +3161,8 @@ skip_info: goto skip_write; } - if (checksum_field != 0 + if (!srv_fast_checksum + && checksum_field != 0 && checksum_field != BUF_NO_CHECKSUM_MAGIC && checksum_field != buf_calc_page_new_checksum(page)) { @@ -3164,6 +3170,17 @@ skip_info: goto skip_write; } + if (srv_fast_checksum + && checksum_field != 0 + && checksum_field != BUF_NO_CHECKSUM_MAGIC + && checksum_field + != buf_calc_page_new_checksum_32(page) + && checksum_field + != buf_calc_page_new_checksum(page)) { + + goto skip_write; + } + if (mach_read_from_4(page + FIL_PAGE_OFFSET) || !offset) { mach_write_to_4(page + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID, id); @@ -3238,7 +3255,9 @@ skip_info: mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, srv_use_checksums - ? buf_calc_page_new_checksum(page) + ? (!srv_fast_checksum + ? buf_calc_page_new_checksum(page) + : buf_calc_page_new_checksum_32(page)) : BUF_NO_CHECKSUM_MAGIC); mach_write_to_4(page + UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM, srv_use_checksums @@ -4577,9 +4596,9 @@ _fil_io( ut_ad(ut_is_2pow(zip_size)); ut_ad(buf); ut_ad(len > 0); -#if (1 << UNIV_PAGE_SIZE_SHIFT) != UNIV_PAGE_SIZE -# error "(1 << UNIV_PAGE_SIZE_SHIFT) != UNIV_PAGE_SIZE" -#endif +//#if (1 << UNIV_PAGE_SIZE_SHIFT) != UNIV_PAGE_SIZE +//# error "(1 << UNIV_PAGE_SIZE_SHIFT) != UNIV_PAGE_SIZE" +//#endif ut_ad(fil_validate()); #ifndef UNIV_HOTBACKUP # ifndef UNIV_LOG_DEBUG @@ -4713,6 +4732,22 @@ _fil_io( ut_a(byte_offset % OS_FILE_LOG_BLOCK_SIZE == 0); ut_a((len % OS_FILE_LOG_BLOCK_SIZE) == 0); + if (srv_pass_corrupt_table && space->is_corrupt) { + /* should ignore i/o for the crashed space */ + mutex_enter(&fil_system->mutex); + fil_node_complete_io(node, fil_system, type); + mutex_exit(&fil_system->mutex); + if (mode == OS_AIO_NORMAL) { + ut_a(space->purpose == FIL_TABLESPACE); + buf_page_io_complete(message, trx); + } + if (type == OS_FILE_READ) { + return(DB_TABLESPACE_DELETED); + } else { + return(DB_SUCCESS); + } + } else { + ut_a(!space->is_corrupt); #ifdef UNIV_HOTBACKUP /* In ibbackup do normal i/o, not aio */ if (type == OS_FILE_READ) { @@ -4727,6 +4762,8 @@ _fil_io( ret = os_aio(type, mode | wake_later, node->name, node->handle, buf, offset_low, offset_high, len, node, message, trx); #endif + } /**/ + ut_a(ret); if (mode == OS_AIO_SYNC) { @@ -4873,7 +4910,7 @@ fil_aio_wait( if (fil_node->space->purpose == FIL_TABLESPACE) { srv_set_io_thread_op_info(segment, "complete io for buf page"); - buf_page_io_complete(message); + buf_page_io_complete(message, NULL); } else { srv_set_io_thread_op_info(segment, "complete io for log"); log_io_complete(message); @@ -5225,3 +5262,46 @@ fil_system_hash_nodes(void) return 0; } } + +/************************************************************************* +functions to access is_corrupt flag of fil_space_t*/ + +ibool +fil_space_is_corrupt( +/*=================*/ + ulint space_id) +{ + fil_space_t* space; + ibool ret = FALSE; + + mutex_enter(&fil_system->mutex); + + space = fil_space_get_by_id(space_id); + + if (space && space->is_corrupt) { + ret = TRUE; + } + + mutex_exit(&fil_system->mutex); + + return(ret); +} + +void +fil_space_set_corrupt( +/*==================*/ + ulint space_id) +{ + fil_space_t* space; + + mutex_enter(&fil_system->mutex); + + space = fil_space_get_by_id(space_id); + + if (space) { + space->is_corrupt = TRUE; + } + + mutex_exit(&fil_system->mutex); +} + === modified file 'storage/xtradb/fsp/fsp0fsp.c' --- a/storage/xtradb/fsp/fsp0fsp.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/fsp/fsp0fsp.c 2010-03-22 20:42:52 +0000 @@ -370,6 +370,12 @@ fsp_get_space_header( ut_ad(id || !zip_size); block = buf_page_get(id, zip_size, 0, RW_X_LATCH, mtr); + + if (srv_pass_corrupt_table && !block) { + return(0); + } + ut_a(block); + header = FSP_HEADER_OFFSET + buf_block_get_frame(block); buf_block_dbg_add_level(block, SYNC_FSP_PAGE); @@ -652,15 +658,16 @@ xdes_calc_descriptor_page( ulint offset) /*!< in: page offset */ { #ifndef DOXYGEN /* Doxygen gets confused of these */ -# if UNIV_PAGE_SIZE <= XDES_ARR_OFFSET \ - + (UNIV_PAGE_SIZE / FSP_EXTENT_SIZE) * XDES_SIZE -# error -# endif +//# if UNIV_PAGE_SIZE <= XDES_ARR_OFFSET \ +// + (UNIV_PAGE_SIZE / FSP_EXTENT_SIZE) * XDES_SIZE +//# error +//# endif # if PAGE_ZIP_MIN_SIZE <= XDES_ARR_OFFSET \ + (PAGE_ZIP_MIN_SIZE / FSP_EXTENT_SIZE) * XDES_SIZE # error # endif #endif /* !DOXYGEN */ + ut_a(UNIV_PAGE_SIZE > XDES_ARR_OFFSET + (UNIV_PAGE_SIZE / FSP_EXTENT_SIZE) * XDES_SIZE); ut_ad(ut_is_2pow(zip_size)); if (!zip_size) { @@ -788,6 +795,12 @@ xdes_get_descriptor( fsp_header_t* sp_header; block = buf_page_get(space, zip_size, 0, RW_X_LATCH, mtr); + + if (srv_pass_corrupt_table && !block) { + return(0); + } + ut_a(block); + buf_block_dbg_add_level(block, SYNC_FSP_PAGE); sp_header = FSP_HEADER_OFFSET + buf_block_get_frame(block); @@ -1457,12 +1470,12 @@ fsp_fill_free_list( mtr); xdes_init(descr, mtr); -#if UNIV_PAGE_SIZE % FSP_EXTENT_SIZE -# error "UNIV_PAGE_SIZE % FSP_EXTENT_SIZE != 0" -#endif -#if PAGE_ZIP_MIN_SIZE % FSP_EXTENT_SIZE -# error "PAGE_ZIP_MIN_SIZE % FSP_EXTENT_SIZE != 0" -#endif +//#if UNIV_PAGE_SIZE % FSP_EXTENT_SIZE +//# error "UNIV_PAGE_SIZE % FSP_EXTENT_SIZE != 0" +//#endif +//#if PAGE_ZIP_MIN_SIZE % FSP_EXTENT_SIZE +//# error "PAGE_ZIP_MIN_SIZE % FSP_EXTENT_SIZE != 0" +//#endif if (UNIV_UNLIKELY(init_xdes)) { @@ -1871,6 +1884,11 @@ fsp_seg_inode_page_find_free( { fseg_inode_t* inode; + if (srv_pass_corrupt_table && !page) { + return(ULINT_UNDEFINED); + } + ut_a(page); + for (; i < FSP_SEG_INODES_PER_PAGE(zip_size); i++) { inode = fsp_seg_inode_page_get_nth_inode( @@ -1984,6 +2002,11 @@ fsp_alloc_seg_inode( page = buf_block_get_frame(block); + if (srv_pass_corrupt_table && !page) { + return(0); + } + ut_a(page); + n = fsp_seg_inode_page_find_free(page, 0, zip_size, mtr); ut_a(n != ULINT_UNDEFINED); @@ -2077,6 +2100,11 @@ fseg_inode_try_get( inode = fut_get_ptr(space, zip_size, inode_addr, RW_X_LATCH, mtr); + if (srv_pass_corrupt_table && !inode) { + return(0); + } + ut_a(inode); + if (UNIV_UNLIKELY (ut_dulint_is_zero(mach_read_from_8(inode + FSEG_ID)))) { @@ -2104,7 +2132,7 @@ fseg_inode_get( { fseg_inode_t* inode = fseg_inode_try_get(header, space, zip_size, mtr); - ut_a(inode); + ut_a(srv_pass_corrupt_table || inode); return(inode); } @@ -3263,6 +3291,11 @@ fseg_free_page_low( descr = xdes_get_descriptor(space, zip_size, page, mtr); + if (srv_pass_corrupt_table && !descr) { + /* The page may be corrupt. pass it. */ + return; + } + ut_a(descr); if (xdes_get_bit(descr, XDES_FREE_BIT, page % FSP_EXTENT_SIZE, mtr)) { fputs("InnoDB: Dump of the tablespace extent descriptor: ", @@ -3515,6 +3548,11 @@ fseg_free_step( descr = xdes_get_descriptor(space, zip_size, header_page, mtr); + if (srv_pass_corrupt_table && !descr) { + /* The page may be corrupt. pass it. */ + return(TRUE); + } + /* Check that the header resides on a page which has not been freed yet */ @@ -3599,6 +3637,12 @@ fseg_free_step_not_header( inode = fseg_inode_get(header, space, zip_size, mtr); + if (srv_pass_corrupt_table && !inode) { + /* ignore the corruption */ + return(TRUE); + } + ut_a(inode); + descr = fseg_get_first_extent(inode, space, zip_size, mtr); if (descr != NULL) { === modified file 'storage/xtradb/handler/ha_innodb.cc' --- a/storage/xtradb/handler/ha_innodb.cc 2010-04-28 13:53:04 +0000 +++ b/storage/xtradb/handler/ha_innodb.cc 2010-04-28 14:35:00 +0000 @@ -167,6 +167,8 @@ static ulong innobase_commit_concurrency static ulong innobase_read_io_threads; static ulong innobase_write_io_threads; +static ulong innobase_page_size; + static my_bool innobase_thread_concurrency_timer_based; static long long innobase_buffer_pool_size, innobase_log_file_size; @@ -200,6 +202,7 @@ static char* innobase_log_arch_dir = N #endif /* UNIV_LOG_ARCHIVE */ static my_bool innobase_use_doublewrite = TRUE; static my_bool innobase_use_checksums = TRUE; +static my_bool innobase_fast_checksum = FALSE; static my_bool innobase_extra_undoslots = FALSE; static my_bool innobase_fast_recovery = FALSE; static my_bool innobase_recovery_stats = TRUE; @@ -2030,6 +2033,36 @@ innobase_init( } #endif /* UNIV_DEBUG */ + srv_page_size = 0; + srv_page_size_shift = 0; + + if (innobase_page_size != (1 << 14)) { + int n_shift; + + fprintf(stderr, + "InnoDB: Warning: innodb_page_size has been changed from default value 16384. (###EXPERIMENTAL### operation)\n"); + for (n_shift = 12; n_shift <= UNIV_PAGE_SIZE_SHIFT_MAX; n_shift++) { + if (innobase_page_size == (1 << n_shift)) { + srv_page_size_shift = n_shift; + srv_page_size = (1 << srv_page_size_shift); + fprintf(stderr, + "InnoDB: The universal page size of the database is set to %lu.\n", + srv_page_size); + break; + } + } + } else { + srv_page_size_shift = 14; + srv_page_size = (1 << srv_page_size_shift); + } + + if (!srv_page_size_shift) { + fprintf(stderr, + "InnoDB: Error: %lu is not valid value for innodb_page_size.\n", + innobase_page_size); + goto error; + } + #ifndef MYSQL_SERVER innodb_overwrite_relay_log_info = FALSE; #endif @@ -2338,6 +2371,7 @@ innobase_change_buffering_inited_ok: srv_use_doublewrite_buf = (ibool) innobase_use_doublewrite; srv_use_checksums = (ibool) innobase_use_checksums; + srv_fast_checksum = (ibool) innobase_fast_checksum; #ifdef HAVE_LARGE_PAGES if ((os_use_large_pages = (ibool) my_use_large_pages)) @@ -3348,6 +3382,12 @@ ha_innobase::open( DBUG_RETURN(1); } + if (share->ib_table && share->ib_table->is_corrupt) { + free_share(share); + + DBUG_RETURN(HA_ERR_CRASHED_ON_USAGE); + } + /* Create buffers for packing the fields of a record. Why table->reclength did not work here? Obviously, because char fields when packed actually became 1 byte longer, when we also @@ -3375,6 +3415,19 @@ retry: /* Get pointer to a table object in InnoDB dictionary cache */ ib_table = dict_table_get(norm_name, TRUE); + if (ib_table && ib_table->is_corrupt) { + free_share(share); + my_free(upd_buff, MYF(0)); + + DBUG_RETURN(HA_ERR_CRASHED_ON_USAGE); + } + + if (share->ib_table) { + ut_a(share->ib_table == ib_table); + } else { + share->ib_table = ib_table; + } + if (NULL == ib_table) { if (is_part && retries < 10) { ++retries; @@ -4541,6 +4594,10 @@ ha_innobase::write_row( ha_statistic_increment(&SSV::ha_write_count); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + if (table->timestamp_field_type & TIMESTAMP_AUTO_SET_ON_INSERT) table->timestamp_field->set_time(); @@ -4653,6 +4710,10 @@ no_commit: error = row_insert_for_mysql((byte*) record, prebuilt); +#ifdef EXTENDED_FOR_USERSTAT + if (error == DB_SUCCESS) rows_changed++; +#endif + /* Handle duplicate key errors */ if (auto_inc_used) { ulint err; @@ -4748,6 +4809,10 @@ report_error: func_exit: innobase_active_small(); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + DBUG_RETURN(error_result); } @@ -4924,6 +4989,10 @@ ha_innobase::update_row( ha_statistic_increment(&SSV::ha_update_count); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + if (table->timestamp_field_type & TIMESTAMP_AUTO_SET_ON_UPDATE) table->timestamp_field->set_time(); @@ -4989,6 +5058,10 @@ ha_innobase::update_row( } } +#ifdef EXTENDED_FOR_USERSTAT + if (error == DB_SUCCESS) rows_changed++; +#endif + innodb_srv_conc_exit_innodb(trx); error = convert_error_code_to_mysql(error, @@ -5009,6 +5082,10 @@ ha_innobase::update_row( innobase_active_small(); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + DBUG_RETURN(error); } @@ -5030,6 +5107,10 @@ ha_innobase::delete_row( ha_statistic_increment(&SSV::ha_delete_count); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + if (!prebuilt->upd_node) { row_get_prebuilt_update_vector(prebuilt); } @@ -5042,6 +5123,10 @@ ha_innobase::delete_row( error = row_update_for_mysql((byte*) record, prebuilt); +#ifdef EXTENDED_FOR_USERSTAT + if (error == DB_SUCCESS) rows_changed++; +#endif + innodb_srv_conc_exit_innodb(trx); error = convert_error_code_to_mysql( @@ -5052,6 +5137,10 @@ ha_innobase::delete_row( innobase_active_small(); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + DBUG_RETURN(error); } @@ -5291,6 +5380,10 @@ ha_innobase::index_read( ha_statistic_increment(&SSV::ha_read_key_count); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + index = prebuilt->index; if (UNIV_UNLIKELY(index == NULL)) { @@ -5353,6 +5446,10 @@ ha_innobase::index_read( ret = DB_UNSUPPORTED; } + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + switch (ret) { case DB_SUCCESS: error = 0; @@ -5447,6 +5544,10 @@ ha_innobase::change_active_index( { DBUG_ENTER("change_active_index"); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + ut_ad(user_thd == ha_thd()); ut_a(prebuilt->trx == thd_to_trx(user_thd)); @@ -5538,6 +5639,10 @@ ha_innobase::general_fetch( DBUG_ENTER("general_fetch"); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + ut_a(prebuilt->trx == thd_to_trx(user_thd)); innodb_srv_conc_enter_innodb(prebuilt->trx); @@ -5547,10 +5652,19 @@ ha_innobase::general_fetch( innodb_srv_conc_exit_innodb(prebuilt->trx); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + switch (ret) { case DB_SUCCESS: error = 0; table->status = 0; +#ifdef EXTENDED_FOR_USERSTAT + rows_read++; + if (active_index >= 0 && active_index < MAX_KEY) + index_rows_read[active_index]++; +#endif break; case DB_RECORD_NOT_FOUND: error = HA_ERR_END_OF_FILE; @@ -6506,9 +6620,9 @@ ha_innobase::create( | DICT_TF_COMPACT | DICT_TF_FORMAT_ZIP << DICT_TF_FORMAT_SHIFT; -#if DICT_TF_ZSSIZE_MAX < 1 -# error "DICT_TF_ZSSIZE_MAX < 1" -#endif +//#if DICT_TF_ZSSIZE_MAX < 1 +//# error "DICT_TF_ZSSIZE_MAX < 1" +//#endif } } @@ -6768,6 +6882,10 @@ ha_innobase::delete_all_rows(void) DBUG_RETURN(my_errno=HA_ERR_WRONG_COMMAND); } + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + /* Truncate the table in InnoDB */ error = row_truncate_table_for_mysql(prebuilt->table, prebuilt->trx); @@ -6776,6 +6894,10 @@ ha_innobase::delete_all_rows(void) goto fallback; } + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + error = convert_error_code_to_mysql(error, prebuilt->table->flags, NULL); @@ -7279,6 +7401,16 @@ ha_innobase::read_time( return(ranges + (double) rows / (double) total_rows * time_for_scan); } +UNIV_INTERN +bool +ha_innobase::is_corrupt() const +{ + if (share->ib_table) + return ((bool)share->ib_table->is_corrupt); + else + return (FALSE); +} + /*********************************************************************//** Returns statistics information of the table to the MySQL interpreter, in various fields of the handle object. */ @@ -7329,9 +7461,9 @@ ha_innobase::info( ib_table = prebuilt->table; if (flag & HA_STATUS_TIME) { - if (innobase_stats_on_metadata - && (thd_sql_command(user_thd) == SQLCOM_ANALYZE - || srv_stats_auto_update)) { + if ((innobase_stats_on_metadata + || thd_sql_command(user_thd) == SQLCOM_ANALYZE) + && !share->ib_table->is_corrupt) { /* In sql_show we call with this flag: update then statistics so that they are up-to-date */ @@ -7558,6 +7690,10 @@ ha_innobase::analyze( THD* thd, /*!< in: connection thread handle */ HA_CHECK_OPT* check_opt) /*!< in: currently ignored */ { + if (share->ib_table->is_corrupt) { + return(HA_ADMIN_CORRUPT); + } + /* Serialize ANALYZE TABLE inside InnoDB, see Bug#38996 Race condition in ANALYZE TABLE */ pthread_mutex_lock(&analyze_mutex); @@ -7567,6 +7703,10 @@ ha_innobase::analyze( pthread_mutex_unlock(&analyze_mutex); + if (share->ib_table->is_corrupt) { + return(HA_ADMIN_CORRUPT); + } + return(0); } @@ -7612,6 +7752,10 @@ ha_innobase::check( ret = row_check_table_for_mysql(prebuilt); + if (ret != DB_INTERRUPTED && share->ib_table->is_corrupt) { + return(HA_ADMIN_CORRUPT); + } + switch (ret) { case DB_SUCCESS: return(HA_ADMIN_OK); @@ -8345,6 +8489,10 @@ ha_innobase::transactional_table_lock( update_thd(thd); + if (share->ib_table->is_corrupt) { + DBUG_RETURN(HA_ERR_CRASHED); + } + if (prebuilt->table->ibd_file_missing && !thd_tablespace_op(thd)) { ut_print_timestamp(stderr); fprintf(stderr, @@ -10209,6 +10357,20 @@ static MYSQL_SYSVAR_BOOL(checksums, inno "Disable with --skip-innodb-checksums.", NULL, NULL, TRUE); +static MYSQL_SYSVAR_BOOL(fast_checksum, innobase_fast_checksum, + PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, + "Change the algorithm of checksum for the whole of datapage to 4-bytes word based. " + "The original checksum is checked after the new one. It may be slow for reading page" + " which has orginal checksum. Overwrite the page or recreate the InnoDB database, " + "if you want the entire benefit for performance at once. " + "#### Attention: The checksum is not compatible for normal or disabled version! ####", + NULL, NULL, FALSE); + +static MYSQL_SYSVAR_ULONG(page_size, innobase_page_size, + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, + "###EXPERIMENTAL###: The universal page size of the database. Changing for created database is not supported. Use on your own risk!", + NULL, NULL, (1 << 14), (1 << 12), (1 << UNIV_PAGE_SIZE_SHIFT_MAX), 0); + static MYSQL_SYSVAR_STR(data_home_dir, innobase_data_home_dir, PLUGIN_VAR_READONLY, "The common part for InnoDB table spaces.", @@ -10665,11 +10827,21 @@ static MYSQL_SYSVAR_ULONG(relax_table_cr "Relax limitation of column size at table creation as builtin InnoDB.", NULL, NULL, 0, 0, 1, 0); +static MYSQL_SYSVAR_ULONG(pass_corrupt_table, srv_pass_corrupt_table, + PLUGIN_VAR_RQCMDARG, + "Pass corruptions of user tables as 'corrupt table' instead of not crashing itself, " + "when used with file_per_table. " + "All file io for the datafile after detected as corrupt are disabled, " + "except for the deletion.", + NULL, NULL, 0, 0, 1, 0); + static struct st_mysql_sys_var* innobase_system_variables[]= { + MYSQL_SYSVAR(page_size), MYSQL_SYSVAR(additional_mem_pool_size), MYSQL_SYSVAR(autoextend_increment), MYSQL_SYSVAR(buffer_pool_size), MYSQL_SYSVAR(checksums), + MYSQL_SYSVAR(fast_checksum), MYSQL_SYSVAR(commit_concurrency), MYSQL_SYSVAR(concurrency_tickets), MYSQL_SYSVAR(data_file_path), @@ -10744,6 +10916,7 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(io_capacity), MYSQL_SYSVAR(use_purge_thread), MYSQL_SYSVAR(relax_table_creation), + MYSQL_SYSVAR(pass_corrupt_table), NULL }; @@ -10776,6 +10949,8 @@ i_s_innodb_cmpmem_reset, i_s_innodb_table_stats, i_s_innodb_index_stats, i_s_innodb_admin_command, +i_s_innodb_sys_tables, +i_s_innodb_sys_indexes, i_s_innodb_patches mysql_declare_plugin_end; === modified file 'storage/xtradb/handler/ha_innodb.h' --- a/storage/xtradb/handler/ha_innodb.h 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/handler/ha_innodb.h 2010-04-28 14:35:00 +0000 @@ -36,6 +36,7 @@ typedef struct st_innobase_share { incremented in get_share() and decremented in free_share() */ void* table_name_hash;/*!< hash table chain node */ + dict_table_t* ib_table; } INNOBASE_SHARE; @@ -119,6 +120,7 @@ class ha_innobase: public handler int close(void); double scan_time(); double read_time(uint index, uint ranges, ha_rows rows); + bool is_corrupt() const; int write_row(uchar * buf); int update_row(const uchar * old_data, uchar * new_data); === modified file 'storage/xtradb/handler/i_s.cc' --- a/storage/xtradb/handler/i_s.cc 2010-03-31 20:50:54 +0000 +++ b/storage/xtradb/handler/i_s.cc 2010-04-28 14:35:00 +0000 @@ -47,6 +47,7 @@ extern "C" { #include "trx0rseg.h" /* for trx_rseg_struct */ #include "trx0sys.h" /* for trx_sys */ #include "dict0dict.h" /* for dict_sys */ +#include "btr0pcur.h" #include "buf0lru.h" /* for XTRA_LRU_[DUMP/RESTORE] */ /* from buf0buf.c */ struct buf_chunk_struct{ @@ -807,7 +808,7 @@ i_s_innodb_buffer_pool_pages_index_fill( p++; } else { field_store_string(table->field[0], NULL); - p = (char *)index->table_name; + p = index->table_name; } strcpy(table_name_raw, (const char*)p); filename_to_tablename(table_name_raw, table_name, sizeof(table_name)); @@ -2664,6 +2665,14 @@ UNIV_INTERN struct st_mysql_plugin i_s_i */ static ST_FIELD_INFO i_s_innodb_table_stats_info[] = { + {STRUCT_FLD(field_name, "table_schema"), + STRUCT_FLD(field_length, NAME_LEN), + STRUCT_FLD(field_type, MYSQL_TYPE_STRING), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, 0), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + {STRUCT_FLD(field_name, "table_name"), STRUCT_FLD(field_length, NAME_LEN), STRUCT_FLD(field_type, MYSQL_TYPE_STRING), @@ -2709,6 +2718,14 @@ static ST_FIELD_INFO i_s_innodb_table_st static ST_FIELD_INFO i_s_innodb_index_stats_info[] = { + {STRUCT_FLD(field_name, "table_schema"), + STRUCT_FLD(field_length, NAME_LEN), + STRUCT_FLD(field_type, MYSQL_TYPE_STRING), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, 0), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + {STRUCT_FLD(field_name, "table_name"), STRUCT_FLD(field_length, NAME_LEN), STRUCT_FLD(field_type, MYSQL_TYPE_STRING), @@ -2784,16 +2801,30 @@ i_s_innodb_table_stats_fill( table = UT_LIST_GET_FIRST(dict_sys->table_LRU); while (table) { + char buf[NAME_LEN * 2 + 2]; + char* ptr; + if (table->stat_clustered_index_size == 0) { table = UT_LIST_GET_NEXT(table_LRU, table); continue; } - field_store_string(i_s_table->field[0], table->name); - i_s_table->field[1]->store(table->stat_n_rows); - i_s_table->field[2]->store(table->stat_clustered_index_size); - i_s_table->field[3]->store(table->stat_sum_of_other_index_sizes); - i_s_table->field[4]->store(table->stat_modified_counter); + buf[NAME_LEN * 2 + 1] = 0; + strncpy(buf, table->name, NAME_LEN * 2 + 1); + ptr = strchr(buf, '/'); + if (ptr) { + *ptr = '\0'; + ++ptr; + } else { + ptr = buf; + } + + field_store_string(i_s_table->field[0], buf); + field_store_string(i_s_table->field[1], ptr); + i_s_table->field[2]->store(table->stat_n_rows); + i_s_table->field[3]->store(table->stat_clustered_index_size); + i_s_table->field[4]->store(table->stat_sum_of_other_index_sizes); + i_s_table->field[5]->store(table->stat_modified_counter); if (schema_table_store_record(thd, i_s_table)) { status = 1; @@ -2849,11 +2880,24 @@ i_s_innodb_index_stats_fill( while (index) { char buff[256+1]; char row_per_keys[256+1]; + char buf[NAME_LEN * 2 + 2]; + char* ptr; ulint i; - field_store_string(i_s_table->field[0], table->name); - field_store_string(i_s_table->field[1], index->name); - i_s_table->field[2]->store(index->n_uniq); + buf[NAME_LEN * 2 + 1] = 0; + strncpy(buf, table->name, NAME_LEN * 2 + 1); + ptr = strchr(buf, '/'); + if (ptr) { + *ptr = '\0'; + ++ptr; + } else { + ptr = buf; + } + + field_store_string(i_s_table->field[0], buf); + field_store_string(i_s_table->field[1], ptr); + field_store_string(i_s_table->field[2], index->name); + i_s_table->field[3]->store(index->n_uniq); row_per_keys[0] = '\0'; if (index->stat_n_diff_key_vals) { @@ -2869,10 +2913,10 @@ i_s_innodb_index_stats_fill( strncat(row_per_keys, buff, 256 - strlen(row_per_keys)); } } - field_store_string(i_s_table->field[3], row_per_keys); + field_store_string(i_s_table->field[4], row_per_keys); - i_s_table->field[4]->store(index->stat_index_size); - i_s_table->field[5]->store(index->stat_n_leaf_pages); + i_s_table->field[5]->store(index->stat_index_size); + i_s_table->field[6]->store(index->stat_n_leaf_pages); if (schema_table_store_record(thd, i_s_table)) { status = 1; @@ -3121,3 +3165,505 @@ UNIV_INTERN struct st_mysql_plugin i_s_i STRUCT_FLD(system_vars, NULL), STRUCT_FLD(__reserved1, NULL) }; + +static ST_FIELD_INFO i_s_innodb_sys_tables_info[] = +{ + {STRUCT_FLD(field_name, "NAME"), + STRUCT_FLD(field_length, NAME_LEN), + STRUCT_FLD(field_type, MYSQL_TYPE_STRING), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, 0), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "ID"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "N_COLS"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "TYPE"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "MIX_ID"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "MIX_LEN"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "CLUSTER_NAME"), + STRUCT_FLD(field_length, NAME_LEN), + STRUCT_FLD(field_type, MYSQL_TYPE_STRING), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, 0), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "SPACE"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + END_OF_ST_FIELD_INFO +}; + +static ST_FIELD_INFO i_s_innodb_sys_indexes_info[] = +{ + {STRUCT_FLD(field_name, "TABLE_ID"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "ID"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "NAME"), + STRUCT_FLD(field_length, NAME_LEN), + STRUCT_FLD(field_type, MYSQL_TYPE_STRING), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, 0), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "N_FIELDS"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "TYPE"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "SPACE"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + {STRUCT_FLD(field_name, "PAGE_NO"), + STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), + STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, MY_I_S_UNSIGNED), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + END_OF_ST_FIELD_INFO +}; + +static +int +copy_string_field( +/*==============*/ + TABLE* table, + int table_field, + const rec_t* rec, + int rec_field) +{ + int status; + const byte* data; + ulint len; + + /*fprintf(stderr, "copy_string_field %d %d\n", table_field, rec_field);*/ + + data = rec_get_nth_field_old(rec, rec_field, &len); + if (len == UNIV_SQL_NULL) { + table->field[table_field]->set_null(); + status = 0; /* success */ + } else { + table->field[table_field]->set_notnull(); + status = table->field[table_field]->store( + (char *) data, len, system_charset_info); + } + + return status; +} + +static +int +copy_int_field( +/*===========*/ + TABLE* table, + int table_field, + const rec_t* rec, + int rec_field) +{ + int status; + const byte* data; + ulint len; + + /*fprintf(stderr, "copy_int_field %d %d\n", table_field, rec_field);*/ + + data = rec_get_nth_field_old(rec, rec_field, &len); + if (len == UNIV_SQL_NULL) { + table->field[table_field]->set_null(); + status = 0; /* success */ + } else { + table->field[table_field]->set_notnull(); + status = table->field[table_field]->store( + mach_read_from_4(data), true); + } + + return status; +} + +static +int +copy_id_field( +/*==========*/ + TABLE* table, + int table_field, + const rec_t* rec, + int rec_field) +{ + int status; + const byte* data; + ulint len; + + /*fprintf(stderr, "copy_id_field %d %d\n", table_field, rec_field);*/ + + data = rec_get_nth_field_old(rec, rec_field, &len); + if (len == UNIV_SQL_NULL) { + table->field[table_field]->set_null(); + status = 0; /* success */ + } else { + table->field[table_field]->set_notnull(); + status = table->field[table_field]->store( + ut_conv_dulint_to_longlong(mach_read_from_8(data)), true); + } + + return status; +} + +static +int +copy_sys_tables_rec( +/*================*/ + TABLE* table, + const dict_index_t* index, + const rec_t* rec +) +{ + int status; + int field; + + /* NAME */ + field = dict_index_get_nth_col_pos(index, 0); + status = copy_string_field(table, 0, rec, field); + if (status) { + return status; + } + /* ID */ + field = dict_index_get_nth_col_pos(index, 1); + status = copy_id_field(table, 1, rec, field); + if (status) { + return status; + } + /* N_COLS */ + field = dict_index_get_nth_col_pos(index, 2); + status = copy_int_field(table, 2, rec, field); + if (status) { + return status; + } + /* TYPE */ + field = dict_index_get_nth_col_pos(index, 3); + status = copy_int_field(table, 3, rec, field); + if (status) { + return status; + } + /* MIX_ID */ + field = dict_index_get_nth_col_pos(index, 4); + status = copy_id_field(table, 4, rec, field); + if (status) { + return status; + } + /* MIX_LEN */ + field = dict_index_get_nth_col_pos(index, 5); + status = copy_int_field(table, 5, rec, field); + if (status) { + return status; + } + /* CLUSTER_NAME */ + field = dict_index_get_nth_col_pos(index, 6); + status = copy_string_field(table, 6, rec, field); + if (status) { + return status; + } + /* SPACE */ + field = dict_index_get_nth_col_pos(index, 7); + status = copy_int_field(table, 7, rec, field); + if (status) { + return status; + } + + return 0; +} + +static +int +copy_sys_indexes_rec( +/*=================*/ + TABLE* table, + const dict_index_t* index, + const rec_t* rec +) +{ + int status; + int field; + + /* TABLE_ID */ + field = dict_index_get_nth_col_pos(index, 0); + status = copy_id_field(table, 0, rec, field); + if (status) { + return status; + } + /* ID */ + field = dict_index_get_nth_col_pos(index, 1); + status = copy_id_field(table, 1, rec, field); + if (status) { + return status; + } + /* NAME */ + field = dict_index_get_nth_col_pos(index, 2); + status = copy_string_field(table, 2, rec, field); + if (status) { + return status; + } + /* N_FIELDS */ + field = dict_index_get_nth_col_pos(index, 3); + status = copy_int_field(table, 3, rec, field); + if (status) { + return status; + } + /* TYPE */ + field = dict_index_get_nth_col_pos(index, 4); + status = copy_int_field(table, 4, rec, field); + if (status) { + return status; + } + /* SPACE */ + field = dict_index_get_nth_col_pos(index, 5); + status = copy_int_field(table, 5, rec, field); + if (status) { + return status; + } + /* PAGE_NO */ + field = dict_index_get_nth_col_pos(index, 6); + status = copy_int_field(table, 6, rec, field); + if (status) { + return status; + } + + return 0; +} + +static +int +i_s_innodb_schema_table_fill( +/*=========================*/ + THD* thd, + TABLE_LIST* tables, + COND* cond) +{ + int status = 0; + TABLE* table = (TABLE *) tables->table; + const char* table_name = tables->schema_table_name; + dict_table_t* innodb_table; + dict_index_t* index; + btr_pcur_t pcur; + const rec_t* rec; + mtr_t mtr; + int id; + + DBUG_ENTER("i_s_innodb_schema_table_fill"); + + /* deny access to non-superusers */ + if (check_global_access(thd, PROCESS_ACL)) { + DBUG_RETURN(0); + } + + if (innobase_strcasecmp(table_name, "innodb_sys_tables") == 0) { + id = 0; + } else if (innobase_strcasecmp(table_name, "innodb_sys_indexes") == 0) { + id = 1; + } else { + DBUG_RETURN(1); + } + + + RETURN_IF_INNODB_NOT_STARTED(tables->schema_table_name); + + mutex_enter(&(dict_sys->mutex)); + + mtr_start(&mtr); + + if (id == 0) { + innodb_table = dict_table_get_low("SYS_TABLES"); + } else { + innodb_table = dict_table_get_low("SYS_INDEXES"); + } + index = UT_LIST_GET_FIRST(innodb_table->indexes); + + btr_pcur_open_at_index_side(TRUE, index, BTR_SEARCH_LEAF, &pcur, + TRUE, &mtr); + for (;;) { + btr_pcur_move_to_next_user_rec(&pcur, &mtr); + + rec = btr_pcur_get_rec(&pcur); + if (!btr_pcur_is_on_user_rec(&pcur)) { + /* end of index */ + btr_pcur_close(&pcur); + mtr_commit(&mtr); + break; + } + if (rec_get_deleted_flag(rec, 0)) { + /* record marked as deleted */ + btr_pcur_close(&pcur); + mtr_commit(&mtr); + continue; + } + + if (id == 0) { + status = copy_sys_tables_rec(table, index, rec); + } else { + status = copy_sys_indexes_rec(table, index, rec); + } + if (status) { + btr_pcur_close(&pcur); + mtr_commit(&mtr); + break; + } + +#if 0 + btr_pcur_store_position(&pcur, &mtr); + mtr_commit(&mtr); + + status = schema_table_store_record(thd, table); + if (status) { + btr_pcur_close(&pcur); + break; + } + + mtr_start(&mtr); + btr_pcur_restore_position(BTR_SEARCH_LEAF, &pcur, &mtr); +#else + status = schema_table_store_record(thd, table); + if (status) { + btr_pcur_close(&pcur); + mtr_commit(&mtr); + break; + } +#endif + } + + mutex_exit(&(dict_sys->mutex)); + + DBUG_RETURN(status); +} + +static +int +i_s_innodb_sys_tables_init( +/*=======================*/ + void* p) +{ + DBUG_ENTER("i_s_innodb_sys_tables_init"); + ST_SCHEMA_TABLE* schema = (ST_SCHEMA_TABLE*) p; + + schema->fields_info = i_s_innodb_sys_tables_info; + schema->fill_table = i_s_innodb_schema_table_fill; + + DBUG_RETURN(0); +} + +static +int +i_s_innodb_sys_indexes_init( +/*========================*/ + void* p) +{ + DBUG_ENTER("i_s_innodb_sys_indexes_init"); + ST_SCHEMA_TABLE* schema = (ST_SCHEMA_TABLE*) p; + + schema->fields_info = i_s_innodb_sys_indexes_info; + schema->fill_table = i_s_innodb_schema_table_fill; + + DBUG_RETURN(0); +} + +UNIV_INTERN struct st_mysql_plugin i_s_innodb_sys_tables = +{ + STRUCT_FLD(type, MYSQL_INFORMATION_SCHEMA_PLUGIN), + STRUCT_FLD(info, &i_s_info), + STRUCT_FLD(name, "INNODB_SYS_TABLES"), + STRUCT_FLD(author, plugin_author), + STRUCT_FLD(descr, "InnoDB SYS_TABLES table"), + STRUCT_FLD(license, PLUGIN_LICENSE_GPL), + STRUCT_FLD(init, i_s_innodb_sys_tables_init), + STRUCT_FLD(deinit, i_s_common_deinit), + STRUCT_FLD(version, 0x0100 /* 1.0 */), + STRUCT_FLD(status_vars, NULL), + STRUCT_FLD(system_vars, NULL), + STRUCT_FLD(__reserved1, NULL) +}; + +UNIV_INTERN struct st_mysql_plugin i_s_innodb_sys_indexes = +{ + STRUCT_FLD(type, MYSQL_INFORMATION_SCHEMA_PLUGIN), + STRUCT_FLD(info, &i_s_info), + STRUCT_FLD(name, "INNODB_SYS_INDEXES"), + STRUCT_FLD(author, plugin_author), + STRUCT_FLD(descr, "InnoDB SYS_INDEXES table"), + STRUCT_FLD(license, PLUGIN_LICENSE_GPL), + STRUCT_FLD(init, i_s_innodb_sys_indexes_init), + STRUCT_FLD(deinit, i_s_common_deinit), + STRUCT_FLD(version, 0x0100 /* 1.0 */), + STRUCT_FLD(status_vars, NULL), + STRUCT_FLD(system_vars, NULL), + STRUCT_FLD(__reserved1, NULL) +}; === modified file 'storage/xtradb/handler/i_s.h' --- a/storage/xtradb/handler/i_s.h 2009-11-04 20:11:12 +0000 +++ b/storage/xtradb/handler/i_s.h 2010-03-22 20:42:52 +0000 @@ -41,5 +41,7 @@ extern struct st_mysql_plugin i_s_innodb extern struct st_mysql_plugin i_s_innodb_table_stats; extern struct st_mysql_plugin i_s_innodb_index_stats; extern struct st_mysql_plugin i_s_innodb_admin_command; +extern struct st_mysql_plugin i_s_innodb_sys_tables; +extern struct st_mysql_plugin i_s_innodb_sys_indexes; #endif /* i_s_h */ === modified file 'storage/xtradb/handler/innodb_patch_info.h' --- a/storage/xtradb/handler/innodb_patch_info.h 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/handler/innodb_patch_info.h 2010-04-28 14:35:00 +0000 @@ -43,5 +43,9 @@ struct innodb_enhancement { {"innodb_extend_slow","Extended statistics in slow.log","It is InnoDB-part only. It needs to patch also to mysqld.","http://www.percona.com/docs/wiki/percona-xtradb"}, {"innodb_relax_table_creation","Relax limitation of column size at table creation as builtin InnoDB.","","http://www.percona.com/docs/wiki/percona-xtradb"}, {"innodb_lru_dump_restore","Dump and restore command for content of buffer pool","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_pass_corrupt_table","Treat tables as corrupt instead of crash, when meet corrupt blocks","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_fast_checksum","Using the checksum on 32bit-unit calculation","incompatible for unpatched ver.","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_files_extend","allow >4GB transaction log files, and can vary universal page size of datafiles","incompatible for unpatched ver.","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_sys_tables_sys_indexes","Expose InnoDB SYS_TABLES and SYS_INDEXES schema tables","","http://www.percona.com/docs/wiki/percona-xtradb"}, {NULL, NULL, NULL, NULL} }; === modified file 'storage/xtradb/include/btr0btr.ic' --- a/storage/xtradb/include/btr0btr.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/btr0btr.ic 2010-03-22 20:42:52 +0000 @@ -28,7 +28,7 @@ Created 6/2/1994 Heikki Tuuri #include "mtr0mtr.h" #include "mtr0log.h" #include "page0zip.h" - +#include "srv0srv.h" #define BTR_MAX_NODE_LEVEL 50 /*!< Maximum B-tree page level (not really a hard limit). Used in debug assertions @@ -52,7 +52,9 @@ btr_block_get( block = buf_page_get(space, zip_size, page_no, mode, mtr); - if (mode != RW_NO_LATCH) { + ut_a(srv_pass_corrupt_table || block); + + if (block && mode != RW_NO_LATCH) { buf_block_dbg_add_level(block, SYNC_TREE_NODE); } === modified file 'storage/xtradb/include/buf0buddy.h' --- a/storage/xtradb/include/buf0buddy.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0buddy.h 2010-03-22 20:42:52 +0000 @@ -83,7 +83,7 @@ typedef struct buf_buddy_stat_struct buf /** Statistics of the buddy system, indexed by block size. Protected by buf_pool_mutex. */ -extern buf_buddy_stat_t buf_buddy_stat[BUF_BUDDY_SIZES + 1]; +extern buf_buddy_stat_t buf_buddy_stat[BUF_BUDDY_SIZES_MAX + 1]; #ifndef UNIV_NONINL # include "buf0buddy.ic" === modified file 'storage/xtradb/include/buf0buf.h' --- a/storage/xtradb/include/buf0buf.h 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/include/buf0buf.h 2010-03-22 20:42:52 +0000 @@ -482,6 +482,11 @@ ulint buf_calc_page_new_checksum( /*=======================*/ const byte* page); /*!< in: buffer page */ +UNIV_INTERN +ulint +buf_calc_page_new_checksum_32( +/*==========================*/ + const byte* page); /*!< in: buffer page */ /********************************************************************//** In versions < 4.0.14 and < 4.1.1 there was a bug that the checksum only looked at the first few bytes of the page. This calculates that old @@ -855,7 +860,7 @@ buf_block_get_frame( const buf_block_t* block) /*!< in: pointer to the control block */ __attribute__((pure)); #else /* UNIV_DEBUG */ -# define buf_block_get_frame(block) (block)->frame +# define buf_block_get_frame(block) (block ? (block)->frame : 0) #endif /* UNIV_DEBUG */ /*********************************************************************//** Gets the space id of a block. @@ -987,7 +992,8 @@ UNIV_INTERN void buf_page_io_complete( /*=================*/ - buf_page_t* bpage); /*!< in: pointer to the block in question */ + buf_page_t* bpage, /*!< in: pointer to the block in question */ + trx_t* trx); /********************************************************************//** Calculates a folded value of a file page address to use in the page hash table. @@ -1155,6 +1161,7 @@ struct buf_page_struct{ 0 if the block was never accessed in the buffer pool */ /* @} */ + ibool is_corrupt; # ifdef UNIV_DEBUG_FILE_ACCESSES ibool file_page_was_freed; /*!< this is set to TRUE when fsp @@ -1422,11 +1429,11 @@ struct buf_pool_struct{ /* @{ */ UT_LIST_BASE_NODE_T(buf_page_t) zip_clean; /*!< unmodified compressed pages */ - UT_LIST_BASE_NODE_T(buf_page_t) zip_free[BUF_BUDDY_SIZES]; + UT_LIST_BASE_NODE_T(buf_page_t) zip_free[BUF_BUDDY_SIZES_MAX]; /*!< buddy free lists */ -#if BUF_BUDDY_HIGH != UNIV_PAGE_SIZE -# error "BUF_BUDDY_HIGH != UNIV_PAGE_SIZE" -#endif +//#if BUF_BUDDY_HIGH != UNIV_PAGE_SIZE +//# error "BUF_BUDDY_HIGH != UNIV_PAGE_SIZE" +//#endif #if BUF_BUDDY_LOW > PAGE_ZIP_MIN_SIZE # error "BUF_BUDDY_LOW > PAGE_ZIP_MIN_SIZE" #endif === modified file 'storage/xtradb/include/buf0buf.ic' --- a/storage/xtradb/include/buf0buf.ic 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/include/buf0buf.ic 2010-04-28 14:35:00 +0000 @@ -35,7 +35,7 @@ Created 11/5/1995 Heikki Tuuri #include "buf0flu.h" #include "buf0lru.h" #include "buf0rea.h" - +#include "srv0srv.h" /********************************************************************//** Reads the freed_page_clock of a buffer block. @return freed_page_clock */ @@ -581,6 +581,12 @@ buf_block_get_frame( /*================*/ const buf_block_t* block) /*!< in: pointer to the control block */ { + ut_a(srv_pass_corrupt_table || block); + + if (srv_pass_corrupt_table && !block) { + return(0); + } + ut_ad(block); switch (buf_block_get_state(block)) { === modified file 'storage/xtradb/include/buf0types.h' --- a/storage/xtradb/include/buf0types.h 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/include/buf0types.h 2010-03-22 20:42:52 +0000 @@ -70,6 +70,7 @@ enum buf_io_fix { buddy system; must be at least sizeof(buf_page_t) */ #define BUF_BUDDY_SIZES (UNIV_PAGE_SIZE_SHIFT - BUF_BUDDY_LOW_SHIFT) +#define BUF_BUDDY_SIZES_MAX (UNIV_PAGE_SIZE_SHIFT_MAX - BUF_BUDDY_LOW_SHIFT) /*!< number of buddy sizes */ /** twice the maximum block size of the buddy system; === modified file 'storage/xtradb/include/dict0dict.h' --- a/storage/xtradb/include/dict0dict.h 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/include/dict0dict.h 2010-03-22 20:42:52 +0000 @@ -1164,6 +1164,15 @@ void dict_close(void); /*============*/ +/************************************************************************* +set is_corrupt flag by space_id*/ + +void +dict_table_set_corrupt_by_space( +/*============================*/ + ulint space_id, + ibool need_mutex); + #ifndef UNIV_NONINL #include "dict0dict.ic" #endif === modified file 'storage/xtradb/include/dict0mem.h' --- a/storage/xtradb/include/dict0mem.h 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/include/dict0mem.h 2010-03-22 20:42:52 +0000 @@ -521,6 +521,7 @@ struct dict_table_struct{ the AUTOINC lock on this table. */ /* @} */ /*----------------------*/ + ibool is_corrupt; #endif /* !UNIV_HOTBACKUP */ #ifdef UNIV_DEBUG === modified file 'storage/xtradb/include/fil0fil.h' --- a/storage/xtradb/include/fil0fil.h 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/include/fil0fil.h 2010-03-22 20:42:52 +0000 @@ -116,6 +116,7 @@ extern fil_addr_t fil_addr_null; #define FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID 34 /*!< starting from 4.1.x this contains the space id of the page */ #define FIL_PAGE_DATA 38 /*!< start of the data on the page */ +#define FIL_PAGE_DATA_ALIGN_32 40 /* @} */ /** File page trailer @{ */ #define FIL_PAGE_END_LSN_OLD_CHKSUM 8 /*!< the low 4 bytes of this are used @@ -748,6 +749,19 @@ ulint fil_system_hash_nodes(void); /*========================*/ +/************************************************************************* +functions to access is_corrupt flag of fil_space_t*/ + +ibool +fil_space_is_corrupt( +/*=================*/ + ulint space_id); + +void +fil_space_set_corrupt( +/*==================*/ + ulint space_id); + typedef struct fil_space_struct fil_space_t; #endif === modified file 'storage/xtradb/include/fut0fut.ic' --- a/storage/xtradb/include/fut0fut.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/fut0fut.ic 2010-03-22 20:42:52 +0000 @@ -23,6 +23,7 @@ File-based utilities Created 12/13/1995 Heikki Tuuri ***********************************************************************/ +#include "srv0srv.h" #include "sync0rw.h" #include "buf0buf.h" @@ -48,6 +49,12 @@ fut_get_ptr( ut_ad((rw_latch == RW_S_LATCH) || (rw_latch == RW_X_LATCH)); block = buf_page_get(space, zip_size, addr.page, rw_latch, mtr); + + if (srv_pass_corrupt_table && !block) { + return(0); + } + ut_a(block); + ptr = buf_block_get_frame(block) + addr.boffset; buf_block_dbg_add_level(block, SYNC_NO_ORDER_CHECK); === modified file 'storage/xtradb/include/page0cur.h' --- a/storage/xtradb/include/page0cur.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0cur.h 2010-03-22 20:42:52 +0000 @@ -293,6 +293,22 @@ page_cur_open_on_rnd_user_rec( /*==========================*/ buf_block_t* block, /*!< in: page */ page_cur_t* cursor);/*!< out: page cursor */ + +UNIV_INTERN +void +page_cur_open_on_nth_user_rec( +/*==========================*/ + buf_block_t* block, /*!< in: page */ + page_cur_t* cursor, /*!< out: page cursor */ + ulint nth); + +UNIV_INTERN +ibool +page_cur_open_on_rnd_user_rec_after_nth( +/*==========================*/ + buf_block_t* block, /*!< in: page */ + page_cur_t* cursor, /*!< out: page cursor */ + ulint nth); #endif /* !UNIV_HOTBACKUP */ /***********************************************************//** Parses a log record of a record insert on a page. === modified file 'storage/xtradb/include/page0types.h' --- a/storage/xtradb/include/page0types.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0types.h 2010-03-22 20:42:52 +0000 @@ -56,8 +56,9 @@ page0*.h includes rem0rec.h and may incl /** Number of supported compressed page sizes */ #define PAGE_ZIP_NUM_SSIZE (UNIV_PAGE_SIZE_SHIFT - PAGE_ZIP_MIN_SIZE_SHIFT + 2) -#if PAGE_ZIP_NUM_SSIZE > (1 << PAGE_ZIP_SSIZE_BITS) -# error "PAGE_ZIP_NUM_SSIZE > (1 << PAGE_ZIP_SSIZE_BITS)" +#define PAGE_ZIP_NUM_SSIZE_MAX (UNIV_PAGE_SIZE_SHIFT_MAX - PAGE_ZIP_MIN_SIZE_SHIFT + 2) +#if PAGE_ZIP_NUM_SSIZE_MAX > (1 << PAGE_ZIP_SSIZE_BITS) +# error "PAGE_ZIP_NUM_SSIZE_MAX > (1 << PAGE_ZIP_SSIZE_BITS)" #endif /** Compressed page descriptor */ @@ -98,7 +99,7 @@ struct page_zip_stat_struct { typedef struct page_zip_stat_struct page_zip_stat_t; /** Statistics on compression, indexed by page_zip_des_struct::ssize - 1 */ -extern page_zip_stat_t page_zip_stat[PAGE_ZIP_NUM_SSIZE - 1]; +extern page_zip_stat_t page_zip_stat[PAGE_ZIP_NUM_SSIZE_MAX - 1]; /**********************************************************************//** Write the "deleted" flag of a record on a compressed page. The flag must === modified file 'storage/xtradb/include/srv0srv.h' --- a/storage/xtradb/include/srv0srv.h 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/include/srv0srv.h 2010-04-28 14:35:00 +0000 @@ -227,6 +227,7 @@ extern ulint srv_stats_update_need_lock; extern ibool srv_use_doublewrite_buf; extern ibool srv_use_checksums; +extern ibool srv_fast_checksum; extern ibool srv_set_thread_priorities; extern int srv_query_thread_priority; @@ -247,6 +248,7 @@ extern ulong srv_adaptive_checkpoint; extern ulong srv_expand_import; extern ulint srv_relax_table_creation; +extern ulint srv_pass_corrupt_table; extern ulong srv_extra_rsegments; extern ulong srv_dict_size_limit; === modified file 'storage/xtradb/include/trx0sys.h' --- a/storage/xtradb/include/trx0sys.h 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/include/trx0sys.h 2010-03-22 20:42:52 +0000 @@ -487,9 +487,9 @@ in size */ /** Contents of TRX_SYS_MYSQL_LOG_MAGIC_N_FLD */ #define TRX_SYS_MYSQL_LOG_MAGIC_N 873422344 -#if UNIV_PAGE_SIZE < 4096 -# error "UNIV_PAGE_SIZE < 4096" -#endif +//#if UNIV_PAGE_SIZE < 4096 +//# error "UNIV_PAGE_SIZE < 4096" +//#endif /** The offset of the MySQL replication info in the trx system header; this contains the same fields as TRX_SYS_MYSQL_LOG_INFO below */ #define TRX_SYS_MYSQL_MASTER_LOG_INFO (UNIV_PAGE_SIZE - 2000) === modified file 'storage/xtradb/include/univ.i' --- a/storage/xtradb/include/univ.i 2010-04-28 13:53:04 +0000 +++ b/storage/xtradb/include/univ.i 2010-04-28 14:35:00 +0000 @@ -47,7 +47,7 @@ Created 1/20/1994 Heikki Tuuri #define INNODB_VERSION_MAJOR 1 #define INNODB_VERSION_MINOR 0 #define INNODB_VERSION_BUGFIX 6 -#define PERCONA_INNODB_VERSION 9.1 +#define PERCONA_INNODB_VERSION 10 /* The following is the InnoDB version as shown in SELECT plugin_version FROM information_schema.plugins; @@ -288,9 +288,13 @@ management to ensure correct alignment f */ /* The 2-logarithm of UNIV_PAGE_SIZE: */ -#define UNIV_PAGE_SIZE_SHIFT 14 +/* #define UNIV_PAGE_SIZE_SHIFT 14 */ +#define UNIV_PAGE_SIZE_SHIFT_MAX 14 +#define UNIV_PAGE_SIZE_SHIFT srv_page_size_shift /* The universal page size of the database */ -#define UNIV_PAGE_SIZE (1u << UNIV_PAGE_SIZE_SHIFT) +/* #define UNIV_PAGE_SIZE (1u << UNIV_PAGE_SIZE_SHIFT) */ +#define UNIV_PAGE_SIZE srv_page_size +#define UNIV_PAGE_SIZE_MAX (1u << UNIV_PAGE_SIZE_SHIFT_MAX) /* Maximum number of parallel threads in a parallelized operation */ #define UNIV_MAX_PARALLELISM 32 @@ -383,7 +387,7 @@ number indicate that a field contains a stored part of the field in the tablespace. The length field then contains the sum of the following flag and the locally stored len. */ -#define UNIV_EXTERN_STORAGE_FIELD (UNIV_SQL_NULL - UNIV_PAGE_SIZE) +#define UNIV_EXTERN_STORAGE_FIELD (UNIV_SQL_NULL - UNIV_PAGE_SIZE_MAX) /* Some macros to improve branch prediction and reduce cache misses */ #if defined(__GNUC__) && (__GNUC__ > 2) && ! defined(__INTEL_COMPILER) @@ -486,4 +490,6 @@ typedef void* os_thread_ret_t; UNIV_MEM_ALLOC(addr, size); \ } while (0) +extern ulint srv_page_size_shift; +extern ulint srv_page_size; #endif === modified file 'storage/xtradb/include/ut0rnd.h' --- a/storage/xtradb/include/ut0rnd.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0rnd.h 2010-03-22 20:42:52 +0000 @@ -124,6 +124,13 @@ ut_fold_binary( const byte* str, /*!< in: string of bytes */ ulint len) /*!< in: length */ __attribute__((pure)); +UNIV_INLINE +ulint +ut_fold_binary_32( +/*==============*/ + const byte* str, /*!< in: string of bytes */ + ulint len) /*!< in: length */ + __attribute__((pure)); /***********************************************************//** Looks for a prime number slightly greater than the given argument. The prime is chosen so that it is not near any power of 2. === modified file 'storage/xtradb/include/ut0rnd.ic' --- a/storage/xtradb/include/ut0rnd.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0rnd.ic 2010-03-22 20:42:52 +0000 @@ -228,3 +228,28 @@ ut_fold_binary( return(fold); } + +UNIV_INLINE +ulint +ut_fold_binary_32( +/*==============*/ + const byte* str, /*!< in: string of bytes */ + ulint len) /*!< in: length */ +{ + const ib_uint32_t* str_end = (const ib_uint32_t*) (str + len); + const ib_uint32_t* str_32 = (const ib_uint32_t*) str; + ulint fold = 0; + + ut_ad(str); + /* This function is only for word-aligned data */ + ut_ad(len % 4 == 0); + ut_ad((ulint)str % 4 == 0); + + while (str_32 < str_end) { + fold = ut_fold_ulint_pair(fold, (ulint)(*str_32)); + + str_32++; + } + + return(fold); +} === modified file 'storage/xtradb/lock/lock0lock.c' --- a/storage/xtradb/lock/lock0lock.c 2010-03-18 15:25:47 +0000 +++ b/storage/xtradb/lock/lock0lock.c 2010-03-22 20:42:52 +0000 @@ -3364,26 +3364,23 @@ lock_deadlock_recursive( bit_no = lock_rec_find_set_bit(wait_lock); ut_a(bit_no != ULINT_UNDEFINED); - - /* get the starting point for the search for row level locks - since we are scanning from the front of the list */ - lock = lock_rec_get_first_on_page_addr(wait_lock->un_member.rec_lock.space, - wait_lock->un_member.rec_lock.page_no); - } - else { - /* table level locks use a two-way linked list so scanning backwards is OK */ - lock = UT_LIST_GET_PREV(un_member.tab_lock.locks, - lock); - } + } /* Look at the locks ahead of wait_lock in the lock queue */ for (;;) { + if (lock_get_type_low(lock) & LOCK_TABLE) { + lock = UT_LIST_GET_PREV(un_member.tab_lock.locks, + lock); + } else { + ut_ad(lock_get_type_low(lock) == LOCK_REC); + ut_a(bit_no != ULINT_UNDEFINED); + + lock = (lock_t*) lock_rec_get_prev(lock, bit_no); + } - /* reached the original lock in the queue for row level locks - or past beginning of the list for table level locks */ - if (lock == NULL || lock == wait_lock) { + if (lock == NULL) { /* We can mark this subtree as searched */ trx->deadlock_mark = 1; @@ -3508,17 +3505,6 @@ lock_deadlock_recursive( } } } - - /* next lock to check */ - if (lock_get_type_low(lock) & LOCK_TABLE) { - lock = UT_LIST_GET_PREV(un_member.tab_lock.locks, - lock); - } else { - ut_ad(lock_get_type_low(lock) == LOCK_REC); - ut_a(bit_no != ULINT_UNDEFINED); - - lock = (lock_t*) lock_rec_get_next(bit_no, lock); - } }/* end of the 'for (;;)'-loop */ } === modified file 'storage/xtradb/log/log0log.c' --- a/storage/xtradb/log/log0log.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/log/log0log.c 2010-03-22 20:42:52 +0000 @@ -608,7 +608,9 @@ log_group_calc_lsn_offset( offset = (gr_lsn_size_offset + difference) % group_size; + if (sizeof(ulint) == 4) { ut_a(offset < (((ib_int64_t) 1) << 32)); /* offset must be < 4 GB */ + } /* fprintf(stderr, "Offset is %lu gr_lsn_offset is %lu difference is %lu\n", @@ -1805,6 +1807,7 @@ log_group_checkpoint( mach_write_to_4(buf + LOG_CHECKPOINT_LOG_BUF_SIZE, log_sys->buf_size); #ifdef UNIV_LOG_ARCHIVE +#error "UNIV_LOG_ARCHIVE could not be enabled" if (log_sys->archiving_state == LOG_ARCH_OFF) { archived_lsn = IB_ULONGLONG_MAX; } else { @@ -1818,7 +1821,9 @@ log_group_checkpoint( mach_write_ull(buf + LOG_CHECKPOINT_ARCHIVED_LSN, archived_lsn); #else /* UNIV_LOG_ARCHIVE */ - mach_write_ull(buf + LOG_CHECKPOINT_ARCHIVED_LSN, IB_ULONGLONG_MAX); + mach_write_ull(buf + LOG_CHECKPOINT_ARCHIVED_LSN, + (ib_uint64_t)log_group_calc_lsn_offset( + log_sys->next_checkpoint_lsn, group)); #endif /* UNIV_LOG_ARCHIVE */ for (i = 0; i < LOG_MAX_N_GROUPS; i++) { === modified file 'storage/xtradb/log/log0recv.c' --- a/storage/xtradb/log/log0recv.c 2010-03-31 20:50:54 +0000 +++ b/storage/xtradb/log/log0recv.c 2010-04-28 14:35:00 +0000 @@ -680,8 +680,22 @@ recv_find_max_checkpoint( group->lsn = mach_read_ull( buf + LOG_CHECKPOINT_LSN); + +#ifdef UNIV_LOG_ARCHIVE +#error "UNIV_LOG_ARCHIVE could not be enabled" +#endif + { + ib_uint64_t tmp_lsn_offset = mach_read_ull( + buf + LOG_CHECKPOINT_ARCHIVED_LSN); + if (sizeof(ulint) != 4 + && tmp_lsn_offset != IB_ULONGLONG_MAX) { + group->lsn_offset = (ulint) tmp_lsn_offset; + } else { group->lsn_offset = mach_read_from_4( buf + LOG_CHECKPOINT_OFFSET); + } + } + checkpoint_no = mach_read_ull( buf + LOG_CHECKPOINT_NO); === modified file 'storage/xtradb/page/page0cur.c' --- a/storage/xtradb/page/page0cur.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/page/page0cur.c 2010-03-22 20:42:52 +0000 @@ -564,6 +564,74 @@ page_cur_open_on_rnd_user_rec( } while (rnd--); } +UNIV_INTERN +void +page_cur_open_on_nth_user_rec( +/*==========================*/ + buf_block_t* block, /*!< in: page */ + page_cur_t* cursor, /*!< out: page cursor */ + ulint nth) +{ + ulint n_recs = page_get_n_recs(buf_block_get_frame(block)); + + page_cur_set_before_first(block, cursor); + + if (UNIV_UNLIKELY(n_recs == 0)) { + + return; + } + + nth--; + + if (nth >= n_recs) { + nth = n_recs - 1; + } + + do { + page_cur_move_to_next(cursor); + } while (nth--); +} + +UNIV_INTERN +ibool +page_cur_open_on_rnd_user_rec_after_nth( +/*==========================*/ + buf_block_t* block, /*!< in: page */ + page_cur_t* cursor, /*!< out: page cursor */ + ulint nth) +{ + ulint rnd; + ulint n_recs = page_get_n_recs(buf_block_get_frame(block)); + ibool ret; + + page_cur_set_before_first(block, cursor); + + if (UNIV_UNLIKELY(n_recs == 0)) { + + return (FALSE); + } + + nth--; + + if (nth >= n_recs) { + nth = n_recs - 1; + } + + rnd = (ulint) (nth + page_cur_lcg_prng() % (n_recs - nth)); + + if (rnd == nth) { + ret = TRUE; + } else { + ret = FALSE; + } + + do { + page_cur_move_to_next(cursor); + } while (rnd--); + + return (ret); +} + /***********************************************************//** Writes the log record of a record insert on a page. */ static === modified file 'storage/xtradb/page/page0zip.c' --- a/storage/xtradb/page/page0zip.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/page/page0zip.c 2010-03-22 20:42:52 +0000 @@ -49,7 +49,7 @@ Created June 2005 by Marko Makela #ifndef UNIV_HOTBACKUP /** Statistics on compression, indexed by page_zip_des_t::ssize - 1 */ -UNIV_INTERN page_zip_stat_t page_zip_stat[PAGE_ZIP_NUM_SSIZE - 1]; +UNIV_INTERN page_zip_stat_t page_zip_stat[PAGE_ZIP_NUM_SSIZE_MAX - 1]; #endif /* !UNIV_HOTBACKUP */ /* Please refer to ../include/page0zip.ic for a description of the === modified file 'storage/xtradb/row/row0ins.c' --- a/storage/xtradb/row/row0ins.c 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/row/row0ins.c 2010-04-28 14:35:00 +0000 @@ -1340,6 +1340,12 @@ run_again: const rec_t* rec = btr_pcur_get_rec(&pcur); const buf_block_t* block = btr_pcur_get_block(&pcur); + if (srv_pass_corrupt_table && !block) { + err = DB_CORRUPTION; + break; + } + ut_a(block); + if (page_rec_is_infimum(rec)) { goto next_rec; === modified file 'storage/xtradb/row/row0merge.c' --- a/storage/xtradb/row/row0merge.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/row/row0merge.c 2010-03-22 20:42:52 +0000 @@ -92,7 +92,7 @@ This buffer is used for writing or readi row_merge_block_t. Thus, it must be able to hold one merge record, whose maximum size is the same as the minimum size of row_merge_block_t. */ -typedef byte mrec_buf_t[UNIV_PAGE_SIZE]; +typedef byte mrec_buf_t[UNIV_PAGE_SIZE_MAX]; /** @brief Merge record in row_merge_block_t. @@ -1216,6 +1216,13 @@ row_merge_read_clustered_index( if (UNIV_LIKELY(has_next)) { rec = btr_pcur_get_rec(&pcur); + + if (srv_pass_corrupt_table && !rec) { + err = DB_CORRUPTION; + goto err_exit; + } + ut_a(rec); + offsets = rec_get_offsets(rec, clust_index, NULL, ULINT_UNDEFINED, &row_heap); === modified file 'storage/xtradb/row/row0sel.c' --- a/storage/xtradb/row/row0sel.c 2010-01-15 21:12:30 +0000 +++ b/storage/xtradb/row/row0sel.c 2010-04-28 14:35:00 +0000 @@ -3766,6 +3766,13 @@ rec_loop: /* PHASE 4: Look for matching records in a loop */ rec = btr_pcur_get_rec(pcur); + + if (srv_pass_corrupt_table && !rec) { + err = DB_CORRUPTION; + goto lock_wait_or_error; + } + ut_a(rec); + ut_ad(!!page_rec_is_comp(rec) == comp); #ifdef UNIV_SEARCH_DEBUG /* === modified file 'storage/xtradb/srv/srv0srv.c' --- a/storage/xtradb/srv/srv0srv.c 2010-01-15 19:48:33 +0000 +++ b/storage/xtradb/srv/srv0srv.c 2010-04-28 14:35:00 +0000 @@ -224,6 +224,10 @@ UNIV_INTERN ulint srv_n_file_io_threads UNIV_INTERN ulint srv_n_read_io_threads = ULINT_MAX; UNIV_INTERN ulint srv_n_write_io_threads = ULINT_MAX; +/* The universal page size of the database */ +UNIV_INTERN ulint srv_page_size_shift = 0; +UNIV_INTERN ulint srv_page_size = 0; + /* User settable value of the number of pages that must be present in the buffer cache and accessed sequentially for InnoDB to trigger a readahead request. */ @@ -386,6 +390,7 @@ UNIV_INTERN ulint srv_stats_update_need_ UNIV_INTERN ibool srv_use_doublewrite_buf = TRUE; UNIV_INTERN ibool srv_use_checksums = TRUE; +UNIV_INTERN ibool srv_fast_checksum = FALSE; UNIV_INTERN ibool srv_set_thread_priorities = TRUE; UNIV_INTERN int srv_query_thread_priority = 0; @@ -406,6 +411,7 @@ UNIV_INTERN ulong srv_adaptive_checkpoin UNIV_INTERN ulong srv_expand_import = 0; /* 0:disable 1:enable */ UNIV_INTERN ulint srv_relax_table_creation = 0; /* 0:disable 1:enable */ +UNIV_INTERN ulint srv_pass_corrupt_table = 0; /* 0:disable 1:enable */ UNIV_INTERN ulong srv_extra_rsegments = 0; /* extra rseg for users */ UNIV_INTERN ulong srv_dict_size_limit = 0; @@ -2559,6 +2565,7 @@ srv_master_thread( srv_main_thread_process_no = os_proc_get_number(); srv_main_thread_id = os_thread_pf(os_thread_get_curr_id()); + mutex_enter(&kernel_mutex); srv_table_reserve_slot(SRV_MASTER); === modified file 'storage/xtradb/srv/srv0start.c' --- a/storage/xtradb/srv/srv0start.c 2010-01-17 08:41:43 +0000 +++ b/storage/xtradb/srv/srv0start.c 2010-04-28 14:35:00 +0000 @@ -1352,10 +1352,12 @@ innobase_start_or_create_for_mysql(void) } #endif /* UNIV_LOG_ARCHIVE */ - if (srv_n_log_files * srv_log_file_size >= 262144) { + if (sizeof(ulint) == 4 + && srv_n_log_files * srv_log_file_size + >= ((ulint)1 << (32 - UNIV_PAGE_SIZE_SHIFT))) { fprintf(stderr, "InnoDB: Error: combined size of log files" - " must be < 4 GB\n"); + " must be < 4 GB on 32-bit systems\n"); return(DB_ERROR); } @@ -1364,7 +1366,7 @@ innobase_start_or_create_for_mysql(void) for (i = 0; i < srv_n_data_files; i++) { #ifndef __WIN__ - if (sizeof(off_t) < 5 && srv_data_file_sizes[i] >= 262144) { + if (sizeof(off_t) < 5 && srv_data_file_sizes[i] >= ((ulint)1 << (32 - UNIV_PAGE_SIZE_SHIFT))) { fprintf(stderr, "InnoDB: Error: file size must be < 4 GB" " with this MySQL binary\n" @@ -1809,6 +1811,13 @@ innobase_start_or_create_for_mysql(void) os_fast_mutex_free(&srv_os_test_mutex); + if (!srv_file_per_table_original_value + && srv_pass_corrupt_table) { + fprintf(stderr, "InnoDB: Warning:" + " innodb_file_per_table is diabled." + " So innodb_pass_corrupt_table doesn't make sence\n"); + } + if (srv_print_verbose_log) { ut_print_timestamp(stderr); fprintf(stderr,

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2853)
by knielsen＠knielsen-hq.org 28 Apr '10

28 Apr '10

#At lp:maria 2853 knielsen(a)knielsen-hq.org 2010-04-28 [merge] Merge XtraDB 9.1 into MariaDB. added: storage/xtradb/build/ storage/xtradb/build/debian/ storage/xtradb/build/debian/README.Maintainer storage/xtradb/build/debian/additions/ storage/xtradb/build/debian/additions/Docs__Images__Makefile.in storage/xtradb/build/debian/additions/Docs__Makefile.in storage/xtradb/build/debian/additions/debian-start storage/xtradb/build/debian/additions/debian-start.inc.sh storage/xtradb/build/debian/additions/echo_stderr storage/xtradb/build/debian/additions/innotop/ storage/xtradb/build/debian/additions/innotop/InnoDBParser.pm storage/xtradb/build/debian/additions/innotop/changelog.innotop storage/xtradb/build/debian/additions/innotop/innotop storage/xtradb/build/debian/additions/innotop/innotop.1 storage/xtradb/build/debian/additions/msql2mysql.1 storage/xtradb/build/debian/additions/my.cnf storage/xtradb/build/debian/additions/my_print_defaults.1 storage/xtradb/build/debian/additions/myisam_ftdump.1 storage/xtradb/build/debian/additions/myisamchk.1 storage/xtradb/build/debian/additions/myisamlog.1 storage/xtradb/build/debian/additions/myisampack.1 storage/xtradb/build/debian/additions/mysql-server.lintian-overrides storage/xtradb/build/debian/additions/mysql_config.1 storage/xtradb/build/debian/additions/mysql_convert_table_format.1 storage/xtradb/build/debian/additions/mysql_find_rows.1 storage/xtradb/build/debian/additions/mysql_fix_extensions.1 storage/xtradb/build/debian/additions/mysql_install_db.1 storage/xtradb/build/debian/additions/mysql_secure_installation.1 storage/xtradb/build/debian/additions/mysql_setpermission.1 storage/xtradb/build/debian/additions/mysql_tableinfo.1 storage/xtradb/build/debian/additions/mysql_waitpid.1 storage/xtradb/build/debian/additions/mysqlbinlog.1 storage/xtradb/build/debian/additions/mysqlbug.1 storage/xtradb/build/debian/additions/mysqlcheck.1 storage/xtradb/build/debian/additions/mysqld_safe_syslog.cnf storage/xtradb/build/debian/additions/mysqldumpslow.1 storage/xtradb/build/debian/additions/mysqlimport.1 storage/xtradb/build/debian/additions/mysqlmanager.1 storage/xtradb/build/debian/additions/mysqlreport storage/xtradb/build/debian/additions/mysqlreport.1 storage/xtradb/build/debian/additions/mysqltest.1 storage/xtradb/build/debian/additions/pack_isam.1 storage/xtradb/build/debian/additions/resolve_stack_dump.1 storage/xtradb/build/debian/additions/resolveip.1 storage/xtradb/build/debian/changelog storage/xtradb/build/debian/compat storage/xtradb/build/debian/control storage/xtradb/build/debian/copyright storage/xtradb/build/debian/libpercona-xtradb-client-dev.README.Maintainer storage/xtradb/build/debian/libpercona-xtradb-client-dev.dirs storage/xtradb/build/debian/libpercona-xtradb-client-dev.docs storage/xtradb/build/debian/libpercona-xtradb-client-dev.examples storage/xtradb/build/debian/libpercona-xtradb-client-dev.files storage/xtradb/build/debian/libpercona-xtradb-client-dev.links storage/xtradb/build/debian/libpercona-xtradb-client16.dirs storage/xtradb/build/debian/libpercona-xtradb-client16.docs storage/xtradb/build/debian/libpercona-xtradb-client16.files storage/xtradb/build/debian/libpercona-xtradb-client16.postinst storage/xtradb/build/debian/patches/ storage/xtradb/build/debian/patches/00list storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Images_Makefile.in.dpatch storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Makefile.in.dpatch storage/xtradb/build/debian/patches/33_scripts__mysql_create_system_tables__no_test.dpatch storage/xtradb/build/debian/patches/38_scripts__mysqld_safe.sh__signals.dpatch storage/xtradb/build/debian/patches/41_scripts__mysql_install_db.sh__no_test.dpatch storage/xtradb/build/debian/patches/44_scripts__mysql_config__libs.dpatch storage/xtradb/build/debian/patches/50_mysql-test__db_test.dpatch storage/xtradb/build/debian/patches/60_percona_support.dpatch storage/xtradb/build/debian/percona-xtradb-client-5.1.README.Debian storage/xtradb/build/debian/percona-xtradb-client-5.1.dirs storage/xtradb/build/debian/percona-xtradb-client-5.1.docs storage/xtradb/build/debian/percona-xtradb-client-5.1.files storage/xtradb/build/debian/percona-xtradb-client-5.1.links storage/xtradb/build/debian/percona-xtradb-client-5.1.lintian-overrides storage/xtradb/build/debian/percona-xtradb-client-5.1.menu storage/xtradb/build/debian/percona-xtradb-common.dirs storage/xtradb/build/debian/percona-xtradb-common.files storage/xtradb/build/debian/percona-xtradb-common.lintian-overrides storage/xtradb/build/debian/percona-xtradb-common.postrm storage/xtradb/build/debian/percona-xtradb-server-5.1.NEWS storage/xtradb/build/debian/percona-xtradb-server-5.1.README.Debian storage/xtradb/build/debian/percona-xtradb-server-5.1.config storage/xtradb/build/debian/percona-xtradb-server-5.1.dirs storage/xtradb/build/debian/percona-xtradb-server-5.1.docs storage/xtradb/build/debian/percona-xtradb-server-5.1.files storage/xtradb/build/debian/percona-xtradb-server-5.1.links storage/xtradb/build/debian/percona-xtradb-server-5.1.lintian-overrides storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.paranoid storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.server storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.workstation storage/xtradb/build/debian/percona-xtradb-server-5.1.mysql.init storage/xtradb/build/debian/percona-xtradb-server-5.1.percona-xtradb-server.logrotate storage/xtradb/build/debian/percona-xtradb-server-5.1.postinst storage/xtradb/build/debian/percona-xtradb-server-5.1.postrm storage/xtradb/build/debian/percona-xtradb-server-5.1.preinst storage/xtradb/build/debian/percona-xtradb-server-5.1.prerm storage/xtradb/build/debian/percona-xtradb-server-5.1.templates storage/xtradb/build/debian/po/ storage/xtradb/build/debian/po/POTFILES.in storage/xtradb/build/debian/po/ar.po storage/xtradb/build/debian/po/ca.po storage/xtradb/build/debian/po/cs.po storage/xtradb/build/debian/po/da.po storage/xtradb/build/debian/po/de.po storage/xtradb/build/debian/po/es.po storage/xtradb/build/debian/po/eu.po storage/xtradb/build/debian/po/fr.po storage/xtradb/build/debian/po/gl.po storage/xtradb/build/debian/po/it.po storage/xtradb/build/debian/po/ja.po storage/xtradb/build/debian/po/nb.po storage/xtradb/build/debian/po/nl.po storage/xtradb/build/debian/po/pt.po storage/xtradb/build/debian/po/pt_BR.po storage/xtradb/build/debian/po/ro.po storage/xtradb/build/debian/po/ru.po storage/xtradb/build/debian/po/sv.po storage/xtradb/build/debian/po/templates.pot storage/xtradb/build/debian/po/tr.po storage/xtradb/build/debian/rules storage/xtradb/build/debian/source.lintian-overrides storage/xtradb/build/debian/watch storage/xtradb/build/percona-sql.spec modified: storage/xtradb/buf/buf0flu.c storage/xtradb/handler/ha_innodb.cc storage/xtradb/include/ha_prototypes.h storage/xtradb/include/univ.i storage/xtradb/lock/lock0lock.c storage/xtradb/trx/trx0i_s.c storage/xtradb/trx/trx0trx.c === modified file 'storage/xtradb/buf/buf0flu.c' --- a/storage/xtradb/buf/buf0flu.c 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/buf/buf0flu.c 2010-04-28 13:53:04 +0000 @@ -1271,7 +1271,7 @@ buf_flush_LRU_recommendation(void) if(UT_LIST_GET_LEN(buf_pool->unzip_LRU)) have_LRU_mutex = TRUE; - +retry: //buf_pool_mutex_enter(); if (have_LRU_mutex) mutex_enter(&LRU_list_mutex); @@ -1313,6 +1313,10 @@ buf_flush_LRU_recommendation(void) if (n_replaceable >= BUF_FLUSH_FREE_BLOCK_MARGIN) { return(0); + } else if (!have_LRU_mutex) { + /* confirm it again with LRU_mutex for exactness */ + have_LRU_mutex = TRUE; + goto retry; } return(BUF_FLUSH_FREE_BLOCK_MARGIN + BUF_FLUSH_EXTRA_MARGIN === added directory 'storage/xtradb/build' === added directory 'storage/xtradb/build/debian' === added file 'storage/xtradb/build/debian/README.Maintainer' --- a/storage/xtradb/build/debian/README.Maintainer 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/README.Maintainer 2010-01-26 18:02:46 +0000 @@ -0,0 +1,116 @@ + +########################### +## FIXME for 5.1 ## +########################### + +* put this trigger-recreation thing into the init scripts -- what?! +* Let debian-i10n-english review all template changes before the translaters start. +* Mark debconf translations as obsolete with debconf-updatepo. + +########################################################################### +# Here are some information that are only of interest for the current and # +# following Debian maintainers of MySQL. # +########################################################################### + +The debian/ directory is under SVN control, see debian/control for URL. + +# +# Preparing a new version +# +The new orig.tar.gz (without non-free documentation) is created in /tmp/ when +running this command: + +debian/rules get-orig-source + +# +# mysqlreport +# +The authors e-mail address is <public(a)codenode.com>. + +# +# Remarks to dependencies +# +libwrap0-dev (>= 7.6-8.3) + According to bug report 114582 where where build problems on + IA-64/sid with at least two prior versions. +psmisc + /usr/bin/killall in the initscript + +zlib1g in libmysqlclient-dev: + "mysql_config --libs" ads "-lz" + +Build-Dep: + +debhelper (>=4.1.16): + See po-debconf(7). + +autoconf (>= 2.13-20), automake1.7 + Try to get rid of them. + +doxygen, tetex-bin, tetex-extra, gs + for ndb/docs/*tex + +# +# Remarks to the start scripts +# + +## initscripts rely on mysqladmin from a different package +We have the problem that "/etc/init.d/mysql stop" relies on mysqladmin which +is in another package (mysql-client) and a passwordless access that's maybe +only available if the user configured his /root/.my.cnf. Can this be a problem? +* normal mode: not because the user is required to have it. Else: +* purge/remove: not, same as normal mode +* upgrade: not, same as normal mode +* first install: not, it depends on mysql-client which at least is unpacked + so mysqladmin is there (to ping). It is not yet configured + passwordles but if there's a server running then there's a + /root/.my.cnf. Anyways, we simply kill anything that's mysqld. + +## Passwordless access for the maintainer scripts +Another issue is that the scripts needs passwordless access. To ensure this +a debian-sys-maint user is configured which has process and shutdown privs. +The file with the randomly (that's important!) generated password must be +present as long as the databases remain installed because else a new install +would have no access. This file should be used like: + mysqladmin --defaults-file=/etc/mysql/debian.cnf restart +to avoid providing the password in plaintext on a commandline where it would +be visible to any user via the "ps" command. + +## When to start the daemon? +We aim to give the admin full control on when MySQL is running. +Issues to be faced here: +OLD: + 1. Debconf asks whether MySQL should be started on boot so update-rc.d is + only run if the answer has been yes. The admin is likely to forget + this decision but update-rc.d checks for an existing line in + /etc/runlevel.conf and leaves it intact. + 2. On initial install, if the answer is yes, the daemon has to be started. + 3. On upgrades it should only be started if it was already running, everything + else is confusing. Especiall relying on an debconf decision made month ago + is considered suboptimal. See bug #274264 + Implementation so far: + prerm (called on upgrade before stopping the server): + check for a running server and set flag if necessary + preinst (called on initial install and before unpacking when upgrading): + check for the debconf variable and set flag if necessary + postinst (called on initial install and after each upgrade after unpacking): + call update-rc.d if debconf says yes + call invoce-rc.d if the flag has been set + Problems remaining: + dpkg-reconfigure and setting mysql start on boot to yes did not start mysql + (ok "start on boot" literally does not mean "start now" so that might have been ok) +NEW: + 1. --- no debconf anymore for the sake of simplicity. We have runlevel.conf, + the admin should use it + 2. On initial install the server is started. + 3. On upgrades the server is started exactly if it was running before so the + runlevel configuration is irrelevant. It will be preserved by the mean of + update-rc.d's builtin check. + Implementation: + prerm (called on upgrade before stopping the server): + check for a running server and set flag if necessary + preinst (called on initial install and before unpacking when upgrading): + check for $1 beeing (initial) "install" and set flag + postinst (called on initial install and after each upgrade after unpacking): + call update-rc.d + call invoce-rc.d if the flag has been set === added directory 'storage/xtradb/build/debian/additions' === added file 'storage/xtradb/build/debian/additions/Docs__Images__Makefile.in' --- a/storage/xtradb/build/debian/additions/Docs__Images__Makefile.in 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/Docs__Images__Makefile.in 2010-01-26 18:02:46 +0000 @@ -0,0 +1,6 @@ +all: + +distclean: + -rm -f Makefile + +.PHONY: all distclean clean install check === added file 'storage/xtradb/build/debian/additions/Docs__Makefile.in' --- a/storage/xtradb/build/debian/additions/Docs__Makefile.in 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/Docs__Makefile.in 2010-01-26 18:02:46 +0000 @@ -0,0 +1,6 @@ +all: + +distclean: + -rm -f Makefile + +.PHONY: all distclean clean install check === added file 'storage/xtradb/build/debian/additions/debian-start' --- a/storage/xtradb/build/debian/additions/debian-start 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/debian-start 2010-01-26 18:02:46 +0000 @@ -0,0 +1,31 @@ +#!/bin/bash +# +# This script is executed by "/etc/init.d/mysql" on every (re)start. +# +# Changes to this file will be preserved when updating the Debian package. +# + +source /usr/share/mysql/debian-start.inc.sh + +MYSQL="/usr/bin/mysql --defaults-file=/etc/mysql/debian.cnf" +MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf" +MYUPGRADE="/usr/bin/mysql_upgrade --defaults-extra-file=/etc/mysql/debian.cnf" +MYCHECK="/usr/bin/mysqlcheck --defaults-file=/etc/mysql/debian.cnf" +MYCHECK_SUBJECT="WARNING: mysqlcheck has found corrupt tables" +MYCHECK_PARAMS="--all-databases --fast --silent" +MYCHECK_RCPT="root" + +# The following commands should be run when the server is up but in background +# where they do not block the server start and in one shell instance so that +# they run sequentially. They are supposed not to echo anything to stdout. +# If you want to disable the check for crashed tables comment +# "check_for_crashed_tables" out. +# (There may be no output to stdout inside the background process!) +echo "Checking for corrupt, not cleanly closed and upgrade needing tables." +( + upgrade_system_tables_if_necessary; + check_root_accounts; + check_for_crashed_tables; +) >&2 & + +exit 0 === added file 'storage/xtradb/build/debian/additions/debian-start.inc.sh' --- a/storage/xtradb/build/debian/additions/debian-start.inc.sh 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/debian-start.inc.sh 2010-01-26 18:02:46 +0000 @@ -0,0 +1,72 @@ +#!/bin/bash +# +# This file is included by /etc/mysql/debian-start +# + +## Check all unclosed tables. +# - Requires the server to be up. +# - Is supposed to run silently in background. +function check_for_crashed_tables() { + set -e + set -u + + # But do it in the background to not stall the boot process. + logger -p daemon.info -i -t$0 "Triggering myisam-recover for all MyISAM tables" + + # Checking for $? is unreliable so the size of the output is checked. + # Some table handlers like HEAP do not support CHECK TABLE. + tempfile=`tempfile` + # We have to use xargs in this case, because a for loop barfs on the + # spaces in the thing to be looped over. + LC_ALL=C $MYSQL --skip-column-names --batch -e ' + select concat("select count(*) into @discard from `", + TABLE_SCHEMA, "`.`", TABLE_NAME, "`") + from information_schema.TABLES where ENGINE="MyISAM"' | \ + xargs -i $MYSQL --skip-column-names --silent --batch \ + --force -e "{}" >$tempfile + if [ -s $tempfile ]; then + ( + /bin/echo -e "\n" \ + "Improperly closed tables are also reported if clients are accessing\n" \ + "the tables *now*. A list of current connections is below.\n"; + $MYADMIN processlist status + ) >> $tempfile + # Check for presence as a dependency on mailx would require an MTA. + if [ -x /usr/bin/mailx ]; then + mailx -e -s"$MYCHECK_SUBJECT" $MYCHECK_RCPT < $tempfile + fi + (echo "$MYCHECK_SUBJECT"; cat $tempfile) | logger -p daemon.warn -i -t$0 + fi + rm $tempfile +} + +## Check for tables needing an upgrade. +# - Requires the server to be up. +# - Is supposed to run silently in background. +function upgrade_system_tables_if_necessary() { + set -e + set -u + + logger -p daemon.info -i -t$0 "Upgrading MySQL tables if necessary." + + # Filter all "duplicate column", "duplicate key" and "unknown column" + # errors as the script is designed to be idempotent. + LC_ALL=C $MYUPGRADE \ + 2>&1 \ + | egrep -v '^(1|@had|ERROR (1054|1060|1061))' \ + | logger -p daemon.warn -i -t$0 +} + +## Check for the presence of both, root accounts with and without password. +# This might have been caused by a bug related to mysql_install_db (#418672). +function check_root_accounts() { + set -e + set -u + + logger -p daemon.info -i -t$0 "Checking for insecure root accounts." + + ret=$( echo "SELECT count(*) FROM mysql.user WHERE user='root' and password='';" | $MYSQL --skip-column-names ) + if [ "$ret" -ne "0" ]; then + logger -p daemon.warn -i -t$0 "WARNING: mysql.user contains $ret root accounts without password!" + fi +} === added file 'storage/xtradb/build/debian/additions/echo_stderr' --- a/storage/xtradb/build/debian/additions/echo_stderr 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/echo_stderr 2010-01-26 18:02:46 +0000 @@ -0,0 +1,2 @@ +#!/bin/bash +echo "$*" 1>&2 === added directory 'storage/xtradb/build/debian/additions/innotop' === added file 'storage/xtradb/build/debian/additions/innotop/InnoDBParser.pm' --- a/storage/xtradb/build/debian/additions/innotop/InnoDBParser.pm 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/innotop/InnoDBParser.pm 2010-01-26 18:02:46 +0000 @@ -0,0 +1,1089 @@ +use strict; +use warnings FATAL => 'all'; + +package InnoDBParser; + +# This program is copyright (c) 2006 Baron Schwartz, baron at xaprb dot com. +# Feedback and improvements are gratefully received. +# +# THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED +# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF +# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. +# +# This program is free software; you can redistribute it and/or modify it under +# the terms of the GNU General Public License as published by the Free Software +# Foundation, version 2; OR the Perl Artistic License. On UNIX and similar +# systems, you can issue `man perlgpl' or `man perlartistic' to read these + +# You should have received a copy of the GNU General Public License along with +# this program; if not, write to the Free Software Foundation, Inc., 59 Temple +# Place, Suite 330, Boston, MA 02111-1307 USA + +our $VERSION = '1.6.0'; + +use Data::Dumper; +$Data::Dumper::Sortkeys = 1; +use English qw(-no_match_vars); +use List::Util qw(max); + +# Some common patterns +my $d = qr/(\d+)/; # Digit +my $f = qr/(\d+\.\d+)/; # Float +my $t = qr/(\d+ \d+)/; # Transaction ID +my $i = qr/((?:\d{1,3}\.){3}\d+)/; # IP address +my $n = qr/([^`\s]+)/; # MySQL object name +my $w = qr/(\w+)/; # Words +my $fl = qr/([\w\.\/]+) line $d/; # Filename and line number +my $h = qr/((?:0x)?[0-9a-f]*)/; # Hex +my $s = qr/(\d{6} .\d:\d\d:\d\d)/; # InnoDB timestamp + +# If you update this variable, also update the SYNOPSIS in the pod. +my %innodb_section_headers = ( + "TRANSACTIONS" => "tx", + "BUFFER POOL AND MEMORY" => "bp", + "SEMAPHORES" => "sm", + "LOG" => "lg", + "ROW OPERATIONS" => "ro", + "INSERT BUFFER AND ADAPTIVE HASH INDEX" => "ib", + "FILE I/O" => "io", + "LATEST DETECTED DEADLOCK" => "dl", + "LATEST FOREIGN KEY ERROR" => "fk", +); + +my %parser_for = ( + tx => \&parse_tx_section, + bp => \&parse_bp_section, + sm => \&parse_sm_section, + lg => \&parse_lg_section, + ro => \&parse_ro_section, + ib => \&parse_ib_section, + io => \&parse_io_section, + dl => \&parse_dl_section, + fk => \&parse_fk_section, +); + +my %fk_parser_for = ( + Transaction => \&parse_fk_transaction_error, + Error => \&parse_fk_bad_constraint_error, + Cannot => \&parse_fk_cant_drop_parent_error, +); + +# A thread's proc_info can be at least 98 different things I've found in the +# source. Fortunately, most of them begin with a gerunded verb. These are +# the ones that don't. +my %is_proc_info = ( + 'After create' => 1, + 'Execution of init_command' => 1, + 'FULLTEXT initialization' => 1, + 'Reopen tables' => 1, + 'Repair done' => 1, + 'Repair with keycache' => 1, + 'System lock' => 1, + 'Table lock' => 1, + 'Thread initialized' => 1, + 'User lock' => 1, + 'copy to tmp table' => 1, + 'discard_or_import_tablespace' => 1, + 'end' => 1, + 'got handler lock' => 1, + 'got old table' => 1, + 'init' => 1, + 'key cache' => 1, + 'locks' => 1, + 'malloc' => 1, + 'query end' => 1, + 'rename result table' => 1, + 'rename' => 1, + 'setup' => 1, + 'statistics' => 1, + 'status' => 1, + 'table cache' => 1, + 'update' => 1, +); + +sub new { + bless {}, shift; +} + +# Parse the status and return it. +# See srv_printf_innodb_monitor in innobase/srv/srv0srv.c +# Pass in the text to parse, whether to be in debugging mode, which sections +# to parse (hashref; if empty, parse all), and whether to parse full info from +# locks and such (probably shouldn't unless you need to). +sub parse_status_text { + my ( $self, $fulltext, $debug, $sections, $full ) = @_; + + die "I can't parse undef" unless defined $fulltext; + $fulltext =~ s/[\r\n]+/\n/g; + + $sections ||= {}; + die '$sections must be a hashref' unless ref($sections) eq 'HASH'; + + my %innodb_data = ( + got_all => 0, # Whether I was able to get the whole thing + ts => '', # Timestamp the server put on it + last_secs => 0, # Num seconds the averages are over + sections => {}, # Parsed values from each section + ); + + if ( $debug ) { + $innodb_data{'fulltext'} = $fulltext; + } + + # Get the most basic info about the status: beginning and end, and whether + # I got the whole thing (if there has been a big deadlock and there are + # too many locks to print, the output might be truncated) + my ( $time_text ) = $fulltext =~ m/^$s INNODB MONITOR OUTPUT$/m; + $innodb_data{'ts'} = [ parse_innodb_timestamp( $time_text ) ]; + $innodb_data{'timestring'} = ts_to_string($innodb_data{'ts'}); + ( $innodb_data{'last_secs'} ) = $fulltext + =~ m/Per second averages calculated from the last $d seconds/; + + ( my $got_all ) = $fulltext =~ m/END OF INNODB MONITOR OUTPUT/; + $innodb_data{'got_all'} = $got_all || 0; + + # Split it into sections. Each section begins with + # ----- + # LABEL + # ----- + my %innodb_sections; + my @matches = $fulltext + =~ m#\n(---+)\n([A-Z /]+)\n\1\n(.*?)(?=\n(---+)\n[A-Z /]+\n\4\n|$)#gs; + while ( my ( $start, $name, $text, $end ) = splice(@matches, 0, 4) ) { + $innodb_sections{$name} = [ $text, $end ? 1 : 0 ]; + } + # The Row Operations section is a special case, because instead of ending + # with the beginning of another section, it ends with the end of the file. + # So this section is complete if the entire file is complete. + $innodb_sections{'ROW OPERATIONS'}->[1] ||= $innodb_data{'got_all'}; + + # Just for sanity's sake, make sure I understand what to do with each + # section + eval { + foreach my $section ( keys %innodb_sections ) { + my $header = $innodb_section_headers{$section}; + die "Unknown section $section in $fulltext\n" + unless $header; + $innodb_data{'sections'}->{ $header } + ->{'fulltext'} = $innodb_sections{$section}->[0]; + $innodb_data{'sections'}->{ $header } + ->{'complete'} = $innodb_sections{$section}->[1]; + } + }; + if ( $EVAL_ERROR ) { + _debug( $debug, $EVAL_ERROR); + } + + # ################################################################ + # Parse the detailed data out of the sections. + # ################################################################ + eval { + foreach my $section ( keys %parser_for ) { + if ( defined $innodb_data{'sections'}->{$section} + && (!%$sections || (defined($sections->{$section} && $sections->{$section})) )) { + $parser_for{$section}->( + $innodb_data{'sections'}->{$section}, + $innodb_data{'sections'}->{$section}->{'complete'}, + $debug, + $full ) + or delete $innodb_data{'sections'}->{$section}; + } + else { + delete $innodb_data{'sections'}->{$section}; + } + } + }; + if ( $EVAL_ERROR ) { + _debug( $debug, $EVAL_ERROR); + } + + return \%innodb_data; +} + +# Parses the status text and returns it flattened out as a single hash. +sub get_status_hash { + my ( $self, $fulltext, $debug, $sections, $full ) = @_; + + # Parse the status text... + my $innodb_status + = $self->parse_status_text($fulltext, $debug, $sections, $full ); + + # Flatten the hierarchical structure into a single list by grabbing desired + # sections from it. + return + (map { 'IB_' . $_ => $innodb_status->{$_} } qw(timestring last_secs got_all)), + (map { 'IB_bp_' . $_ => $innodb_status->{'sections'}->{'bp'}->{$_} } + qw( writes_pending buf_pool_hit_rate total_mem_alloc buf_pool_reads + awe_mem_alloc pages_modified writes_pending_lru page_creates_sec + reads_pending pages_total buf_pool_hits writes_pending_single_page + page_writes_sec pages_read pages_written page_reads_sec + writes_pending_flush_list buf_pool_size add_pool_alloc + dict_mem_alloc pages_created buf_free complete )), + (map { 'IB_tx_' . $_ => $innodb_status->{'sections'}->{'tx'}->{$_} } + qw( num_lock_structs history_list_len purge_done_for transactions + purge_undo_for is_truncated trx_id_counter complete )), + (map { 'IB_ib_' . $_ => $innodb_status->{'sections'}->{'ib'}->{$_} } + qw( hash_table_size hash_searches_s non_hash_searches_s + bufs_in_node_heap used_cells size free_list_len seg_size inserts + merged_recs merges complete )), + (map { 'IB_lg_' . $_ => $innodb_status->{'sections'}->{'lg'}->{$_} } + qw( log_ios_done pending_chkp_writes last_chkp log_ios_s + log_flushed_to log_seq_no pending_log_writes complete )), + (map { 'IB_sm_' . $_ => $innodb_status->{'sections'}->{'sm'}->{$_} } + qw( wait_array_size rw_shared_spins rw_excl_os_waits mutex_os_waits + mutex_spin_rounds mutex_spin_waits rw_excl_spins rw_shared_os_waits + waits signal_count reservation_count complete )), + (map { 'IB_ro_' . $_ => $innodb_status->{'sections'}->{'ro'}->{$_} } + qw( queries_in_queue n_reserved_extents main_thread_state + main_thread_proc_no main_thread_id read_sec del_sec upd_sec ins_sec + read_views_open num_rows_upd num_rows_ins num_rows_read + queries_inside num_rows_del complete )), + (map { 'IB_fk_' . $_ => $innodb_status->{'sections'}->{'fk'}->{$_} } + qw( trigger parent_table child_index parent_index attempted_op + child_db timestring fk_name records col_name reason txn parent_db + type child_table parent_col complete )), + (map { 'IB_io_' . $_ => $innodb_status->{'sections'}->{'io'}->{$_} } + qw( pending_buffer_pool_flushes pending_pwrites pending_preads + pending_normal_aio_reads fsyncs_s os_file_writes pending_sync_ios + reads_s flush_type avg_bytes_s pending_ibuf_aio_reads writes_s + threads os_file_reads pending_aio_writes pending_log_ios os_fsyncs + pending_log_flushes complete )), + (map { 'IB_dl_' . $_ => $innodb_status->{'sections'}->{'dl'}->{$_} } + qw( timestring rolled_back txns complete )); + +} + +sub ts_to_string { + my $parts = shift; + return sprintf('%02d-%02d-%02d %02d:%02d:%02d', @$parts); +} + +sub parse_innodb_timestamp { + my $text = shift; + my ( $y, $m, $d, $h, $i, $s ) + = $text =~ m/^(\d\d)(\d\d)(\d\d) +(\d+):(\d+):(\d+)$/; + die("Can't get timestamp from $text\n") unless $y; + $y += 2000; + return ( $y, $m, $d, $h, $i, $s ); +} + +sub parse_fk_section { + my ( $section, $complete, $debug, $full ) = @_; + my $fulltext = $section->{'fulltext'}; + + return 0 unless $fulltext; + + my ( $ts, $type ) = $fulltext =~ m/^$s\s+(\w+)/m; + $section->{'ts'} = [ parse_innodb_timestamp( $ts ) ]; + $section->{'timestring'} = ts_to_string($section->{'ts'}); + $section->{'type'} = $type; + + # Decide which type of FK error happened, and dispatch to the right parser. + if ( $type && $fk_parser_for{$type} ) { + $fk_parser_for{$type}->( $section, $complete, $debug, $fulltext, $full ); + } + + delete $section->{'fulltext'} unless $debug; + + return 1; +} + +sub parse_fk_cant_drop_parent_error { + my ( $section, $complete, $debug, $fulltext, $full ) = @_; + + # Parse the parent/child table info out + @{$section}{ qw(attempted_op parent_db parent_table) } = $fulltext + =~ m{Cannot $w table `(.*)/(.*)`}m; + @{$section}{ qw(child_db child_table) } = $fulltext + =~ m{because it is referenced by `(.*)/(.*)`}m; + + ( $section->{'reason'} ) = $fulltext =~ m/(Cannot .*)/s; + $section->{'reason'} =~ s/\n(?:InnoDB: )?/ /gm + if $section->{'reason'}; + + # Certain data may not be present. Make them '' if not present. + map { $section->{$_} ||= "" } + qw(child_index fk_name col_name parent_col); +} + +# See dict/dict0dict.c, function dict_foreign_error_report +# I don't care much about these. There are lots of different messages, and +# they come from someone trying to create a foreign key, or similar +# statements. They aren't indicative of some transaction trying to insert, +# delete or update data. Sometimes it is possible to parse out a lot of +# information about the tables and indexes involved, but often the message +# contains the DDL string the user entered, which is way too much for this +# module to try to handle. +sub parse_fk_bad_constraint_error { + my ( $section, $complete, $debug, $fulltext, $full ) = @_; + + # Parse the parent/child table and index info out + @{$section}{ qw(child_db child_table) } = $fulltext + =~ m{Error in foreign key constraint of table (.*)/(.*):$}m; + $section->{'attempted_op'} = 'DDL'; + + # FK name, parent info... if possible. + @{$section}{ qw(fk_name col_name parent_db parent_table parent_col) } + = $fulltext + =~ m/CONSTRAINT `?$n`? FOREIGN KEY $`?$n`?$ REFERENCES (?:`?$n`?\.)?`?$n`? $`?$n`?$/; + + if ( !defined($section->{'fk_name'}) ) { + # Try to parse SQL a user might have typed in a CREATE statement or such + @{$section}{ qw(col_name parent_db parent_table parent_col) } + = $fulltext + =~ m/FOREIGN\s+KEY\s*$`?$n`?$\s+REFERENCES\s+(?:`?$n`?\.)?`?$n`?\s*$`?$n`?$/i; + } + $section->{'parent_db'} ||= $section->{'child_db'}; + + # Name of the child index (index in the same table where the FK is, see + # definition of dict_foreign_struct in include/dict0mem.h, where it is + # called foreign_index, as opposed to referenced_index which is in the + # parent table. This may not be possible to find. + @{$section}{ qw(child_index) } = $fulltext + =~ m/^The index in the foreign key in table is $n$/m; + + @{$section}{ qw(reason) } = $fulltext =~ m/:\s*([^:]+)(?= Constraint:|$)/ms; + $section->{'reason'} =~ s/\s+/ /g + if $section->{'reason'}; + + # Certain data may not be present. Make them '' if not present. + map { $section->{$_} ||= "" } + qw(child_index fk_name col_name parent_table parent_col); +} + +# see source file row/row0ins.c +sub parse_fk_transaction_error { + my ( $section, $complete, $debug, $fulltext, $full ) = @_; + + # Parse the txn info out + my ( $txn ) = $fulltext + =~ m/Transaction:\n(TRANSACTION.*)\nForeign key constraint fails/s; + if ( $txn ) { + $section->{'txn'} = parse_tx_text( $txn, $complete, $debug, $full ); + } + + # Parse the parent/child table and index info out. There are two types: an + # update or a delete of a parent record leaves a child orphaned + # (row_ins_foreign_report_err), and an insert or update of a child record has + # no matching parent record (row_ins_foreign_report_add_err). + + @{$section}{ qw(reason child_db child_table) } + = $fulltext =~ m{^(Foreign key constraint fails for table `(.*)/(.*)`:)$}m; + + @{$section}{ qw(fk_name col_name parent_db parent_table parent_col) } + = $fulltext + =~ m/CONSTRAINT `$n` FOREIGN KEY $`$n`$ REFERENCES (?:`$n`\.)?`$n` $`$n`$/; + $section->{'parent_db'} ||= $section->{'child_db'}; + + # Special case, which I don't know how to trigger, but see + # innobase/row/row0ins.c row_ins_check_foreign_constraint + if ( $fulltext =~ m/ibd file does not currently exist!/ ) { + my ( $attempted_op, $index, $records ) + = $fulltext =~ m/^Trying to (add to index) `$n` tuple:\n(.*))?/sm; + $section->{'child_index'} = $index; + $section->{'attempted_op'} = $attempted_op || ''; + if ( $records && $full ) { + ( $section->{'records'} ) + = parse_innodb_record_dump( $records, $complete, $debug ); + } + @{$section}{qw(parent_db parent_table)} + =~ m/^But the parent table `$n`\.`$n`$/m; + } + else { + my ( $attempted_op, $which, $index ) + = $fulltext =~ m/^Trying to ([\w ]*) in (child|parent) table, in index `$n` tuple:$/m; + if ( $which ) { + $section->{$which . '_index'} = $index; + $section->{'attempted_op'} = $attempted_op || ''; + + # Parse out the related records in the other table. + my ( $search_index, $records ); + if ( $which eq 'child' ) { + ( $search_index, $records ) = $fulltext + =~ m/^But in parent table [^,]*, in index `$n`,\nthe closest match we can find is record:\n(.*)/ms; + $section->{'parent_index'} = $search_index; + } + else { + ( $search_index, $records ) = $fulltext + =~ m/^But in child table [^,]*, in index `$n`, (?:the record is not available|there is a record:\n(.*))?/ms; + $section->{'child_index'} = $search_index; + } + if ( $records && $full ) { + $section->{'records'} + = parse_innodb_record_dump( $records, $complete, $debug ); + } + else { + $section->{'records'} = ''; + } + } + } + + # Parse out the tuple trying to be updated, deleted or inserted. + my ( $trigger ) = $fulltext =~ m/^(DATA TUPLE: \d+ fields;\n.*)$/m; + if ( $trigger ) { + $section->{'trigger'} = parse_innodb_record_dump( $trigger, $complete, $debug ); + } + + # Certain data may not be present. Make them '' if not present. + map { $section->{$_} ||= "" } + qw(child_index fk_name col_name parent_table parent_col); +} + +# There are new-style and old-style record formats. See rem/rem0rec.c +# TODO: write some tests for this +sub parse_innodb_record_dump { + my ( $dump, $complete, $debug ) = @_; + return undef unless $dump; + + my $result = {}; + + if ( $dump =~ m/PHYSICAL RECORD/ ) { + my $style = $dump =~ m/compact format/ ? 'new' : 'old'; + $result->{'style'} = $style; + + # This is a new-style record. + if ( $style eq 'new' ) { + @{$result}{qw( heap_no type num_fields info_bits )} + = $dump + =~ m/^(?:Record lock, heap no $d )?([A-Z ]+): n_fields $d; compact format; info bits $d$/m; + } + + # OK, it's old-style. Unfortunately there are variations here too. + elsif ( $dump =~ m/-byte offs / ) { + # Older-old style. + @{$result}{qw( heap_no type num_fields byte_offset info_bits )} + = $dump + =~ m/^(?:Record lock, heap no $d )?([A-Z ]+): n_fields $d; $d-byte offs [A-Z]+; info bits $d$/m; + if ( $dump !~ m/-byte offs TRUE/ ) { + $result->{'byte_offset'} = 0; + } + } + else { + # Newer-old style. + @{$result}{qw( heap_no type num_fields byte_offset info_bits )} + = $dump + =~ m/^(?:Record lock, heap no $d )?([A-Z ]+): n_fields $d; $d-byte offsets; info bits $d$/m; + } + + } + else { + $result->{'style'} = 'tuple'; + @{$result}{qw( type num_fields )} + = $dump =~ m/^(DATA TUPLE): $d fields;$/m; + } + + # Fill in default values for things that couldn't be parsed. + map { $result->{$_} ||= 0 } + qw(heap_no num_fields byte_offset info_bits); + map { $result->{$_} ||= '' } + qw(style type ); + + my @fields = $dump =~ m/ (\d+:.*?;?);(?=$| \d+:)/gm; + $result->{'fields'} = [ map { parse_field($_, $complete, $debug ) } @fields ]; + + return $result; +} + +# New/old-style applies here. See rem/rem0rec.c +# $text should not include the leading space or the second trailing semicolon. +sub parse_field { + my ( $text, $complete, $debug ) = @_; + + # Sample fields: + # '4: SQL NULL, size 4 ' + # '1: len 6; hex 000000005601; asc V ;' + # '6: SQL NULL' + # '5: len 30; hex 687474703a2f2f7777772e737765657477617465722e636f6d2f73746f72; asc http://www.sweetwater.com/stor;...(truncated)' + my ( $id, $nullsize, $len, $hex, $asc, $truncated ); + ( $id, $nullsize ) = $text =~ m/^$d: SQL NULL, size $d $/; + if ( !defined($id) ) { + ( $id ) = $text =~ m/^$d: SQL NULL$/; + } + if ( !defined($id) ) { + ( $id, $len, $hex, $asc, $truncated ) + = $text =~ m/^$d: len $d; hex $h; asc (.*);(\.\.\.$truncated$)?$/; + } + + die "Could not parse this field: '$text'" unless defined $id; + return { + id => $id, + len => defined($len) ? $len : defined($nullsize) ? $nullsize : 0, + 'hex' => defined($hex) ? $hex : '', + asc => defined($asc) ? $asc : '', + trunc => $truncated ? 1 : 0, + }; + +} + +sub parse_dl_section { + my ( $dl, $complete, $debug, $full ) = @_; + return unless $dl; + my $fulltext = $dl->{'fulltext'}; + return 0 unless $fulltext; + + my ( $ts ) = $fulltext =~ m/^$s$/m; + return 0 unless $ts; + + $dl->{'ts'} = [ parse_innodb_timestamp( $ts ) ]; + $dl->{'timestring'} = ts_to_string($dl->{'ts'}); + $dl->{'txns'} = {}; + + my @sections + = $fulltext + =~ m{ + ^\*{3}\s([^\n]*) # *** (1) WAITING FOR THIS... + (.*?) # Followed by anything, non-greedy + (?=(?:^\*{3})|\z) # Followed by another three stars or EOF + }gmsx; + + + # Loop through each section. There are no assumptions about how many + # there are, who holds and wants what locks, and who gets rolled back. + while ( my ($header, $body) = splice(@sections, 0, 2) ) { + my ( $txn_id, $what ) = $header =~ m/^$$d$ (.*):$/; + next unless $txn_id; + $dl->{'txns'}->{$txn_id} ||= {}; + my $txn = $dl->{'txns'}->{$txn_id}; + + if ( $what eq 'TRANSACTION' ) { + $txn->{'tx'} = parse_tx_text( $body, $complete, $debug, $full ); + } + else { + push @{$txn->{'locks'}}, parse_innodb_record_locks( $body, $complete, $debug, $full ); + } + } + + @{ $dl }{ qw(rolled_back) } + = $fulltext =~ m/^\*\*\* WE ROLL BACK TRANSACTION $$d$$/m; + + # Make sure certain values aren't undef + map { $dl->{$_} ||= '' } qw(rolled_back); + + delete $dl->{'fulltext'} unless $debug; + return 1; +} + +sub parse_innodb_record_locks { + my ( $text, $complete, $debug, $full ) = @_; + my @result; + + foreach my $lock ( $text =~ m/(^(?:RECORD|TABLE) LOCKS?.*$)/gm ) { + my $hash = {}; + @{$hash}{ qw(lock_type space_id page_no n_bits index db table txn_id lock_mode) } + = $lock + =~ m{^(RECORD|TABLE) LOCKS? (?:space id $d page no $d n bits $d index `?$n`? of )?table `$n(?:/|`\.`)$n` trx id $t lock.mode (\S+)}m; + ( $hash->{'special'} ) + = $lock =~ m/^(?:RECORD|TABLE) .*? locks (rec but not gap|gap before rec)/m; + $hash->{'insert_intention'} + = $lock =~ m/^(?:RECORD|TABLE) .*? insert intention/m ? 1 : 0; + $hash->{'waiting'} + = $lock =~ m/^(?:RECORD|TABLE) .*? waiting/m ? 1 : 0; + + # Some things may not be in the text, so make sure they are not + # undef. + map { $hash->{$_} ||= 0 } qw(n_bits page_no space_id); + map { $hash->{$_} ||= "" } qw(index special); + push @result, $hash; + } + + return @result; +} + +sub parse_tx_text { + my ( $txn, $complete, $debug, $full ) = @_; + + my ( $txn_id, $txn_status, $active_secs, $proc_no, $os_thread_id ) + = $txn + =~ m/^(?:---)?TRANSACTION $t, (\D*?)(?: $d sec)?, (?:process no $d, )?OS thread id $d/m; + my ( $thread_status, $thread_decl_inside ) + = $txn + =~ m/OS thread id \d+(?: ([^,]+?))?(?:, thread declared inside InnoDB $d)?$/m; + + # Parsing the line that begins 'MySQL thread id' is complicated. The only + # thing always in the line is the thread and query id. See function + # innobase_mysql_print_thd in InnoDB source file sql/ha_innodb.cc. + my ( $thread_line ) = $txn =~ m/^(MySQL thread id .*)$/m; + my ( $mysql_thread_id, $query_id, $hostname, $ip, $user, $query_status ); + + if ( $thread_line ) { + # These parts can always be gotten. + ( $mysql_thread_id, $query_id ) = $thread_line =~ m/^MySQL thread id $d, query id $d/m; + + # If it's a master/slave thread, "Has (read|sent) all" may be the thread's + # proc_info. In these cases, there won't be any host/ip/user info + ( $query_status ) = $thread_line =~ m/(Has (?:read|sent) all .*$)/m; + if ( defined($query_status) ) { + $user = 'system user'; + } + + # It may be the case that the query id is the last thing in the line. + elsif ( $thread_line =~ m/query id \d+ / ) { + # The IP address is the only non-word thing left, so it's the most + # useful marker for where I have to start guessing. + ( $hostname, $ip ) = $thread_line =~ m/query id \d+(?: ([A-Za-z]\S+))? $i/m; + if ( defined $ip ) { + ( $user, $query_status ) = $thread_line =~ m/$ip $w(?: (.*))?$/; + } + else { # OK, there wasn't an IP address. + # There might not be ANYTHING except the query status. + ( $query_status ) = $thread_line =~ m/query id \d+ (.*)$/; + if ( $query_status !~ m/^\w+ing/ && !exists($is_proc_info{$query_status}) ) { + # The remaining tokens are, in order: hostname, user, query_status. + # It's basically impossible to know which is which. + ( $hostname, $user, $query_status ) = $thread_line + =~ m/query id \d+(?: ([A-Za-z]\S+))?(?: $w(?: (.*))?)?$/m; + } + else { + $user = 'system user'; + } + } + } + } + + my ( $lock_wait_status, $lock_structs, $heap_size, $row_locks, $undo_log_entries ) + = $txn + =~ m/^(?:(\D*) )?$d lock struct$s$, heap size $d(?:, $d row lock$s$)?(?:, undo log entries $d)?$/m; + my ( $lock_wait_time ) + = $txn + =~ m/^------- TRX HAS BEEN WAITING $d SEC/m; + + my $locks; + # If the transaction has locks, grab the locks. + if ( $txn =~ m/^TABLE LOCK|RECORD LOCKS/ ) { + $locks = [parse_innodb_record_locks($txn, $complete, $debug, $full)]; + } + + my ( $tables_in_use, $tables_locked ) + = $txn + =~ m/^mysql tables in use $d, locked $d$/m; + my ( $txn_doesnt_see_ge, $txn_sees_lt ) + = $txn + =~ m/^Trx read view will not see trx with id >= $t, sees < $t$/m; + my $has_read_view = defined($txn_doesnt_see_ge); + # Only a certain number of bytes of the query text are included here, at least + # under some circumstances. Some versions include 300, some 600. + my ( $query_text ) + = $txn + =~ m{ + ^MySQL\sthread\sid\s[^\n]+\n # This comes before the query text + (.*?) # The query text + (?= # Followed by any of... + ^Trx\sread\sview + |^-------\sTRX\sHAS\sBEEN\sWAITING + |^TABLE\sLOCK + |^RECORD\sLOCKS\sspace\sid + |^(?:---)?TRANSACTION + |^\*\*\*\s$\d$ + |\Z + ) + }xms; + if ( $query_text ) { + $query_text =~ s/\s+$//; + } + else { + $query_text = ''; + } + + my %stuff = ( + active_secs => $active_secs, + has_read_view => $has_read_view, + heap_size => $heap_size, + hostname => $hostname, + ip => $ip, + lock_structs => $lock_structs, + lock_wait_status => $lock_wait_status, + lock_wait_time => $lock_wait_time, + mysql_thread_id => $mysql_thread_id, + os_thread_id => $os_thread_id, + proc_no => $proc_no, + query_id => $query_id, + query_status => $query_status, + query_text => $query_text, + row_locks => $row_locks, + tables_in_use => $tables_in_use, + tables_locked => $tables_locked, + thread_decl_inside => $thread_decl_inside, + thread_status => $thread_status, + txn_doesnt_see_ge => $txn_doesnt_see_ge, + txn_id => $txn_id, + txn_sees_lt => $txn_sees_lt, + txn_status => $txn_status, + undo_log_entries => $undo_log_entries, + user => $user, + ); + $stuff{'fulltext'} = $txn if $debug; + $stuff{'locks'} = $locks if $locks; + + # Some things may not be in the txn text, so make sure they are not + # undef. + map { $stuff{$_} ||= 0 } qw(active_secs heap_size lock_structs + tables_in_use undo_log_entries tables_locked has_read_view + thread_decl_inside lock_wait_time proc_no row_locks); + map { $stuff{$_} ||= "" } qw(thread_status txn_doesnt_see_ge + txn_sees_lt query_status ip query_text lock_wait_status user); + $stuff{'hostname'} ||= $stuff{'ip'}; + + return \%stuff; +} + +sub parse_tx_section { + my ( $section, $complete, $debug, $full ) = @_; + return unless $section && $section->{'fulltext'}; + my $fulltext = $section->{'fulltext'}; + $section->{'transactions'} = []; + + # Handle the individual transactions + my @transactions = $fulltext =~ m/(---TRANSACTION \d.*?)(?=\n---TRANSACTION|$)/gs; + foreach my $txn ( @transactions ) { + my $stuff = parse_tx_text( $txn, $complete, $debug, $full ); + delete $stuff->{'fulltext'} unless $debug; + push @{$section->{'transactions'}}, $stuff; + } + + # Handle the general info + @{$section}{ 'trx_id_counter' } + = $fulltext =~ m/^Trx id counter $t$/m; + @{$section}{ 'purge_done_for', 'purge_undo_for' } + = $fulltext =~ m/^Purge done for trx's n:o < $t undo n:o < $t$/m; + @{$section}{ 'history_list_len' } # This isn't present in some 4.x versions + = $fulltext =~ m/^History list length $d$/m; + @{$section}{ 'num_lock_structs' } + = $fulltext =~ m/^Total number of lock structs in row lock hash table $d$/m; + @{$section}{ 'is_truncated' } + = $fulltext =~ m/^\.\.\. truncated\.\.\.$/m ? 1 : 0; + + # Fill in things that might not be present + foreach ( qw(history_list_len) ) { + $section->{$_} ||= 0; + } + + delete $section->{'fulltext'} unless $debug; + return 1; +} + +# I've read the source for this section. +sub parse_ro_section { + my ( $section, $complete, $debug, $full ) = @_; + return unless $section && $section->{'fulltext'}; + my $fulltext = $section->{'fulltext'}; + + # Grab the info + @{$section}{ 'queries_inside', 'queries_in_queue' } + = $fulltext =~ m/^$d queries inside InnoDB, $d queries in queue$/m; + ( $section->{ 'read_views_open' } ) + = $fulltext =~ m/^$d read views open inside InnoDB$/m; + ( $section->{ 'n_reserved_extents' } ) + = $fulltext =~ m/^$d tablespace extents now reserved for B-tree/m; + @{$section}{ 'main_thread_proc_no', 'main_thread_id', 'main_thread_state' } + = $fulltext =~ m/^Main thread (?:process no. $d, )?id $d, state: (.*)$/m; + @{$section}{ 'num_rows_ins', 'num_rows_upd', 'num_rows_del', 'num_rows_read' } + = $fulltext =~ m/^Number of rows inserted $d, updated $d, deleted $d, read $d$/m; + @{$section}{ 'ins_sec', 'upd_sec', 'del_sec', 'read_sec' } + = $fulltext =~ m#^$f inserts/s, $f updates/s, $f deletes/s, $f reads/s$#m; + $section->{'main_thread_proc_no'} ||= 0; + + map { $section->{$_} ||= 0 } qw(read_views_open n_reserved_extents); + delete $section->{'fulltext'} unless $debug; + return 1; +} + +sub parse_lg_section { + my ( $section, $complete, $debug, $full ) = @_; + return unless $section; + my $fulltext = $section->{'fulltext'}; + + # Grab the info + ( $section->{ 'log_seq_no' } ) + = $fulltext =~ m/Log sequence number \s*(\d.*)$/m; + ( $section->{ 'log_flushed_to' } ) + = $fulltext =~ m/Log flushed up to \s*(\d.*)$/m; + ( $section->{ 'last_chkp' } ) + = $fulltext =~ m/Last checkpoint at \s*(\d.*)$/m; + @{$section}{ 'pending_log_writes', 'pending_chkp_writes' } + = $fulltext =~ m/$d pending log writes, $d pending chkp writes/; + @{$section}{ 'log_ios_done', 'log_ios_s' } + = $fulltext =~ m#$d log i/o's done, $f log i/o's/second#; + + delete $section->{'fulltext'} unless $debug; + return 1; +} + +sub parse_ib_section { + my ( $section, $complete, $debug, $full ) = @_; + return unless $section && $section->{'fulltext'}; + my $fulltext = $section->{'fulltext'}; + + # Some servers will output ibuf information for tablespace 0, as though there + # might be many tablespaces with insert buffers. (In practice I believe + # the source code shows there will only ever be one). I have to parse both + # cases here, but I assume there will only be one. + @{$section}{ 'size', 'free_list_len', 'seg_size' } + = $fulltext =~ m/^Ibuf(?: for space 0)?: size $d, free list len $d, seg size $d,$/m; + @{$section}{ 'inserts', 'merged_recs', 'merges' } + = $fulltext =~ m/^$d inserts, $d merged recs, $d merges$/m; + + @{$section}{ 'hash_table_size', 'used_cells', 'bufs_in_node_heap' } + = $fulltext =~ m/^Hash table size $d, used cells $d, node heap has $d buffer$s$$/m; + @{$section}{ 'hash_searches_s', 'non_hash_searches_s' } + = $fulltext =~ m{^$f hash searches/s, $f non-hash searches/s$}m; + + delete $section->{'fulltext'} unless $debug; + return 1; +} + +sub parse_wait_array { + my ( $text, $complete, $debug, $full ) = @_; + my %result; + + @result{ qw(thread waited_at_filename waited_at_line waited_secs) } + = $text =~ m/^--Thread $d has waited at $fl for $f seconds/m; + + # Depending on whether it's a SYNC_MUTEX,RW_LOCK_EX,RW_LOCK_SHARED, + # there will be different text output + if ( $text =~ m/^Mutex at/m ) { + $result{'request_type'} = 'M'; + @result{ qw( lock_mem_addr lock_cfile_name lock_cline lock_var) } + = $text =~ m/^Mutex at $h created file $fl, lock var $d$/m; + @result{ qw( waiters_flag )} + = $text =~ m/^waiters flag $d$/m; + } + else { + @result{ qw( request_type lock_mem_addr lock_cfile_name lock_cline) } + = $text =~ m/^(.)-lock on RW-latch at $h created in file $fl$/m; + @result{ qw( writer_thread writer_lock_mode ) } + = $text =~ m/^a writer $thread id $d$ has reserved it in mode (.*)$/m; + @result{ qw( num_readers waiters_flag )} + = $text =~ m/^number of readers $d, waiters flag $d$/m; + @result{ qw(last_s_file_name last_s_line ) } + = $text =~ m/Last time read locked in file $fl$/m; + @result{ qw(last_x_file_name last_x_line ) } + = $text =~ m/Last time write locked in file $fl$/m; + } + + $result{'cell_waiting'} = $text =~ m/^wait has ended$/m ? 0 : 1; + $result{'cell_event_set'} = $text =~ m/^wait is ending$/m ? 1 : 0; + + # Because there are two code paths, some things won't get set. + map { $result{$_} ||= '' } + qw(last_s_file_name last_x_file_name writer_lock_mode); + map { $result{$_} ||= 0 } + qw(num_readers lock_var last_s_line last_x_line writer_thread); + + return \%result; +} + +sub parse_sm_section { + my ( $section, $complete, $debug, $full ) = @_; + return 0 unless $section && $section->{'fulltext'}; + my $fulltext = $section->{'fulltext'}; + + # Grab the info + @{$section}{ 'reservation_count', 'signal_count' } + = $fulltext =~ m/^OS WAIT ARRAY INFO: reservation count $d, signal count $d$/m; + @{$section}{ 'mutex_spin_waits', 'mutex_spin_rounds', 'mutex_os_waits' } + = $fulltext =~ m/^Mutex spin waits $d, rounds $d, OS waits $d$/m; + @{$section}{ 'rw_shared_spins', 'rw_shared_os_waits', 'rw_excl_spins', 'rw_excl_os_waits' } + = $fulltext =~ m/^RW-shared spins $d, OS waits $d; RW-excl spins $d, OS waits $d$/m; + + # Look for info on waits. + my @waits = $fulltext =~ m/^(--Thread.*?)^(?=Mutex spin|--Thread)/gms; + $section->{'waits'} = [ map { parse_wait_array($_, $complete, $debug) } @waits ]; + $section->{'wait_array_size'} = scalar(@waits); + + delete $section->{'fulltext'} unless $debug; + return 1; +} + +# I've read the source for this section. +sub parse_bp_section { + my ( $section, $complete, $debug, $full ) = @_; + return unless $section && $section->{'fulltext'}; + my $fulltext = $section->{'fulltext'}; + + # Grab the info + @{$section}{ 'total_mem_alloc', 'add_pool_alloc' } + = $fulltext =~ m/^Total memory allocated $d; in additional pool allocated $d$/m; + @{$section}{'dict_mem_alloc'} = $fulltext =~ m/Dictionary memory allocated $d/; + @{$section}{'awe_mem_alloc'} = $fulltext =~ m/$d MB of AWE memory/; + @{$section}{'buf_pool_size'} = $fulltext =~ m/^Buffer pool size\s*$d$/m; + @{$section}{'buf_free'} = $fulltext =~ m/^Free buffers\s*$d$/m; + @{$section}{'pages_total'} = $fulltext =~ m/^Database pages\s*$d$/m; + @{$section}{'pages_modified'} = $fulltext =~ m/^Modified db pages\s*$d$/m; + @{$section}{'pages_read', 'pages_created', 'pages_written'} + = $fulltext =~ m/^Pages read $d, created $d, written $d$/m; + @{$section}{'page_reads_sec', 'page_creates_sec', 'page_writes_sec'} + = $fulltext =~ m{^$f reads/s, $f creates/s, $f writes/s$}m; + @{$section}{'buf_pool_hits', 'buf_pool_reads'} + = $fulltext =~ m{Buffer pool hit rate $d / $d$}m; + if ($fulltext =~ m/^No buffer pool page gets since the last printout$/m) { + @{$section}{'buf_pool_hits', 'buf_pool_reads'} = (0, 0); + @{$section}{'buf_pool_hit_rate'} = '--'; + } + else { + @{$section}{'buf_pool_hit_rate'} + = $fulltext =~ m{Buffer pool hit rate (\d+ / \d+)$}m; + } + @{$section}{'reads_pending'} = $fulltext =~ m/^Pending reads $d/m; + @{$section}{'writes_pending_lru', 'writes_pending_flush_list', 'writes_pending_single_page' } + = $fulltext =~ m/^Pending writes: LRU $d, flush list $d, single page $d$/m; + + map { $section->{$_} ||= 0 } + qw(writes_pending_lru writes_pending_flush_list writes_pending_single_page + awe_mem_alloc dict_mem_alloc); + @{$section}{'writes_pending'} = List::Util::sum( + @{$section}{ qw(writes_pending_lru writes_pending_flush_list writes_pending_single_page) }); + + delete $section->{'fulltext'} unless $debug; + return 1; +} + +# I've read the source for this. +sub parse_io_section { + my ( $section, $complete, $debug, $full ) = @_; + return unless $section && $section->{'fulltext'}; + my $fulltext = $section->{'fulltext'}; + $section->{'threads'} = {}; + + # Grab the I/O thread info + my @threads = $fulltext =~ m<^(I/O thread \d+ .*)$>gm; + foreach my $thread (@threads) { + my ( $tid, $state, $purpose, $event_set ) + = $thread =~ m{I/O thread $d state: (.+?) $(.*)$(?: ev set)?$}m; + if ( defined $tid ) { + $section->{'threads'}->{$tid} = { + thread => $tid, + state => $state, + purpose => $purpose, + event_set => $event_set ? 1 : 0, + }; + } + } + + # Grab the reads/writes/flushes info + @{$section}{ 'pending_normal_aio_reads', 'pending_aio_writes' } + = $fulltext =~ m/^Pending normal aio reads: $d, aio writes: $d,$/m; + @{$section}{ 'pending_ibuf_aio_reads', 'pending_log_ios', 'pending_sync_ios' } + = $fulltext =~ m{^ ibuf aio reads: $d, log i/o's: $d, sync i/o's: $d$}m; + @{$section}{ 'flush_type', 'pending_log_flushes', 'pending_buffer_pool_flushes' } + = $fulltext =~ m/^Pending flushes $$w$ log: $d; buffer pool: $d$/m; + @{$section}{ 'os_file_reads', 'os_file_writes', 'os_fsyncs' } + = $fulltext =~ m/^$d OS file reads, $d OS file writes, $d OS fsyncs$/m; + @{$section}{ 'reads_s', 'avg_bytes_s', 'writes_s', 'fsyncs_s' } + = $fulltext =~ m{^$f reads/s, $d avg bytes/read, $f writes/s, $f fsyncs/s$}m; + @{$section}{ 'pending_preads', 'pending_pwrites' } + = $fulltext =~ m/$d pending preads, $d pending pwrites$/m; + @{$section}{ 'pending_preads', 'pending_pwrites' } = (0, 0) + unless defined($section->{'pending_preads'}); + + delete $section->{'fulltext'} unless $debug; + return 1; +} + +sub _debug { + my ( $debug, $msg ) = @_; + if ( $debug ) { + die $msg; + } + else { + warn $msg; + } + return 1; +} + +1; + +# end_of_package +# ############################################################################ +# Perldoc section. I put this last as per the Dog book. +# ############################################################################ +=pod + +=head1 NAME + +InnoDBParser - Parse InnoDB monitor text. + +=head1 DESCRIPTION + +InnoDBParser tries to parse the output of the InnoDB monitor. One way to get +this output is to connect to a MySQL server and issue the command SHOW ENGINE +INNODB STATUS (omit 'ENGINE' on earlier versions of MySQL). The goal is to +turn text into data that something else (e.g. innotop) can use. + +The output comes from all over, but the place to start in the source is +innobase/srv/srv0srv.c. + +=head1 SYNOPSIS + + use InnoDBParser; + use DBI; + + # Get the status text. + my $dbh = DBI->connect( + "DBI::mysql:test;host=localhost", + 'user', + 'password' + ); + my $query = 'SHOW /*!5 ENGINE */ INNODB STATUS'; + my $text = $dbh->selectcol_arrayref($query)->[0]; + + # 1 or 0 + my $debug = 1; + + # Choose sections of the monitor text you want. Possible values: + # TRANSACTIONS => tx + # BUFFER POOL AND MEMORY => bp + # SEMAPHORES => sm + # LOG => lg + # ROW OPERATIONS => ro + # INSERT BUFFER AND ADAPTIVE HASH INDEX => ib + # FILE I/O => io + # LATEST DETECTED DEADLOCK => dl + # LATEST FOREIGN KEY ERROR => fk + + my $required_sections = { + tx => 1, + }; + + # Parse the status text. + my $parser = InnoDBParser->new; + $innodb_status = $parser->parse_status_text( + $text, + $debug, + # Omit the following parameter to get all sections. + $required_sections, + ); + +=head1 COPYRIGHT, LICENSE AND WARRANTY + +This package is copyright (c) 2006 Baron Schwartz, baron at xaprb dot com. +Feedback and improvements are gratefully received. + +THIS PACKAGE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED +WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF +MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation, version 2; OR the Perl Artistic License. On UNIX and similar +systems, you can issue `man perlgpl' or `man perlartistic' to read these +licenses. + +You should have received a copy of the GNU General Public License along with +this program; if not, write to the Free Software Foundation, Inc., 59 Temple +Place, Suite 330, Boston, MA 02111-1307 USA + +=head1 AUTHOR + +Baron Schwartz, baron at xaprb dot com. + +=head1 BUGS + +None known, but I bet there are some. The InnoDB monitor text wasn't really +designed to be parsable. + +=head1 SEE ALSO + +innotop - a program that can format the parsed status information for humans +to read and enjoy. + +=cut === added file 'storage/xtradb/build/debian/additions/innotop/changelog.innotop' --- a/storage/xtradb/build/debian/additions/innotop/changelog.innotop 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/innotop/changelog.innotop 2010-01-26 18:02:46 +0000 @@ -0,0 +1,318 @@ +Changelog for innotop and InnoDBParser: + +2007-11-09: version 1.6.0 + + * S mode crashed on non-numeric values. + * New user-defined columns crashed upon restart. + * Added --color option to control terminal coloring. + +2007-09-18: version 1.5.2 + + * Added the ability to monitor InnoDB status from a file. + * Changed W mode to L mode; it monitors all locks, not just lock waits. + +2007-09-16: version 1.5.1 + + * Added C (Command Summary) mode. + * Fixed a bug in the 'avg' aggregate function. + +2007-09-10: version 1.5.0 + + Changes: + * Added plugin functionality. + * Added group-by functionality. + * Moved the configuration file to a directory. + * Enhanced filtering and sorting on pivoted tables. + * Many small bug fixes. + +2007-07-16: version 1.4.3 + + Changes: + * Added standard --version command-line option + * Changed colors to cyan instead of blue; more visible on dark terminals. + * Added information to the filter-choosing dialog. + * Added column auto-completion when entering a filter expression. + * Changed Term::ReadKey from optional to mandatory. + * Clarified username in password prompting. + * Ten thousand words of documentation! + + Bugs fixed: + * innotop crashed in W mode when InnoDB status data was truncated. + * innotop didn't display errors in tables if debug was enabled. + * The colored() subroutine wasn't being created in non-interactive mode. + * Don't prompt to save password except the first time. + +2007-05-03: version 1.4.2 + + This version contains all changes to the trunk until revision 239; some + changes in revisions 240:250 are included. + + MAJOR CHANGES: + + * Quick-filters to easily filter any column in any display + * Compatibility with MySQL 3.23 through 6.0 + * Improved error handling when a server is down, permissions denied, etc + * Use additional SHOW INNODB STATUS information in 5.1.x + * Make all modes use tables consistently, so they can all be edited, + filtered, colored and sorted consistently + * Combine V, G and S modes into S mode, with v, g, and s hot-keys + * Let DBD driver read MySQL option files; permit connections without + user/pass/etc + * Compile SQL-like expressions into Perl subroutines; eliminate need to + know Perl + * Do not save all config data to config file, only save user's customizations + * Rewritten and improved command-line option handling + * Added --count, --delay, and other command-line options to support + run-and-exit operation + * Improve built-in variable sets + * Improve help screen with three-part balanced-column layout + * Simplify table-editor and improve hotkey support + * Require Perl to have high-resolution time support (Time::HiRes) + * Help the user choose a query to analyze or kill + * Enable EXPLAIN, show-full-query in T mode just like Q mode + * Let data-extraction access current, previous and incremental data sets + all at once + + MINOR CHANGES: + + * Column stabilizing for Q mode + * New color rules for T, Q, W modes + * Apply slave I/O filter to Q mode + * Improve detection of server version and other meta-data + * Make connection timeout a config variable + * Improve cross-version-compatible SQL syntax + * Get some information from the DBD driver instead of asking MySQL for it + * Improved error messages + * Improve server group creation/editing + * Improve connection/thread killing + * Fix broken key bindings and restore previously mapped hot-keys for + choosing columns + * Some documentation updates (but not nearly enough) + * Allow the user to specify graphing char in S mode (formerly G mode) + * Allow easy switching between variable sets in S mode + * Bind 'n' key globally to choose the 'next' server connection + * Bind '%' key globally to filter displayed tables + * Allow aligning columns on the decimal place for easy readability + * Add hide_hdr config variable to hide column headers in tables + * Add a feature to smartly run PURGE MASTER LOGS in Replication mode + * Enable debug mode as a globally configurable variable + * Improve error messages when an expression or filter doesn't compile or has + a run-time error; die on error when debug is enabled + * Allow user-configurable delays after executing SQL (to let the server + settle down before taking another measurement) + * Add an expression to show how long until a transaction is finished + * Add skip_innodb as a global config variable + * Add '%' after percentages to help disambiguate (user-configurable) + * Add column to M mode to help see how fast slave is catching up to master + + BUG FIXES: + + * T and W modes had wrong value for wait_status column + * Error tracking on connections didn't reset when the connection recovered + * wait_timeout on connections couldn't be set before MySQL 4.0.3 + * There was a crash on 3.23 when wiping deadlocks + * Lettercase changes in some result sets (SHOW MASTER/SLAVE STATUS) between + MySQL versions crashed innotop + * Inactive connections crashed innotop upon access to DBD driver + * set_precision did not respect user defaults for number of digits + * --inc command-line option could not be negated + * InnoDB status parsing was not always parsing all needed information + * S mode (formerly G mode) could crash trying to divide non-numeric data + * M table didn't show Slave_open_temp_tables variable; incorrect lettercase + * DBD drivers with broken AutoCommit would crash innotop + * Some key bindings had incorrect labels + * Some config-file loading routines could load data for things that didn't + exist + * Headers printed too often in S mode + * High-resolution time was not used even when the user had it + * Non-interactive mode printed blank lines sometimes + * Q-mode header and statusbar showed different QPS numbers + * Formulas for key-cache and query-cache hit ratios were wrong + * Mac OS "Darwin" machines were mis-identified as Microsoft Windows + * Some multiplications crashed when given undefined input + * The commify transformation did not check its input and could crash + * Specifying an invalid mode on the command line or config file could crash + innotop + +2007-03-29: version 1.4.1 + + * More tweaks to display of connection errors. + * Fixed a problem with skip-innodb in MySQL 5.1. + * Fix a bug with dead connections in single-connection mode. + * Fix a regex to allow parsing more data from truncated deadlocks. + * Don't load active cxns from the config file if the cxn isn't defined. + +2007-03-03: version 1.4.0 + + * Further tweak error handling and display of connection errors + * More centralization of querying + * Fix forking so it doesn't kill all database connections + * Allow user to run innotop without permissions for GLOBAL variables and status + +2007-02-11: version 1.3.6 + + * Handle some connection failures so innotop doesn't crash because of one server. + * Enable incremental display in more modes. + * Tweaks to colorizing, color editor, and default color rules. + * Tweaks to default sorting rules. + * Use prepared statements for efficiency. + * Bug fixes and code cleanups. + * Data storage is keyed on clock ticks now. + +2007-02-03: version 1.3.5 + + * Bug fixes. + * More tools for editing configuration from within innotop. + * Filters and transformations are constrained to valid values. + * Support for colorizing rows. + * Sorting by multiple columns. + * Compress headers when display is very wide. + * Stabilize and limit column widths. + * Check config file formats when upgrading so upgrades go smoothly. + * Make D mode handle many connections at once. + * Extract simple expressions from data sets in column src property. + This makes innotop more awk-ish. + +2007-01-16: version 1.3 + + * Readline support. + * Can be used unattended, or in a pipe-and-filter mode + where it outputs tab-separated data to standard output. + * You can specify a config file on the command line. + Config files can be marked read-only. + * Monitor multiple servers simultaneously. + * Server groups to help manage many servers conveniently. + * Monitor master/slave status, and control slaves. + * Columns can have user-defined expressions as their data sources. + * Better configuration tools. + * InnoDB status information is merged into SHOW VARIABLES and + SHOW STATUS information, so you can access it all together. + * High-precision time support in more places. + * Lots of tweaks to make things display more readably and compactly. + * Column transformations and filters. + +2007-01-16: version 1.0.1 + * NOTE: innotop is now hosted at Sourceforge, in Subversion not CVS. + The new project homepage is http://sourceforge.net/projects/innotop/ + * Tweak default T/Q mode sort columns to match what people expect. + * Fix broken InnoDBParser.pm documentation (and hence man page). + +2007-01-06: version 1.0 + * NOTE: innotop is now hosted at Sourceforge, in Subversion not CVS. + The new project homepage is http://sourceforge.net/projects/innotop/ + * Prevent control characters from freaking terminal out. + * Set timeout to keep busy servers from closing connection. + * There is only one InnoDB insert buffer. + * Make licenses clear and consistent. + +2006-11-14: innotop 0.1.160, InnoDBParser version 1.69 + * Support for ANSI color on Microsoft Windows (more readable, compact + display; thanks Gisbert W. Selke). + * Better handling of $ENV{HOME} on Windows. + * Added a LICENSE file to the package as per Gentoo bug: + http://bugs.gentoo.org/show_bug.cgi?id=147600 + +2006-11-11: innotop 0.1.157, InnoDBParser version 1.69 + * Add Microsoft Windows support. + +2006-10-19: innotop 0.1.154, InnoDBParser version 1.69 + * Add O (Open Tables) mode + * Add some more checks to handle incomplete InnoDB status information + +2006-09-30: innotop 0.1.152, InnoDBParser version 1.69 + * Figured out what was wrong with package $VERSION variable: it wasn't + after the package declaration! + +2006-09-28: innotop 0.1.152, InnoDBParser version 1.67 + * Make more efforts towards crash-resistance and tolerance of completely + messed-up inputs. If innotop itself is broken, it is now much harder to + tell, because it just keeps on running without complaining. + * Fix a small bug parsing out some information and displaying it. + +2006-09-05: innotop 0.1.149, InnoDBParser version 1.64 + * Try to find and eliminate any parsing code that assumes pattern matches + will succeed. + +2006-09-05: innotop 0.1.149, InnoDBParser version 1.62 + * Make innotop crash-resistant, so I can declare it STABLE finally. + * Instead of using SQL conditional comments, detect MySQL version. + +2006-08-22: innotop 0.1.147, InnoDBParser version 1.60 + * Fix some innotop bugs with undefined values, bad formatting etc. + +2006-08-19: innotop 0.1.146, InnoDBParser version 1.60 + * Make innotop handle some unexpected NULL values in Q mode. + * Add OS wait information to W mode, so it is now "everything that waits." + * Center section captions better. + * Make R mode more readable and compact. + * Make InnoDBParser parse lock waits even when they've been waiting 0 secs. + +2006-08-12: innotop 0.1.139, InnoDBParser version 1.59 + * Add more documentation + * Tweak V mode to show more info in less space. + * Fix a bug in G mode. + +2006-08-10: innotop 0.1.132, InnoDBParser version 1.58 + * Handle yet more types of FK error... it will never end! + * Handle some special cases when DEADLOCK info truncated + * Add a bit more FK info to F mode in innotop + * More tests added to the test suite + +2006-08-07: innotop 0.1.131, InnoDBParser version 1.55 + * Fix another issue with configuration + * Handle another type of FK error + +2006-08-03: innotop 0.1.130, InnoDBParser version 1.54 + * Fix an issue loading config file + * Add heap_no to 'D' (InnoDB Deadlock) mode to ease deadlock debugging. + +2006-08-02: innotop 0.1.128, InnoDBParser version 1.54 + * Parse lock wait information from the TRANSACTION section. + * Even more OS-specific parsing... pain in the butt... + * Add 'W' (InnoDB Lock Wait) mode. + * Fix some minor display issues with statusbar. + +2006-08-02: innotop 0.1.125, InnoDBParser version 1.50 + * Don't try to get references to Perl built-in functions like time() + * Handle more OS-specific variations of InnoDB status text + * Add some more information to various places in innotop + +2006-08-01: innotop 0.1.123, InnoDBParser version 1.47 + + * Enhance S and G modes: clear screen and re-print headers + * Don't crash when deadlock data is truncated + * Make Analyze mode say how to get back to whatever you came from + * Display 'nothing to display' when there is nothing + * Add ability to read InnoDB status text from a file (mostly helps test) + * Add table of Wait Array Information in Row Op/Semaphore mode + * Add table of lock information in InnoDB deadlock mode + * Ensure new features in upgrades don't get masked by existing config files + * Tweak default column choices for T mode + * Enhance foreign key parsing + * Enhance physical record and data tuple parsing + * Enhance lock parsing (handle old-style and new-style formats) + +2006-07-24: innotop 0.1.112, InnoDBParser version 1.36 + + * InnoDBParser enhancements for FK error messages. + * A fix to innotop to prevent it from crashing while trying to display a FK + error message. + * Some minor cosmetic changes to number formatting in innotop. + +2006-07-22: innotop 0.1.106, InnoDBParser version 1.35 + + * InnoDBParser is much more complete and accurate. + * Tons of bug fixes. + * Add partitions to EXPLAIN mode. + * Enhance Q mode header, add T mode header. + * Share some configuration variables across modes. + * Add formatted time columns to Q, T modes. + * Add command-line argument parsing. + * Turn off echo when asking for password. + * Add option to specify port when connecting. + * Let display-optimized-query display multiple notes. + * Lots of small improvements, such as showing more info in statusbar. + +2006-07-02: innotop 0.1.74, InnoDBParser version 1.24 + + * Initial release for public consumption. === added file 'storage/xtradb/build/debian/additions/innotop/innotop' --- a/storage/xtradb/build/debian/additions/innotop/innotop 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/innotop/innotop 2010-01-26 18:02:46 +0000 @@ -0,0 +1,9485 @@ +#!/usr/bin/perl + +# vim: tw=160:nowrap:expandtab:tabstop=3:shiftwidth=3:softtabstop=3 + +use strict; +use warnings FATAL => 'all'; +use sigtrap qw(handler finish untrapped normal-signals); + +use Data::Dumper; +use DBI; +use English qw(-no_match_vars); +use File::Basename qw(dirname); +use Getopt::Long; +use List::Util qw(max min maxstr sum); +use InnoDBParser; +use POSIX qw(ceil); +use Time::HiRes qw(time sleep); +use Term::ReadKey qw(ReadMode ReadKey); + +# Version, license and warranty information. {{{1 +# ########################################################################### +our $VERSION = '1.6.0'; +our $SVN_REV = sprintf("%d", q$Revision: 383 $ =~ m/(\d+)/g); +our $SVN_URL = sprintf("%s", q$URL: https://innotop.svn.sourceforge.net/svnroot/innotop/trunk/innotop $ =~ m$svnroot/innotop/(\S+)$g); + +my $innotop_license = <<"LICENSE"; + +This is innotop version $VERSION, a MySQL and InnoDB monitor. + +This program is copyright (c) 2006 Baron Schwartz. +Feedback and improvements are welcome. + +THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED +WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF +MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation, version 2; OR the Perl Artistic License. On UNIX and similar +systems, you can issue `man perlgpl' or `man perlartistic' to read these +licenses. + +You should have received a copy of the GNU General Public License along with +this program; if not, write to the Free Software Foundation, Inc., 59 Temple +Place, Suite 330, Boston, MA 02111-1307 USA. +LICENSE + +# Configuration information and global setup {{{1 +# ########################################################################### + +# Really, really, super-global variables. +my @config_versions = ( + "000-000-000", "001-003-000", # config file was one big name-value hash. + "001-003-000", "001-004-002", # config file contained non-user-defined stuff. +); + +my $clear_screen_sub; + +# This defines expected properties and defaults for the column definitions that +# eventually end up in tbl_meta. +my %col_props = ( + hdr => '', + just => '-', + dec => 0, # Whether to align the column on the decimal point + num => 0, + label => '', + user => 0, + src => '', + tbl => '', # Helps when writing/reading custom columns in config files + minw => 0, + maxw => 0, + trans => [], + agg => 'first', # Aggregate function + aggonly => 0, # Whether to show only when tbl_meta->{aggregate} is true +); + +# Actual DBI connections to MySQL servers. +my %dbhs; + +# Command-line parameters {{{2 +# ########################################################################### + +my @opt_spec = ( + { s => 'help', d => 'Show this help message' }, + { s => 'color|C!', d => 'Use terminal coloring (default)', c => 'color' }, + { s => 'config|c=s', d => 'Config file to read' }, + { s => 'nonint|n', d => 'Non-interactive, output tab-separated fields' }, + { s => 'count=i', d => 'Number of updates before exiting' }, + { s => 'delay|d=f', d => 'Delay between updates in seconds', c => 'interval' }, + { s => 'mode|m=s', d => 'Operating mode to start in', c => 'mode' }, + { s => 'inc|i!', d => 'Measure incremental differences', c => 'status_inc' }, + { s => 'version', d => 'Output version information and exit' }, +); + +# This is the container for the command-line options' values to be stored in +# after processing. Initial values are defaults. +my %opts = ( + n => !( -t STDIN && -t STDOUT ), # If in/out aren't to terminals, we're interactive +); +# Post-process... +my %opt_seen; +foreach my $spec ( @opt_spec ) { + my ( $long, $short ) = $spec->{s} =~ m/^(\w+)(?:\|([^!+=]*))?/; + $spec->{k} = $short || $long; + $spec->{l} = $long; + $spec->{t} = $short; + $spec->{n} = $spec->{s} =~ m/!/; + $opts{$spec->{k}} = undef unless defined $opts{$spec->{k}}; + die "Duplicate option $spec->{k}" if $opt_seen{$spec->{k}}++; +} + +Getopt::Long::Configure('no_ignore_case', 'bundling'); +GetOptions( map { $_->{s} => \$opts{$_->{k}} } @opt_spec) or $opts{help} = 1; + +if ( $opts{version} ) { + print "innotop Ver $VERSION Changeset $SVN_REV from $SVN_URL\n"; + exit(0); +} + +if ( $opts{'help'} ) { + print "Usage: innotop <options> <innodb-status-file>\n\n"; + my $maxw = max(map { length($_->{l}) + ($_->{n} ? 4 : 0)} @opt_spec); + foreach my $spec ( sort { $a->{l} cmp $b->{l} } @opt_spec ) { + my $long = $spec->{n} ? "[no]$spec->{l}" : $spec->{l}; + my $short = $spec->{t} ? "-$spec->{t}" : ''; + printf(" --%-${maxw}s %-4s %s\n", $long, $short, $spec->{d}); + } + print <<USAGE; + +innotop is a MySQL and InnoDB transaction/status monitor, like 'top' for +MySQL. It displays queries, InnoDB transactions, lock waits, deadlocks, +foreign key errors, open tables, replication status, buffer information, +row operations, logs, I/O operations, load graph, and more. You can +monitor many servers at once with innotop. + +USAGE + exit(1); +} + +# Meta-data (table definitions etc) {{{2 +# ########################################################################### + +# Expressions {{{3 +# Convenience so I can copy/paste these in several places... +# ########################################################################### +my %exprs = ( + Host => q{my $host = host || hostname || ''; ($host) = $host =~ m/^((?:[\d.]+(?=:))|(?:[a-zA-Z]\w+))/; return $host || ''}, + Port => q{my ($p) = host =~ m/:(.*)$/; return $p || 0}, + OldVersions => q{dulint_to_int(IB_tx_trx_id_counter) - dulint_to_int(IB_tx_purge_done_for)}, + MaxTxnTime => q/max(map{ $_->{active_secs} } @{ IB_tx_transactions }) || 0/, + NumTxns => q{scalar @{ IB_tx_transactions } }, + DirtyBufs => q{ $cur->{IB_bp_pages_modified} / ($cur->{IB_bp_buf_pool_size} || 1) }, + BufPoolFill => q{ $cur->{IB_bp_pages_total} / ($cur->{IB_bp_buf_pool_size} || 1) }, + ServerLoad => q{ $cur->{Threads_connected}/(Questions||1)/Uptime_hires }, + TxnTimeRemain => q{ defined undo_log_entries && defined $pre->{undo_log_entries} && undo_log_entries < $pre->{undo_log_entries} ? undo_log_entries / (($pre->{undo_log_entries} - undo_log_entries)/((active_secs-$pre->{active_secs})||1))||1 : 0}, + SlaveCatchupRate => ' defined $cur->{seconds_behind_master} && defined $pre->{seconds_behind_master} && $cur->{seconds_behind_master} < $pre->{seconds_behind_master} ? ($pre->{seconds_behind_master}-$cur->{seconds_behind_master})/($cur->{Uptime_hires}-$pre->{Uptime_hires}) : 0', + QcacheHitRatio => q{(Qcache_hits||0)/(((Com_select||0)+(Qcache_hits||0))||1)}, +); + +# ########################################################################### +# Column definitions {{{3 +# Defines every column in every table. A named column has the following +# properties: +# * hdr Column header/title +# * label Documentation for humans. +# * num Whether it's numeric (for sorting). +# * just Alignment; generated from num, user-overridable in tbl_meta +# * minw, maxw Auto-generated, user-overridable. +# Values from this hash are just copied to tbl_meta, which is where everything +# else in the program should read from. +# ########################################################################### + +my %columns = ( + active_secs => { hdr => 'SecsActive', num => 1, label => 'Seconds transaction has been active', }, + add_pool_alloc => { hdr => 'Add\'l Pool', num => 1, label => 'Additonal pool allocated' }, + attempted_op => { hdr => 'Action', num => 0, label => 'The action that caused the error' }, + awe_mem_alloc => { hdr => 'AWE Memory', num => 1, label => '[Windows] AWE memory allocated' }, + binlog_cache_overflow => { hdr => 'Binlog Cache', num => 1, label => 'Transactions too big for binlog cache that went to disk' }, + binlog_do_db => { hdr => 'Binlog Do DB', num => 0, label => 'binlog-do-db setting' }, + binlog_ignore_db => { hdr => 'Binlog Ignore DB', num => 0, label => 'binlog-ignore-db setting' }, + bps_in => { hdr => 'BpsIn', num => 1, label => 'Bytes per second received by the server', }, + bps_out => { hdr => 'BpsOut', num => 1, label => 'Bytes per second sent by the server', }, + buf_free => { hdr => 'Free Bufs', num => 1, label => 'Buffers free in the buffer pool' }, + buf_pool_hit_rate => { hdr => 'Hit Rate', num => 0, label => 'Buffer pool hit rate' }, + buf_pool_hits => { hdr => 'Hits', num => 1, label => 'Buffer pool hits' }, + buf_pool_reads => { hdr => 'Reads', num => 1, label => 'Buffer pool reads' }, + buf_pool_size => { hdr => 'Size', num => 1, label => 'Buffer pool size' }, + bufs_in_node_heap => { hdr => 'Node Heap Bufs', num => 1, label => 'Buffers in buffer pool node heap' }, + bytes_behind_master => { hdr => 'ByteLag', num => 1, label => 'Bytes the slave lags the master in binlog' }, + cell_event_set => { hdr => 'Ending?', num => 1, label => 'Whether the cell event is set' }, + cell_waiting => { hdr => 'Waiting?', num => 1, label => 'Whether the cell is waiting' }, + child_db => { hdr => 'Child DB', num => 0, label => 'The database of the child table' }, + child_index => { hdr => 'Child Index', num => 0, label => 'The index in the child table' }, + child_table => { hdr => 'Child Table', num => 0, label => 'The child table' }, + cmd => { hdr => 'Cmd', num => 0, label => 'Type of command being executed', }, + cnt => { hdr => 'Cnt', num => 0, label => 'Count', agg => 'count', aggonly => 1 }, + connect_retry => { hdr => 'Connect Retry', num => 1, label => 'Slave connect-retry timeout' }, + cxn => { hdr => 'CXN', num => 0, label => 'Connection from which the data came', }, + db => { hdr => 'DB', num => 0, label => 'Current database', }, + dict_mem_alloc => { hdr => 'Dict Mem', num => 1, label => 'Dictionary memory allocated' }, + dirty_bufs => { hdr => 'Dirty Buf', num => 1, label => 'Dirty buffer pool pages' }, + dl_txn_num => { hdr => 'Num', num => 0, label => 'Deadlocked transaction number', }, + event_set => { hdr => 'Evt Set?', num => 1, label => '[Win32] if a wait event is set', }, + exec_master_log_pos => { hdr => 'Exec Master Log Pos', num => 1, label => 'Exec Master Log Position' }, + fk_name => { hdr => 'Constraint', num => 0, label => 'The name of the FK constraint' }, + free_list_len => { hdr => 'Free List Len', num => 1, label => 'Length of the free list' }, + has_read_view => { hdr => 'Rd View', num => 1, label => 'Whether the transaction has a read view' }, + hash_searches_s => { hdr => 'Hash/Sec', num => 1, label => 'Number of hash searches/sec' }, + hash_table_size => { hdr => 'Size', num => 1, label => 'Number of non-hash searches/sec' }, + heap_no => { hdr => 'Heap', num => 1, label => 'Heap number' }, + heap_size => { hdr => 'Heap', num => 1, label => 'Heap size' }, + history_list_len => { hdr => 'History', num => 1, label => 'History list length' }, + host_and_domain => { hdr => 'Host', num => 0, label => 'Hostname/IP and domain' }, + host_and_port => { hdr => 'Host/IP', num => 0, label => 'Hostname or IP address, and port number', }, + hostname => { hdr => 'Host', num => 0, label => 'Hostname' }, + index => { hdr => 'Index', num => 0, label => 'The index involved' }, + index_ref => { hdr => 'Index Ref', num => 0, label => 'Index referenced' }, + info => { hdr => 'Query', num => 0, label => 'Info or the current query', }, + insert_intention => { hdr => 'Ins Intent', num => 1, label => 'Whether the thread was trying to insert' }, + inserts => { hdr => 'Inserts', num => 1, label => 'Inserts' }, + io_bytes_s => { hdr => 'Bytes/Sec', num => 1, label => 'Average I/O bytes/sec' }, + io_flush_type => { hdr => 'Flush Type', num => 0, label => 'I/O Flush Type' }, + io_fsyncs_s => { hdr => 'fsyncs/sec', num => 1, label => 'I/O fsyncs/sec' }, + io_reads_s => { hdr => 'Reads/Sec', num => 1, label => 'Average I/O reads/sec' }, + io_writes_s => { hdr => 'Writes/Sec', num => 1, label => 'Average I/O writes/sec' }, + ip => { hdr => 'IP', num => 0, label => 'IP address' }, + is_name_locked => { hdr => 'Locked', num => 1, label => 'Whether table is name locked', }, + key_buffer_hit => { hdr => 'KCacheHit', num => 1, label => 'Key cache hit ratio', }, + key_len => { hdr => 'Key Length', num => 1, label => 'Number of bytes used in the key' }, + last_chkp => { hdr => 'Last Checkpoint', num => 0, label => 'Last log checkpoint' }, + last_errno => { hdr => 'Last Errno', num => 1, label => 'Last error number' }, + last_error => { hdr => 'Last Error', num => 0, label => 'Last error' }, + last_s_file_name => { hdr => 'S-File', num => 0, label => 'Filename where last read locked' }, + last_s_line => { hdr => 'S-Line', num => 1, label => 'Line where last read locked' }, + last_x_file_name => { hdr => 'X-File', num => 0, label => 'Filename where last write locked' }, + last_x_line => { hdr => 'X-Line', num => 1, label => 'Line where last write locked' }, + last_pct => { hdr => 'Pct', num => 1, label => 'Last Percentage' }, + last_total => { hdr => 'Last Total', num => 1, label => 'Last Total' }, + last_value => { hdr => 'Last Incr', num => 1, label => 'Last Value' }, + load => { hdr => 'Load', num => 1, label => 'Server load' }, + lock_cfile_name => { hdr => 'Crtd File', num => 0, label => 'Filename where lock created' }, + lock_cline => { hdr => 'Crtd Line', num => 1, label => 'Line where lock created' }, + lock_mem_addr => { hdr => 'Addr', num => 0, label => 'The lock memory address' }, + lock_mode => { hdr => 'Mode', num => 0, label => 'The lock mode' }, + lock_structs => { hdr => 'LStrcts', num => 1, label => 'Number of lock structs' }, + lock_type => { hdr => 'Type', num => 0, label => 'The lock type' }, + lock_var => { hdr => 'Lck Var', num => 1, label => 'The lock variable' }, + lock_wait_time => { hdr => 'Wait', num => 1, label => 'How long txn has waited for a lock' }, + log_flushed_to => { hdr => 'Flushed To', num => 0, label => 'Log position flushed to' }, + log_ios_done => { hdr => 'IO Done', num => 1, label => 'Log I/Os done' }, + log_ios_s => { hdr => 'IO/Sec', num => 1, label => 'Average log I/Os per sec' }, + log_seq_no => { hdr => 'Sequence No.', num => 0, label => 'Log sequence number' }, + main_thread_id => { hdr => 'Main Thread ID', num => 1, label => 'Main thread ID' }, + main_thread_proc_no => { hdr => 'Main Thread Proc', num => 1, label => 'Main thread process number' }, + main_thread_state => { hdr => 'Main Thread State', num => 0, label => 'Main thread state' }, + master_file => { hdr => 'File', num => 0, label => 'Master file' }, + master_host => { hdr => 'Master', num => 0, label => 'Master server hostname' }, + master_log_file => { hdr => 'Master Log File', num => 0, label => 'Master log file' }, + master_port => { hdr => 'Master Port', num => 1, label => 'Master port' }, + master_pos => { hdr => 'Position', num => 1, label => 'Master position' }, + master_ssl_allowed => { hdr => 'Master SSL Allowed', num => 0, label => 'Master SSL Allowed' }, + master_ssl_ca_file => { hdr => 'Master SSL CA File', num => 0, label => 'Master SSL Cert Auth File' }, + master_ssl_ca_path => { hdr => 'Master SSL CA Path', num => 0, label => 'Master SSL Cert Auth Path' }, + master_ssl_cert => { hdr => 'Master SSL Cert', num => 0, label => 'Master SSL Cert' }, + master_ssl_cipher => { hdr => 'Master SSL Cipher', num => 0, label => 'Master SSL Cipher' }, + master_ssl_key => { hdr => 'Master SSL Key', num => 0, label => 'Master SSL Key' }, + master_user => { hdr => 'Master User', num => 0, label => 'Master username' }, + max_txn => { hdr => 'MaxTxnTime', num => 1, label => 'MaxTxn' }, + merged_recs => { hdr => 'Merged Recs', num => 1, label => 'Merged records' }, + merges => { hdr => 'Merges', num => 1, label => 'Merges' }, + mutex_os_waits => { hdr => 'Waits', num => 1, label => 'Mutex OS Waits' }, + mutex_spin_rounds => { hdr => 'Rounds', num => 1, label => 'Mutex Spin Rounds' }, + mutex_spin_waits => { hdr => 'Spins', num => 1, label => 'Mutex Spin Waits' }, + mysql_thread_id => { hdr => 'ID', num => 1, label => 'MySQL connection (thread) ID', }, + name => { hdr => 'Name', num => 0, label => 'Variable Name' }, + n_bits => { hdr => '# Bits', num => 1, label => 'Number of bits' }, + non_hash_searches_s => { hdr => 'Non-Hash/Sec', num => 1, label => 'Non-hash searches/sec' }, + num_deletes => { hdr => 'Del', num => 1, label => 'Number of deletes' }, + num_deletes_sec => { hdr => 'Del/Sec', num => 1, label => 'Number of deletes' }, + num_inserts => { hdr => 'Ins', num => 1, label => 'Number of inserts' }, + num_inserts_sec => { hdr => 'Ins/Sec', num => 1, label => 'Number of inserts' }, + num_readers => { hdr => 'Readers', num => 1, label => 'Number of readers' }, + num_reads => { hdr => 'Read', num => 1, label => 'Number of reads' }, + num_reads_sec => { hdr => 'Read/Sec', num => 1, label => 'Number of reads' }, + num_res_ext => { hdr => 'BTree Extents', num => 1, label => 'Number of extents reserved for B-Tree' }, + num_rows => { hdr => 'Row Count', num => 1, label => 'Number of rows estimated to examine' }, + num_times_open => { hdr => 'In Use', num => 1, label => '# times table is opened', }, + num_txns => { hdr => 'Txns', num => 1, label => 'Number of transactions' }, + num_updates => { hdr => 'Upd', num => 1, label => 'Number of updates' }, + num_updates_sec => { hdr => 'Upd/Sec', num => 1, label => 'Number of updates' }, + os_file_reads => { hdr => 'OS Reads', num => 1, label => 'OS file reads' }, + os_file_writes => { hdr => 'OS Writes', num => 1, label => 'OS file writes' }, + os_fsyncs => { hdr => 'OS fsyncs', num => 1, label => 'OS fsyncs' }, + os_thread_id => { hdr => 'OS Thread', num => 1, label => 'The operating system thread ID' }, + p_aio_writes => { hdr => 'Async Wrt', num => 1, label => 'Pending asynchronous I/O writes' }, + p_buf_pool_flushes => { hdr => 'Buffer Pool Flushes', num => 1, label => 'Pending buffer pool flushes' }, + p_ibuf_aio_reads => { hdr => 'IBuf Async Rds', num => 1, label => 'Pending insert buffer asynch I/O reads' }, + p_log_flushes => { hdr => 'Log Flushes', num => 1, label => 'Pending log flushes' }, + p_log_ios => { hdr => 'Log I/Os', num => 1, label => 'Pending log I/O operations' }, + p_normal_aio_reads => { hdr => 'Async Rds', num => 1, label => 'Pending asynchronous I/O reads' }, + p_preads => { hdr => 'preads', num => 1, label => 'Pending p-reads' }, + p_pwrites => { hdr => 'pwrites', num => 1, label => 'Pending p-writes' }, + p_sync_ios => { hdr => 'Sync I/Os', num => 1, label => 'Pending synchronous I/O operations' }, + page_creates_sec => { hdr => 'Creates/Sec', num => 1, label => 'Page creates/sec' }, + page_no => { hdr => 'Page', num => 1, label => 'Page number' }, + page_reads_sec => { hdr => 'Reads/Sec', num => 1, label => 'Page reads per second' }, + page_writes_sec => { hdr => 'Writes/Sec', num => 1, label => 'Page writes per second' }, + pages_created => { hdr => 'Created', num => 1, label => 'Pages created' }, + pages_modified => { hdr => 'Dirty Pages', num => 1, label => 'Pages modified (dirty)' }, + pages_read => { hdr => 'Reads', num => 1, label => 'Pages read' }, + pages_total => { hdr => 'Pages', num => 1, label => 'Pages total' }, + pages_written => { hdr => 'Writes', num => 1, label => 'Pages written' }, + parent_col => { hdr => 'Parent Column', num => 0, label => 'The referred column in the parent table', }, + parent_db => { hdr => 'Parent DB', num => 0, label => 'The database of the parent table' }, + parent_index => { hdr => 'Parent Index', num => 0, label => 'The referred index in the parent table' }, + parent_table => { hdr => 'Parent Table', num => 0, label => 'The parent table' }, + part_id => { hdr => 'Part ID', num => 1, label => 'Sub-part ID of the query' }, + partitions => { hdr => 'Partitions', num => 0, label => 'Query partitions used' }, + pct => { hdr => 'Pct', num => 1, label => 'Percentage' }, + pending_chkp_writes => { hdr => 'Chkpt Writes', num => 1, label => 'Pending log checkpoint writes' }, + pending_log_writes => { hdr => 'Log Writes', num => 1, label => 'Pending log writes' }, + port => { hdr => 'Port', num => 1, label => 'Client port number', }, + possible_keys => { hdr => 'Poss. Keys', num => 0, label => 'Possible keys' }, + proc_no => { hdr => 'Proc', num => 1, label => 'Process number' }, + q_cache_hit => { hdr => 'QCacheHit', num => 1, label => 'Query cache hit ratio', }, + qps => { hdr => 'QPS', num => 1, label => 'How many queries/sec', }, + queries_in_queue => { hdr => 'Queries Queued', num => 1, label => 'Queries in queue' }, + queries_inside => { hdr => 'Queries Inside', num => 1, label => 'Queries inside InnoDB' }, + query_id => { hdr => 'Query ID', num => 1, label => 'Query ID' }, + query_status => { hdr => 'Query Status', num => 0, label => 'The query status' }, + query_text => { hdr => 'Query Text', num => 0, label => 'The query text' }, + questions => { hdr => 'Questions', num => 1, label => 'How many queries the server has gotten', }, + read_master_log_pos => { hdr => 'Read Master Pos', num => 1, label => 'Read master log position' }, + read_views_open => { hdr => 'Rd Views', num => 1, label => 'Number of read views open' }, + reads_pending => { hdr => 'Pending Reads', num => 1, label => 'Reads pending' }, + relay_log_file => { hdr => 'Relay File', num => 0, label => 'Relay log file' }, + relay_log_pos => { hdr => 'Relay Pos', num => 1, label => 'Relay log position' }, + relay_log_size => { hdr => 'Relay Size', num => 1, label => 'Relay log size' }, + relay_master_log_file => { hdr => 'Relay Master File', num => 0, label => 'Relay master log file' }, + replicate_do_db => { hdr => 'Do DB', num => 0, label => 'Replicate-do-db setting' }, + replicate_do_table => { hdr => 'Do Table', num => 0, label => 'Replicate-do-table setting' }, + replicate_ignore_db => { hdr => 'Ignore DB', num => 0, label => 'Replicate-ignore-db setting' }, + replicate_ignore_table => { hdr => 'Ignore Table', num => 0, label => 'Replicate-do-table setting' }, + replicate_wild_do_table => { hdr => 'Wild Do Table', num => 0, label => 'Replicate-wild-do-table setting' }, + replicate_wild_ignore_table => { hdr => 'Wild Ignore Table', num => 0, label => 'Replicate-wild-ignore-table setting' }, + request_type => { hdr => 'Type', num => 0, label => 'Type of lock the thread waits for' }, + reservation_count => { hdr => 'ResCnt', num => 1, label => 'Reservation Count' }, + row_locks => { hdr => 'RLocks', num => 1, label => 'Number of row locks' }, + rw_excl_os_waits => { hdr => 'RW Waits', num => 1, label => 'R/W Excl. OS Waits' }, + rw_excl_spins => { hdr => 'RW Spins', num => 1, label => 'R/W Excl. Spins' }, + rw_shared_os_waits => { hdr => 'Sh Waits', num => 1, label => 'R/W Shared OS Waits' }, + rw_shared_spins => { hdr => 'Sh Spins', num => 1, label => 'R/W Shared Spins' }, + scan_type => { hdr => 'Type', num => 0, label => 'Scan type in chosen' }, + seg_size => { hdr => 'Seg. Size', num => 1, label => 'Segment size' }, + select_type => { hdr => 'Select Type', num => 0, label => 'Type of select used' }, + signal_count => { hdr => 'Signals', num => 1, label => 'Signal Count' }, + size => { hdr => 'Size', num => 1, label => 'Size of the tablespace' }, + skip_counter => { hdr => 'Skip Counter', num => 1, label => 'Skip counter' }, + slave_catchup_rate => { hdr => 'Catchup', num => 1, label => 'How fast the slave is catching up in the binlog' }, + slave_io_running => { hdr => 'Slave-IO', num => 0, label => 'Whether the slave I/O thread is running' }, + slave_io_state => { hdr => 'Slave IO State', num => 0, label => 'Slave I/O thread state' }, + slave_open_temp_tables => { hdr => 'Temp', num => 1, label => 'Slave open temp tables' }, + slave_sql_running => { hdr => 'Slave-SQL', num => 0, label => 'Whether the slave SQL thread is running' }, + slow => { hdr => 'Slow', num => 1, label => 'How many slow queries', }, + space_id => { hdr => 'Space', num => 1, label => 'Tablespace ID' }, + special => { hdr => 'Special', num => 0, label => 'Special/Other info' }, + state => { hdr => 'State', num => 0, label => 'Connection state', maxw => 18, }, + tables_in_use => { hdr => 'Tbl Used', num => 1, label => 'Number of tables in use' }, + tables_locked => { hdr => 'Tbl Lck', num => 1, label => 'Number of tables locked' }, + tbl => { hdr => 'Table', num => 0, label => 'Table', }, + thread => { hdr => 'Thread', num => 1, label => 'Thread number' }, + thread_decl_inside => { hdr => 'Thread Inside', num => 0, label => 'What the thread is declared inside' }, + thread_purpose => { hdr => 'Purpose', num => 0, label => "The thread's purpose" }, + thread_status => { hdr => 'Thread Status', num => 0, label => 'The thread status' }, + time => { hdr => 'Time', num => 1, label => 'Time since the last event', }, + time_behind_master => { hdr => 'TimeLag', num => 1, label => 'Time slave lags master' }, + timestring => { hdr => 'Timestring', num => 0, label => 'Time the event occurred' }, + total => { hdr => 'Total', num => 1, label => 'Total' }, + total_mem_alloc => { hdr => 'Memory', num => 1, label => 'Total memory allocated' }, + truncates => { hdr => 'Trunc', num => 0, label => 'Whether the deadlock is truncating InnoDB status' }, + txn_doesnt_see_ge => { hdr => "Txn Won't See", num => 0, label => 'Where txn read view is limited' }, + txn_id => { hdr => 'ID', num => 0, label => 'Transaction ID' }, + txn_sees_lt => { hdr => 'Txn Sees', num => 1, label => 'Where txn read view is limited' }, + txn_status => { hdr => 'Txn Status', num => 0, label => 'Transaction status' }, + txn_time_remain => { hdr => 'Remaining', num => 1, label => 'Time until txn rollback/commit completes' }, + undo_log_entries => { hdr => 'Undo', num => 1, label => 'Number of undo log entries' }, + undo_for => { hdr => 'Undo', num => 0, label => 'Undo for' }, + until_condition => { hdr => 'Until Condition', num => 0, label => 'Slave until condition' }, + until_log_file => { hdr => 'Until Log File', num => 0, label => 'Slave until log file' }, + until_log_pos => { hdr => 'Until Log Pos', num => 1, label => 'Slave until log position' }, + used_cells => { hdr => 'Cells Used', num => 1, label => 'Number of cells used' }, + used_bufs => { hdr => 'Used Bufs', num => 1, label => 'Number of buffer pool pages used' }, + user => { hdr => 'User', num => 0, label => 'Database username', }, + value => { hdr => 'Value', num => 1, label => 'Value' }, + versions => { hdr => 'Versions', num => 1, label => 'Number of InnoDB MVCC versions unpurged' }, + victim => { hdr => 'Victim', num => 0, label => 'Whether this txn was the deadlock victim' }, + wait_array_size => { hdr => 'Wait Array Size', num => 1, label => 'Wait Array Size' }, + wait_status => { hdr => 'Lock Status', num => 0, label => 'Status of txn locks' }, + waited_at_filename => { hdr => 'File', num => 0, label => 'Filename at which thread waits' }, + waited_at_line => { hdr => 'Line', num => 1, label => 'Line at which thread waits' }, + waiters_flag => { hdr => 'Waiters', num => 1, label => 'Waiters Flag' }, + waiting => { hdr => 'Waiting', num => 1, label => 'Whether lock is being waited for' }, + when => { hdr => 'When', num => 0, label => 'Time scale' }, + writer_lock_mode => { hdr => 'Wrtr Lck Mode', num => 0, label => 'Writer lock mode' }, + writer_thread => { hdr => 'Wrtr Thread', num => 1, label => 'Writer thread ID' }, + writes_pending => { hdr => 'Writes', num => 1, label => 'Number of writes pending' }, + writes_pending_flush_list => { hdr => 'Flush List Writes', num => 1, label => 'Number of flush list writes pending' }, + writes_pending_lru => { hdr => 'LRU Writes', num => 1, label => 'Number of LRU writes pending' }, + writes_pending_single_page => { hdr => '1-Page Writes', num => 1, label => 'Number of 1-page writes pending' }, +); + +# Apply a default property or three. By default, columns are not width-constrained, +# aligned left, and sorted alphabetically, not numerically. +foreach my $col ( values %columns ) { + map { $col->{$_} ||= 0 } qw(num minw maxw); + $col->{just} = $col->{num} ? '' : '-'; +} + +# Filters {{{3 +# This hash defines every filter that can be applied to a table. These +# become part of tbl_meta as well. Each filter is just an expression that +# returns true or false. +# Properties of each entry: +# * func: the subroutine +# * name: the name, repeated +# * user: whether it's a user-defined filter (saved in config) +# * text: text of the subroutine +# * note: explanation +my %filters = (); + +# These are pre-processed to live in %filters above, by compiling them. +my %builtin_filters = ( + hide_self => { + text => <<' END', + return ( !$set->{info} || $set->{info} ne 'SHOW FULL PROCESSLIST' ) + && ( !$set->{query_text} || $set->{query_text} !~ m/INNODB STATUS$/ ); + END + note => 'Removes the innotop processes from the list', + tbls => [qw(innodb_transactions processlist)], + }, + hide_inactive => { + text => <<' END', + return ( !defined($set->{txn_status}) || $set->{txn_status} ne 'not started' ) + && ( !defined($set->{cmd}) || $set->{cmd} !~ m/Sleep|Binlog Dump/ ) + && ( !defined($set->{info}) || $set->{info} =~ m/\S/ ); + END + note => 'Removes processes which are not doing anything', + tbls => [qw(innodb_transactions processlist)], + }, + hide_slave_io => { + text => <<' END', + return !$set->{state} || $set->{state} !~ m/^(?:Waiting for master|Has read all relay)/; + END + note => 'Removes slave I/O threads from the list', + tbls => [qw(processlist slave_io_status)], + }, + table_is_open => { + text => <<' END', + return $set->{num_times_open} + $set->{is_name_locked}; + END + note => 'Removes tables that are not in use or locked', + tbls => [qw(open_tables)], + }, + cxn_is_master => { + text => <<' END', + return $set->{master_file} ? 1 : 0; + END + note => 'Removes servers that are not masters', + tbls => [qw(master_status)], + }, + cxn_is_slave => { + text => <<' END', + return $set->{master_host} ? 1 : 0; + END + note => 'Removes servers that are not slaves', + tbls => [qw(slave_io_status slave_sql_status)], + }, + thd_is_not_waiting => { + text => <<' END', + return $set->{thread_status} !~ m#waiting for i/o request#; + END + note => 'Removes idle I/O threads', + tbls => [qw(io_threads)], + }, +); +foreach my $key ( keys %builtin_filters ) { + my ( $sub, $err ) = compile_filter($builtin_filters{$key}->{text}); + $filters{$key} = { + func => $sub, + text => $builtin_filters{$key}->{text}, + user => 0, + name => $key, # useful for later + note => $builtin_filters{$key}->{note}, + tbls => $builtin_filters{$key}->{tbls}, + } +} + +# Variable sets {{{3 +# Sets (arrayrefs) of variables that are used in S mode. They are read/written to +# the config file. +my %var_sets = ( + general => { + text => join( + ', ', + 'set_precision(Questions/Uptime_hires) as QPS', + 'set_precision(Com_commit/Uptime_hires) as Commit_PS', + 'set_precision((Com_rollback||0)/(Com_commit||1)) as Rollback_Commit', + 'set_precision((' + . join('+', map { "($_||0)" } + qw(Com_delete Com_delete_multi Com_insert Com_insert_select Com_replace + Com_replace_select Com_select Com_update Com_update_multi)) + . ')/(Com_commit||1)) as Write_Commit', + 'set_precision((Com_select+(Qcache_hits||0))/((' + . join('+', map { "($_||0)" } + qw(Com_delete Com_delete_multi Com_insert Com_insert_select Com_replace + Com_replace_select Com_select Com_update Com_update_multi)) + . ')||1)) as R_W_Ratio', + 'set_precision(Opened_tables/Uptime_hires) as Opens_PS', + 'percent($cur->{Open_tables}/($cur->{table_cache})) as Table_Cache_Used', + 'set_precision(Threads_created/Uptime_hires) as Threads_PS', + 'percent($cur->{Threads_cached}/($cur->{thread_cache_size}||1)) as Thread_Cache_Used', + 'percent($cur->{Max_used_connections}/($cur->{max_connections}||1)) as CXN_Used_Ever', + 'percent($cur->{Threads_connected}/($cur->{max_connections}||1)) as CXN_Used_Now', + ), + }, + commands => { + text => join( + ', ', + qw(Uptime Questions Com_delete Com_delete_multi Com_insert + Com_insert_select Com_replace Com_replace_select Com_select Com_update + Com_update_multi) + ), + }, + query_status => { + text => join( + ',', + qw( Uptime Select_full_join Select_full_range_join Select_range + Select_range_check Select_scan Slow_queries Sort_merge_passes + Sort_range Sort_rows Sort_scan) + ), + }, + innodb => { + text => join( + ',', + qw( Uptime Innodb_row_lock_current_waits Innodb_row_lock_time + Innodb_row_lock_time_avg Innodb_row_lock_time_max Innodb_row_lock_waits + Innodb_rows_deleted Innodb_rows_inserted Innodb_rows_read + Innodb_rows_updated) + ), + }, + txn => { + text => join( + ',', + qw( Uptime Com_begin Com_commit Com_rollback Com_savepoint + Com_xa_commit Com_xa_end Com_xa_prepare Com_xa_recover Com_xa_rollback + Com_xa_start) + ), + }, + key_cache => { + text => join( + ',', + qw( Uptime Key_blocks_not_flushed Key_blocks_unused Key_blocks_used + Key_read_requests Key_reads Key_write_requests Key_writes ) + ), + }, + query_cache => { + text => join( + ',', + "percent($exprs{QcacheHitRatio}) as Hit_Pct", + 'set_precision((Qcache_hits||0)/(Qcache_inserts||1)) as Hit_Ins', + 'set_precision((Qcache_lowmem_prunes||0)/Uptime_hires) as Lowmem_Prunes_sec', + 'percent(1-((Qcache_free_blocks||0)/(Qcache_total_blocks||1))) as Blocks_used', + qw( Qcache_free_blocks Qcache_free_memory Qcache_not_cached Qcache_queries_in_cache) + ), + }, + handler => { + text => join( + ',', + qw( Uptime Handler_read_key Handler_read_first Handler_read_next + Handler_read_prev Handler_read_rnd Handler_read_rnd_next Handler_delete + Handler_update Handler_write) + ), + }, + cxns_files_threads => { + text => join( + ',', + qw( Uptime Aborted_clients Aborted_connects Bytes_received Bytes_sent + Compression Connections Created_tmp_disk_tables Created_tmp_files + Created_tmp_tables Max_used_connections Open_files Open_streams + Open_tables Opened_tables Table_locks_immediate Table_locks_waited + Threads_cached Threads_connected Threads_created Threads_running) + ), + }, + prep_stmt => { + text => join( + ',', + qw( Uptime Com_dealloc_sql Com_execute_sql Com_prepare_sql Com_reset + Com_stmt_close Com_stmt_execute Com_stmt_fetch Com_stmt_prepare + Com_stmt_reset Com_stmt_send_long_data ) + ), + }, + innodb_health => { + text => join( + ',', + "$exprs{OldVersions} as OldVersions", + qw(IB_sm_mutex_spin_waits IB_sm_mutex_spin_rounds IB_sm_mutex_os_waits), + "$exprs{NumTxns} as NumTxns", + "$exprs{MaxTxnTime} as MaxTxnTime", + qw(IB_ro_queries_inside IB_ro_queries_in_queue), + "set_precision($exprs{DirtyBufs} * 100) as dirty_bufs", + "set_precision($exprs{BufPoolFill} * 100) as buf_fill", + qw(IB_bp_pages_total IB_bp_pages_read IB_bp_pages_written IB_bp_pages_created) + ), + }, + innodb_health2 => { + text => join( + ', ', + 'percent(1-((Innodb_buffer_pool_pages_free||0)/($cur->{Innodb_buffer_pool_pages_total}||1))) as BP_page_cache_usage', + 'percent(1-((Innodb_buffer_pool_reads||0)/(Innodb_buffer_pool_read_requests||1))) as BP_cache_hit_ratio', + 'Innodb_buffer_pool_wait_free', + 'Innodb_log_waits', + ), + }, + slow_queries => { + text => join( + ', ', + 'set_precision(Slow_queries/Uptime_hires) as Slow_PS', + 'set_precision(Select_full_join/Uptime_hires) as Full_Join_PS', + 'percent(Select_full_join/(Com_select||1)) as Full_Join_Ratio', + ), + }, +); + +# Server sets {{{3 +# Defines sets of servers between which the user can quickly switch. +my %server_groups; + +# Connections {{{3 +# This hash defines server connections. Each connection is a string that can be passed to +# the DBI connection. These are saved in the connections section in the config file. +my %connections; +# Defines the parts of connections. +my @conn_parts = qw(user have_user pass have_pass dsn savepass dl_table); + +# Graph widths {{{3 +# This hash defines the max values seen for various status/variable values, for graphing. +# These are stored in their own section in the config file. These are just initial values: +my %mvs = ( + Com_select => 50, + Com_insert => 50, + Com_update => 50, + Com_delete => 50, + Questions => 100, +); + +# ########################################################################### +# Valid Term::ANSIColor color strings. +# ########################################################################### +my %ansicolors = map { $_ => 1 } + qw( black blink blue bold clear concealed cyan dark green magenta on_black + on_blue on_cyan on_green on_magenta on_red on_white on_yellow red reset + reverse underline underscore white yellow); + +# ########################################################################### +# Valid comparison operators for color rules +# ########################################################################### +my %comp_ops = ( + '==' => 'Numeric equality', + '>' => 'Numeric greater-than', + '<' => 'Numeric less-than', + '>=' => 'Numeric greater-than/equal', + '<=' => 'Numeric less-than/equal', + '!=' => 'Numeric not-equal', + 'eq' => 'String equality', + 'gt' => 'String greater-than', + 'lt' => 'String less-than', + 'ge' => 'String greater-than/equal', + 'le' => 'String less-than/equal', + 'ne' => 'String not-equal', + '=~' => 'Pattern match', + '!~' => 'Negated pattern match', +); + +# ########################################################################### +# Valid aggregate functions. +# ########################################################################### +my %agg_funcs = ( + first => sub { + return $_[0] + }, + count => sub { + return 0 + @_; + }, + avg => sub { + my @args = grep { defined $_ } @_; + return (sum(map { m/([\d\.-]+)/g } @args) || 0) / (scalar(@args) || 1); + }, + sum => \&sum, +); + +# ########################################################################### +# Valid functions for transformations. +# ########################################################################### +my %trans_funcs = ( + shorten => \&shorten, + secs_to_time => \&secs_to_time, + no_ctrl_char => \&no_ctrl_char, + percent => \&percent, + commify => \&commify, + dulint_to_int => \&dulint_to_int, + set_precision => \&set_precision, +); + +# Table definitions {{{3 +# This hash defines every table that can get displayed in every mode. Each +# table specifies columns and column data sources. The column is +# defined by the %columns hash. +# +# Example: foo => { src => 'bar' } means the foo column (look at +# $columns{foo} for its definition) gets its data from the 'bar' element of +# the current data set, whatever that is. +# +# These columns are post-processed after being defined, because they get stuff +# from %columns. After all the config is loaded for columns, there's more +# post-processing too; the subroutines compiled from src get added to +# the hash elements for extract_values to use. +# ########################################################################### + +my %tbl_meta = ( + adaptive_hash_index => { + capt => 'Adaptive Hash Index', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + hash_table_size => { src => 'IB_ib_hash_table_size', trans => [qw(shorten)], }, + used_cells => { src => 'IB_ib_used_cells' }, + bufs_in_node_heap => { src => 'IB_ib_bufs_in_node_heap' }, + hash_searches_s => { src => 'IB_ib_hash_searches_s' }, + non_hash_searches_s => { src => 'IB_ib_non_hash_searches_s' }, + }, + visible => [ qw(cxn hash_table_size used_cells bufs_in_node_heap hash_searches_s non_hash_searches_s) ], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'ib', + group_by => [], + aggregate => 0, + }, + buffer_pool => { + capt => 'Buffer Pool', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + total_mem_alloc => { src => 'IB_bp_total_mem_alloc', trans => [qw(shorten)], }, + awe_mem_alloc => { src => 'IB_bp_awe_mem_alloc', trans => [qw(shorten)], }, + add_pool_alloc => { src => 'IB_bp_add_pool_alloc', trans => [qw(shorten)], }, + buf_pool_size => { src => 'IB_bp_buf_pool_size', trans => [qw(shorten)], }, + buf_free => { src => 'IB_bp_buf_free' }, + buf_pool_hit_rate => { src => 'IB_bp_buf_pool_hit_rate' }, + buf_pool_reads => { src => 'IB_bp_buf_pool_reads' }, + buf_pool_hits => { src => 'IB_bp_buf_pool_hits' }, + dict_mem_alloc => { src => 'IB_bp_dict_mem_alloc' }, + pages_total => { src => 'IB_bp_pages_total' }, + pages_modified => { src => 'IB_bp_pages_modified' }, + reads_pending => { src => 'IB_bp_reads_pending' }, + writes_pending => { src => 'IB_bp_writes_pending' }, + writes_pending_lru => { src => 'IB_bp_writes_pending_lru' }, + writes_pending_flush_list => { src => 'IB_bp_writes_pending_flush_list' }, + writes_pending_single_page => { src => 'IB_bp_writes_pending_single_page' }, + page_creates_sec => { src => 'IB_bp_page_creates_sec' }, + page_reads_sec => { src => 'IB_bp_page_reads_sec' }, + page_writes_sec => { src => 'IB_bp_page_writes_sec' }, + pages_created => { src => 'IB_bp_pages_created' }, + pages_read => { src => 'IB_bp_pages_read' }, + pages_written => { src => 'IB_bp_pages_written' }, + }, + visible => [ qw(cxn buf_pool_size buf_free pages_total pages_modified buf_pool_hit_rate total_mem_alloc add_pool_alloc)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'bp', + group_by => [], + aggregate => 0, + }, + # TODO: a new step in set_to_tbl: join result to itself, grouped? + # TODO: this would also enable pulling Q and T data together. + # TODO: using a SQL-ish language would also allow pivots to be easier -- treat the pivoted data as a view and SELECT from it. + cmd_summary => { + capt => 'Command Summary', + cust => {}, + cols => { + name => { src => 'name' }, + total => { src => 'total' }, + value => { src => 'value', agg => 'sum'}, + pct => { src => 'value/total', trans => [qw(percent)] }, + last_total => { src => 'last_total' }, + last_value => { src => 'last_value', agg => 'sum'}, + last_pct => { src => 'last_value/last_total', trans => [qw(percent)] }, + }, + visible => [qw(name value pct last_value last_pct)], + filters => [qw()], + sort_cols => '-value', + sort_dir => '1', + innodb => '', + group_by => [qw(name)], + aggregate => 1, + }, + deadlock_locks => { + capt => 'Deadlock Locks', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + mysql_thread_id => { src => 'mysql_thread_id' }, + dl_txn_num => { src => 'dl_txn_num' }, + lock_type => { src => 'lock_type' }, + space_id => { src => 'space_id' }, + page_no => { src => 'page_no' }, + heap_no => { src => 'heap_no' }, + n_bits => { src => 'n_bits' }, + index => { src => 'index' }, + db => { src => 'db' }, + tbl => { src => 'table' }, + lock_mode => { src => 'lock_mode' }, + special => { src => 'special' }, + insert_intention => { src => 'insert_intention' }, + waiting => { src => 'waiting' }, + }, + visible => [ qw(cxn mysql_thread_id waiting lock_mode db tbl index special insert_intention)], + filters => [], + sort_cols => 'cxn mysql_thread_id', + sort_dir => '1', + innodb => 'dl', + group_by => [], + aggregate => 0, + }, + deadlock_transactions => { + capt => 'Deadlock Transactions', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + active_secs => { src => 'active_secs' }, + dl_txn_num => { src => 'dl_txn_num' }, + has_read_view => { src => 'has_read_view' }, + heap_size => { src => 'heap_size' }, + host_and_domain => { src => 'hostname' }, + hostname => { src => $exprs{Host} }, + ip => { src => 'ip' }, + lock_structs => { src => 'lock_structs' }, + lock_wait_time => { src => 'lock_wait_time', trans => [ qw(secs_to_time) ] }, + mysql_thread_id => { src => 'mysql_thread_id' }, + os_thread_id => { src => 'os_thread_id' }, + proc_no => { src => 'proc_no' }, + query_id => { src => 'query_id' }, + query_status => { src => 'query_status' }, + query_text => { src => 'query_text', trans => [ qw(no_ctrl_char) ] }, + row_locks => { src => 'row_locks' }, + tables_in_use => { src => 'tables_in_use' }, + tables_locked => { src => 'tables_locked' }, + thread_decl_inside => { src => 'thread_decl_inside' }, + thread_status => { src => 'thread_status' }, + 'time' => { src => 'active_secs', trans => [ qw(secs_to_time) ] }, + timestring => { src => 'timestring' }, + txn_doesnt_see_ge => { src => 'txn_doesnt_see_ge' }, + txn_id => { src => 'txn_id' }, + txn_sees_lt => { src => 'txn_sees_lt' }, + txn_status => { src => 'txn_status' }, + truncates => { src => 'truncates' }, + undo_log_entries => { src => 'undo_log_entries' }, + user => { src => 'user' }, + victim => { src => 'victim' }, + wait_status => { src => 'lock_wait_status' }, + }, + visible => [ qw(cxn mysql_thread_id timestring user hostname victim time undo_log_entries lock_structs query_text)], + filters => [], + sort_cols => 'cxn mysql_thread_id', + sort_dir => '1', + innodb => 'dl', + group_by => [], + aggregate => 0, + }, + explain => { + capt => 'EXPLAIN Results', + cust => {}, + cols => { + part_id => { src => 'id' }, + select_type => { src => 'select_type' }, + tbl => { src => 'table' }, + partitions => { src => 'partitions' }, + scan_type => { src => 'type' }, + possible_keys => { src => 'possible_keys' }, + index => { src => 'key' }, + key_len => { src => 'key_len' }, + index_ref => { src => 'ref' }, + num_rows => { src => 'rows' }, + special => { src => 'extra' }, + }, + visible => [ qw(select_type tbl partitions scan_type possible_keys index key_len index_ref num_rows special)], + filters => [], + sort_cols => '', + sort_dir => '1', + innodb => '', + group_by => [], + aggregate => 0, + }, + file_io_misc => { + capt => 'File I/O Misc', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + io_bytes_s => { src => 'IB_io_avg_bytes_s' }, + io_flush_type => { src => 'IB_io_flush_type' }, + io_fsyncs_s => { src => 'IB_io_fsyncs_s' }, + io_reads_s => { src => 'IB_io_reads_s' }, + io_writes_s => { src => 'IB_io_writes_s' }, + os_file_reads => { src => 'IB_io_os_file_reads' }, + os_file_writes => { src => 'IB_io_os_file_writes' }, + os_fsyncs => { src => 'IB_io_os_fsyncs' }, + }, + visible => [ qw(cxn os_file_reads os_file_writes os_fsyncs io_reads_s io_writes_s io_bytes_s)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'io', + group_by => [], + aggregate => 0, + }, + fk_error => { + capt => 'Foreign Key Error Info', + cust => {}, + cols => { + timestring => { src => 'IB_fk_timestring' }, + child_db => { src => 'IB_fk_child_db' }, + child_table => { src => 'IB_fk_child_table' }, + child_index => { src => 'IB_fk_child_index' }, + fk_name => { src => 'IB_fk_fk_name' }, + parent_db => { src => 'IB_fk_parent_db' }, + parent_table => { src => 'IB_fk_parent_table' }, + parent_col => { src => 'IB_fk_parent_col' }, + parent_index => { src => 'IB_fk_parent_index' }, + attempted_op => { src => 'IB_fk_attempted_op' }, + }, + visible => [ qw(timestring child_db child_table child_index parent_db parent_table parent_col parent_index fk_name attempted_op)], + filters => [], + sort_cols => '', + sort_dir => '1', + innodb => 'fk', + group_by => [], + aggregate => 0, + }, + insert_buffers => { + capt => 'Insert Buffers', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + inserts => { src => 'IB_ib_inserts' }, + merged_recs => { src => 'IB_ib_merged_recs' }, + merges => { src => 'IB_ib_merges' }, + size => { src => 'IB_ib_size' }, + free_list_len => { src => 'IB_ib_free_list_len' }, + seg_size => { src => 'IB_ib_seg_size' }, + }, + visible => [ qw(cxn inserts merged_recs merges size free_list_len seg_size)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'ib', + group_by => [], + aggregate => 0, + }, + innodb_locks => { + capt => 'InnoDB Locks', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + db => { src => 'db' }, + index => { src => 'index' }, + insert_intention => { src => 'insert_intention' }, + lock_mode => { src => 'lock_mode' }, + lock_type => { src => 'lock_type' }, + lock_wait_time => { src => 'lock_wait_time', trans => [ qw(secs_to_time) ] }, + mysql_thread_id => { src => 'mysql_thread_id' }, + n_bits => { src => 'n_bits' }, + page_no => { src => 'page_no' }, + space_id => { src => 'space_id' }, + special => { src => 'special' }, + tbl => { src => 'table' }, + 'time' => { src => 'active_secs', hdr => 'Active', trans => [ qw(secs_to_time) ] }, + txn_id => { src => 'txn_id' }, + waiting => { src => 'waiting' }, + }, + visible => [ qw(cxn mysql_thread_id lock_type waiting lock_wait_time time lock_mode db tbl index insert_intention special)], + filters => [], + sort_cols => 'cxn -lock_wait_time', + sort_dir => '1', + innodb => 'tx', + colors => [ + { col => 'lock_wait_time', op => '>', arg => 60, color => 'red' }, + { col => 'lock_wait_time', op => '>', arg => 30, color => 'yellow' }, + { col => 'lock_wait_time', op => '>', arg => 10, color => 'green' }, + ], + group_by => [], + aggregate => 0, + }, + innodb_transactions => { + capt => 'InnoDB Transactions', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + active_secs => { src => 'active_secs' }, + has_read_view => { src => 'has_read_view' }, + heap_size => { src => 'heap_size' }, + hostname => { src => $exprs{Host} }, + ip => { src => 'ip' }, + wait_status => { src => 'lock_wait_status' }, + lock_wait_time => { src => 'lock_wait_time', trans => [ qw(secs_to_time) ] }, + lock_structs => { src => 'lock_structs' }, + mysql_thread_id => { src => 'mysql_thread_id' }, + os_thread_id => { src => 'os_thread_id' }, + proc_no => { src => 'proc_no' }, + query_id => { src => 'query_id' }, + query_status => { src => 'query_status' }, + query_text => { src => 'query_text', trans => [ qw(no_ctrl_char) ] }, + txn_time_remain => { src => $exprs{TxnTimeRemain}, trans => [ qw(secs_to_time) ] }, + row_locks => { src => 'row_locks' }, + tables_in_use => { src => 'tables_in_use' }, + tables_locked => { src => 'tables_locked' }, + thread_decl_inside => { src => 'thread_decl_inside' }, + thread_status => { src => 'thread_status' }, + 'time' => { src => 'active_secs', trans => [ qw(secs_to_time) ], agg => 'sum' }, + txn_doesnt_see_ge => { src => 'txn_doesnt_see_ge' }, + txn_id => { src => 'txn_id' }, + txn_sees_lt => { src => 'txn_sees_lt' }, + txn_status => { src => 'txn_status', minw => 10, maxw => 10 }, + undo_log_entries => { src => 'undo_log_entries' }, + user => { src => 'user', maxw => 10 }, + cnt => { src => 'mysql_thread_id', minw => 0 }, + }, + visible => [ qw(cxn cnt mysql_thread_id user hostname txn_status time undo_log_entries query_text)], + filters => [ qw( hide_self hide_inactive ) ], + sort_cols => '-active_secs txn_status cxn mysql_thread_id', + sort_dir => '1', + innodb => 'tx', + hide_caption => 1, + colors => [ + { col => 'wait_status', op => 'eq', arg => 'LOCK WAIT', color => 'black on_red' }, + { col => 'time', op => '>', arg => 600, color => 'red' }, + { col => 'time', op => '>', arg => 300, color => 'yellow' }, + { col => 'time', op => '>', arg => 60, color => 'green' }, + { col => 'time', op => '>', arg => 30, color => 'cyan' }, + { col => 'txn_status', op => 'eq', arg => 'not started', color => 'white' }, + ], + group_by => [ qw(cxn txn_status) ], + aggregate => 0, + }, + io_threads => { + capt => 'I/O Threads', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + thread => { src => 'thread' }, + thread_purpose => { src => 'purpose' }, + event_set => { src => 'event_set' }, + thread_status => { src => 'state' }, + }, + visible => [ qw(cxn thread thread_purpose thread_status)], + filters => [ qw() ], + sort_cols => 'cxn thread', + sort_dir => '1', + innodb => 'io', + group_by => [], + aggregate => 0, + }, + log_statistics => { + capt => 'Log Statistics', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + last_chkp => { src => 'IB_lg_last_chkp' }, + log_flushed_to => { src => 'IB_lg_log_flushed_to' }, + log_ios_done => { src => 'IB_lg_log_ios_done' }, + log_ios_s => { src => 'IB_lg_log_ios_s' }, + log_seq_no => { src => 'IB_lg_log_seq_no' }, + pending_chkp_writes => { src => 'IB_lg_pending_chkp_writes' }, + pending_log_writes => { src => 'IB_lg_pending_log_writes' }, + }, + visible => [ qw(cxn log_seq_no log_flushed_to last_chkp log_ios_done log_ios_s)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'lg', + group_by => [], + aggregate => 0, + }, + master_status => { + capt => 'Master Status', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + binlog_do_db => { src => 'binlog_do_db' }, + binlog_ignore_db => { src => 'binlog_ignore_db' }, + master_file => { src => 'file' }, + master_pos => { src => 'position' }, + binlog_cache_overflow => { src => '(Binlog_cache_disk_use||0)/(Binlog_cache_use||1)', trans => [ qw(percent) ] }, + }, + visible => [ qw(cxn master_file master_pos binlog_cache_overflow)], + filters => [ qw(cxn_is_master) ], + sort_cols => 'cxn', + sort_dir => '1', + innodb => '', + group_by => [], + aggregate => 0, + }, + pending_io => { + capt => 'Pending I/O', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + p_normal_aio_reads => { src => 'IB_io_pending_normal_aio_reads' }, + p_aio_writes => { src => 'IB_io_pending_aio_writes' }, + p_ibuf_aio_reads => { src => 'IB_io_pending_ibuf_aio_reads' }, + p_sync_ios => { src => 'IB_io_pending_sync_ios' }, + p_buf_pool_flushes => { src => 'IB_io_pending_buffer_pool_flushes' }, + p_log_flushes => { src => 'IB_io_pending_log_flushes' }, + p_log_ios => { src => 'IB_io_pending_log_ios' }, + p_preads => { src => 'IB_io_pending_preads' }, + p_pwrites => { src => 'IB_io_pending_pwrites' }, + }, + visible => [ qw(cxn p_normal_aio_reads p_aio_writes p_ibuf_aio_reads p_sync_ios p_log_flushes p_log_ios)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'io', + group_by => [], + aggregate => 0, + }, + open_tables => { + capt => 'Open Tables', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + db => { src => 'database' }, + tbl => { src => 'table' }, + num_times_open => { src => 'in_use' }, + is_name_locked => { src => 'name_locked' }, + }, + visible => [ qw(cxn db tbl num_times_open is_name_locked)], + filters => [ qw(table_is_open) ], + sort_cols => '-num_times_open cxn db tbl', + sort_dir => '1', + innodb => '', + group_by => [], + aggregate => 0, + }, + page_statistics => { + capt => 'Page Statistics', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + pages_read => { src => 'IB_bp_pages_read' }, + pages_written => { src => 'IB_bp_pages_written' }, + pages_created => { src => 'IB_bp_pages_created' }, + page_reads_sec => { src => 'IB_bp_page_reads_sec' }, + page_writes_sec => { src => 'IB_bp_page_writes_sec' }, + page_creates_sec => { src => 'IB_bp_page_creates_sec' }, + }, + visible => [ qw(cxn pages_read pages_written pages_created page_reads_sec page_writes_sec page_creates_sec)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'bp', + group_by => [], + aggregate => 0, + }, + processlist => { + capt => 'MySQL Process List', + cust => {}, + cols => { + cxn => { src => 'cxn', minw => 6, maxw => 10 }, + mysql_thread_id => { src => 'id', minw => 6, maxw => 0 }, + user => { src => 'user', minw => 5, maxw => 8 }, + hostname => { src => $exprs{Host}, minw => 13, maxw => 8, }, + port => { src => $exprs{Port}, minw => 0, maxw => 0, }, + host_and_port => { src => 'host', minw => 0, maxw => 0 }, + db => { src => 'db', minw => 6, maxw => 12 }, + cmd => { src => 'command', minw => 5, maxw => 0 }, + time => { src => 'time', minw => 5, maxw => 0, trans => [ qw(secs_to_time) ], agg => 'sum' }, + state => { src => 'state', minw => 0, maxw => 0 }, + info => { src => 'info', minw => 0, maxw => 0, trans => [ qw(no_ctrl_char) ] }, + cnt => { src => 'id', minw => 0, maxw => 0 }, + }, + visible => [ qw(cxn cmd cnt mysql_thread_id user hostname db time info)], + filters => [ qw(hide_self hide_inactive hide_slave_io) ], + sort_cols => '-time cxn hostname mysql_thread_id', + sort_dir => '1', + innodb => '', + hide_caption => 1, + colors => [ + { col => 'state', op => 'eq', arg => 'Locked', color => 'black on_red' }, + { col => 'cmd', op => 'eq', arg => 'Sleep', color => 'white' }, + { col => 'user', op => 'eq', arg => 'system user', color => 'white' }, + { col => 'cmd', op => 'eq', arg => 'Connect', color => 'white' }, + { col => 'cmd', op => 'eq', arg => 'Binlog Dump', color => 'white' }, + { col => 'time', op => '>', arg => 600, color => 'red' }, + { col => 'time', op => '>', arg => 120, color => 'yellow' }, + { col => 'time', op => '>', arg => 60, color => 'green' }, + { col => 'time', op => '>', arg => 30, color => 'cyan' }, + ], + group_by => [qw(cxn cmd)], + aggregate => 0, + }, + + # TODO: some more columns: + # kb_used=hdr='BufUsed' minw='0' num='0' src='percent(1 - ((Key_blocks_unused * key_cache_block_size) / (key_buffer_size||1)))' dec='0' trans='' tbl='q_header' just='-' user='1' maxw='0' label='User-defined' + # retries=hdr='Retries' minw='0' num='0' src='Slave_retried_transactions' dec='0' trans='' tbl='slave_sql_status' just='-' user='1' maxw='0' label='User-defined' + # thd=hdr='Thd' minw='0' num='0' src='Threads_connected' dec='0' trans='' tbl='slave_sql_status' just='-' user='1' maxw='0' label='User-defined' + + q_header => { + capt => 'Q-mode Header', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + questions => { src => 'Questions' }, + qps => { src => 'Questions/Uptime_hires', dec => 1, trans => [qw(shorten)] }, + load => { src => $exprs{ServerLoad}, dec => 1, trans => [qw(shorten)] }, + slow => { src => 'Slow_queries', dec => 1, trans => [qw(shorten)] }, + q_cache_hit => { src => $exprs{QcacheHitRatio}, dec => 1, trans => [qw(percent)] }, + key_buffer_hit => { src => '1-(Key_reads/(Key_read_requests||1))', dec => 1, trans => [qw(percent)] }, + bps_in => { src => 'Bytes_received/Uptime_hires', dec => 1, trans => [qw(shorten)] }, + bps_out => { src => 'Bytes_sent/Uptime_hires', dec => 1, trans => [qw(shorten)] }, + when => { src => 'when' }, + }, + visible => [ qw(cxn when load qps slow q_cache_hit key_buffer_hit bps_in bps_out)], + filters => [], + sort_cols => 'when cxn', + sort_dir => '1', + innodb => '', + hide_caption => 1, + group_by => [], + aggregate => 0, + }, + row_operations => { + capt => 'InnoDB Row Operations', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + num_inserts => { src => 'IB_ro_num_rows_ins' }, + num_updates => { src => 'IB_ro_num_rows_upd' }, + num_reads => { src => 'IB_ro_num_rows_read' }, + num_deletes => { src => 'IB_ro_num_rows_del' }, + num_inserts_sec => { src => 'IB_ro_ins_sec' }, + num_updates_sec => { src => 'IB_ro_upd_sec' }, + num_reads_sec => { src => 'IB_ro_read_sec' }, + num_deletes_sec => { src => 'IB_ro_del_sec' }, + }, + visible => [ qw(cxn num_inserts num_updates num_reads num_deletes num_inserts_sec + num_updates_sec num_reads_sec num_deletes_sec)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'ro', + group_by => [], + aggregate => 0, + }, + row_operation_misc => { + capt => 'Row Operation Misc', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + queries_in_queue => { src => 'IB_ro_queries_in_queue' }, + queries_inside => { src => 'IB_ro_queries_inside' }, + read_views_open => { src => 'IB_ro_read_views_open' }, + main_thread_id => { src => 'IB_ro_main_thread_id' }, + main_thread_proc_no => { src => 'IB_ro_main_thread_proc_no' }, + main_thread_state => { src => 'IB_ro_main_thread_state' }, + num_res_ext => { src => 'IB_ro_n_reserved_extents' }, + }, + visible => [ qw(cxn queries_in_queue queries_inside read_views_open main_thread_state)], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'ro', + group_by => [], + aggregate => 0, + }, + semaphores => { + capt => 'InnoDB Semaphores', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + mutex_os_waits => { src => 'IB_sm_mutex_os_waits' }, + mutex_spin_rounds => { src => 'IB_sm_mutex_spin_rounds' }, + mutex_spin_waits => { src => 'IB_sm_mutex_spin_waits' }, + reservation_count => { src => 'IB_sm_reservation_count' }, + rw_excl_os_waits => { src => 'IB_sm_rw_excl_os_waits' }, + rw_excl_spins => { src => 'IB_sm_rw_excl_spins' }, + rw_shared_os_waits => { src => 'IB_sm_rw_shared_os_waits' }, + rw_shared_spins => { src => 'IB_sm_rw_shared_spins' }, + signal_count => { src => 'IB_sm_signal_count' }, + wait_array_size => { src => 'IB_sm_wait_array_size' }, + }, + visible => [ qw(cxn mutex_os_waits mutex_spin_waits mutex_spin_rounds + rw_excl_os_waits rw_excl_spins rw_shared_os_waits rw_shared_spins + signal_count reservation_count )], + filters => [], + sort_cols => 'cxn', + sort_dir => '1', + innodb => 'sm', + group_by => [], + aggregate => 0, + }, + slave_io_status => { + capt => 'Slave I/O Status', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + connect_retry => { src => 'connect_retry' }, + master_host => { src => 'master_host', hdr => 'Master'}, + master_log_file => { src => 'master_log_file', hdr => 'File' }, + master_port => { src => 'master_port' }, + master_ssl_allowed => { src => 'master_ssl_allowed' }, + master_ssl_ca_file => { src => 'master_ssl_ca_file' }, + master_ssl_ca_path => { src => 'master_ssl_ca_path' }, + master_ssl_cert => { src => 'master_ssl_cert' }, + master_ssl_cipher => { src => 'master_ssl_cipher' }, + master_ssl_key => { src => 'master_ssl_key' }, + master_user => { src => 'master_user' }, + read_master_log_pos => { src => 'read_master_log_pos', hdr => 'Pos' }, + relay_log_size => { src => 'relay_log_space', trans => [qw(shorten)] }, + slave_io_running => { src => 'slave_io_running', hdr => 'On?' }, + slave_io_state => { src => 'slave_io_state', hdr => 'State' }, + }, + visible => [ qw(cxn master_host slave_io_running master_log_file relay_log_size read_master_log_pos slave_io_state)], + filters => [ qw( cxn_is_slave ) ], + sort_cols => 'slave_io_running cxn', + colors => [ + { col => 'slave_io_running', op => 'ne', arg => 'Yes', color => 'black on_red' }, + ], + sort_dir => '1', + innodb => '', + group_by => [], + aggregate => 0, + }, + slave_sql_status => { + capt => 'Slave SQL Status', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + exec_master_log_pos => { src => 'exec_master_log_pos', hdr => 'Master Pos' }, + last_errno => { src => 'last_errno' }, + last_error => { src => 'last_error' }, + master_host => { src => 'master_host', hdr => 'Master' }, + relay_log_file => { src => 'relay_log_file' }, + relay_log_pos => { src => 'relay_log_pos' }, + relay_log_size => { src => 'relay_log_space', trans => [qw(shorten)] }, + relay_master_log_file => { src => 'relay_master_log_file', hdr => 'Master File' }, + replicate_do_db => { src => 'replicate_do_db' }, + replicate_do_table => { src => 'replicate_do_table' }, + replicate_ignore_db => { src => 'replicate_ignore_db' }, + replicate_ignore_table => { src => 'replicate_ignore_table' }, + replicate_wild_do_table => { src => 'replicate_wild_do_table' }, + replicate_wild_ignore_table => { src => 'replicate_wild_ignore_table' }, + skip_counter => { src => 'skip_counter' }, + slave_sql_running => { src => 'slave_sql_running', hdr => 'On?' }, + until_condition => { src => 'until_condition' }, + until_log_file => { src => 'until_log_file' }, + until_log_pos => { src => 'until_log_pos' }, + time_behind_master => { src => 'seconds_behind_master', trans => [ qw(secs_to_time) ] }, + bytes_behind_master => { src => 'master_log_file && master_log_file eq relay_master_log_file ? read_master_log_pos - exec_master_log_pos : 0', trans => [qw(shorten)] }, + slave_catchup_rate => { src => $exprs{SlaveCatchupRate}, trans => [ qw(set_precision) ] }, + slave_open_temp_tables => { src => 'Slave_open_temp_tables' }, + }, + visible => [ qw(cxn master_host slave_sql_running time_behind_master slave_catchup_rate slave_open_temp_tables relay_log_pos last_error)], + filters => [ qw( cxn_is_slave ) ], + sort_cols => 'slave_sql_running cxn', + sort_dir => '1', + innodb => '', + colors => [ + { col => 'slave_sql_running', op => 'ne', arg => 'Yes', color => 'black on_red' }, + { col => 'time_behind_master', op => '>', arg => 600, color => 'red' }, + { col => 'time_behind_master', op => '>', arg => 60, color => 'yellow' }, + { col => 'time_behind_master', op => '==', arg => 0, color => 'white' }, + ], + group_by => [], + aggregate => 0, + }, + t_header => { + capt => 'T-Mode Header', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + dirty_bufs => { src => $exprs{DirtyBufs}, trans => [qw(percent)] }, + history_list_len => { src => 'IB_tx_history_list_len' }, + lock_structs => { src => 'IB_tx_num_lock_structs' }, + num_txns => { src => $exprs{NumTxns} }, + max_txn => { src => $exprs{MaxTxnTime}, trans => [qw(secs_to_time)] }, + undo_for => { src => 'IB_tx_purge_undo_for' }, + used_bufs => { src => $exprs{BufPoolFill}, trans => [qw(percent)]}, + versions => { src => $exprs{OldVersions} }, + }, + visible => [ qw(cxn history_list_len versions undo_for dirty_bufs used_bufs num_txns max_txn lock_structs)], + filters => [ ], + sort_cols => 'cxn', + sort_dir => '1', + innodb => '', + colors => [], + hide_caption => 1, + group_by => [], + aggregate => 0, + }, + var_status => { + capt => 'Variables & Status', + cust => {}, + cols => {}, # Generated from current varset + visible => [], # Generated from current varset + filters => [], + sort_cols => '', + sort_dir => 1, + innodb => '', + temp => 1, # Do not persist to config file. + hide_caption => 1, + pivot => 0, + group_by => [], + aggregate => 0, + }, + wait_array => { + capt => 'InnoDB Wait Array', + cust => {}, + cols => { + cxn => { src => 'cxn' }, + thread => { src => 'thread' }, + waited_at_filename => { src => 'waited_at_filename' }, + waited_at_line => { src => 'waited_at_line' }, + 'time' => { src => 'waited_secs', trans => [ qw(secs_to_time) ] }, + request_type => { src => 'request_type' }, + lock_mem_addr => { src => 'lock_mem_addr' }, + lock_cfile_name => { src => 'lock_cfile_name' }, + lock_cline => { src => 'lock_cline' }, + writer_thread => { src => 'writer_thread' }, + writer_lock_mode => { src => 'writer_lock_mode' }, + num_readers => { src => 'num_readers' }, + lock_var => { src => 'lock_var' }, + waiters_flag => { src => 'waiters_flag' }, + last_s_file_name => { src => 'last_s_file_name' }, + last_s_line => { src => 'last_s_line' }, + last_x_file_name => { src => 'last_x_file_name' }, + last_x_line => { src => 'last_x_line' }, + cell_waiting => { src => 'cell_waiting' }, + cell_event_set => { src => 'cell_event_set' }, + }, + visible => [ qw(cxn thread time waited_at_filename waited_at_line request_type num_readers lock_var waiters_flag cell_waiting cell_event_set)], + filters => [], + sort_cols => 'cxn -time', + sort_dir => '1', + innodb => 'sm', + group_by => [], + aggregate => 0, + }, +); + +# Initialize %tbl_meta from %columns and do some checks. +foreach my $table_name ( keys %tbl_meta ) { + my $table = $tbl_meta{$table_name}; + my $cols = $table->{cols}; + + foreach my $col_name ( keys %$cols ) { + my $col_def = $table->{cols}->{$col_name}; + die "I can't find a column named '$col_name' for '$table_name'" unless $columns{$col_name}; + $columns{$col_name}->{referenced} = 1; + + foreach my $prop ( keys %col_props ) { + # Each column gets non-existing values set from %columns or defaults from %col_props. + if ( !$col_def->{$prop} ) { + $col_def->{$prop} + = defined($columns{$col_name}->{$prop}) + ? $columns{$col_name}->{$prop} + : $col_props{$prop}; + } + } + + # Ensure transformations and aggregate functions are valid + die "Unknown aggregate function '$col_def->{agg}' " + . "for column '$col_name' in table '$table_name'" + unless exists $agg_funcs{$col_def->{agg}}; + foreach my $trans ( @{$col_def->{trans}} ) { + die "Unknown transformation '$trans' " + . "for column '$col_name' in table '$table_name'" + unless exists $trans_funcs{$trans}; + } + } + + # Ensure each column in visible and group_by exists in cols + foreach my $place ( qw(visible group_by) ) { + foreach my $col_name ( @{$table->{$place}} ) { + if ( !exists $cols->{$col_name} ) { + die "Column '$col_name' is listed in '$place' for '$table_name', but doesn't exist"; + } + } + } + + # Compile sort and color subroutines + $table->{sort_func} = make_sort_func($table); + $table->{color_func} = make_color_func($table); +} + +# This is for code cleanup: +{ + my @unused_cols = grep { !$columns{$_}->{referenced} } sort keys %columns; + if ( @unused_cols ) { + die "The following columns are not used: " + . join(' ', @unused_cols); + } +} + +# ########################################################################### +# Operating modes {{{3 +# ########################################################################### +my %modes = ( + B => { + hdr => 'InnoDB Buffers', + cust => {}, + note => 'Shows buffer info from InnoDB', + action_for => { + i => { + action => sub { toggle_config('status_inc') }, + label => 'Toggle incremental status display', + }, + }, + display_sub => \&display_B, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(buffer_pool page_statistics insert_buffers adaptive_hash_index)], + visible_tables => [qw(buffer_pool page_statistics insert_buffers adaptive_hash_index)], + }, + C => { + hdr => 'Command Summary', + cust => {}, + note => 'Shows relative magnitude of variables', + action_for => { + s => { + action => sub { get_config_interactive('cmd_filter') }, + label => 'Choose variable prefix', + }, + }, + display_sub => \&display_C, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(cmd_summary)], + visible_tables => [qw(cmd_summary)], + }, + D => { + hdr => 'InnoDB Deadlocks', + cust => {}, + note => 'View InnoDB deadlock information', + action_for => { + c => { + action => sub { edit_table('deadlock_transactions') }, + label => 'Choose visible columns', + }, + w => { + action => \&create_deadlock, + label => 'Wipe deadlock status info by creating a deadlock', + }, + }, + display_sub => \&display_D, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(deadlock_transactions deadlock_locks)], + visible_tables => [qw(deadlock_transactions deadlock_locks)], + }, + F => { + hdr => 'InnoDB FK Err', + cust => {}, + note => 'View the latest InnoDB foreign key error', + action_for => {}, + display_sub => \&display_F, + connections => [], + server_group => '', + one_connection => 1, + tables => [qw(fk_error)], + visible_tables => [qw(fk_error)], + }, + I => { + hdr => 'InnoDB I/O Info', + cust => {}, + note => 'Shows I/O info (i/o, log...) from InnoDB', + action_for => { + i => { + action => sub { toggle_config('status_inc') }, + label => 'Toggle incremental status display', + }, + }, + display_sub => \&display_I, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(io_threads pending_io file_io_misc log_statistics)], + visible_tables => [qw(io_threads pending_io file_io_misc log_statistics)], + }, + L => { + hdr => 'Locks', + cust => {}, + note => 'Shows transaction locks', + action_for => { + a => { + action => sub { send_cmd_to_servers('CREATE TABLE IF NOT EXISTS test.innodb_lock_monitor(a int) ENGINE=InnoDB', 0, '', []); }, + label => 'Start the InnoDB Lock Monitor', + }, + o => { + action => sub { send_cmd_to_servers('DROP TABLE IF EXISTS test.innodb_lock_monitor', 0, '', []); }, + label => 'Stop the InnoDB Lock Monitor', + }, + }, + display_sub => \&display_L, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(innodb_locks)], + visible_tables => [qw(innodb_locks)], + }, + M => { + hdr => 'Replication Status', + cust => {}, + note => 'Shows replication (master and slave) status', + action_for => { + a => { + action => sub { send_cmd_to_servers('START SLAVE', 0, 'START SLAVE SQL_THREAD UNTIL MASTER_LOG_FILE = ?, MASTER_LOG_POS = ?', []); }, + label => 'Start slave(s)', + }, + i => { + action => sub { toggle_config('status_inc') }, + label => 'Toggle incremental status display', + }, + o => { + action => sub { send_cmd_to_servers('STOP SLAVE', 0, '', []); }, + label => 'Stop slave(s)', + }, + b => { + action => sub { purge_master_logs() }, + label => 'Purge unused master logs', + }, + }, + display_sub => \&display_M, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(slave_sql_status slave_io_status master_status)], + visible_tables => [qw(slave_sql_status slave_io_status master_status)], + }, + O => { + hdr => 'Open Tables', + cust => {}, + note => 'Shows open tables in MySQL', + action_for => { + r => { + action => sub { reverse_sort('open_tables'); }, + label => 'Reverse sort order', + }, + s => { + action => sub { choose_sort_cols('open_tables'); }, + label => "Choose sort column", + }, + }, + display_sub => \&display_O, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(open_tables)], + visible_tables => [qw(open_tables)], + }, + Q => { + hdr => 'Query List', + cust => {}, + note => 'Shows queries from SHOW FULL PROCESSLIST', + action_for => { + a => { + action => sub { toggle_filter('processlist', 'hide_self') }, + label => 'Toggle the innotop process', + }, + c => { + action => sub { edit_table('processlist') }, + label => 'Choose visible columns', + }, + e => { + action => sub { analyze_query('e'); }, + label => "Explain a thread's query", + }, + f => { + action => sub { analyze_query('f'); }, + label => "Show a thread's full query", + }, + h => { + action => sub { toggle_visible_table('Q', 'q_header') }, + label => 'Toggle the header on and off', + }, + i => { + action => sub { toggle_filter('processlist', 'hide_inactive') }, + label => 'Toggle idle processes', + }, + k => { + action => sub { kill_query('CONNECTION') }, + label => "Kill a query's connection", + }, + r => { + action => sub { reverse_sort('processlist'); }, + label => 'Reverse sort order', + }, + s => { + action => sub { choose_sort_cols('processlist'); }, + label => "Change the display's sort column", + }, + x => { + action => sub { kill_query('QUERY') }, + label => "Kill a query", + }, + }, + display_sub => \&display_Q, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(q_header processlist)], + visible_tables => [qw(q_header processlist)], + }, + R => { + hdr => 'InnoDB Row Ops', + cust => {}, + note => 'Shows InnoDB row operation and semaphore info', + action_for => { + i => { + action => sub { toggle_config('status_inc') }, + label => 'Toggle incremental status display', + }, + }, + display_sub => \&display_R, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(row_operations row_operation_misc semaphores wait_array)], + visible_tables => [qw(row_operations row_operation_misc semaphores wait_array)], + }, + S => { + hdr => 'Variables & Status', + cust => {}, + note => 'Shows query load statistics a la vmstat', + action_for => { + '>' => { + action => sub { switch_var_set('S_set', 1) }, + label => 'Switch to next variable set', + }, + '<' => { + action => sub { switch_var_set('S_set', -1) }, + label => 'Switch to prev variable set', + }, + c => { + action => sub { + choose_var_set('S_set'); + start_S_mode(); + }, + label => "Choose which set to display", + }, + e => { + action => \&edit_current_var_set, + label => 'Edit the current set of variables', + }, + i => { + action => sub { $clear_screen_sub->(); toggle_config('status_inc') }, + label => 'Toggle incremental status display', + }, + '-' => { + action => sub { set_display_precision(-1) }, + label => 'Decrease fractional display precision', + }, + '+' => { + action => sub { set_display_precision(1) }, + label => 'Increase fractional display precision', + }, + g => { + action => sub { set_s_mode('g') }, + label => 'Switch to graph (tload) view', + }, + s => { + action => sub { set_s_mode('s') }, + label => 'Switch to standard (vmstat) view', + }, + v => { + action => sub { set_s_mode('v') }, + label => 'Switch to pivoted view', + }, + }, + display_sub => \&display_S, + no_clear_screen => 1, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(var_status)], + visible_tables => [qw(var_status)], + }, + T => { + hdr => 'InnoDB Txns', + cust => {}, + note => 'Shows InnoDB transactions in top-like format', + action_for => { + a => { + action => sub { toggle_filter('innodb_transactions', 'hide_self') }, + label => 'Toggle the innotop process', + }, + c => { + action => sub { edit_table('innodb_transactions') }, + label => 'Choose visible columns', + }, + e => { + action => sub { analyze_query('e'); }, + label => "Explain a thread's query", + }, + f => { + action => sub { analyze_query('f'); }, + label => "Show a thread's full query", + }, + h => { + action => sub { toggle_visible_table('T', 't_header') }, + label => 'Toggle the header on and off', + }, + i => { + action => sub { toggle_filter('innodb_transactions', 'hide_inactive') }, + label => 'Toggle inactive transactions', + }, + k => { + action => sub { kill_query('CONNECTION') }, + label => "Kill a transaction's connection", + }, + r => { + action => sub { reverse_sort('innodb_transactions'); }, + label => 'Reverse sort order', + }, + s => { + action => sub { choose_sort_cols('innodb_transactions'); }, + label => "Change the display's sort column", + }, + x => { + action => sub { kill_query('QUERY') }, + label => "Kill a query", + }, + }, + display_sub => \&display_T, + connections => [], + server_group => '', + one_connection => 0, + tables => [qw(t_header innodb_transactions)], + visible_tables => [qw(t_header innodb_transactions)], + }, +); + +# ########################################################################### +# Global key mappings {{{3 +# Keyed on a single character, which is read from the keyboard. Uppercase +# letters switch modes. Lowercase letters access commands when in a mode. +# These can be overridden by action_for in %modes. +# ########################################################################### +my %action_for = ( + '$' => { + action => \&edit_configuration, + label => 'Edit configuration settings', + }, + '?' => { + action => \&display_help, + label => 'Show help', + }, + '!' => { + action => \&display_license, + label => 'Show license and warranty', + }, + '^' => { + action => \&edit_table, + label => "Edit the displayed table(s)", + }, + '#' => { + action => \&choose_server_groups, + label => 'Select/create server groups', + }, + '@' => { + action => \&choose_servers, + label => 'Select/create server connections', + }, + '/' => { + action => \&add_quick_filter, + label => 'Quickly filter what you see', + }, + '\\' => { + action => \&clear_quick_filters, + label => 'Clear quick-filters', + }, + '%' => { + action => \&choose_filters, + label => 'Choose and edit table filters', + }, + "\t" => { + action => \&next_server_group, + label => 'Switch to the next server group', + key => 'TAB', + }, + '=' => { + action => \&toggle_aggregate, + label => 'Toggle aggregation', + }, + # TODO: can these be auto-generated from %modes? + B => { + action => sub { switch_mode('B') }, + label => '', + }, + C => { + action => sub { switch_mode('C') }, + label => '', + }, + D => { + action => sub { switch_mode('D') }, + label => '', + }, + F => { + action => sub { switch_mode('F') }, + label => '', + }, + I => { + action => sub { switch_mode('I') }, + label => '', + }, + L => { + action => sub { switch_mode('L') }, + label => '', + }, + M => { + action => sub { switch_mode('M') }, + label => '', + }, + O => { + action => sub { switch_mode('O') }, + label => '', + }, + Q => { + action => sub { switch_mode('Q') }, + label => '', + }, + R => { + action => sub { switch_mode('R') }, + label => '', + }, + S => { + action => \&start_S_mode, + label => '', + }, + T => { + action => sub { switch_mode('T') }, + label => '', + }, + d => { + action => sub { get_config_interactive('interval') }, + label => 'Change refresh interval', + }, + n => { action => \&next_server, label => 'Switch to the next connection' }, + p => { action => \&pause, label => 'Pause innotop', }, + q => { action => \&finish, label => 'Quit innotop', }, +); + +# ########################################################################### +# Sleep times after certain statements {{{3 +# ########################################################################### +my %stmt_sleep_time_for = (); + +# ########################################################################### +# Config editor key mappings {{{3 +# ########################################################################### +my %cfg_editor_action = ( + c => { + note => 'Edit columns, etc in the displayed table(s)', + func => \&edit_table, + }, + g => { + note => 'Edit general configuration', + func => \&edit_configuration_variables, + }, + k => { + note => 'Edit row-coloring rules', + func => \&edit_color_rules, + }, + p => { + note => 'Manage plugins', + func => \&edit_plugins, + }, + s => { + note => 'Edit server groups', + func => \&edit_server_groups, + }, + S => { + note => 'Edit SQL statement sleep delays', + func => \&edit_stmt_sleep_times, + }, + t => { + note => 'Choose which table(s) to display in this mode', + func => \&choose_mode_tables, + }, +); + +# ########################################################################### +# Color editor key mappings {{{3 +# ########################################################################### +my %color_editor_action = ( + n => { + note => 'Create a new color rule', + func => sub { + my ( $tbl, $idx ) = @_; + my $meta = $tbl_meta{$tbl}; + + $clear_screen_sub->(); + my $col; + do { + $col = prompt_list( + 'Choose the target column for the rule', + '', + sub { return keys %{$meta->{cols}} }, + { map { $_ => $meta->{cols}->{$_}->{label} } keys %{$meta->{cols}} }); + } while ( !$col ); + ( $col ) = grep { $_ } split(/\W+/, $col); + return $idx unless $col && exists $meta->{cols}->{$col}; + + $clear_screen_sub->(); + my $op; + do { + $op = prompt_list( + 'Choose the comparison operator for the rule', + '', + sub { return keys %comp_ops }, + { map { $_ => $comp_ops{$_} } keys %comp_ops } ); + } until ( $op ); + $op =~ s/\s+//g; + return $idx unless $op && exists $comp_ops{$op}; + + my $arg; + do { + $arg = prompt('Specify an argument for the comparison'); + } until defined $arg; + + my $color; + do { + $color = prompt_list( + 'Choose the color(s) the row should be when the rule matches', + '', + sub { return keys %ansicolors }, + { map { $_ => $_ } keys %ansicolors } ); + } until defined $color; + $color = join(' ', unique(grep { exists $ansicolors{$_} } split(/\W+/, $color))); + return $idx unless $color; + + push @{$tbl_meta{$tbl}->{colors}}, { + col => $col, + op => $op, + arg => $arg, + color => $color + }; + $tbl_meta{$tbl}->{cust}->{colors} = 1; + + return $idx; + }, + }, + d => { + note => 'Remove the selected rule', + func => sub { + my ( $tbl, $idx ) = @_; + my @rules = @{ $tbl_meta{$tbl}->{colors} }; + return 0 unless @rules > 0 && $idx < @rules && $idx >= 0; + splice(@{$tbl_meta{$tbl}->{colors}}, $idx, 1); + $tbl_meta{$tbl}->{cust}->{colors} = 1; + return $idx == @rules ? $#rules : $idx; + }, + }, + j => { + note => 'Move highlight down one', + func => sub { + my ( $tbl, $idx ) = @_; + my $num_rules = scalar @{$tbl_meta{$tbl}->{colors}}; + return ($idx + 1) % $num_rules; + }, + }, + k => { + note => 'Move highlight up one', + func => sub { + my ( $tbl, $idx ) = @_; + my $num_rules = scalar @{$tbl_meta{$tbl}->{colors}}; + return ($idx - 1) % $num_rules; + }, + }, + '+' => { + note => 'Move selected rule up one', + func => sub { + my ( $tbl, $idx ) = @_; + my $meta = $tbl_meta{$tbl}; + my $dest = $idx == 0 ? scalar(@{$meta->{colors}} - 1) : $idx - 1; + my $temp = $meta->{colors}->[$idx]; + $meta->{colors}->[$idx] = $meta->{colors}->[$dest]; + $meta->{colors}->[$dest] = $temp; + $meta->{cust}->{colors} = 1; + return $dest; + }, + }, + '-' => { + note => 'Move selected rule down one', + func => sub { + my ( $tbl, $idx ) = @_; + my $meta = $tbl_meta{$tbl}; + my $dest = $idx == scalar(@{$meta->{colors}} - 1) ? 0 : $idx + 1; + my $temp = $meta->{colors}->[$idx]; + $meta->{colors}->[$idx] = $meta->{colors}->[$dest]; + $meta->{colors}->[$dest] = $temp; + $meta->{cust}->{colors} = 1; + return $dest; + }, + }, +); + +# ########################################################################### +# Plugin editor key mappings {{{3 +# ########################################################################### +my %plugin_editor_action = ( + '*' => { + note => 'Toggle selected plugin active/inactive', + func => sub { + my ( $plugins, $idx ) = @_; + my $plugin = $plugins->[$idx]; + $plugin->{active} = $plugin->{active} ? 0 : 1; + return $idx; + }, + }, + j => { + note => 'Move highlight down one', + func => sub { + my ( $plugins, $idx ) = @_; + return ($idx + 1) % scalar(@$plugins); + }, + }, + k => { + note => 'Move highlight up one', + func => sub { + my ( $plugins, $idx ) = @_; + return $idx == 0 ? @$plugins - 1 : $idx - 1; + }, + }, +); + +# ########################################################################### +# Table editor key mappings {{{3 +# ########################################################################### +my %tbl_editor_action = ( + a => { + note => 'Add a column to the table', + func => sub { + my ( $tbl, $col ) = @_; + my @visible_cols = @{ $tbl_meta{$tbl}->{visible} }; + my %all_cols = %{ $tbl_meta{$tbl}->{cols} }; + delete @all_cols{@visible_cols}; + my $choice = prompt_list( + 'Choose a column', + '', + sub { return keys %all_cols; }, + { map { $_ => $all_cols{$_}->{label} || $all_cols{$_}->{hdr} } keys %all_cols }); + if ( $all_cols{$choice} ) { + push @{$tbl_meta{$tbl}->{visible}}, $choice; + $tbl_meta{$tbl}->{cust}->{visible} = 1; + return $choice; + } + return $col; + }, + }, + n => { + note => 'Create a new column and add it to the table', + func => sub { + my ( $tbl, $col ) = @_; + + $clear_screen_sub->(); + print word_wrap("Choose a name for the column. This name is not displayed, and is used only " + . "for internal reference. It can contain only lowercase letters, numbers, " + . "and underscores."); + print "\n\n"; + do { + $col = prompt("Enter column name"); + $col = '' if $col =~ m/[^a-z0-9_]/; + } while ( !$col ); + + $clear_screen_sub->(); + my $hdr; + do { + $hdr = prompt("Enter column header"); + } while ( !$hdr ); + + $clear_screen_sub->(); + print "Choose a source for the column's data\n\n"; + my ( $src, $sub, $err ); + do { + if ( $err ) { + print "Error: $err\n\n"; + } + $src = prompt("Enter column source"); + if ( $src ) { + ( $sub, $err ) = compile_expr($src); + } + } until ( !$err); + + # TODO: this duplicates %col_props. + $tbl_meta{$tbl}->{cols}->{$col} = { + hdr => $hdr, + src => $src, + just => '-', + num => 0, + label => 'User-defined', + user => 1, + tbl => $tbl, + minw => 0, + maxw => 0, + trans => [], + func => $sub, + dec => 0, + agg => 0, + aggonly => 0, + }; + + $tbl_meta{$tbl}->{visible} = [ unique(@{$tbl_meta{$tbl}->{visible}}, $col) ]; + $tbl_meta{$tbl}->{cust}->{visible} = 1; + return $col; + }, + }, + d => { + note => 'Remove selected column', + func => sub { + my ( $tbl, $col ) = @_; + my @visible_cols = @{ $tbl_meta{$tbl}->{visible} }; + my $idx = 0; + return $col unless @visible_cols > 1; + while ( $visible_cols[$idx] ne $col ) { + $idx++; + } + $tbl_meta{$tbl}->{visible} = [ grep { $_ ne $col } @visible_cols ]; + $tbl_meta{$tbl}->{cust}->{visible} = 1; + return $idx == $#visible_cols ? $visible_cols[$idx - 1] : $visible_cols[$idx + 1]; + }, + }, + e => { + note => 'Edit selected column', + func => sub { + # TODO: make this editor hotkey-driven and give readline support. + my ( $tbl, $col ) = @_; + $clear_screen_sub->(); + my $meta = $tbl_meta{$tbl}->{cols}->{$col}; + my @prop = qw(hdr label src just num minw maxw trans agg); # TODO redundant + + my $answer; + do { + # Do what the user asked... + if ( $answer && grep { $_ eq $answer } @prop ) { + # Some properties are arrays, others scalars. + my $ini = ref $col_props{$answer} ? join(' ', @{$meta->{$answer}}) : $meta->{$answer}; + my $val = prompt("New value for $answer", undef, $ini); + $val = [ split(' ', $val) ] if ref($col_props{$answer}); + if ( $answer eq 'trans' ) { + $val = [ unique(grep{ exists $trans_funcs{$_} } @$val) ]; + } + @{$meta}{$answer, 'user', 'tbl' } = ( $val, 1, $tbl ); + } + + my @display_lines = ( + '', + "You are editing column $tbl.$col.\n", + ); + + push @display_lines, create_table2( + \@prop, + { map { $_ => $_ } @prop }, + { map { $_ => ref $meta->{$_} eq 'ARRAY' ? join(' ', @{$meta->{$_}}) + : ref $meta->{$_} ? '[expression code]' + : $meta->{$_} + } @prop + }, + { sep => ' ' }); + draw_screen(\@display_lines, { raw => 1 }); + print "\n\n"; # One to add space, one to clear readline artifacts + $answer = prompt('Edit what? (q to quit)'); + } while ( $answer ne 'q' ); + + return $col; + }, + }, + j => { + note => 'Move highlight down one', + func => sub { + my ( $tbl, $col ) = @_; + my @visible_cols = @{ $tbl_meta{$tbl}->{visible} }; + my $idx = 0; + while ( $visible_cols[$idx] ne $col ) { + $idx++; + } + return $visible_cols[ ($idx + 1) % @visible_cols ]; + }, + }, + k => { + note => 'Move highlight up one', + func => sub { + my ( $tbl, $col ) = @_; + my @visible_cols = @{ $tbl_meta{$tbl}->{visible} }; + my $idx = 0; + while ( $visible_cols[$idx] ne $col ) { + $idx++; + } + return $visible_cols[ $idx - 1 ]; + }, + }, + '+' => { + note => 'Move selected column up one', + func => sub { + my ( $tbl, $col ) = @_; + my $meta = $tbl_meta{$tbl}; + my @visible_cols = @{$meta->{visible}}; + my $idx = 0; + while ( $visible_cols[$idx] ne $col ) { + $idx++; + } + if ( $idx ) { + $visible_cols[$idx] = $visible_cols[$idx - 1]; + $visible_cols[$idx - 1] = $col; + $meta->{visible} = \@visible_cols; + } + else { + shift @{$meta->{visible}}; + push @{$meta->{visible}}, $col; + } + $meta->{cust}->{visible} = 1; + return $col; + }, + }, + '-' => { + note => 'Move selected column down one', + func => sub { + my ( $tbl, $col ) = @_; + my $meta = $tbl_meta{$tbl}; + my @visible_cols = @{$meta->{visible}}; + my $idx = 0; + while ( $visible_cols[$idx] ne $col ) { + $idx++; + } + if ( $idx == $#visible_cols ) { + unshift @{$meta->{visible}}, $col; + pop @{$meta->{visible}}; + } + else { + $visible_cols[$idx] = $visible_cols[$idx + 1]; + $visible_cols[$idx + 1] = $col; + $meta->{visible} = \@visible_cols; + } + $meta->{cust}->{visible} = 1; + return $col; + }, + }, + f => { + note => 'Choose filters', + func => sub { + my ( $tbl, $col ) = @_; + choose_filters($tbl); + return $col; + }, + }, + o => { + note => 'Edit color rules', + func => sub { + my ( $tbl, $col ) = @_; + edit_color_rules($tbl); + return $col; + }, + }, + s => { + note => 'Choose sort columns', + func => sub { + my ( $tbl, $col ) = @_; + choose_sort_cols($tbl); + return $col; + }, + }, + g => { + note => 'Choose group-by (aggregate) columns', + func => sub { + my ( $tbl, $col ) = @_; + choose_group_cols($tbl); + return $col; + }, + }, +); + +# ########################################################################### +# Global variables and environment {{{2 +# ########################################################################### + +my @this_term_size; # w_chars, h_chars, w_pix, h_pix +my @last_term_size; # w_chars, h_chars, w_pix, h_pix +my $char; +my $windows = $OSNAME =~ m/MSWin/; +my $have_color = 0; +my $MAX_ULONG = 4294967295; # 2^32-1 +my $num_regex = qr/^[+-]?(?=\d|\.)\d*(?:\.\d+)?(?:E[+-]?\d+|)$/i; +my $int_regex = qr/^\d+$/; +my $bool_regex = qr/^[01]$/; +my $term = undef; +my $file = undef; # File to watch for InnoDB monitor output +my $file_mtime = undef; # Status of watched file +my $file_data = undef; # Last chunk of text read from file +my $innodb_parser = InnoDBParser->new; + +my $nonfatal_errs = join('|', + 'Access denied for user', + 'Unknown MySQL server host', + 'Unknown database', + 'Can\'t connect to local MySQL server through socket', + 'Can\'t connect to MySQL server on', + 'MySQL server has gone away', + 'Cannot call SHOW INNODB STATUS', + 'Access denied', + 'AutoCommit', +); + +if ( !$opts{n} ) { + require Term::ReadLine; + $term = Term::ReadLine->new('innotop'); +} + +# Stores status, variables, innodb status, master/slave status etc. +# Keyed on connection name. Each entry is a hashref of current and past data sets, +# keyed on clock tick. +my %vars; +my %info_gotten = (); # Which things have been retrieved for the current clock tick. + +# Stores info on currently displayed queries: cxn, connection ID, query text. +my @current_queries; + +my $lines_printed = 0; +my $clock = 0; # Incremented with every wake-sleep cycle +my $clearing_deadlocks = 0; + +# Find the home directory; it's different on different OSes. +my $homepath = $ENV{HOME} || $ENV{HOMEPATH} || $ENV{USERPROFILE} || '.'; + +# If terminal coloring is available, use it. The only function I want from +# the module is the colored() function. +eval { + if ( !$opts{n} ) { + if ( $windows ) { + require Win32::Console::ANSI; + } + require Term::ANSIColor; + import Term::ANSIColor qw(colored); + $have_color = 1; + } +}; +if ( $EVAL_ERROR || $opts{n} ) { + # If there was an error, manufacture my own colored() function that does no + # coloring. + *colored = sub { pop @_; @_; }; +} + +if ( $opts{n} ) { + $clear_screen_sub = sub {}; +} +elsif ( $windows ) { + $clear_screen_sub = sub { $lines_printed = 0; system("cls") }; +} +else { + my $clear = `clear`; + $clear_screen_sub = sub { $lines_printed = 0; print $clear }; +} + +# ########################################################################### +# Config storage. {{{2 +# ########################################################################### +my %config = ( + color => { + val => $have_color, + note => 'Whether to use terminal coloring', + conf => 'ALL', + pat => $bool_regex, + }, + cmd_filter => { + val => 'Com_', + note => 'Prefix for values in C mode', + conf => [qw(C)], + }, + plugin_dir => { + val => "$homepath/.innotop/plugins", + note => 'Directory where plugins can be found', + conf => 'ALL', + }, + show_percent => { + val => 1, + note => 'Show the % symbol after percentages', + conf => 'ALL', + pat => $bool_regex, + }, + skip_innodb => { + val => 0, + note => 'Disable SHOW INNODB STATUS', + conf => 'ALL', + pat => $bool_regex, + }, + S_func => { + val => 's', + note => 'What to display in S mode: graph, status, pivoted status', + conf => [qw(S)], + pat => qr/^[gsv]$/, + }, + cxn_timeout => { + val => 28800, + note => 'Connection timeout for keeping unused connections alive', + conf => 'ALL', + pat => $int_regex, + }, + graph_char => { + val => '*', + note => 'Character for drawing graphs', + conf => [ qw(S) ], + pat => qr/^.$/, + }, + show_cxn_errors_in_tbl => { + val => 1, + note => 'Whether to display connection errors as rows in the table', + conf => 'ALL', + pat => $bool_regex, + }, + hide_hdr => { + val => 0, + note => 'Whether to show column headers', + conf => 'ALL', + pat => $bool_regex, + }, + show_cxn_errors => { + val => 1, + note => 'Whether to print connection errors to STDOUT', + conf => 'ALL', + pat => $bool_regex, + }, + readonly => { + val => 0, + note => 'Whether the config file is read-only', + conf => [ qw() ], + pat => $bool_regex, + }, + global => { + val => 1, + note => 'Whether to show GLOBAL variables and status', + conf => 'ALL', + pat => $bool_regex, + }, + header_highlight => { + val => 'bold', + note => 'How to highlight table column headers', + conf => 'ALL', + pat => qr/^(?:bold|underline)$/, + }, + display_table_captions => { + val => 1, + note => 'Whether to put captions on tables', + conf => 'ALL', + pat => $bool_regex, + }, + charset => { + val => 'ascii', + note => 'What type of characters should be displayed in queries (ascii, unicode, none)', + conf => 'ALL', + pat => qr/^(?:ascii|unicode|none)$/, + }, + auto_wipe_dl => { + val => 0, + note => 'Whether to auto-wipe InnoDB deadlocks', + conf => 'ALL', + pat => $bool_regex, + }, + max_height => { + val => 30, + note => '[Win32] Max window height', + conf => 'ALL', + }, + debug => { + val => 0, + pat => $bool_regex, + note => 'Debug mode (more verbose errors, uses more memory)', + conf => 'ALL', + }, + num_digits => { + val => 2, + pat => $int_regex, + note => 'How many digits to show in fractional numbers and percents', + conf => 'ALL', + }, + debugfile => { + val => "$homepath/.innotop/core_dump", + note => 'A debug file in case you are interested in error output', + }, + show_statusbar => { + val => 1, + pat => $bool_regex, + note => 'Whether to show the status bar in the display', + conf => 'ALL', + }, + mode => { + val => "T", + note => "Which mode to start in", + cmdline => 1, + }, + status_inc => { + val => 0, + note => 'Whether to show raw or incremental values for status variables', + pat => $bool_regex, + }, + interval => { + val => 10, + pat => qr/^(?:(?:\d*?[1-9]\d*(?:\.\d*)?)|(?:\d*\.\d*?[1-9]\d*))$/, + note => "The interval at which the display will be refreshed. Fractional values allowed.", + }, + num_status_sets => { + val => 9, + pat => $int_regex, + note => 'How many sets of STATUS and VARIABLES values to show', + conf => [ qw(S) ], + }, + S_set => { + val => 'general', + pat => qr/^\w+$/, + note => 'Which set of variables to display in S (Variables & Status) mode', + conf => [ qw(S) ], + }, +); + +# ########################################################################### +# Config file sections {{{2 +# The configuration file is broken up into sections like a .ini file. This +# variable defines those sections and the subroutines responsible for reading +# and writing them. +# ########################################################################### +my %config_file_sections = ( + plugins => { + reader => \&load_config_plugins, + writer => \&save_config_plugins, + }, + group_by => { + reader => \&load_config_group_by, + writer => \&save_config_group_by, + }, + filters => { + reader => \&load_config_filters, + writer => \&save_config_filters, + }, + active_filters => { + reader => \&load_config_active_filters, + writer => \&save_config_active_filters, + }, + visible_tables => { + reader => \&load_config_visible_tables, + writer => \&save_config_visible_tables, + }, + sort_cols => { + reader => \&load_config_sort_cols, + writer => \&save_config_sort_cols, + }, + active_columns => { + reader => \&load_config_active_columns, + writer => \&save_config_active_columns, + }, + tbl_meta => { + reader => \&load_config_tbl_meta, + writer => \&save_config_tbl_meta, + }, + general => { + reader => \&load_config_config, + writer => \&save_config_config, + }, + connections => { + reader => \&load_config_connections, + writer => \&save_config_connections, + }, + active_connections => { + reader => \&load_config_active_connections, + writer => \&save_config_active_connections, + }, + server_groups => { + reader => \&load_config_server_groups, + writer => \&save_config_server_groups, + }, + active_server_groups => { + reader => \&load_config_active_server_groups, + writer => \&save_config_active_server_groups, + }, + max_values_seen => { + reader => \&load_config_mvs, + writer => \&save_config_mvs, + }, + varsets => { + reader => \&load_config_varsets, + writer => \&save_config_varsets, + }, + colors => { + reader => \&load_config_colors, + writer => \&save_config_colors, + }, + stmt_sleep_times => { + reader => \&load_config_stmt_sleep_times, + writer => \&save_config_stmt_sleep_times, + }, +); + +# Config file sections have some dependencies, so they have to be read/written in order. +my @ordered_config_file_sections = qw(general plugins filters active_filters tbl_meta + connections active_connections server_groups active_server_groups max_values_seen + active_columns sort_cols visible_tables varsets colors stmt_sleep_times + group_by); + +# All events for which plugins may register themselves. Entries are arrayrefs. +my %event_listener_for = map { $_ => [] } + qw( + extract_values + set_to_tbl_pre_filter set_to_tbl_pre_sort set_to_tbl_pre_group + set_to_tbl_pre_colorize set_to_tbl_pre_transform set_to_tbl_pre_pivot + set_to_tbl_pre_create set_to_tbl_post_create + draw_screen + ); + +# All variables to which plugins have access. +my %pluggable_vars = ( + action_for => \%action_for, + agg_funcs => \%agg_funcs, + config => \%config, + connections => \%connections, + dbhs => \%dbhs, + filters => \%filters, + modes => \%modes, + server_groups => \%server_groups, + tbl_meta => \%tbl_meta, + trans_funcs => \%trans_funcs, + var_sets => \%var_sets, +); + +# ########################################################################### +# Contains logic to generate prepared statements for a given function for a +# given DB connection. Returns a $sth. +# ########################################################################### +my %stmt_maker_for = ( + INNODB_STATUS => sub { + my ( $dbh ) = @_; + return $dbh->prepare(version_ge( $dbh, '5.0.0' ) + ? 'SHOW ENGINE INNODB STATUS' + : 'SHOW INNODB STATUS'); + }, + SHOW_VARIABLES => sub { + my ( $dbh ) = @_; + return $dbh->prepare($config{global}->{val} && version_ge( $dbh, '4.0.3' ) + ? 'SHOW GLOBAL VARIABLES' + : 'SHOW VARIABLES'); + }, + SHOW_STATUS => sub { + my ( $dbh ) = @_; + return $dbh->prepare($config{global}->{val} && version_ge( $dbh, '5.0.2' ) + ? 'SHOW GLOBAL STATUS' + : 'SHOW STATUS'); + }, + KILL_QUERY => sub { + my ( $dbh ) = @_; + return $dbh->prepare(version_ge( $dbh, '5.0.0' ) + ? 'KILL QUERY ?' + : 'KILL ?'); + }, + SHOW_MASTER_LOGS => sub { + my ( $dbh ) = @_; + return $dbh->prepare('SHOW MASTER LOGS'); + }, + SHOW_MASTER_STATUS => sub { + my ( $dbh ) = @_; + return $dbh->prepare('SHOW MASTER STATUS'); + }, + SHOW_SLAVE_STATUS => sub { + my ( $dbh ) = @_; + return $dbh->prepare('SHOW SLAVE STATUS'); + }, + KILL_CONNECTION => sub { + my ( $dbh ) = @_; + return $dbh->prepare(version_ge( $dbh, '5.0.0' ) + ? 'KILL CONNECTION ?' + : 'KILL ?'); + }, + OPEN_TABLES => sub { + my ( $dbh ) = @_; + return version_ge($dbh, '4.0.0') + ? $dbh->prepare('SHOW OPEN TABLES') + : undef; + }, + PROCESSLIST => sub { + my ( $dbh ) = @_; + return $dbh->prepare('SHOW FULL PROCESSLIST'); + }, +); + +# Plugins! +my %plugins = ( +); + +# ########################################################################### +# Run the program {{{1 +# ########################################################################### + +# This config variable is only useful for MS Windows because its terminal +# can't tell how tall it is. +if ( !$windows ) { + delete $config{max_height}; +} + +# Try to lower my priority. +eval { setpriority(0, 0, getpriority(0, 0) + 10); }; + +# Print stuff to the screen immediately, don't wait for a newline. +$OUTPUT_AUTOFLUSH = 1; + +# Clear the screen and load the configuration. +$clear_screen_sub->(); +load_config(); +post_process_tbl_meta(); + +# Make sure no changes are written to config file in non-interactive mode. +if ( $opts{n} ) { + $config{readonly}->{val} = 1; +} + +eval { + + # Open the file for InnoDB status + if ( @ARGV ) { + my $filename = shift @ARGV; + open $file, "<", $filename + or die "Cannot open '$filename': $OS_ERROR"; + } + + # In certain modes we might have to collect data for two cycles + # before printing anything out, so we need to bump up the count one. + if ( $opts{n} && $opts{count} && $config{status_inc}->{val} + && $config{mode}->{val} =~ m/[S]/ ) + { + $opts{count}++; + } + + while (++$clock) { + + my $mode = $config{mode}->{val} || 'T'; + if ( !$modes{$mode} ) { + die "Mode '$mode' doesn't exist; try one of these:\n" + . join("\n", map { " $_ $modes{$_}->{hdr}" } sort keys %modes) + . "\n"; + } + + if ( !$opts{n} ) { + @last_term_size = @this_term_size; + @this_term_size = Term::ReadKey::GetTerminalSize(\*STDOUT); + if ( $windows ) { + $this_term_size[0]--; + $this_term_size[1] + = min($this_term_size[1], $config{max_height}->{val}); + } + die("Can't read terminal size") unless @this_term_size; + } + + # If there's no connection to a database server, we need to fix that... + if ( !%connections ) { + print "You have not defined any database connections.\n\n"; + add_new_dsn(); + } + + # See whether there are any connections defined for this mode. If there's only one + # connection total, assume the user wants to just use innotop for a single server + # and don't ask which server to connect to. Also, if we're monitoring from a file, + # we just use the first connection. + if ( !get_connections() ) { + if ( $file || 1 == scalar keys %connections ) { + $modes{$config{mode}->{val}}->{connections} = [ keys %connections ]; + } + else { + choose_connections(); + } + } + + # Term::ReadLine might have re-set $OUTPUT_AUTOFLUSH. + $OUTPUT_AUTOFLUSH = 1; + + # Prune old data + my $sets = $config{num_status_sets}->{val}; + foreach my $store ( values %vars ) { + delete @{$store}{ grep { $_ < $clock - $sets } keys %$store }; + } + %info_gotten = (); + + # Call the subroutine to display this mode. + $modes{$mode}->{display_sub}->(); + + # It may be time to quit now. + if ( $opts{count} && $clock >= $opts{count} ) { + finish(); + } + + # Wait for a bit. + if ( $opts{n} ) { + sleep($config{interval}->{val}); + } + else { + ReadMode('cbreak'); + $char = ReadKey($config{interval}->{val}); + ReadMode('normal'); + } + + # Handle whatever action the key indicates. + do_key_action(); + + } +}; +if ( $EVAL_ERROR ) { + core_dump( $EVAL_ERROR ); +} +finish(); + +# Subroutines {{{1 +# Mode functions{{{2 +# switch_mode {{{3 +sub switch_mode { + my $mode = shift; + $config{mode}->{val} = $mode; +} + +# Prompting functions {{{2 +# prompt_list {{{3 +# Prompts the user for a value, given a question, initial value, +# a completion function and a hashref of hints. +sub prompt_list { + die "Can't call in non-interactive mode" if $opts{n}; + my ( $question, $init, $completion, $hints ) = @_; + if ( $hints ) { + # Figure out how wide the table will be + my $max_name = max(map { length($_) } keys %$hints ); + $max_name ||= 0; + $max_name += 3; + my @meta_rows = create_table2( + [ sort keys %$hints ], + { map { $_ => $_ } keys %$hints }, + { map { $_ => trunc($hints->{$_}, $this_term_size[0] - $max_name) } keys %$hints }, + { sep => ' ' }); + if (@meta_rows > 10) { + # Try to split and stack the meta rows next to each other + my $split = int(@meta_rows / 2); + @meta_rows = stack_next( + [@meta_rows[0..$split - 1]], + [@meta_rows[$split..$#meta_rows]], + { pad => ' | '}, + ); + } + print join( "\n", + '', + map { ref $_ ? colored(@$_) : $_ } create_caption('Choose from', @meta_rows), ''), + "\n"; + } + $term->Attribs->{completion_function} = $completion; + my $answer = $term->readline("$question: ", $init); + $OUTPUT_AUTOFLUSH = 1; + $answer = '' if !defined($answer); + $answer =~ s/\s+$//; + return $answer; +} + +# prompt {{{3 +# Prints out a prompt and reads from the keyboard, then validates with the +# validation regex until the input is correct. +sub prompt { + die "Can't call in non-interactive mode" if $opts{n}; + my ( $prompt, $regex, $init, $completion ) = @_; + my $response; + my $success = 0; + do { + if ( $completion ) { + $term->Attribs->{completion_function} = $completion; + } + $response = $term->readline("$prompt: ", $init); + if ( $regex && $response !~ m/$regex/ ) { + print "Invalid response.\n\n"; + } + else { + $success = 1; + } + } while ( !$success ); + $OUTPUT_AUTOFLUSH = 1; + $response =~ s/\s+$//; + return $response; +} + +# prompt_noecho {{{3 +# Unfortunately, suppressing echo with Term::ReadLine isn't reliable; the user might not +# have that library, or it might not support that feature. +sub prompt_noecho { + my ( $prompt ) = @_; + print colored("$prompt: ", 'underline'); + my $response; + ReadMode('noecho'); + $response = <STDIN>; + chomp($response); + ReadMode('normal'); + return $response; +} + +# do_key_action {{{3 +# Depending on whether a key was read, do something. Keys have certain +# actions defined in lookup tables. Each mode may have its own lookup table, +# which trumps the global table -- so keys can be context-sensitive. The key +# may be read and written in a subroutine, so it's a global. +sub do_key_action { + if ( defined $char ) { + my $mode = $config{mode}->{val}; + my $action + = defined($modes{$mode}->{action_for}->{$char}) + ? $modes{$mode}->{action_for}->{$char}->{action} + : defined($action_for{$char}) + ? $action_for{$char}->{action} + : sub{}; + $action->(); + } +} + +# pause {{{3 +sub pause { + die "Can't call in non-interactive mode" if $opts{n}; + my $msg = shift; + print defined($msg) ? "\n$msg" : "\nPress any key to continue"; + ReadMode('cbreak'); + my $char = ReadKey(0); + ReadMode('normal'); + return $char; +} + +# reverse_sort {{{3 +sub reverse_sort { + my $tbl = shift; + $tbl_meta{$tbl}->{sort_dir} *= -1; +} + +# select_cxn {{{3 +# Selects connection(s). If the mode (or argument list) has only one, returns +# it without prompt. +sub select_cxn { + my ( $prompt, @cxns ) = @_; + if ( !@cxns ) { + @cxns = get_connections(); + } + if ( @cxns == 1 ) { + return $cxns[0]; + } + my $choices = prompt_list( + $prompt, + $cxns[0], + sub{ return @cxns }, + { map { $_ => $connections{$_}->{dsn} } @cxns }); + my @result = unique(grep { my $a = $_; grep { $_ eq $a } @cxns } split(/\s+/, $choices)); + return @result; +} + +# kill_query {{{3 +# Kills a connection, or on new versions, optionally a query but not connection. +sub kill_query { + my ( $q_or_c ) = @_; + + my $info = choose_thread( + sub { 1 }, + 'Select a thread to kill the ' . $q_or_c, + ); + return unless $info; + return unless pause("Kill $info->{id}?") =~ m/y/i; + + eval { + do_stmt($info->{cxn}, $q_or_c eq 'QUERY' ? 'KILL_QUERY' : 'KILL_CONNECTION', $info->{id} ); + }; + + if ( $EVAL_ERROR ) { + print "\nError: $EVAL_ERROR"; + pause(); + } +} + +# set_display_precision {{{3 +sub set_display_precision { + my $dir = shift; + $config{num_digits}->{val} = min(9, max(0, $config{num_digits}->{val} + $dir)); +} + +sub toggle_visible_table { + my ( $mode, $table ) = @_; + my $visible = $modes{$mode}->{visible_tables}; + if ( grep { $_ eq $table } @$visible ) { + $modes{$mode}->{visible_tables} = [ grep { $_ ne $table } @$visible ]; + } + else { + unshift @$visible, $table; + } + $modes{$mode}->{cust}->{visible_tables} = 1; +} + +# toggle_filter{{{3 +sub toggle_filter { + my ( $tbl, $filter ) = @_; + my $filters = $tbl_meta{$tbl}->{filters}; + if ( grep { $_ eq $filter } @$filters ) { + $tbl_meta{$tbl}->{filters} = [ grep { $_ ne $filter } @$filters ]; + } + else { + push @$filters, $filter; + } + $tbl_meta{$tbl}->{cust}->{filters} = 1; +} + +# toggle_config {{{3 +sub toggle_config { + my ( $key ) = @_; + $config{$key}->{val} ^= 1; +} + +# create_deadlock {{{3 +sub create_deadlock { + $clear_screen_sub->(); + + print "This function will deliberately cause a small deadlock, " + . "clearing deadlock information from the InnoDB monitor.\n\n"; + + my $answer = prompt("Are you sure you want to proceed? Say 'y' if you do"); + return 0 unless $answer eq 'y'; + + my ( $cxn ) = select_cxn('Clear on which server? '); + return unless $cxn && exists($connections{$cxn}); + + clear_deadlock($cxn); +} + +# deadlock_thread {{{3 +sub deadlock_thread { + my ( $id, $tbl, $cxn ) = @_; + + eval { + my $dbh = get_new_db_connection($cxn, 1); + my @stmts = ( + "set transaction isolation level serializable", + (version_ge($dbh, '4.0.11') ? "start transaction" : 'begin'), + "select * from $tbl where a = $id", + "update $tbl set a = $id where a <> $id", + ); + + foreach my $stmt (@stmts[0..2]) { + $dbh->do($stmt); + } + sleep(1 + $id); + $dbh->do($stmts[-1]); + }; + if ( $EVAL_ERROR ) { + if ( $EVAL_ERROR !~ m/Deadlock found/ ) { + die $EVAL_ERROR; + } + } + exit(0); +} + +# Purges unused binlogs on the master, up to but not including the latest log. +# TODO: guess which connections are slaves of a given master. +sub purge_master_logs { + my @cxns = get_connections(); + + get_master_slave_status(@cxns); + + # Toss out the rows that don't have master/slave status... + my @vars = + grep { $_ && ($_->{file} || $_->{master_host}) } + map { $vars{$_}->{$clock} } @cxns; + @cxns = map { $_->{cxn} } @vars; + + # Figure out which master to purge ons. + my @masters = map { $_->{cxn} } grep { $_->{file} } @vars; + my ( $master ) = select_cxn('Which master?', @masters ); + return unless $master; + my ($master_status) = grep { $_->{cxn} eq $master } @vars; + + # Figure out the result order (not lexical order) of master logs. + my @master_logs = get_master_logs($master); + my $i = 0; + my %master_logs = map { $_->{log_name} => $i++ } @master_logs; + + # Ask which slave(s) are reading from this master. + my @slave_status = grep { $_->{master_host} } @vars; + my @slaves = map { $_->{cxn} } @slave_status; + @slaves = select_cxn("Which slaves are reading from $master?", @slaves); + @slave_status = grep { my $item = $_; grep { $item->{cxn} eq $_ } @slaves } @slave_status; + return unless @slave_status; + + # Find the minimum binary log in use. + my $min_log = min(map { $master_logs{$_->{master_log_file}} } @slave_status); + my $log_name = $master_logs[$min_log]->{log_name}; + + my $stmt = "PURGE MASTER LOGS TO '$log_name'"; + send_cmd_to_servers($stmt, 0, 'PURGE {MASTER | BINARY} LOGS {TO "log_name" | BEFORE "date"}', [$master]); +} + +sub send_cmd_to_servers { + my ( $cmd, $all, $hint, $cxns ) = @_; + if ( $all ) { + @$cxns = get_connections(); + } + elsif ( !@$cxns ) { + @$cxns = select_cxn('Which servers?', @$cxns); + } + if ( $hint ) { + print "\nHint: $hint\n"; + } + $cmd = prompt('Command to send', undef, $cmd); + foreach my $cxn ( @$cxns ) { + eval { + my $sth = do_query($cxn, $cmd); + }; + if ( $EVAL_ERROR ) { + print "Error from $cxn: $EVAL_ERROR\n"; + } + else { + print "Success on $cxn\n"; + } + } + pause(); +} + +# Display functions {{{2 + +sub set_s_mode { + my ( $func ) = @_; + $clear_screen_sub->(); + $config{S_func}->{val} = $func; +} + +# start_S_mode {{{3 +sub start_S_mode { + $clear_screen_sub->(); + switch_mode('S'); +} + +# display_B {{{3 +sub display_B { + my @display_lines; + my @cxns = get_connections(); + get_innodb_status(\@cxns); + + my @buffer_pool; + my @page_statistics; + my @insert_buffers; + my @adaptive_hash_index; + my %rows_for = ( + buffer_pool => \@buffer_pool, + page_statistics => \@page_statistics, + insert_buffers => \@insert_buffers, + adaptive_hash_index => \@adaptive_hash_index, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + foreach my $cxn ( @cxns ) { + my $set = $vars{$cxn}->{$clock}; + my $pre = $vars{$cxn}->{$clock-1} || $set; + + if ( $set->{IB_bp_complete} ) { + if ( $wanted{buffer_pool} ) { + push @buffer_pool, extract_values($set, $set, $pre, 'buffer_pool'); + } + if ( $wanted{page_statistics} ) { + push @page_statistics, extract_values($set, $set, $pre, 'page_statistics'); + } + } + if ( $set->{IB_ib_complete} ) { + if ( $wanted{insert_buffers} ) { + push @insert_buffers, extract_values( + $config{status_inc}->{val} ? inc(0, $cxn) : $set, $set, $pre, + 'insert_buffers'); + } + if ( $wanted{adaptive_hash_index} ) { + push @adaptive_hash_index, extract_values($set, $set, $pre, 'adaptive_hash_index'); + } + } + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + draw_screen(\@display_lines); +} + +# display_C {{{3 +sub display_C { + my @display_lines; + my @cxns = get_connections(); + get_status_info(@cxns); + + my @cmd_summary; + my %rows_for = ( + cmd_summary => \@cmd_summary, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + # For now, I'm manually pulling these variables out and pivoting. Eventually a SQL-ish + # dialect should let me join a table to a grouped and pivoted table and do this more easily. + # TODO: make it so. + my $prefix = qr/^$config{cmd_filter}->{val}/; # TODO: this is a total hack + my @values; + my ($total, $last_total) = (0, 0); + foreach my $cxn ( @cxns ) { + my $set = $vars{$cxn}->{$clock}; + my $pre = $vars{$cxn}->{$clock-1} || $set; + foreach my $key ( keys %$set ) { + next unless $key =~ m/$prefix/i; + my $val = $set->{$key}; + next unless defined $val && $val =~ m/^\d+$/; + my $last_val = $val - ($pre->{$key} || 0); + $total += $val; + $last_total += $last_val; + push @values, { + name => $key, + value => $val, + last_value => $last_val, + }; + } + } + + # Add aggregation and turn into a real set TODO: total hack + if ( $wanted{cmd_summary} ) { + foreach my $value ( @values ) { + @{$value}{qw(total last_total)} = ($total, $last_total); + push @cmd_summary, extract_values($value, $value, $value, 'cmd_summary'); + } + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + draw_screen(\@display_lines); +} + +# display_D {{{3 +sub display_D { + my @display_lines; + my @cxns = get_connections(); + get_innodb_status(\@cxns); + + my @deadlock_transactions; + my @deadlock_locks; + my %rows_for = ( + deadlock_transactions => \@deadlock_transactions, + deadlock_locks => \@deadlock_locks, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + foreach my $cxn ( @cxns ) { + my $innodb_status = $vars{$cxn}->{$clock}; + my $prev_status = $vars{$cxn}->{$clock-1} || $innodb_status; + + if ( $innodb_status->{IB_dl_timestring} ) { + + my $victim = $innodb_status->{IB_dl_rolled_back} || 0; + + if ( %wanted ) { + foreach my $txn_id ( keys %{$innodb_status->{IB_dl_txns}} ) { + my $txn = $innodb_status->{IB_dl_txns}->{$txn_id}; + my $pre = $prev_status->{IB_dl_txns}->{$txn_id} || $txn; + + if ( $wanted{deadlock_transactions} ) { + my $hash = extract_values($txn->{tx}, $txn->{tx}, $pre->{tx}, 'deadlock_transactions'); + $hash->{cxn} = $cxn; + $hash->{dl_txn_num} = $txn_id; + $hash->{victim} = $txn_id == $victim ? 'Yes' : 'No'; + $hash->{timestring} = $innodb_status->{IB_dl_timestring}; + $hash->{truncates} = $innodb_status->{IB_dl_complete} ? 'No' : 'Yes'; + push @deadlock_transactions, $hash; + } + + if ( $wanted{deadlock_locks} ) { + foreach my $lock ( @{$txn->{locks}} ) { + my $hash = extract_values($lock, $lock, $lock, 'deadlock_locks'); + $hash->{dl_txn_num} = $txn_id; + $hash->{cxn} = $cxn; + $hash->{mysql_thread_id} = $txn->{tx}->{mysql_thread_id}; + push @deadlock_locks, $hash; + } + } + + } + } + } + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + draw_screen(\@display_lines); +} + +# display_F {{{3 +sub display_F { + my @display_lines; + my ( $cxn ) = get_connections(); + get_innodb_status([$cxn]); + my $innodb_status = $vars{$cxn}->{$clock}; + + if ( $innodb_status->{IB_fk_timestring} ) { + + push @display_lines, 'Reason: ' . $innodb_status->{IB_fk_reason}; + + # Display FK errors caused by invalid DML. + if ( $innodb_status->{IB_fk_txn} ) { + my $txn = $innodb_status->{IB_fk_txn}; + push @display_lines, + '', + "User $txn->{user} from $txn->{hostname}, thread $txn->{mysql_thread_id} was executing:", + '', no_ctrl_char($txn->{query_text}); + } + + my @fk_table = create_table2( + $tbl_meta{fk_error}->{visible}, + meta_to_hdr('fk_error'), + extract_values($innodb_status, $innodb_status, $innodb_status, 'fk_error'), + { just => '-', sep => ' '}); + push @display_lines, '', @fk_table; + + } + else { + push @display_lines, '', 'No foreign key error data.'; + } + draw_screen(\@display_lines, { raw => 1 } ); +} + +# display_I {{{3 +sub display_I { + my @display_lines; + my @cxns = get_connections(); + get_innodb_status(\@cxns); + + my @io_threads; + my @pending_io; + my @file_io_misc; + my @log_statistics; + my %rows_for = ( + io_threads => \@io_threads, + pending_io => \@pending_io, + file_io_misc => \@file_io_misc, + log_statistics => \@log_statistics, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + foreach my $cxn ( @cxns ) { + my $set = $vars{$cxn}->{$clock}; + my $pre = $vars{$cxn}->{$clock-1} || $set; + + if ( $set->{IB_io_complete} ) { + if ( $wanted{io_threads} ) { + my $cur_threads = $set->{IB_io_threads}; + my $pre_threads = $pre->{IB_io_threads} || $cur_threads; + foreach my $key ( sort keys %$cur_threads ) { + my $cur_thd = $cur_threads->{$key}; + my $pre_thd = $pre_threads->{$key} || $cur_thd; + my $hash = extract_values($cur_thd, $cur_thd, $pre_thd, 'io_threads'); + $hash->{cxn} = $cxn; + push @io_threads, $hash; + } + } + if ( $wanted{pending_io} ) { + push @pending_io, extract_values($set, $set, $pre, 'pending_io'); + } + if ( $wanted{file_io_misc} ) { + push @file_io_misc, extract_values( + $config{status_inc}->{val} ? inc(0, $cxn) : $set, + $set, $pre, 'file_io_misc'); + } + } + if ( $set->{IB_lg_complete} && $wanted{log_statistics} ) { + push @log_statistics, extract_values($set, $set, $pre, 'log_statistics'); + } + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + draw_screen(\@display_lines); +} + +# display_L {{{3 +sub display_L { + my @display_lines; + my @cxns = get_connections(); + get_innodb_status(\@cxns); + + my @innodb_locks; + my %rows_for = ( + innodb_locks => \@innodb_locks, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + # Get info on locks + foreach my $cxn ( @cxns ) { + my $set = $vars{$cxn}->{$clock} or next; + my $pre = $vars{$cxn}->{$clock-1} || $set; + + if ( $wanted{innodb_locks} && defined $set->{IB_tx_transactions} && @{$set->{IB_tx_transactions}} ) { + + my $cur_txns = $set->{IB_tx_transactions}; + my $pre_txns = $pre->{IB_tx_transactions} || $cur_txns; + my %cur_txns = map { $_->{mysql_thread_id} => $_ } @$cur_txns; + my %pre_txns = map { $_->{mysql_thread_id} => $_ } @$pre_txns; + foreach my $txn ( @$cur_txns ) { + foreach my $lock ( @{$txn->{locks}} ) { + my %hash = map { $_ => $txn->{$_} } qw(txn_id mysql_thread_id lock_wait_time active_secs); + map { $hash{$_} = $lock->{$_} } qw(lock_type space_id page_no n_bits index db table txn_id lock_mode special insert_intention waiting); + $hash{cxn} = $cxn; + push @innodb_locks, extract_values(\%hash, \%hash, \%hash, 'innodb_locks'); + } + } + } + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + draw_screen(\@display_lines); +} + +# display_M {{{3 +sub display_M { + my @display_lines; + my @cxns = get_connections(); + get_master_slave_status(@cxns); + get_status_info(@cxns); + + my @slave_sql_status; + my @slave_io_status; + my @master_status; + my %rows_for = ( + slave_sql_status => \@slave_sql_status, + slave_io_status => \@slave_io_status, + master_status => \@master_status, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + foreach my $cxn ( @cxns ) { + my $set = $config{status_inc}->{val} ? inc(0, $cxn) : $vars{$cxn}->{$clock}; + my $pre = $vars{$cxn}->{$clock - 1} || $set; + if ( $wanted{slave_sql_status} ) { + push @slave_sql_status, extract_values($set, $set, $pre, 'slave_sql_status'); + } + if ( $wanted{slave_io_status} ) { + push @slave_io_status, extract_values($set, $set, $pre, 'slave_io_status'); + } + if ( $wanted{master_status} ) { + push @master_status, extract_values($set, $set, $pre, 'master_status'); + } + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + draw_screen(\@display_lines); +} + +# display_O {{{3 +sub display_O { + my @display_lines = (''); + my @cxns = get_connections(); + my @open_tables = get_open_tables(@cxns); + my @tables = map { extract_values($_, $_, $_, 'open_tables') } @open_tables; + push @display_lines, set_to_tbl(\@tables, 'open_tables'), get_cxn_errors(@cxns); + draw_screen(\@display_lines); +} + +# display_Q {{{3 +sub display_Q { + my @display_lines; + + my @q_header; + my @processlist; + my %rows_for = ( + q_header => \@q_header, + processlist => \@processlist, + ); + + my @visible = $opts{n} ? 'processlist' : get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + # Get the data + my @cxns = get_connections(); + my @full_processlist = get_full_processlist(@cxns); + + # Create header + if ( $wanted{q_header} ) { + get_status_info(@cxns); + foreach my $cxn ( @cxns ) { + my $set = $vars{$cxn}->{$clock}; + my $pre = $vars{$cxn}->{$clock-1} || $set; + my $hash = extract_values($set, $set, $pre, 'q_header'); + $hash->{cxn} = $cxn; + $hash->{when} = 'Total'; + push @q_header, $hash; + + if ( exists $vars{$cxn}->{$clock - 1} ) { + my $inc = inc(0, $cxn); + my $hash = extract_values($inc, $set, $pre, 'q_header'); + $hash->{cxn} = $cxn; + $hash->{when} = 'Now'; + push @q_header, $hash; + } + } + } + + if ( $wanted{processlist} ) { + # TODO: save prev values + push @processlist, map { extract_values($_, $_, $_, 'processlist') } @full_processlist; + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + next unless $wanted{$tbl}; + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + # Save queries in global variable for analysis. The rows in %rows_for have been + # filtered, etc as a side effect of set_to_tbl(), so they are the same as the rows + # that get pushed to the screen. + @current_queries = map { + my %hash; + @hash{ qw(cxn id db query secs) } = @{$_}{ qw(cxn mysql_thread_id db info secs) }; + \%hash; + } @{$rows_for{processlist}}; + + draw_screen(\@display_lines); +} + +# display_R {{{3 +sub display_R { + my @display_lines; + my @cxns = get_connections(); + get_innodb_status(\@cxns); + + my @row_operations; + my @row_operation_misc; + my @semaphores; + my @wait_array; + my %rows_for = ( + row_operations => \@row_operations, + row_operation_misc => \@row_operation_misc, + semaphores => \@semaphores, + wait_array => \@wait_array, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + my $incvar = $config{status_inc}->{val}; + + foreach my $cxn ( @cxns ) { + my $set = $vars{$cxn}->{$clock}; + my $pre = $vars{$cxn}->{$clock-1} || $set; + my $inc; # Only assigned to if wanted + + if ( $set->{IB_ro_complete} ) { + if ( $wanted{row_operations} ) { + $inc ||= $incvar ? inc(0, $cxn) : $set; + push @row_operations, extract_values($inc, $set, $pre, 'row_operations'); + } + if ( $wanted{row_operation_misc} ) { + push @row_operation_misc, extract_values($set, $set, $pre, 'row_operation_misc'), + } + } + + if ( $set->{IB_sm_complete} && $wanted{semaphores} ) { + $inc ||= $incvar ? inc(0, $cxn) : $set; + push @semaphores, extract_values($inc, $set, $pre, 'semaphores'); + } + + if ( $set->{IB_sm_wait_array_size} && $wanted{wait_array} ) { + foreach my $wait ( @{$set->{IB_sm_waits}} ) { + my $hash = extract_values($wait, $wait, $wait, 'wait_array'); + $hash->{cxn} = $cxn; + push @wait_array, $hash; + } + } + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + draw_screen(\@display_lines); +} + +# display_T {{{3 +sub display_T { + my @display_lines; + + my @t_header; + my @innodb_transactions; + my %rows_for = ( + t_header => \@t_header, + innodb_transactions => \@innodb_transactions, + ); + + my @visible = $opts{n} ? 'innodb_transactions' : get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + + my @cxns = get_connections(); + + # If the header is to be shown, buffer pool data is required. + get_innodb_status( \@cxns, [ $wanted{t_header} ? qw(bp) : () ] ); + + foreach my $cxn ( get_connections() ) { + my $set = $vars{$cxn}->{$clock}; + my $pre = $vars{$cxn}->{$clock-1} || $set; + + next unless $set->{IB_tx_transactions}; + + if ( $wanted{t_header} ) { + my $hash = extract_values($set, $set, $pre, 't_header'); + push @t_header, $hash; + } + + if ( $wanted{innodb_transactions} ) { + my $cur_txns = $set->{IB_tx_transactions}; + my $pre_txns = $pre->{IB_tx_transactions} || $cur_txns; + my %cur_txns = map { $_->{mysql_thread_id} => $_ } @$cur_txns; + my %pre_txns = map { $_->{mysql_thread_id} => $_ } @$pre_txns; + foreach my $thd_id ( sort keys %cur_txns ) { + my $cur_txn = $cur_txns{$thd_id}; + my $pre_txn = $pre_txns{$thd_id} || $cur_txn; + my $hash = extract_values($cur_txn, $cur_txn, $pre_txn, 'innodb_transactions'); + $hash->{cxn} = $cxn; + push @innodb_transactions, $hash; + } + } + + } + + my $first_table = 0; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + + # Save queries in global variable for analysis. The rows in %rows_for have been + # filtered, etc as a side effect of set_to_tbl(), so they are the same as the rows + # that get pushed to the screen. + @current_queries = map { + my %hash; + @hash{ qw(cxn id db query secs) } = @{$_}{ qw(cxn mysql_thread_id db query_text active_secs) }; + \%hash; + } @{$rows_for{innodb_transactions}}; + + draw_screen(\@display_lines); +} + +# display_S {{{3 +sub display_S { + my $fmt = get_var_set('S_set'); + my $func = $config{S_func}->{val}; + my $inc = $func eq 'g' || $config{status_inc}->{val}; + + # The table's meta-data is generated from the compiled var_set. + my ( $cols, $visible ); + if ( $tbl_meta{var_status}->{fmt} && $fmt eq $tbl_meta{var_status}->{fmt} ) { + ( $cols, $visible ) = @{$tbl_meta{var_status}}{qw(cols visible)}; + } + else { + ( $cols, $visible ) = compile_select_stmt($fmt); + + # Apply missing values to columns. Always apply averages across all connections. + map { + $_->{agg} = 'avg'; + $_->{label} = $_->{hdr}; + } values %$cols; + + $tbl_meta{var_status}->{cols} = $cols; + $tbl_meta{var_status}->{visible} = $visible; + $tbl_meta{var_status}->{fmt} = $fmt; + map { $tbl_meta{var_status}->{cols}->{$_}->{just} = ''} @$visible; + } + + my @var_status; + my %rows_for = ( + var_status => \@var_status, + ); + + my @visible = get_visible_tables(); + my %wanted = map { $_ => 1 } @visible; + my @cxns = get_connections(); + + get_status_info(@cxns); + get_innodb_status(\@cxns); + + # Set up whether to pivot and how many sets to extract. + $tbl_meta{var_status}->{pivot} = $func eq 'v'; + + my $num_sets + = $func eq 'v' + ? $config{num_status_sets}->{val} + : 0; + foreach my $set ( 0 .. $num_sets ) { + my @rows; + foreach my $cxn ( @cxns ) { + my $vars = $inc ? inc($set, $cxn) : $vars{$cxn}->{$clock - $set}; + my $cur = $vars{$cxn}->{$clock-$set}; + my $pre = $vars{$cxn}->{$clock-$set-1} || $cur; + next unless $vars && %$vars; + my $hash = extract_values($vars, $cur, $pre, 'var_status'); + push @rows, $hash; + } + @rows = apply_group_by('var_status', [], @rows); + push @var_status, @rows; + } + + # Recompile the sort func. TODO: avoid recompiling at every refresh. + # Figure out whether the data is all numeric and decide on a sort type. + # my $cmp + # = scalar( + # grep { !defined $_ || $_ !~ m/^\d+$/ } + # map { my $col = $_; map { $_->{$col} } @var_status } + # $tbl_meta{var_status}->{sort_cols} =~ m/(\w+)/g) + # ? 'cmp' + # : '<=>'; + $tbl_meta{var_status}->{sort_func} = make_sort_func($tbl_meta{var_status}); + + # ################################################################ + # Now there is specific display code based on $config{S_func} + # ################################################################ + if ( $func =~ m/s|g/ ) { + my $min_width = 4; + + # Clear the screen if the display width changed. + if ( @last_term_size && $this_term_size[0] != $last_term_size[0] ) { + $lines_printed = 0; + $clear_screen_sub->(); + } + + if ( $func eq 's' ) { + # Decide how wide columns should be. + my $num_cols = scalar(@$visible); + my $width = $opts{n} ? 0 : max($min_width, int(($this_term_size[0] - $num_cols + 1) / $num_cols)); + my $g_format = $opts{n} ? ( "%s\t" x $num_cols ) : ( "%-${width}s " x $num_cols ); + + # Print headers every now and then. Headers can get really long, so compact them. + my @hdr = @$visible; + if ( $opts{n} ) { + if ( $lines_printed == 0 ) { + print join("\t", @hdr), "\n"; + $lines_printed++; + } + } + elsif ( $lines_printed == 0 || $lines_printed > $this_term_size[1] - 2 ) { + @hdr = map { donut(crunch($_, $width), $width) } @hdr; + print join(' ', map { sprintf( "%${width}s", donut($_, $width)) } @hdr) . "\n"; + $lines_printed = 1; + } + + # Design a column format for the values. + my $format + = $opts{n} + ? join("\t", map { '%s' } @$visible) . "\n" + : join(' ', map { "%${width}s" } @hdr) . "\n"; + + foreach my $row ( @var_status ) { + printf($format, map { defined $_ ? $_ : '' } @{$row}{ @$visible }); + $lines_printed++; + } + } + else { # 'g' mode + # Design a column format for the values. + my $num_cols = scalar(@$visible); + my $width = $opts{n} ? 0 : int(($this_term_size[0] - $num_cols + 1) / $num_cols); + my $format = $opts{n} ? ( "%s\t" x $num_cols ) : ( "%-${width}s " x $num_cols ); + $format =~ s/\s$/\n/; + + # Print headers every now and then. + if ( $opts{n} ) { + if ( $lines_printed == 0 ) { + print join("\t", @$visible), "\n"; + print join("\t", map { shorten($mvs{$_}) } @$visible), "\n"; + } + } + elsif ( $lines_printed == 0 || $lines_printed > $this_term_size[1] - 2 ) { + printf($format, map { donut(crunch($_, $width), $width) } @$visible); + printf($format, map { shorten($mvs{$_} || 0) } @$visible); + $lines_printed = 2; + } + + # Update the max ever seen, and scale by the max ever seen. + my $set = $var_status[0]; + foreach my $col ( @$visible ) { + $set->{$col} = 1 unless defined $set->{$col} && $set->{$col} =~ m/$num_regex/; + $set->{$col} = ($set->{$col} || 1) / ($set->{Uptime_hires} || 1); + $mvs{$col} = max($mvs{$col} || 1, $set->{$col}); + $set->{$col} /= $mvs{$col}; + } + printf($format, map { ( $config{graph_char}->{val} x int( $width * $set->{$_} )) || '.' } @$visible ); + $lines_printed++; + + } + } + else { # 'v' + my $first_table = 0; + my @display_lines; + foreach my $tbl ( @visible ) { + push @display_lines, '', set_to_tbl($rows_for{$tbl}, $tbl); + push @display_lines, get_cxn_errors(@cxns) + if ( $config{debug}->{val} || !$first_table++ ); + } + $clear_screen_sub->(); + draw_screen( \@display_lines ); + } +} + +# display_explain {{{3 +sub display_explain { + my $info = shift; + my $cxn = $info->{cxn}; + my $db = $info->{db}; + + my ( $mods, $query ) = rewrite_for_explain($info->{query}); + + my @display_lines; + + if ( $query ) { + + my $part = version_ge($dbhs{$cxn}->{dbh}, '5.1.5') ? 'PARTITIONS' : ''; + $query = "EXPLAIN $part\n" . $query; + + eval { + if ( $db ) { + do_query($cxn, "use $db"); + } + my $sth = do_query($cxn, $query); + + my $res; + while ( $res = $sth->fetchrow_hashref() ) { + map { $res->{$_} ||= '' } ( 'partitions', keys %$res); + my @this_table = create_caption("Sub-Part $res->{id}", + create_table2( + $tbl_meta{explain}->{visible}, + meta_to_hdr('explain'), + extract_values($res, $res, $res, 'explain'))); + @display_lines = stack_next(\@display_lines, \@this_table, { pad => ' ', vsep => 2 }); + } + }; + + if ( $EVAL_ERROR ) { + push @display_lines, + '', + "The query could not be explained. Only SELECT queries can be " + . "explained; innotop tries to rewrite certain REPLACE and INSERT queries " + . "into SELECT, but this doesn't always succeed."; + } + + } + else { + push @display_lines, '', 'The query could not be explained.'; + } + + if ( $mods ) { + push @display_lines, '', '[This query has been re-written to be explainable]'; + } + + unshift @display_lines, no_ctrl_char($query); + draw_screen(\@display_lines, { raw => 1 } ); +} + +# rewrite_for_explain {{{3 +sub rewrite_for_explain { + my $query = shift; + + my $mods = 0; + my $orig = $query; + $mods += $query =~ s/^\s*(?:replace|insert).*?select/select/is; + $mods += $query =~ s/^ + \s*create\s+(?:temporary\s+)?table + \s+(?:\S+\s+)as\s+select/select/xis; + $mods += $query =~ s/\s+on\s+duplicate\s+key\s+update.*$//is; + return ( $mods, $query ); +} + +# show_optimized_query {{{3 +sub show_optimized_query { + my $info = shift; + my $cxn = $info->{cxn}; + my $db = $info->{db}; + my $meta = $dbhs{$cxn}; + + my @display_lines; + + my ( $mods, $query ) = rewrite_for_explain($info->{query}); + + if ( $mods ) { + push @display_lines, '[This query has been re-written to be explainable]'; + } + + if ( $query ) { + push @display_lines, no_ctrl_char($info->{query}); + + eval { + if ( $db ) { + do_query($cxn, "use $db"); + } + do_query( $cxn, 'EXPLAIN EXTENDED ' . $query ) or die "Can't explain query"; + my $sth = do_query($cxn, 'SHOW WARNINGS'); + my $res = $sth->fetchall_arrayref({}); + + if ( $res ) { + foreach my $result ( @$res ) { + push @display_lines, 'Note:', no_ctrl_char($result->{message}); + } + } + else { + push @display_lines, '', 'The query optimization could not be generated.'; + } + }; + + if ( $EVAL_ERROR ) { + push @display_lines, '', "The optimization could not be generated: $EVAL_ERROR"; + } + + } + else { + push @display_lines, '', 'The query optimization could not be generated.'; + } + + draw_screen(\@display_lines, { raw => 1 } ); +} + +# display_help {{{3 +sub display_help { + my $mode = $config{mode}->{val}; + + # Get globally mapped keys, then overwrite them with mode-specific ones. + my %keys = map { + $_ => $action_for{$_}->{label} + } keys %action_for; + foreach my $key ( keys %{$modes{$mode}->{action_for}} ) { + $keys{$key} = $modes{$mode}->{action_for}->{$key}->{label}; + } + delete $keys{'?'}; + + # Split them into three kinds of keys: MODE keys, action keys, and + # magic (special character) keys. + my @modes = sort grep { m/[A-Z]/ } keys %keys; + my @actions = sort grep { m/[a-z]/ } keys %keys; + my @magic = sort grep { m/[^A-Z]/i } keys %keys; + + my @display_lines = ( '', 'Switch to a different mode:' ); + + # Mode keys + my @all_modes = map { "$_ $modes{$_}->{hdr}" } @modes; + my @col1 = splice(@all_modes, 0, ceil(@all_modes/3)); + my @col2 = splice(@all_modes, 0, ceil(@all_modes/2)); + my $max1 = max(map {length($_)} @col1); + my $max2 = max(map {length($_)} @col2); + while ( @col1 ) { + push @display_lines, sprintf(" %-${max1}s %-${max2}s %s", + (shift @col1 || ''), + (shift @col2 || ''), + (shift @all_modes || '')); + } + + # Action keys + my @all_actions = map { "$_ $keys{$_}" } @actions; + @col1 = splice(@all_actions, 0, ceil(@all_actions/2)); + $max1 = max(map {length($_)} @col1); + push @display_lines, '', 'Actions:'; + while ( @col1 ) { + push @display_lines, sprintf(" %-${max1}s %s", + (shift @col1 || ''), + (shift @all_actions || '')); + } + + # Magic keys + my @all_magic = map { sprintf('%4s', $action_for{$_}->{key} || $_) . " $keys{$_}" } @magic; + @col1 = splice(@all_magic, 0, ceil(@all_magic/2)); + $max1 = max(map {length($_)} @col1); + push @display_lines, '', 'Other:'; + while ( @col1 ) { + push @display_lines, sprintf("%-${max1}s%s", + (shift @col1 || ''), + (shift @all_magic || '')); + } + + $clear_screen_sub->(); + draw_screen(\@display_lines, { show_all => 1 } ); + pause(); + $clear_screen_sub->(); +} + +# show_full_query {{{3 +sub show_full_query { + my $info = shift; + my @display_lines = no_ctrl_char($info->{query}); + draw_screen(\@display_lines, { raw => 1 }); +} + +# Formatting functions {{{2 + +# create_table2 {{{3 +# Makes a two-column table, labels on left, data on right. +# Takes refs of @cols, %labels and %data, %user_prefs +sub create_table2 { + my ( $cols, $labels, $data, $user_prefs ) = @_; + my @rows; + + if ( @$cols && %$data ) { + + # Override defaults + my $p = { + just => '', + sep => ':', + just1 => '-', + }; + if ( $user_prefs ) { + map { $p->{$_} = $user_prefs->{$_} } keys %$user_prefs; + } + + # Fix undef values + map { $data->{$_} = '' unless defined $data->{$_} } @$cols; + + # Format the table + my $max_l = max(map{ length($labels->{$_}) } @$cols); + my $max_v = max(map{ length($data->{$_}) } @$cols); + my $format = "%$p->{just}${max_l}s$p->{sep} %$p->{just1}${max_v}s"; + foreach my $col ( @$cols ) { + push @rows, sprintf($format, $labels->{$col}, $data->{$col}); + } + } + return @rows; +} + +# stack_next {{{3 +# Stacks one display section next to the other. Accepts left-hand arrayref, +# right-hand arrayref, and options hashref. Tries to stack as high as +# possible, so +# aaaaaa +# bbb +# can stack ccc next to the bbb. +# NOTE: this DOES modify its arguments, even though it returns a new array. +sub stack_next { + my ( $left, $right, $user_prefs ) = @_; + my @result; + + my $p = { + pad => ' ', + vsep => 0, + }; + if ( $user_prefs ) { + map { $p->{$_} = $user_prefs->{$_} } keys %$user_prefs; + } + + # Find out how wide the LHS can be and still let the RHS fit next to it. + my $pad = $p->{pad}; + my $max_r = max( map { length($_) } @$right) || 0; + my $max_l = $this_term_size[0] - $max_r - length($pad); + + # Find the minimum row on the LHS that the RHS will fit next to. + my $i = scalar(@$left) - 1; + while ( $i >= 0 && length($left->[$i]) <= $max_l ) { + $i--; + } + $i++; + my $offset = $i; + + if ( $i < scalar(@$left) ) { + # Find the max width of the section of the LHS against which the RHS + # will sit. + my $max_i_in_common = min($i + scalar(@$right) - 1, scalar(@$left) - 1); + my $max_width = max( map { length($_) } @{$left}[$i..$max_i_in_common]); + + # Append the RHS onto the LHS until one runs out. + while ( $i < @$left && $i - $offset < @$right ) { + my $format = "%-${max_width}s$pad%${max_r}s"; + $left->[$i] = sprintf($format, $left->[$i], $right->[$i - $offset]); + $i++; + } + while ( $i - $offset < @$right ) { + # There is more RHS to push on the end of the array + push @$left, + sprintf("%${max_width}s$pad%${max_r}s", ' ', $right->[$i - $offset]); + $i++; + } + push @result, @$left; + } + else { + # There is no room to put them side by side. Add them below, with + # a blank line above them if specified. + push @result, @$left; + push @result, (' ' x $this_term_size[0]) if $p->{vsep} && @$left; + push @result, @$right; + } + return @result; +} + +# create_caption {{{3 +sub create_caption { + my ( $caption, @rows ) = @_; + if ( @rows ) { + + # Calculate the width of what will be displayed, so it can be centered + # in that space. When the thing is wider than the display, center the + # caption in the display. + my $width = min($this_term_size[0], max(map { length(ref($_) ? $_->[0] : $_) } @rows)); + + my $cap_len = length($caption); + + # It may be narrow enough to pad the sides with underscores and save a + # line on the screen. + if ( $cap_len <= $width - 6 ) { + my $left = int(($width - 2 - $cap_len) / 2); + unshift @rows, + ("_" x $left) . " $caption " . ("_" x ($width - $left - $cap_len - 2)); + } + + # The caption is too wide to add underscores on each side. + else { + + # Color is supported, so we can use terminal underlining. + if ( $config{color}->{val} ) { + my $left = int(($width - $cap_len) / 2); + unshift @rows, [ + (" " x $left) . $caption . (" " x ($width - $left - $cap_len)), + 'underline', + ]; + } + + # Color is not supported, so we have to add a line underneath to separate the + # caption from whatever it's captioning. + else { + my $left = int(($width - $cap_len) / 2); + unshift @rows, ('-' x $width); + unshift @rows, (" " x $left) . $caption . (" " x ($width - $left - $cap_len)); + } + + # The caption is wider than the thing it labels, so we have to pad the + # thing it labels to a consistent width. + if ( $cap_len > $width ) { + @rows = map { + ref($_) + ? [ sprintf('%-' . $cap_len . 's', $_->[0]), $_->[1] ] + : sprintf('%-' . $cap_len . 's', $_); + } @rows; + } + + } + } + return @rows; +} + +# create_table {{{3 +# Input: an arrayref of columns, hashref of col info, and an arrayref of hashes +# Example: [ 'a', 'b' ] +# { a => spec, b => spec } +# [ { a => 1, b => 2}, { a => 3, b => 4 } ] +# The 'spec' is a hashref of hdr => label, just => ('-' or ''). It also supports min and max-widths +# vi the minw and maxw params. +# Output: an array of strings, one per row. +# Example: +# Column One Column Two +# ---------- ---------- +# 1 2 +# 3 4 +sub create_table { + my ( $cols, $info, $data, $prefs ) = @_; + $prefs ||= {}; + $prefs->{no_hdr} ||= ($opts{n} && $clock != 1); + + # Truncate rows that will surely be off screen even if this is the only table. + if ( !$opts{n} && !$prefs->{raw} && !$prefs->{show_all} && $this_term_size[1] < @$data-1 ) { + $data = [ @$data[0..$this_term_size[1] - 1] ]; + } + + my @rows = (); + + if ( @$cols && %$info ) { + + # Fix undef values, collapse whitespace. + foreach my $row ( @$data ) { + map { $row->{$_} = collapse_ws($row->{$_}) } @$cols; + } + + my $col_sep = $opts{n} ? "\t" : ' '; + + # Find each column's max width. + my %width_for; + if ( !$opts{n} ) { + %width_for = map { + my $col_name = $_; + if ( $info->{$_}->{dec} ) { + # Align along the decimal point + my $max_rodp = max(0, map { $_->{$col_name} =~ m/([^\s\d-].*)$/ ? length($1) : 0 } @$data); + foreach my $row ( @$data ) { + my $col = $row->{$col_name}; + my ( $l, $r ) = $col =~ m/^([\s\d]*)(.*)$/; + $row->{$col_name} = sprintf("%s%-${max_rodp}s", $l, $r); + } + } + my $max_width = max( length($info->{$_}->{hdr}), map { length($_->{$col_name}) } @$data); + if ( $info->{$col_name}->{maxw} ) { + $max_width = min( $max_width, $info->{$col_name}->{maxw} ); + } + if ( $info->{$col_name}->{minw} ) { + $max_width = max( $max_width, $info->{$col_name}->{minw} ); + } + $col_name => $max_width; + } @$cols; + } + + # The table header. + if ( !$config{hide_hdr}->{val} && !$prefs->{no_hdr} ) { + push @rows, $opts{n} + ? join( $col_sep, @$cols ) + : join( $col_sep, map { sprintf( "%-$width_for{$_}s", trunc($info->{$_}->{hdr}, $width_for{$_}) ) } @$cols ); + if ( $config{color}->{val} && $config{header_highlight}->{val} ) { + push @rows, [ pop @rows, $config{header_highlight}->{val} ]; + } + elsif ( !$opts{n} ) { + push @rows, join( $col_sep, map { "-" x $width_for{$_} } @$cols ); + } + } + + # The table data. + if ( $opts{n} ) { + foreach my $item ( @$data ) { + push @rows, join($col_sep, map { $item->{$_} } @$cols ); + } + } + else { + my $format = join( $col_sep, + map { "%$info->{$_}->{just}$width_for{$_}s" } @$cols ); + foreach my $item ( @$data ) { + my $row = sprintf($format, map { trunc($item->{$_}, $width_for{$_}) } @$cols ); + if ( $config{color}->{val} && $item->{_color} ) { + push @rows, [ $row, $item->{_color} ]; + } + else { + push @rows, $row; + } + } + } + } + + return @rows; +} + +# Aggregates a table. If $group_by is an arrayref of columns, the grouping key +# is the specified columns; otherwise it's just the empty string (e.g. +# everything is grouped as one group). +sub apply_group_by { + my ( $tbl, $group_by, @rows ) = @_; + my $meta = $tbl_meta{$tbl}; + my %is_group = map { $_ => 1 } @$group_by; + my @non_grp = grep { !$is_group{$_} } keys %{$meta->{cols}}; + + my %temp_table; + foreach my $row ( @rows ) { + my $group_key + = @$group_by + ? '{' . join('}{', map { defined $_ ? $_ : '' } @{$row}{@$group_by}) . '}' + : ''; + $temp_table{$group_key} ||= []; + push @{$temp_table{$group_key}}, $row; + } + + # Crush the rows together... + my @new_rows; + foreach my $key ( sort keys %temp_table ) { + my $group = $temp_table{$key}; + my %new_row; + @new_row{@$group_by} = @{$group->[0]}{@$group_by}; + foreach my $col ( @non_grp ) { + my $agg = $meta->{cols}->{$col}->{agg} || 'first'; + $new_row{$col} = $agg_funcs{$agg}->( map { $_->{$col} } @$group ); + } + push @new_rows, \%new_row; + } + return @new_rows; +} + +# set_to_tbl {{{3 +# Unifies all the work of filtering, sorting etc. Alters the input. +# TODO: pull all the little pieces out into subroutines and stick events in each of them. +sub set_to_tbl { + my ( $rows, $tbl ) = @_; + my $meta = $tbl_meta{$tbl} or die "No such table $tbl in tbl_meta"; + + if ( !$meta->{pivot} ) { + + # Hook in event listeners + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_filter}} ) { + $listener->set_to_tbl_pre_filter($rows, $tbl); + } + + # Apply filters. Note that if the table is pivoted, filtering and sorting + # are applied later. + foreach my $filter ( @{$meta->{filters}} ) { + eval { + @$rows = grep { $filters{$filter}->{func}->($_) } @$rows; + }; + if ( $EVAL_ERROR && $config{debug}->{val} ) { + die $EVAL_ERROR; + } + } + + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_sort}} ) { + $listener->set_to_tbl_pre_sort($rows, $tbl); + } + + # Sort. Note that if the table is pivoted, sorting might have the wrong + # columns and it could crash. This will only be an issue if it's possible + # to toggle pivoting on and off, which it's not at the moment. + if ( @$rows && $meta->{sort_func} && !$meta->{aggregate} ) { + if ( $meta->{sort_dir} > 0 ) { + @$rows = $meta->{sort_func}->( @$rows ); + } + else { + @$rows = reverse $meta->{sort_func}->( @$rows ); + } + } + + } + + # Stop altering arguments now. + my @rows = @$rows; + + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_group}} ) { + $listener->set_to_tbl_pre_group(\@rows, $tbl); + } + + # Apply group-by. + if ( $meta->{aggregate} ) { + @rows = apply_group_by($tbl, $meta->{group_by}, @rows); + + # Sort. Note that if the table is pivoted, sorting might have the wrong + # columns and it could crash. This will only be an issue if it's possible + # to toggle pivoting on and off, which it's not at the moment. + if ( @rows && $meta->{sort_func} ) { + if ( $meta->{sort_dir} > 0 ) { + @rows = $meta->{sort_func}->( @rows ); + } + else { + @rows = reverse $meta->{sort_func}->( @rows ); + } + } + + } + + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_colorize}} ) { + $listener->set_to_tbl_pre_colorize(\@rows, $tbl); + } + + if ( !$meta->{pivot} ) { + # Colorize. Adds a _color column to rows. + if ( @rows && $meta->{color_func} ) { + eval { + foreach my $row ( @rows ) { + $row->{_color} = $meta->{color_func}->($row); + } + }; + if ( $EVAL_ERROR ) { + pause($EVAL_ERROR); + } + } + } + + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_transform}} ) { + $listener->set_to_tbl_pre_transform(\@rows, $tbl); + } + + # Apply_transformations. + if ( @rows ) { + my $cols = $meta->{cols}; + foreach my $col ( keys %{$rows->[0]} ) { + # Don't auto-vivify $tbl_meta{tbl}-{cols}->{_color}->{trans} + next if $col eq '_color'; + foreach my $trans ( @{$cols->{$col}->{trans}} ) { + map { $_->{$col} = $trans_funcs{$trans}->($_->{$col}) } @rows; + } + } + } + + my ($fmt_cols, $fmt_meta); + + # Pivot. + if ( $meta->{pivot} ) { + + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_pivot}} ) { + $listener->set_to_tbl_pre_pivot(\@rows, $tbl); + } + + my @vars = @{$meta->{visible}}; + my @tmp = map { { name => $_ } } @vars; + my @cols = 'name'; + foreach my $i ( 0..@$rows-1 ) { + my $col = "set_$i"; + push @cols, $col; + foreach my $j ( 0..@vars-1 ) { + $tmp[$j]->{$col} = $rows[$i]->{$vars[$j]}; + } + } + $fmt_meta = { map { $_ => { hdr => $_, just => '-' } } @cols }; + $fmt_cols = \@cols; + @rows = @tmp; + + # Hook in event listeners + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_filter}} ) { + $listener->set_to_tbl_pre_filter($rows, $tbl); + } + + # Apply filters. + foreach my $filter ( @{$meta->{filters}} ) { + eval { + @rows = grep { $filters{$filter}->{func}->($_) } @rows; + }; + if ( $EVAL_ERROR && $config{debug}->{val} ) { + die $EVAL_ERROR; + } + } + + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_sort}} ) { + $listener->set_to_tbl_pre_sort($rows, $tbl); + } + + # Sort. + if ( @rows && $meta->{sort_func} ) { + if ( $meta->{sort_dir} > 0 ) { + @rows = $meta->{sort_func}->( @rows ); + } + else { + @rows = reverse $meta->{sort_func}->( @rows ); + } + } + + } + else { + # If the table isn't pivoted, just show all columns that are supposed to + # be shown; but eliminate aggonly columns if the table isn't aggregated. + my $aggregated = $meta->{aggregate}; + $fmt_cols = [ grep { $aggregated || !$meta->{cols}->{$_}->{aggonly} } @{$meta->{visible}} ]; + $fmt_meta = { map { $_ => $meta->{cols}->{$_} } @$fmt_cols }; + + # If the table is aggregated, re-order the group_by columns to the left of + # the display. + if ( $aggregated ) { + my %is_group = map { $_ => 1 } @{$meta->{group_by}}; + $fmt_cols = [ @{$meta->{group_by}}, grep { !$is_group{$_} } @$fmt_cols ]; + } + } + + foreach my $listener ( @{$event_listener_for{set_to_tbl_pre_create}} ) { + $listener->set_to_tbl_pre_create(\@rows, $tbl); + } + + @rows = create_table( $fmt_cols, $fmt_meta, \@rows); + if ( !$meta->{hide_caption} && !$opts{n} && $config{display_table_captions}->{val} ) { + @rows = create_caption($meta->{capt}, @rows) + } + + foreach my $listener ( @{$event_listener_for{set_to_tbl_post_create}} ) { + $listener->set_to_tbl_post_create(\@rows, $tbl); + } + + return @rows; +} + +# meta_to_hdr {{{3 +sub meta_to_hdr { + my $tbl = shift; + my $meta = $tbl_meta{$tbl}; + my %labels = map { $_ => $meta->{cols}->{$_}->{hdr} } @{$meta->{visible}}; + return \%labels; +} + +# commify {{{3 +# From perlfaq5: add commas. +sub commify { + my ( $num ) = @_; + $num = 0 unless defined $num; + $num =~ s/(^[-+]?\d+?(?=(?>(?:\d{3})+)(?!\d))|\G\d{3}(?=\d))/$1,/g; + return $num; +} + +# set_precision {{{3 +# Trim to desired precision. +sub set_precision { + my ( $num, $precision ) = @_; + $precision = $config{num_digits}->{val} if !defined $precision; + sprintf("%.${precision}f", $num); +} + +# percent {{{3 +# Convert to percent +sub percent { + my ( $num ) = @_; + $num = 0 unless defined $num; + my $digits = $config{num_digits}->{val}; + return sprintf("%.${digits}f", $num * 100) + . ($config{show_percent}->{val} ? '%' : ''); +} + +# shorten {{{3 +sub shorten { + my ( $num, $opts ) = @_; + + return $num if !defined($num) || $opts{n} || $num !~ m/$num_regex/; + + $opts ||= {}; + my $pad = defined $opts->{pad} ? $opts->{pad} : ''; + my $num_digits = defined $opts->{num_digits} + ? $opts->{num_digits} + : $config{num_digits}->{val}; + my $force = defined $opts->{force}; + + my $n = 0; + while ( $num >= 1_024 ) { + $num /= 1_024; + ++$n; + } + return sprintf( + $num =~ m/\./ || $n || $force + ? "%.${num_digits}f%s" + : '%d', + $num, ($pad,'k','M','G', 'T')[$n]); + +} + +# Utility functions {{{2 +# unique {{{3 +sub unique { + my %seen; + return grep { !$seen{$_}++ } @_; +} + +# make_color_func {{{3 +sub make_color_func { + my ( $tbl ) = @_; + my @criteria; + foreach my $spec ( @{$tbl->{colors}} ) { + next unless exists $comp_ops{$spec->{op}}; + my $val = $spec->{op} =~ m/^(?:eq|ne|le|ge|lt|gt)$/ ? "'$spec->{arg}'" + : $spec->{op} =~ m/^(?:=~|!~)$/ ? "m/" . quotemeta($spec->{arg}) . "/" + : $spec->{arg}; + push @criteria, + "( defined \$set->{$spec->{col}} && \$set->{$spec->{col}} $spec->{op} $val ) { return '$spec->{color}'; }"; + } + return undef unless @criteria; + my $sub = eval 'sub { my ( $set ) = @_; if ' . join(" elsif ", @criteria) . '}'; + die if $EVAL_ERROR; + return $sub; +} + +# make_sort_func {{{3 +# Gets a list of sort columns from the table, like "+cxn -time" and returns a +# subroutine that will sort that way. +sub make_sort_func { + my ( $tbl ) = @_; + my @criteria; + + # Pivoted tables can be sorted by 'name' and set_x columns; others must be + # sorted by existing columns. TODO: this will crash if you toggle between + # pivoted and nonpivoted. I have several other 'crash' notes about this if + # this ever becomes possible. + + if ( $tbl->{pivot} ) { + # Sort type is not really possible on pivoted columns, because a 'column' + # contains data from an entire non-pivoted row, so there could be a mix of + # numeric and non-numeric data. Thus everything has to be 'cmp' type. + foreach my $col ( split(/\s+/, $tbl->{sort_cols} ) ) { + next unless $col; + my ( $dir, $name ) = $col =~ m/([+-])?(\w+)$/; + next unless $name && $name =~ m/^(?:name|set_\d+)$/; + $dir ||= '+'; + my $op = 'cmp'; + my $df = "''"; + push @criteria, + $dir eq '+' + ? "(\$a->{$name} || $df) $op (\$b->{$name} || $df)" + : "(\$b->{$name} || $df) $op (\$a->{$name} || $df)"; + } + } + else { + foreach my $col ( split(/\s+/, $tbl->{sort_cols} ) ) { + next unless $col; + my ( $dir, $name ) = $col =~ m/([+-])?(\w+)$/; + next unless $name && $tbl->{cols}->{$name}; + $dir ||= '+'; + my $op = $tbl->{cols}->{$name}->{num} ? "<=>" : "cmp"; + my $df = $tbl->{cols}->{$name}->{num} ? "0" : "''"; + push @criteria, + $dir eq '+' + ? "(\$a->{$name} || $df) $op (\$b->{$name} || $df)" + : "(\$b->{$name} || $df) $op (\$a->{$name} || $df)"; + } + } + return sub { return @_ } unless @criteria; + my $sub = eval 'sub { sort {' . join("||", @criteria) . '} @_; }'; + die if $EVAL_ERROR; + return $sub; +} + +# trunc {{{3 +# Shortens text to specified length. +sub trunc { + my ( $text, $len ) = @_; + if ( length($text) <= $len ) { + return $text; + } + return substr($text, 0, $len); +} + +# donut {{{3 +# Takes out the middle of text to shorten it. +sub donut { + my ( $text, $len ) = @_; + return $text if length($text) <= $len; + my $max = length($text) - $len; + my $min = $max - 1; + + # Try to remove a single "word" from somewhere in the center + if ( $text =~ s/_[^_]{$min,$max}_/_/ ) { + return $text; + } + + # Prefer removing the end of a "word" + if ( $text =~ s/([^_]+)[^_]{$max}_/$1_/ ) { + return $text; + } + + $text = substr($text, 0, int($len/2)) + . "_" + . substr($text, int($len/2) + $max + 1); + return $text; +} + +# crunch {{{3 +# Removes vowels and compacts repeated letters to shorten text. +sub crunch { + my ( $text, $len ) = @_; + return $text if $len && length($text) <= $len; + $text =~ s/^IB_\w\w_//; + $text =~ s/(?<![_ ])[aeiou]//g; + $text =~ s/(.)\1+/$1/g; + return $text; +} + +# collapse_ws {{{3 +# Collapses all whitespace to a single space. +sub collapse_ws { + my ( $text ) = @_; + return '' unless defined $text; + $text =~ s/\s+/ /g; + return $text; +} + +# Strips out non-printable characters within fields, which freak terminals out. +sub no_ctrl_char { + my ( $text ) = @_; + return '' unless defined $text; + my $charset = $config{charset}->{val}; + if ( $charset && $charset eq 'unicode' ) { + $text =~ s/ + ("(?:(?!(?<!\\)").)*" # Double-quoted string + |'(?:(?!(?<!\\)').)*') # Or single-quoted string + /$1 =~ m#\p{IsC}# ? "[BINARY]" : $1/egx; + } + elsif ( $charset && $charset eq 'none' ) { + $text =~ s/ + ("(?:(?!(?<!\\)").)*" + |'(?:(?!(?<!\\)').)*') + /[TEXT]/gx; + } + else { # The default is 'ascii' + $text =~ s/ + ("(?:(?!(?<!\\)").)*" + |'(?:(?!(?<!\\)').)*') + /$1 =~ m#[^\040-\176]# ? "[BINARY]" : $1/egx; + } + return $text; +} + +# word_wrap {{{3 +# Wraps text at word boundaries so it fits the screen. +sub word_wrap { + my ( $text, $width) = @_; + $width ||= $this_term_size[0]; + $text =~ s/(.{0,$width})(?:\s+|$)/$1\n/g; + $text =~ s/ +$//mg; + return $text; +} + +# draw_screen {{{3 +# Prints lines to the screen. The first argument is an arrayref. Each +# element of the array is either a string or an arrayref. If it's a string it +# just gets printed. If it's an arrayref, the first element is the string to +# print, and the second is args to colored(). +sub draw_screen { + my ( $display_lines, $prefs ) = @_; + if ( !$opts{n} && $config{show_statusbar}->{val} ) { + unshift @$display_lines, create_statusbar(); + } + + foreach my $listener ( @{$event_listener_for{draw_screen}} ) { + $listener->draw_screen($display_lines); + } + + $clear_screen_sub->() + if $prefs->{clear} || !$modes{$config{mode}->{val}}->{no_clear_screen}; + if ( $opts{n} || $prefs->{raw} ) { + my $num_lines = 0; + print join("\n", + map { + $num_lines++; + ref $_ + ? colored($_->[0], $_->[1]) + : $_; + } + grep { !$opts{n} || $_ } # Suppress empty lines + @$display_lines); + if ( $opts{n} && $num_lines ) { + print "\n"; + } + } + else { + my $max_lines = $prefs->{show_all} + ? scalar(@$display_lines)- 1 + : min(scalar(@$display_lines), $this_term_size[1]); + print join("\n", + map { + ref $_ + ? colored(substr($_->[0], 0, $this_term_size[0]), $_->[1]) + : substr($_, 0, $this_term_size[0]); + } @$display_lines[0..$max_lines - 1]); + } +} + +# secs_to_time {{{3 +sub secs_to_time { + my ( $secs, $fmt ) = @_; + $secs ||= 0; + return '00:00' unless $secs; + + # Decide what format to use, if not given + $fmt ||= $secs >= 86_400 ? 'd' + : $secs >= 3_600 ? 'h' + : 'm'; + + return + $fmt eq 'd' ? sprintf( + "%d+%02d:%02d:%02d", + int($secs / 86_400), + int(($secs % 86_400) / 3_600), + int(($secs % 3_600) / 60), + $secs % 60) + : $fmt eq 'h' ? sprintf( + "%02d:%02d:%02d", + int(($secs % 86_400) / 3_600), + int(($secs % 3_600) / 60), + $secs % 60) + : sprintf( + "%02d:%02d", + int(($secs % 3_600) / 60), + $secs % 60); +} + +# dulint_to_int {{{3 +# Takes a number that InnoDB formats as two ulint integers, like transaction IDs +# and such, and turns it into a single integer +sub dulint_to_int { + my $num = shift; + return 0 unless $num; + my ( $high, $low ) = $num =~ m/^(\d+) (\d+)$/; + return $low unless $high; + return $low + ( $high * $MAX_ULONG ); +} + +# create_statusbar {{{3 +sub create_statusbar { + my $mode = $config{mode}->{val}; + my @cxns = sort { $a cmp $b } get_connections(); + + my $modeline = ( $config{readonly}->{val} ? '[RO] ' : '' ) + . $modes{$mode}->{hdr} . " (? for help)"; + my $mode_width = length($modeline); + my $remaining_width = $this_term_size[0] - $mode_width - 1; + my $result; + + # The thingie in top-right that says what we're monitoring. + my $cxn = ''; + + if ( 1 == @cxns && $dbhs{$cxns[0]} && $dbhs{$cxns[0]}->{dbh} ) { + $cxn = $dbhs{$cxns[0]}->{dbh}->{mysql_serverinfo} || ''; + } + else { + if ( $modes{$mode}->{server_group} ) { + $cxn = "Servers: " . $modes{$mode}->{server_group}; + my $err_count = grep { $dbhs{$_} && $dbhs{$_}->{err_count} } @cxns; + if ( $err_count ) { + $cxn .= "(" . ( scalar(@cxns) - $err_count ) . "/" . scalar(@cxns) . ")"; + } + } + else { + $cxn = join(' ', map { ($dbhs{$_}->{err_count} ? '!' : '') . $_ } + grep { $dbhs{$_} } @cxns); + } + } + + if ( 1 == @cxns ) { + get_driver_status(@cxns); + my $vars = $vars{$cxns[0]}->{$clock}; + my $inc = inc(0, $cxns[0]); + + # Format server uptime human-readably, calculate QPS... + my $uptime = secs_to_time( $vars->{Uptime_hires} ); + my $qps = ($inc->{Questions}||0) / ($inc->{Uptime_hires}||1); + my $ibinfo = ''; + + if ( exists $vars->{IB_last_secs} ) { + $ibinfo .= "InnoDB $vars->{IB_last_secs}s "; + if ( $vars->{IB_got_all} ) { + if ( ($mode eq 'T' || $mode eq 'W') + && $vars->{IB_tx_is_truncated} ) { + $ibinfo .= ':^|'; + } + else { + $ibinfo .= ':-)'; + } + } + else { + $ibinfo .= ':-('; + } + } + $result = sprintf( + "%-${mode_width}s %${remaining_width}s", + $modeline, + join(', ', grep { $_ } ( + $cxns[0], + $uptime, + $ibinfo, + shorten($qps) . " QPS", + ($vars->{Threads} || 0) . " thd", + $cxn))); + } + else { + $result = sprintf( + "%-${mode_width}s %${remaining_width}s", + $modeline, + $cxn); + } + + return $config{color}->{val} ? [ $result, 'bold reverse' ] : $result; +} + +# Database connections {{{3 +sub add_new_dsn { + my ( $name ) = @_; + + if ( defined $name ) { + $name =~ s/[\s:;]//g; + } + + if ( !$name ) { + print word_wrap("Choose a name for the connection. It cannot contain " + . "whitespace, colons or semicolons."), "\n\n"; + do { + $name = prompt("Enter a name"); + $name =~ s/[\s:;]//g; + } until ( $name ); + } + + my $dsn; + do { + $clear_screen_sub->(); + print "Typical DSN strings look like\n DBI:mysql:;host=hostname;port=port\n" + . "The db and port are optional and can usually be omitted.\n" + . "If you specify 'mysql_read_default_group=mysql' many options can be read\n" + . "from your mysql options files (~/.my.cnf, /etc/my.cnf).\n\n"; + $dsn = prompt("Enter a DSN string", undef, "DBI:mysql:;mysql_read_default_group=mysql;host=$name"); + } until ( $dsn ); + + $clear_screen_sub->(); + my $dl_table = prompt("Optional: enter a table (must not exist) to use when resetting InnoDB deadlock information", + undef, 'test.innotop_dl'); + + $connections{$name} = { + dsn => $dsn, + dl_table => $dl_table, + }; +} + +sub add_new_server_group { + my ( $name ) = @_; + + if ( defined $name ) { + $name =~ s/[\s:;]//g; + } + + if ( !$name ) { + print word_wrap("Choose a name for the group. It cannot contain " + . "whitespace, colons or semicolons."), "\n\n"; + do { + $name = prompt("Enter a name"); + $name =~ s/[\s:;]//g; + } until ( $name ); + } + + my @cxns; + do { + $clear_screen_sub->(); + @cxns = select_cxn("Choose servers for $name", keys %connections); + } until ( @cxns ); + + $server_groups{$name} = \@cxns; + return $name; +} + +sub get_var_set { + my ( $name ) = @_; + while ( !$name || !exists($var_sets{$config{$name}->{val}}) ) { + $name = choose_var_set($name); + } + return $var_sets{$config{$name}->{val}}->{text}; +} + +sub add_new_var_set { + my ( $name ) = @_; + + if ( defined $name ) { + $name =~ s/\W//g; + } + + if ( !$name ) { + do { + $name = prompt("Enter a name"); + $name =~ s/\W//g; + } until ( $name ); + } + + my $variables; + do { + $clear_screen_sub->(); + $variables = prompt("Enter variables for $name", undef ); + } until ( $variables ); + + $var_sets{$name} = { text => $variables, user => 1 }; +} + +sub next_server { + my $mode = $config{mode}->{val}; + my @cxns = sort keys %connections; + my ($cur) = get_connections($mode); + $cur ||= $cxns[0]; + my $pos = grep { $_ lt $cur } @cxns; + my $newpos = ($pos + 1) % @cxns; + $modes{$mode}->{server_group} = ''; + $modes{$mode}->{connections} = [ $cxns[$newpos] ]; + $clear_screen_sub->(); +} + +sub next_server_group { + my $mode = shift || $config{mode}->{val}; + my @grps = sort keys %server_groups; + my $curr = $modes{$mode}->{server_group}; + + return unless @grps; + + if ( $curr ) { + # Find the current group's position. + my $pos = 0; + while ( $curr ne $grps[$pos] ) { + $pos++; + } + $modes{$mode}->{server_group} = $grps[ ($pos + 1) % @grps ]; + } + else { + $modes{$mode}->{server_group} = $grps[0]; + } +} + +# Get a list of connection names used in this mode. +sub get_connections { + if ( $file ) { + return qw(file); + } + my $mode = shift || $config{mode}->{val}; + my @connections = $modes{$mode}->{server_group} + ? @{$server_groups{$modes{$mode}->{server_group}}} + : @{$modes{$mode}->{connections}}; + if ( $modes{$mode}->{one_connection} ) { + @connections = @connections ? $connections[0] : (); + } + return unique(@connections); +} + +# Get a list of tables used in this mode. If innotop is running non-interactively, just use the first. +sub get_visible_tables { + my $mode = shift || $config{mode}->{val}; + my @tbls = @{$modes{$mode}->{visible_tables}}; + if ( $opts{n} ) { + return $tbls[0]; + } + else { + return @tbls; + } +} + +# Choose from among available connections or server groups. +# If the mode has a server set in use, prefers that instead. +sub choose_connections { + $clear_screen_sub->(); + my $mode = $config{mode}->{val}; + my $meta = { map { $_ => $connections{$_}->{dsn} } keys %connections }; + foreach my $group ( keys %server_groups ) { + $meta->{"#$group"} = join(' ', @{$server_groups{$group}}); + } + + my $choices = prompt_list("Choose connections or a group for $mode mode", + undef, sub { return keys %$meta }, $meta); + + my @choices = unique(grep { $_ } split(/\s+/, $choices)); + if ( @choices ) { + if ( $choices[0] =~ s/^#// && exists $server_groups{$choices[0]} ) { + $modes{$mode}->{server_group} = $choices[0]; + } + else { + $modes{$mode}->{connections} = [ grep { exists $connections{$_} } @choices ]; + } + } +} + +# Accepts a DB connection name and the name of a prepared query (e.g. status, kill). +# Also a list of params for the prepared query. This allows not storing prepared +# statements globally. Returns a $sth that's been executed. +# ERROR-HANDLING SEMANTICS: if the statement throws an error, propagate, but if the +# connection has gone away or can't connect, DO NOT. Just return undef. +sub do_stmt { + my ( $cxn, $stmt_name, @args ) = @_; + + return undef if $file; + + # Test if the cxn should not even be tried + return undef if $dbhs{$cxn} + && $dbhs{$cxn}->{err_count} + && ( !$dbhs{$cxn}->{dbh} || !$dbhs{$cxn}->{dbh}->{Active} || $dbhs{$cxn}->{mode} eq $config{mode}->{val} ) + && $dbhs{$cxn}->{wake_up} > $clock; + + my $sth; + my $retries = 1; + my $success = 0; + TRY: + while ( $retries-- >= 0 && !$success ) { + + eval { + my $dbh = connect_to_db($cxn); + + # If the prepared query doesn't exist, make it. + if ( !exists $dbhs{$cxn}->{stmts}->{$stmt_name} ) { + $dbhs{$cxn}->{stmts}->{$stmt_name} = $stmt_maker_for{$stmt_name}->($dbh); + } + + $sth = $dbhs{$cxn}->{stmts}->{$stmt_name}; + if ( $sth ) { + $sth->execute(@args); + } + $success = 1; + }; + if ( $EVAL_ERROR ) { + if ( $EVAL_ERROR =~ m/$nonfatal_errs/ ) { + handle_cxn_error($cxn, $EVAL_ERROR); + } + else { + die "$cxn $stmt_name: $EVAL_ERROR"; + } + if ( $retries < 0 ) { + $sth = undef; + } + } + } + + if ( $sth && $sth->{NUM_OF_FIELDS} ) { + sleep($stmt_sleep_time_for{$stmt_name}) if $stmt_sleep_time_for{$stmt_name}; + return $sth; + } +} + +# Keeps track of error count, sleep times till retries, etc etc. +# When there's an error we retry the connection every so often, increasing in +# Fibonacci series to prevent too much banging on the server. +sub handle_cxn_error { + my ( $cxn, $err ) = @_; + my $meta = $dbhs{$cxn}; + $meta->{err_count}++; + + # This is used so errors that have to do with permissions needed by the current + # mode will get displayed as long as we're in this mode, but get ignored if the + # mode changes. + $meta->{mode} = $config{mode}->{val}; + + # Strip garbage from the error text if possible. + $err =~ s/\s+/ /g; + if ( $err =~ m/failed: (.*?) at \S*innotop line/ ) { + $err = $1; + } + + $meta->{last_err} = $err; + my $sleep_time = $meta->{this_sleep} + $meta->{prev_sleep}; + $meta->{prev_sleep} = $meta->{this_sleep}; + $meta->{this_sleep} = $sleep_time; + $meta->{wake_up} = $clock + $sleep_time; + if ( $config{show_cxn_errors}->{val} ) { + print STDERR "Error at tick $clock $cxn $err" if $config{debug}->{val}; + } +} + +# Accepts a DB connection name and a (string) query. Returns a $sth that's been +# executed. +sub do_query { + my ( $cxn, $query ) = @_; + + return undef if $file; + + # Test if the cxn should not even be tried + return undef if $dbhs{$cxn} + && $dbhs{$cxn}->{err_count} + && ( !$dbhs{$cxn}->{dbh} || !$dbhs{$cxn}->{dbh}->{Active} || $dbhs{$cxn}->{mode} eq $config{mode}->{val} ) + && $dbhs{$cxn}->{wake_up} > $clock; + + my $sth; + my $retries = 1; + my $success = 0; + TRY: + while ( $retries-- >= 0 && !$success ) { + + eval { + my $dbh = connect_to_db($cxn); + + $sth = $dbh->prepare($query); + $sth->execute(); + $success = 1; + }; + if ( $EVAL_ERROR ) { + if ( $EVAL_ERROR =~ m/$nonfatal_errs/ ) { + handle_cxn_error($cxn, $EVAL_ERROR); + } + else { + die $EVAL_ERROR; + } + if ( $retries < 0 ) { + $sth = undef; + } + } + } + + return $sth; +} + +sub get_uptime { + my ( $cxn ) = @_; + $dbhs{$cxn}->{start_time} ||= time(); + # Avoid dividing by zero + return (time() - $dbhs{$cxn}->{start_time}) || .001; +} + +sub connect_to_db { + my ( $cxn ) = @_; + + $dbhs{$cxn} ||= { + stmts => {}, # bucket for prepared statements. + prev_sleep => 0, + this_sleep => 1, + wake_up => 0, + start_time => 0, + dbh => undef, + }; + my $href = $dbhs{$cxn}; + + if ( !$href->{dbh} || ref($href->{dbh}) !~ m/DBI/ || !$href->{dbh}->ping ) { + my $dbh = get_new_db_connection($cxn); + @{$href}{qw(dbh err_count wake_up this_sleep start_time prev_sleep)} + = ($dbh, 0, 0, 1, 0, 0); + + # Derive and store the server's start time in hi-res + my $uptime = $dbh->selectrow_hashref("show status like 'Uptime'")->{value}; + $href->{start_time} = time() - $uptime; + + # Set timeouts so an unused connection stays alive. + # For example, a connection might be used in Q mode but idle in T mode. + if ( version_ge($dbh, '4.0.3')) { + my $timeout = $config{cxn_timeout}->{val}; + $dbh->do("set session wait_timeout=$timeout, interactive_timeout=$timeout"); + } + } + return $href->{dbh}; +} + +# Compares versions like 5.0.27 and 4.1.15-standard-log +sub version_ge { + my ( $dbh, $target ) = @_; + my $version = sprintf('%03d%03d%03d', $dbh->{mysql_serverinfo} =~ m/(\d+)/g); + return $version ge sprintf('%03d%03d%03d', $target =~ m/(\d+)/g); +} + +# Extracts status values that can be gleaned from the DBD driver without doing a whole query. +sub get_driver_status { + my @cxns = @_; + if ( !$info_gotten{driver_status}++ ) { + foreach my $cxn ( @cxns ) { + next unless $dbhs{$cxn} && $dbhs{$cxn}->{dbh} && $dbhs{$cxn}->{dbh}->{Active}; + $vars{$cxn}->{$clock} ||= {}; + my $vars = $vars{$cxn}->{$clock}; + my %res = map { $_ =~ s/ +/_/g; $_ } $dbhs{$cxn}->{dbh}->{mysql_stat} =~ m/(\w[^:]+): ([\d\.]+)/g; + map { $vars->{$_} ||= $res{$_} } keys %res; + $vars->{Uptime_hires} ||= get_uptime($cxn); + $vars->{cxn} = $cxn; + } + } +} + +sub get_new_db_connection { + my ( $connection, $destroy ) = @_; + if ( $file ) { + die "You can't connect to a MySQL server while monitoring a file. This is probably a bug."; + } + + my $dsn = $connections{$connection} + or die "No connection named '$connection' is defined in your configuration"; + + if ( !defined $dsn->{have_user} ) { + my $answer = prompt("Do you want to specify a username for $connection?", undef, 'n'); + $dsn->{have_user} = $answer && $answer =~ m/1|y/i; + } + + if ( !defined $dsn->{have_pass} ) { + my $answer = prompt("Do you want to specify a password for $connection?", undef, 'n'); + $dsn->{have_pass} = $answer && $answer =~ m/1|y/i; + } + + if ( !$dsn->{user} && $dsn->{have_user} ) { + my $user = $ENV{USERNAME} || $ENV{USER} || getlogin() || getpwuid($REAL_USER_ID) || undef; + $dsn->{user} = prompt("Enter username for $connection", undef, $user); + } + + if ( !defined $dsn->{user} ) { + $dsn->{user} = ''; + } + + if ( !$dsn->{pass} && !$dsn->{savepass} && $dsn->{have_pass} ) { + $dsn->{pass} = prompt_noecho("Enter password for '$dsn->{user}' on $connection"); + print "\n"; + if ( !defined($dsn->{savepass}) ) { + my $answer = prompt("Save password in plain text in the config file?", undef, 'y'); + $dsn->{savepass} = $answer && $answer =~ m/1|y/i; + } + } + + my $dbh = DBI->connect( + $dsn->{dsn}, $dsn->{user}, $dsn->{pass}, + { RaiseError => 1, PrintError => 0, AutoCommit => 1 }); + $dbh->{InactiveDestroy} = 1 unless $destroy; # Can't be set in $db_options + $dbh->{FetchHashKeyName} = 'NAME_lc'; # Lowercases all column names for fetchrow_hashref + return $dbh; +} + +sub get_cxn_errors { + my @cxns = @_; + return () unless $config{show_cxn_errors_in_tbl}->{val}; + return + map { [ $_ . ': ' . $dbhs{$_}->{last_err}, 'red' ] } + grep { $dbhs{$_} && $dbhs{$_}->{err_count} && $dbhs{$_}->{mode} eq $config{mode}->{val} } + @cxns; +} + +# Setup and tear-down functions {{{2 + +# Takes a string and turns it into a hashref you can apply to %tbl_meta tables. The string +# can be in the form 'foo, bar, foo/bar, foo as bar' much like a SQL SELECT statement. +sub compile_select_stmt { + my ($str) = @_; + my @exps = $str =~ m/\s*([^,]+(?i:\s+as\s+[^,\s]+)?)\s*(?=,|$)/g; + my %cols; + my @visible; + foreach my $exp ( @exps ) { + my ( $text, $colname ); + if ( $exp =~ m/as\s+(\w+)\s*/ ) { + $colname = $1; + $exp =~ s/as\s+(\w+)\s*//; + $text = $exp; + } + else { + $text = $colname = $exp; + } + my ($func, $err) = compile_expr($text); + $cols{$colname} = { + src => $text, + hdr => $colname, + num => 0, + func => $func, + }; + push @visible, $colname; + } + return (\%cols, \@visible); +} + +# compile_filter {{{3 +sub compile_filter { + my ( $text ) = @_; + my ( $sub, $err ); + eval "\$sub = sub { my \$set = shift; $text }"; + if ( $EVAL_ERROR ) { + $EVAL_ERROR =~ s/at \(eval.*$//; + $sub = sub { return $EVAL_ERROR }; + $err = $EVAL_ERROR; + } + return ( $sub, $err ); +} + +# compile_expr {{{3 +sub compile_expr { + my ( $expr ) = @_; + # Leave built-in functions alone so they get called as Perl functions, unless + # they are the only word in $expr, in which case treat them as hash keys. + if ( $expr =~ m/\W/ ) { + $expr =~ s/(?<!\{|\$)\b([A-Za-z]\w{2,})\b/is_func($1) ? $1 : "\$set->{$1}"/eg; + } + else { + $expr = "\$set->{$expr}"; + } + my ( $sub, $err ); + my $quoted = quotemeta($expr); + eval qq{ + \$sub = sub { + my (\$set, \$cur, \$pre) = \@_; + my \$val = eval { $expr }; + if ( \$EVAL_ERROR && \$config{debug}->{val} ) { + \$EVAL_ERROR =~ s/ at \\(eval.*//s; + die "\$EVAL_ERROR in expression $quoted"; + } + return \$val; + } + }; + if ( $EVAL_ERROR ) { + if ( $config{debug}->{val} ) { + die $EVAL_ERROR; + } + $EVAL_ERROR =~ s/ at \(eval.*$//; + $sub = sub { return $EVAL_ERROR }; + $err = $EVAL_ERROR; + } + return ( $sub, $err ); +} + +# finish {{{3 +# This is a subroutine because it's called from a key to quit the program. +sub finish { + save_config(); + ReadMode('normal') unless $opts{n}; + print "\n"; + exit(0); +} + +# core_dump {{{3 +sub core_dump { + my $msg = shift; + if ($config{debugfile}->{val} && $config{debug}->{val}) { + eval { + open my $file, '>>', $config{debugfile}->{val}; + if ( %vars ) { + print $file "Current variables:\n" . Dumper(\%vars); + } + close $file; + }; + } + print $msg; +} + +# load_config {{{3 +sub load_config { + + my $filename = $opts{c} || "$homepath/.innotop/innotop.ini"; + my $dirname = dirname($filename); + if ( -f $dirname && !$opts{c} ) { + # innotop got upgraded and this is the old config file. + my $answer = pause("Innotop's default config location has moved to $filename. Move old config file $dirname there now? y/n"); + if ( lc $answer eq 'y' ) { + rename($dirname, "$homepath/innotop.ini") + or die "Can't rename '$dirname': $OS_ERROR"; + mkdir($dirname) or die "Can't create directory '$dirname': $OS_ERROR"; + mkdir("$dirname/plugins") or die "Can't create directory '$dirname/plugins': $OS_ERROR"; + rename("$homepath/innotop.ini", $filename) + or die "Can't rename '$homepath/innotop.ini' to '$filename': $OS_ERROR"; + } + else { + print "\nInnotop will now exit so you can fix the config file.\n"; + exit(0); + } + } + + if ( ! -d $dirname ) { + mkdir $dirname + or die "Can't create directory '$dirname': $OS_ERROR"; + } + if ( ! -d "$dirname/plugins" ) { + mkdir "$dirname/plugins" + or die "Can't create directory '$dirname/plugins': $OS_ERROR"; + } + + if ( -f $filename ) { + open my $file, "<", $filename or die("Can't open '$filename': $OS_ERROR"); + + # Check config file version. Just ignore if either innotop or the file has + # garbage in the version number. + if ( defined(my $line = <$file>) && $VERSION =~ m/\d/ ) { + chomp $line; + if ( my ($maj, $min, $rev) = $line =~ m/^version=(\d+)\.(\d+)(?:\.(\d+))?$/ ) { + $rev ||= 0; + my $cfg_ver = sprintf('%03d-%03d-%03d', $maj, $min, $rev); + ( $maj, $min, $rev ) = $VERSION =~ m/^(\d+)\.(\d+)(?:\.(\d+))?$/; + $rev ||= 0; + my $innotop_ver = sprintf('%03d-%03d-%03d', $maj, $min, $rev); + + if ( $cfg_ver gt $innotop_ver ) { + pause("The config file is for a newer version of innotop and may not be read correctly."); + } + else { + my @ver_history = @config_versions; + while ( my ($start, $end) = splice(@ver_history, 0, 2) ) { + # If the config file is between the endpoints and innotop is greater than + # the endpoint, innotop has a newer config file format than the file. + if ( $cfg_ver ge $start && $cfg_ver lt $end && $innotop_ver ge $end ) { + my $msg = "innotop's config file format has changed. Overwrite $filename? y or n"; + if ( pause($msg) eq 'n' ) { + $config{readonly}->{val} = 1; + print "\ninnotop will not save any configuration changes you make."; + pause(); + print "\n"; + } + close $file; + return; + } + } + } + } + } + + while ( my $line = <$file> ) { + chomp $line; + next unless $line =~ m/^\[([a-z_]+)\]$/; + if ( exists $config_file_sections{$1} ) { + $config_file_sections{$1}->{reader}->($file); + } + else { + warn "Unknown config file section '$1'"; + } + } + close $file or die("Can't close $filename: $OS_ERROR"); + } + +} + +# Do some post-processing on %tbl_meta: compile src properties into func etc. +sub post_process_tbl_meta { + foreach my $table ( values %tbl_meta ) { + foreach my $col_name ( keys %{$table->{cols}} ) { + my $col_def = $table->{cols}->{$col_name}; + my ( $sub, $err ) = compile_expr($col_def->{src}); + $col_def->{func} = $sub; + } + } +} + +# load_config_plugins {{{3 +sub load_config_plugins { + my ( $file ) = @_; + + # First, find a list of all plugins that exist on disk, and get information about them. + my $dir = $config{plugin_dir}->{val}; + foreach my $p_file ( <$dir/*.pm> ) { + my ($package, $desc); + eval { + open my $p_in, "<", $p_file or die $OS_ERROR; + while ( my $line = <$p_in> ) { + chomp $line; + if ( $line =~ m/^package\s+(.*?);/ ) { + $package = $1; + } + elsif ( $line =~ m/^# description: (.*)/ ) { + $desc = $1; + } + last if $package && $desc; + } + close $p_in; + }; + if ( $package ) { + $plugins{$package} = { + file => $p_file, + desc => $desc, + class => $package, + active => 0, + }; + if ( $config{debug}->{val} && $EVAL_ERROR ) { + die $EVAL_ERROR; + } + } + } + + # Now read which ones the user has activated. Each line simply represents an active plugin. + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + next unless $line && $plugins{$line}; + + my $obj; + eval { + require $plugins{$line}->{file}; + $obj = $line->new(%pluggable_vars); + foreach my $event ( $obj->register_for_events() ) { + my $queue = $event_listener_for{$event}; + if ( $queue ) { + push @$queue, $obj; + } + } + }; + if ( $config{debug}->{val} && $EVAL_ERROR ) { + die $EVAL_ERROR; + } + if ( $obj ) { + $plugins{$line}->{active} = 1; + $plugins{$line}->{object} = $obj; + } + } +} + +# save_config_plugins {{{3 +sub save_config_plugins { + my $file = shift; + foreach my $class ( sort keys %plugins ) { + next unless $plugins{$class}->{active}; + print $file "$class\n"; + } +} + +# load_config_active_server_groups {{{3 +sub load_config_active_server_groups { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $mode, $group ) = $line =~ m/^(.*?)=(.*)$/; + next unless $mode && $group + && exists $modes{$mode} && exists $server_groups{$group}; + $modes{$mode}->{server_group} = $group; + } +} + +# save_config_active_server_groups {{{3 +sub save_config_active_server_groups { + my $file = shift; + foreach my $mode ( sort keys %modes ) { + print $file "$mode=$modes{$mode}->{server_group}\n"; + } +} + +# load_config_server_groups {{{3 +sub load_config_server_groups { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $name, $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $name && $rest; + my @vars = unique(grep { $_ && exists $connections{$_} } split(/\s+/, $rest)); + next unless @vars; + $server_groups{$name} = \@vars; + } +} + +# save_config_server_groups {{{3 +sub save_config_server_groups { + my $file = shift; + foreach my $set ( sort keys %server_groups ) { + print $file "$set=", join(' ', @{$server_groups{$set}}), "\n"; + } +} + +# load_config_varsets {{{3 +sub load_config_varsets { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $name, $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $name && $rest; + $var_sets{$name} = { + text => $rest, + user => 1, + }; + } +} + +# save_config_varsets {{{3 +sub save_config_varsets { + my $file = shift; + foreach my $varset ( sort keys %var_sets ) { + next unless $var_sets{$varset}->{user}; + print $file "$varset=$var_sets{$varset}->{text}\n"; + } +} + +# load_config_group_by {{{3 +sub load_config_group_by { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $tbl , $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $tbl && exists $tbl_meta{$tbl}; + my @parts = unique(grep { exists($tbl_meta{$tbl}->{cols}->{$_}) } split(/\s+/, $rest)); + $tbl_meta{$tbl}->{group_by} = [ @parts ]; + $tbl_meta{$tbl}->{cust}->{group_by} = 1; + } +} + +# save_config_group_by {{{3 +sub save_config_group_by { + my $file = shift; + foreach my $tbl ( sort keys %tbl_meta ) { + next if $tbl_meta{$tbl}->{temp}; + next unless $tbl_meta{$tbl}->{cust}->{group_by}; + my $aref = $tbl_meta{$tbl}->{group_by}; + print $file "$tbl=", join(' ', @$aref), "\n"; + } +} + +# load_config_filters {{{3 +sub load_config_filters { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $key, $rest ) = $line =~ m/^(.+?)=(.*)$/; + next unless $key && $rest; + + my %parts = $rest =~ m/(\w+)='((?:(?!(?<!\\)').)*)'/g; # Properties are single-quoted + next unless $parts{text} && $parts{tbls}; + + foreach my $prop ( keys %parts ) { + # Un-escape escaping + $parts{$prop} =~ s/\\\\/\\/g; + $parts{$prop} =~ s/\\'/'/g; + } + + my ( $sub, $err ) = compile_filter($parts{text}); + my @tbls = unique(split(/\s+/, $parts{tbls})); + @tbls = grep { exists $tbl_meta{$_} } @tbls; + $filters{$key} = { + func => $sub, + text => $parts{text}, + user => 1, + name => $key, + note => 'User-defined filter', + tbls => \@tbls, + } + } +} + +# save_config_filters {{{3 +sub save_config_filters { + my $file = shift; + foreach my $key ( sort keys %filters ) { + next if !$filters{$key}->{user} || $filters{$key}->{quick}; + my $text = $filters{$key}->{text}; + $text =~ s/([\\'])/\\$1/g; + my $tbls = join(" ", @{$filters{$key}->{tbls}}); + print $file "$key=text='$text' tbls='$tbls'\n"; + } +} + +# load_config_visible_tables {{{3 +sub load_config_visible_tables { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $mode, $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $mode && exists $modes{$mode}; + $modes{$mode}->{visible_tables} = + [ unique(grep { $_ && exists $tbl_meta{$_} } split(/\s+/, $rest)) ]; + $modes{$mode}->{cust}->{visible_tables} = 1; + } +} + +# save_config_visible_tables {{{3 +sub save_config_visible_tables { + my $file = shift; + foreach my $mode ( sort keys %modes ) { + next unless $modes{$mode}->{cust}->{visible_tables}; + my $tables = $modes{$mode}->{visible_tables}; + print $file "$mode=", join(' ', @$tables), "\n"; + } +} + +# load_config_sort_cols {{{3 +sub load_config_sort_cols { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $key , $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $key && exists $tbl_meta{$key}; + $tbl_meta{$key}->{sort_cols} = $rest; + $tbl_meta{$key}->{cust}->{sort_cols} = 1; + $tbl_meta{$key}->{sort_func} = make_sort_func($tbl_meta{$key}); + } +} + +# save_config_sort_cols {{{3 +sub save_config_sort_cols { + my $file = shift; + foreach my $tbl ( sort keys %tbl_meta ) { + next unless $tbl_meta{$tbl}->{cust}->{sort_cols}; + my $col = $tbl_meta{$tbl}->{sort_cols}; + print $file "$tbl=$col\n"; + } +} + +# load_config_active_filters {{{3 +sub load_config_active_filters { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $tbl , $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $tbl && exists $tbl_meta{$tbl}; + my @parts = unique(grep { exists($filters{$_}) } split(/\s+/, $rest)); + @parts = grep { grep { $tbl eq $_ } @{$filters{$_}->{tbls}} } @parts; + $tbl_meta{$tbl}->{filters} = [ @parts ]; + $tbl_meta{$tbl}->{cust}->{filters} = 1; + } +} + +# save_config_active_filters {{{3 +sub save_config_active_filters { + my $file = shift; + foreach my $tbl ( sort keys %tbl_meta ) { + next if $tbl_meta{$tbl}->{temp}; + next unless $tbl_meta{$tbl}->{cust}->{filters}; + my $aref = $tbl_meta{$tbl}->{filters}; + print $file "$tbl=", join(' ', @$aref), "\n"; + } +} + +# load_config_active_columns {{{3 +sub load_config_active_columns { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $key , $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $key && exists $tbl_meta{$key}; + my @parts = grep { exists($tbl_meta{$key}->{cols}->{$_}) } unique split(/ /, $rest); + $tbl_meta{$key}->{visible} = [ @parts ]; + $tbl_meta{$key}->{cust}->{visible} = 1; + } +} + +# save_config_active_columns {{{3 +sub save_config_active_columns { + my $file = shift; + foreach my $tbl ( sort keys %tbl_meta ) { + next unless $tbl_meta{$tbl}->{cust}->{visible}; + my $aref = $tbl_meta{$tbl}->{visible}; + print $file "$tbl=", join(' ', @$aref), "\n"; + } +} + +# save_config_tbl_meta {{{3 +sub save_config_tbl_meta { + my $file = shift; + foreach my $tbl ( sort keys %tbl_meta ) { + foreach my $col ( keys %{$tbl_meta{$tbl}->{cols}} ) { + my $meta = $tbl_meta{$tbl}->{cols}->{$col}; + next unless $meta->{user}; + print $file "$col=", join( + " ", + map { + # Some properties (trans) are arrays, others scalars + my $val = ref($meta->{$_}) ? join(',', @{$meta->{$_}}) : $meta->{$_}; + $val =~ s/([\\'])/\\$1/g; # Escape backslashes and single quotes + "$_='$val'"; # Enclose in single quotes + } + grep { $_ ne 'func' } + keys %$meta + ), "\n"; + } + } +} + +# save_config_config {{{3 +sub save_config_config { + my $file = shift; + foreach my $key ( sort keys %config ) { + eval { + if ( $key ne 'password' || $config{savepass}->{val} ) { + print $file "# $config{$key}->{note}\n" + or die "Cannot print to file: $OS_ERROR"; + my $val = $config{$key}->{val}; + $val = '' unless defined($val); + if ( ref( $val ) eq 'ARRAY' ) { + print $file "$key=" + . join( " ", @$val ) . "\n" + or die "Cannot print to file: $OS_ERROR"; + } + elsif ( ref( $val ) eq 'HASH' ) { + print $file "$key=" + . join( " ", + map { "$_:$val->{$_}" } keys %$val + ) . "\n"; + } + else { + print $file "$key=$val\n"; + } + } + }; + if ( $EVAL_ERROR ) { print "$EVAL_ERROR in $key"; }; + } + +} + +# load_config_config {{{3 +sub load_config_config { + my ( $file ) = @_; + + # Look in the command-line parameters for things stored in the same slot. + my %cmdline = + map { $_->{c} => $opts{$_->{k}} } + grep { exists $_->{c} && exists $opts{$_->{k}} } + @opt_spec; + + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $name, $val ) = $line =~ m/^(.+?)=(.*)$/; + next unless defined $name && defined $val; + + # Values might already have been set at the command line. + $val = defined($cmdline{$name}) ? $cmdline{$name} : $val; + + # Validate the incoming values... + if ( $name && exists( $config{$name} ) ) { + if ( !$config{$name}->{pat} || $val =~ m/$config{$name}->{pat}/ ) { + $config{$name}->{val} = $val; + $config{$name}->{read} = 1; + } + } + } +} + +# load_config_tbl_meta {{{3 +sub load_config_tbl_meta { + my ( $file ) = @_; + + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + # Each tbl_meta section has all the properties defined in %col_props. + my ( $col , $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $col; + my %parts = $rest =~ m/(\w+)='((?:(?!(?<!\\)').)*)'/g; # Properties are single-quoted + + # Each section read from the config file has one extra property: which table it + # goes in. + my $tbl = $parts{tbl} or die "There's no table for tbl_meta $col"; + my $meta = $tbl_meta{$tbl} or die "There's no table in tbl_meta named $tbl"; + + # The section is user-defined by definition (if that makes sense). + $parts{user} = 1; + + # The column may already exist in the table, in which case this is just a + # customization. + $meta->{cols}->{$col} ||= {}; + + foreach my $prop ( keys %col_props ) { + if ( !defined($parts{$prop}) ) { + die "Undefined property $prop for column $col in table $tbl"; + } + + # Un-escape escaping + $parts{$prop} =~ s/\\\\/\\/g; + $parts{$prop} =~ s/\\'/'/g; + + if ( ref $col_props{$prop} ) { + if ( $prop eq 'trans' ) { + $meta->{cols}->{$col}->{trans} + = [ unique(grep { exists $trans_funcs{$_} } split(',', $parts{$prop})) ]; + } + else { + $meta->{cols}->{$col}->{$prop} = [ split(',', $parts{$prop}) ]; + } + } + else { + $meta->{cols}->{$col}->{$prop} = $parts{$prop}; + } + } + + } +} + +# save_config {{{3 +sub save_config { + return if $config{readonly}->{val}; + # Save to a temp file first, so a crash doesn't destroy the main config file + my $newname = $opts{c} || "$homepath/.innotop/innotop.ini"; + my $filename = $newname . '_tmp'; + open my $file, "+>", $filename + or die("Can't write to $filename: $OS_ERROR"); + print $file "version=$VERSION\n"; + + foreach my $section ( @ordered_config_file_sections ) { + die "No such config file section $section" unless $config_file_sections{$section}; + print $file "\n[$section]\n\n"; + $config_file_sections{$section}->{writer}->($file); + print $file "\n[/$section]\n"; + } + + # Now clobber the main config file with the temp. + close $file or die("Can't close $filename: $OS_ERROR"); + rename($filename, $newname) or die("Can't rename $filename to $newname: $OS_ERROR"); +} + +# load_config_connections {{{3 +sub load_config_connections { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $key , $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $key; + my %parts = $rest =~ m/(\S+?)=(\S*)/g; + my %conn = map { $_ => $parts{$_} || '' } @conn_parts; + $connections{$key} = \%conn; + } +} + +# save_config_connections {{{3 +sub save_config_connections { + my $file = shift; + foreach my $conn ( sort keys %connections ) { + my $href = $connections{$conn}; + my @keys = $href->{savepass} ? @conn_parts : grep { $_ ne 'pass' } @conn_parts; + print $file "$conn=", join(' ', map { "$_=$href->{$_}" } grep { defined $href->{$_} } @keys), "\n"; + } +} + +sub load_config_colors { + my ( $file ) = @_; + my %rule_set_for; + + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $tbl, $rule ) = $line =~ m/^(.*?)=(.*)$/; + next unless $tbl && $rule; + next unless exists $tbl_meta{$tbl}; + my %parts = $rule =~ m/(\w+)='((?:(?!(?<!\\)').)*)'/g; # Properties are single-quoted + next unless $parts{col} && exists $tbl_meta{$tbl}->{cols}->{$parts{col}}; + next unless $parts{op} && exists $comp_ops{$parts{op}}; + next unless defined $parts{arg}; + next unless defined $parts{color}; + my @colors = unique(grep { exists $ansicolors{$_} } split(/\W+/, $parts{color})); + next unless @colors; + + # Finally! Enough validation... + $rule_set_for{$tbl} ||= []; + push @{$rule_set_for{$tbl}}, \%parts; + } + + foreach my $tbl ( keys %rule_set_for ) { + $tbl_meta{$tbl}->{colors} = $rule_set_for{$tbl}; + $tbl_meta{$tbl}->{color_func} = make_color_func($tbl_meta{$tbl}); + $tbl_meta{$tbl}->{cust}->{colors} = 1; + } +} + +# save_config_colors {{{3 +sub save_config_colors { + my $file = shift; + foreach my $tbl ( sort keys %tbl_meta ) { + my $meta = $tbl_meta{$tbl}; + next unless $meta->{cust}->{colors}; + foreach my $rule ( @{$meta->{colors}} ) { + print $file "$tbl=", join( + ' ', + map { + my $val = $rule->{$_}; + $val =~ s/([\\'])/\\$1/g; # Escape backslashes and single quotes + "$_='$val'"; # Enclose in single quotes + } + qw(col op arg color) + ), "\n"; + } + } +} + +# load_config_active_connections {{{3 +sub load_config_active_connections { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $key , $rest ) = $line =~ m/^(.*?)=(.*)$/; + next unless $key && exists $modes{$key}; + my @parts = grep { exists $connections{$_} } split(/ /, $rest); + $modes{$key}->{connections} = [ @parts ] if exists $modes{$key}; + } +} + +# save_config_active_connections {{{3 +sub save_config_active_connections { + my $file = shift; + foreach my $mode ( sort keys %modes ) { + my @connections = get_connections($mode); + print $file "$mode=", join(' ', @connections), "\n"; + } +} + +# load_config_stmt_sleep_times {{{3 +sub load_config_stmt_sleep_times { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $key , $val ) = split('=', $line); + next unless $key && defined $val && $val =~ m/$num_regex/; + $stmt_sleep_time_for{$key} = $val; + } +} + +# save_config_stmt_sleep_times {{{3 +sub save_config_stmt_sleep_times { + my $file = shift; + foreach my $key ( sort keys %stmt_sleep_time_for ) { + print $file "$key=$stmt_sleep_time_for{$key}\n"; + } +} + +# load_config_mvs {{{3 +sub load_config_mvs { + my ( $file ) = @_; + while ( my $line = <$file> ) { + chomp $line; + next if $line =~ m/^#/; + last if $line =~ m/^\[/; + + my ( $key , $val ) = split('=', $line); + next unless $key && defined $val && $val =~ m/$num_regex/; + $mvs{$key} = $val; + } +} + +# save_config_mvs {{{3 +sub save_config_mvs { + my $file = shift; + foreach my $key ( sort keys %mvs ) { + print $file "$key=$mvs{$key}\n"; + } +} + +# edit_configuration {{{3 +sub edit_configuration { + my $key = ''; + while ( $key ne 'q' ) { + $clear_screen_sub->(); + my @display_lines = ''; + + if ( $key && $cfg_editor_action{$key} ) { + $cfg_editor_action{$key}->{func}->(); + } + + # Show help + push @display_lines, create_caption('What configuration do you want to edit?', + create_table2( + [ sort keys %cfg_editor_action ], + { map { $_ => $_ } keys %cfg_editor_action }, + { map { $_ => $cfg_editor_action{$_}->{note} } keys %cfg_editor_action }, + { sep => ' ' })); + + draw_screen(\@display_lines); + $key = pause(''); + } +} + +# edit_configuration_variables {{{3 +sub edit_configuration_variables { + $clear_screen_sub->(); + my $mode = $config{mode}->{val}; + + my %config_choices + = map { $_ => $config{$_}->{note} || '' } + # Only config values that are marked as applying to this mode. + grep { + my $key = $_; + $config{$key}->{conf} && + ( $config{$key}->{conf} eq 'ALL' + || grep { $mode eq $_ } @{$config{$key}->{conf}} ) + } keys %config; + + my $key = prompt_list( + "Enter the name of the variable you wish to configure", + '', + sub{ return keys %config_choices }, + \%config_choices); + + if ( exists($config_choices{$key}) ) { + get_config_interactive($key); + } +} + +# edit_color_rules {{{3 +sub edit_color_rules { + my ( $tbl ) = @_; + $clear_screen_sub->(); + $tbl ||= choose_visible_table(); + if ( $tbl && exists($tbl_meta{$tbl}) ) { + my $meta = $tbl_meta{$tbl}; + my @cols = ('', qw(col op arg color)); + my $info = { map { $_ => { hdr => $_, just => '-', } } @cols }; + $info->{label}->{maxw} = 30; + my $key; + my $selected_rule; + + # This loop builds a tabular view of the rules. + do { + + # Show help + if ( $key && $key eq '?' ) { + my @display_lines = ''; + push @display_lines, create_caption('Editor key mappings', + create_table2( + [ sort keys %color_editor_action ], + { map { $_ => $_ } keys %color_editor_action }, + { map { $_ => $color_editor_action{$_}->{note} } keys %color_editor_action }, + { sep => ' ' })); + draw_screen(\@display_lines); + pause(); + $key = ''; + } + else { + + # Do the action specified + $selected_rule ||= 0; + if ( $key && $color_editor_action{$key} ) { + $selected_rule = $color_editor_action{$key}->{func}->($tbl, $selected_rule); + $selected_rule ||= 0; + } + + # Build the table of rules. If the terminal has color, the selected rule + # will be highlighted; otherwise a > at the left will indicate. + my $data = $meta->{colors} || []; + foreach my $i ( 0..@$data - 1 ) { + $data->[$i]->{''} = $i == $selected_rule ? '>' : ''; + } + my @display_lines = create_table(\@cols, $info, $data); + + # Highlight selected entry + for my $i ( 0 .. $#display_lines ) { + if ( $display_lines[$i] =~ m/^>/ ) { + $display_lines[$i] = [ $display_lines[$i], 'reverse' ]; + } + } + + # Draw the screen and wait for a command. + unshift @display_lines, '', + "Editing color rules for $meta->{capt}. Press ? for help, q to " + . "quit.", ''; + draw_screen(\@display_lines); + print "\n\n", word_wrap('Rules are applied in order from top to ' + . 'bottom. The first matching rule wins and prevents the ' + . 'rest of the rules from being applied.'); + $key = pause(''); + } + } while ( $key ne 'q' ); + $meta->{color_func} = make_color_func($meta); + } +} + +# add_quick_filter {{{3 +sub add_quick_filter { + my $tbl = choose_visible_table(); + if ( $tbl && exists($tbl_meta{$tbl}) ) { + print "\n"; + my $response = prompt_list( + "Enter column name and filter text", + '', + sub { return keys %{$tbl_meta{$tbl}->{cols}} }, + () + ); + my ( $col, $text ) = split(/\s+/, $response, 2); + + # You can't filter on a nonexistent column. But if you filter on a pivoted + # table, the columns are different, so on a pivoted table, allow filtering + # on the 'name' column. + # NOTE: if a table is pivoted and un-pivoted, this will likely cause crashes. + # Currently not an issue since there's no way to toggle pivot/nopivot. + return unless $col && $text && + (exists($tbl_meta{$tbl}->{cols}->{$col}) + || ($tbl_meta{$tbl}->{pivot} && $col eq 'name')); + + my ( $sub, $err ) = compile_filter( "defined \$set->{$col} && \$set->{$col} =~ m/$text/" ); + return if !$sub || $err; + my $name = "quick_$tbl.$col"; + $filters{$name} = { + func => $sub, + text => $text, + user => 1, + quick => 1, + name => $name, + note => 'Quick-filter', + tbls => [$tbl], + }; + push @{$tbl_meta{$tbl}->{filters}}, $name; + } +} + +# clear_quick_filters {{{3 +sub clear_quick_filters { + my $tbl = choose_visible_table( + # Only tables that have quick-filters + sub { + my ( $tbl ) = @_; + return scalar grep { $filters{$_}->{quick} } @{ $tbl_meta{$tbl}->{filters} }; + } + ); + if ( $tbl && exists($tbl_meta{$tbl}) ) { + my @current = @{$tbl_meta{$tbl}->{filters}}; + @current = grep { !$filters{$_}->{quick} } @current; + $tbl_meta{$tbl}->{filters} = \@current; + } +} + +sub edit_plugins { + $clear_screen_sub->(); + + my @cols = ('', qw(class desc active)); + my $info = { map { $_ => { hdr => $_, just => '-', } } @cols }; + my @rows = map { $plugins{$_} } sort keys %plugins; + my $key; + my $selected; + + # This loop builds a tabular view of the plugins. + do { + + # Show help + if ( $key && $key eq '?' ) { + my @display_lines = ''; + push @display_lines, create_caption('Editor key mappings', + create_table2( + [ sort keys %plugin_editor_action ], + { map { $_ => $_ } keys %plugin_editor_action }, + { map { $_ => $plugin_editor_action{$_}->{note} } keys %plugin_editor_action }, + { sep => ' ' })); + draw_screen(\@display_lines); + pause(); + $key = ''; + } + + # Do the action specified + else { + $selected ||= 0; + if ( $key && $plugin_editor_action{$key} ) { + $selected = $plugin_editor_action{$key}->{func}->(\@rows, $selected); + $selected ||= 0; + } + + # Build the table of plugins. + foreach my $row ( 0.. $#rows ) { + $rows[$row]->{''} = $row eq $selected ? '>' : ' '; + } + my @display_lines = create_table(\@cols, $info, \@rows); + + # Highlight selected entry + for my $i ( 0 .. $#display_lines ) { + if ( $display_lines[$i] =~ m/^>/ ) { + $display_lines[$i] = [ $display_lines[$i], 'reverse' ]; + } + } + + # Draw the screen and wait for a command. + unshift @display_lines, '', + "Plugin Management. Press ? for help, q to quit.", ''; + draw_screen(\@display_lines); + $key = pause(''); + } + } while ( $key ne 'q' ); +} + +# edit_table {{{3 +sub edit_table { + $clear_screen_sub->(); + my ( $tbl ) = @_; + $tbl ||= choose_visible_table(); + if ( $tbl && exists($tbl_meta{$tbl}) ) { + my $meta = $tbl_meta{$tbl}; + my @cols = ('', qw(name hdr label src)); + my $info = { map { $_ => { hdr => $_, just => '-', } } @cols }; + $info->{label}->{maxw} = 30; + my $key; + my $selected_column; + + # This loop builds a tabular view of the tbl_meta's structure, showing each column + # in the entry as a row. + do { + + # Show help + if ( $key && $key eq '?' ) { + my @display_lines = ''; + push @display_lines, create_caption('Editor key mappings', + create_table2( + [ sort keys %tbl_editor_action ], + { map { $_ => $_ } keys %tbl_editor_action }, + { map { $_ => $tbl_editor_action{$_}->{note} } keys %tbl_editor_action }, + { sep => ' ' })); + draw_screen(\@display_lines); + pause(); + $key = ''; + } + else { + + # Do the action specified + $selected_column ||= $meta->{visible}->[0]; + if ( $key && $tbl_editor_action{$key} ) { + $selected_column = $tbl_editor_action{$key}->{func}->($tbl, $selected_column); + $selected_column ||= $meta->{visible}->[0]; + } + + # Build the pivoted view of the table's meta-data. If the terminal has color, + # The selected row will be highlighted; otherwise a > at the left will indicate. + my $data = []; + foreach my $row ( @{$meta->{visible}} ) { + my %hash; + @hash{ @cols } = @{$meta->{cols}->{$row}}{@cols}; + $hash{src} = '' if ref $hash{src}; + $hash{name} = $row; + $hash{''} = $row eq $selected_column ? '>' : ' '; + push @$data, \%hash; + } + my @display_lines = create_table(\@cols, $info, $data); + + # Highlight selected entry + for my $i ( 0 .. $#display_lines ) { + if ( $display_lines[$i] =~ m/^>/ ) { + $display_lines[$i] = [ $display_lines[$i], 'reverse' ]; + } + } + + # Draw the screen and wait for a command. + unshift @display_lines, '', + "Editing table definition for $meta->{capt}. Press ? for help, q to quit.", ''; + draw_screen(\@display_lines, { clear => 1 }); + $key = pause(''); + } + } while ( $key ne 'q' ); + } +} + +# choose_mode_tables {{{3 +# Choose which table(s), and in what order, to display in a given mode. +sub choose_mode_tables { + my $mode = $config{mode}->{val}; + my @tbls = @{$modes{$mode}->{visible_tables}}; + my $new = prompt_list( + "Choose tables to display", + join(' ', @tbls), + sub { return @{$modes{$mode}->{tables}} }, + { map { $_ => $tbl_meta{$_}->{capt} } @{$modes{$mode}->{tables}} } + ); + $modes{$mode}->{visible_tables} = + [ unique(grep { $_ && exists $tbl_meta{$_} } split(/\s+/, $new)) ]; + $modes{$mode}->{cust}->{visible_tables} = 1; +} + +# choose_visible_table {{{3 +sub choose_visible_table { + my ( $grep_cond ) = @_; + my $mode = $config{mode}->{val}; + my @tbls + = grep { $grep_cond ? $grep_cond->($_) : 1 } + @{$modes{$mode}->{visible_tables}}; + my $tbl = $tbls[0]; + if ( @tbls > 1 ) { + $tbl = prompt_list( + "Choose a table", + '', + sub { return @tbls }, + { map { $_ => $tbl_meta{$_}->{capt} } @tbls } + ); + } + return $tbl; +} + +sub toggle_aggregate { + my ( $tbl ) = @_; + $tbl ||= choose_visible_table(); + return unless $tbl && exists $tbl_meta{$tbl}; + my $meta = $tbl_meta{$tbl}; + $meta->{aggregate} ^= 1; +} + +sub choose_filters { + my ( $tbl ) = @_; + $tbl ||= choose_visible_table(); + return unless $tbl && exists $tbl_meta{$tbl}; + my $meta = $tbl_meta{$tbl}; + $clear_screen_sub->(); + + print "Choose filters for $meta->{capt}:\n"; + + my $ini = join(' ', @{$meta->{filters}}); + my $val = prompt_list( + 'Choose filters', + $ini, + sub { return keys %filters }, + { + map { $_ => $filters{$_}->{note} } + grep { grep { $tbl eq $_ } @{$filters{$_}->{tbls}} } + keys %filters + } + ); + + my @choices = unique(split(/\s+/, $val)); + foreach my $new ( grep { !exists($filters{$_}) } @choices ) { + my $answer = prompt("There is no filter called '$new'. Create it?", undef, 'y'); + if ( $answer eq 'y' ) { + create_new_filter($new, $tbl); + } + } + @choices = grep { exists $filters{$_} } @choices; + @choices = grep { grep { $tbl eq $_ } @{$filters{$_}->{tbls}} } @choices; + $meta->{filters} = [ @choices ]; + $meta->{cust}->{filters} = 1; +} + +sub choose_group_cols { + my ( $tbl ) = @_; + $tbl ||= choose_visible_table(); + return unless $tbl && exists $tbl_meta{$tbl}; + $clear_screen_sub->(); + my $meta = $tbl_meta{$tbl}; + my $curr = join(', ', @{$meta->{group_by}}); + my $val = prompt_list( + 'Group-by columns', + $curr, + sub { return keys %{$meta->{cols}} }, + { map { $_ => $meta->{cols}->{$_}->{label} } keys %{$meta->{cols}} }); + if ( $curr ne $val ) { + $meta->{group_by} = [ grep { exists $meta->{cols}->{$_} } $val =~ m/(\w+)/g ]; + $meta->{cust}->{group_by} = 1; + } +} + +sub choose_sort_cols { + my ( $tbl ) = @_; + $tbl ||= choose_visible_table(); + return unless $tbl && exists $tbl_meta{$tbl}; + $clear_screen_sub->(); + my $meta = $tbl_meta{$tbl}; + + my ( $cols, $hints ); + if ( $meta->{pivot} ) { + $cols = sub { qw(name set_0) }; + $hints = { name => 'name', set_0 => 'set_0' }; + } + else { + $cols = sub { return keys %{$meta->{cols}} }; + $hints = { map { $_ => $meta->{cols}->{$_}->{label} } keys %{$meta->{cols}} }; + } + + my $val = prompt_list( + 'Sort columns (reverse sort with -col)', + $meta->{sort_cols}, + $cols, + $hints ); + if ( $meta->{sort_cols} ne $val ) { + $meta->{sort_cols} = $val; + $meta->{cust}->{sort_cols} = 1; + $tbl_meta{$tbl}->{sort_func} = make_sort_func($tbl_meta{$tbl}); + } +} + +# create_new_filter {{{3 +sub create_new_filter { + my ( $filter, $tbl ) = @_; + $clear_screen_sub->(); + + if ( !$filter || $filter =~ m/\W/ ) { + print word_wrap("Choose a name for the filter. This name is not displayed, and is only used " + . "for internal reference. It can only contain lowercase letters, numbers, and underscores."); + print "\n\n"; + do { + $filter = prompt("Enter filter name"); + } while ( !$filter || $filter =~ m/\W/ ); + } + + my $completion = sub { keys %{$tbl_meta{$tbl}->{cols}} }; + my ( $err, $sub, $body ); + do { + $clear_screen_sub->(); + print word_wrap("A filter is a Perl subroutine that accepts a hashref of columns " + . "called \$set, and returns a true value if the filter accepts the row. Example:\n" + . " \$set->{active_secs} > 5\n" + . "will only allow rows if their active_secs column is greater than 5."); + print "\n\n"; + if ( $err ) { + print "There's an error in your filter expression: $err\n\n"; + } + $body = prompt("Enter subroutine body", undef, undef, $completion); + ( $sub, $err ) = compile_filter($body); + } while ( $err ); + + $filters{$filter} = { + func => $sub, + text => $body, + user => 1, + name => $filter, + note => 'User-defined filter', + tbls => [$tbl], + }; +} + +# get_config_interactive {{{3 +sub get_config_interactive { + my $key = shift; + $clear_screen_sub->(); + + # Print help first. + print "Enter a new value for '$key' ($config{$key}->{note}).\n"; + + my $current = ref($config{$key}->{val}) ? join(" ", @{$config{$key}->{val}}) : $config{$key}->{val}; + + my $new_value = prompt('Enter a value', $config{$key}->{pat}, $current); + $config{$key}->{val} = $new_value; +} + +sub edit_current_var_set { + my $mode = $config{mode}->{val}; + my $name = $config{"${mode}_set"}->{val}; + my $variables = $var_sets{$name}->{text}; + + my $new = $variables; + do { + $clear_screen_sub->(); + $new = prompt("Enter variables for $name", undef, $variables); + } until ( $new ); + + if ( $new ne $variables ) { + @{$var_sets{$name}}{qw(text user)} = ( $new, 1); + } +} + + +sub choose_var_set { + my ( $key ) = @_; + $clear_screen_sub->(); + + my $new_value = prompt_list( + 'Choose a set of values to display, or enter the name of a new one', + $config{$key}->{val}, + sub { return keys %var_sets }, + { map { $_ => $var_sets{$_}->{text} } keys %var_sets }); + + if ( !exists $var_sets{$new_value} ) { + add_new_var_set($new_value); + } + + $config{$key}->{val} = $new_value if exists $var_sets{$new_value}; +} + +sub switch_var_set { + my ( $cfg_var, $dir ) = @_; + my @var_sets = sort keys %var_sets; + my $cur = $config{$cfg_var}->{val}; + my $pos = grep { $_ lt $cur } @var_sets; + my $newpos = ($pos + $dir) % @var_sets; + $config{$cfg_var}->{val} = $var_sets[$newpos]; + $clear_screen_sub->(); +} + +# Online configuration and prompting functions {{{2 + +# edit_stmt_sleep_times {{{3 +sub edit_stmt_sleep_times { + $clear_screen_sub->(); + my $stmt = prompt_list('Specify a statement', '', sub { return sort keys %stmt_maker_for }); + return unless $stmt && exists $stmt_maker_for{$stmt}; + $clear_screen_sub->(); + my $curr_val = $stmt_sleep_time_for{$stmt} || 0; + my $new_val = prompt('Specify a sleep delay after calling this SQL', $num_regex, $curr_val); + if ( $new_val ) { + $stmt_sleep_time_for{$stmt} = $new_val; + } + else { + delete $stmt_sleep_time_for{$stmt}; + } +} + +# edit_server_groups {{{3 +# Choose which server connections are in a server group. First choose a group, +# then choose which connections are in it. +sub edit_server_groups { + $clear_screen_sub->(); + my $mode = $config{mode}->{val}; + my $group = $modes{$mode}->{server_group}; + my %curr = %server_groups; + my $new = choose_or_create_server_group($group, 'to edit'); + $clear_screen_sub->(); + if ( exists $curr{$new} ) { + # Don't do this step if the user just created a new server group, + # because part of that process was to choose connections. + my $cxns = join(' ', @{$server_groups{$new}}); + my @conns = choose_or_create_connection($cxns, 'for this group'); + $server_groups{$new} = \@conns; + } +} + +# choose_server_groups {{{3 +sub choose_server_groups { + $clear_screen_sub->(); + my $mode = $config{mode}->{val}; + my $group = $modes{$mode}->{server_group}; + my $new = choose_or_create_server_group($group, 'for this mode'); + $modes{$mode}->{server_group} = $new if exists $server_groups{$new}; +} + +sub choose_or_create_server_group { + my ( $group, $prompt ) = @_; + my $new = ''; + + my @available = sort keys %server_groups; + + if ( @available ) { + print "You can enter the name of a new group to create it.\n"; + + $new = prompt_list( + "Choose a server group $prompt", + $group, + sub { return @available }, + { map { $_ => join(' ', @{$server_groups{$_}}) } @available }); + + $new =~ s/\s.*//; + + if ( !exists $server_groups{$new} ) { + my $answer = prompt("There is no server group called '$new'. Create it?", undef, "y"); + if ( $answer eq 'y' ) { + add_new_server_group($new); + } + } + } + else { + $new = add_new_server_group(); + } + return $new; +} + +sub choose_or_create_connection { + my ( $cxns, $prompt ) = @_; + print "You can enter the name of a new connection to create it.\n"; + + my @available = sort keys %connections; + my $new_cxns = prompt_list( + "Choose connections $prompt", + $cxns, + sub { return @available }, + { map { $_ => $connections{$_}->{dsn} } @available }); + + my @new = unique(grep { !exists $connections{$_} } split(/\s+/, $new_cxns)); + foreach my $new ( @new ) { + my $answer = prompt("There is no connection called '$new'. Create it?", undef, "y"); + if ( $answer eq 'y' ) { + add_new_dsn($new); + } + } + + return unique(grep { exists $connections{$_} } split(/\s+/, $new_cxns)); +} + +# choose_servers {{{3 +sub choose_servers { + $clear_screen_sub->(); + my $mode = $config{mode}->{val}; + my $cxns = join(' ', get_connections()); + my @chosen = choose_or_create_connection($cxns, 'for this mode'); + $modes{$mode}->{connections} = \@chosen; + $modes{$mode}->{server_group} = ''; # Clear this because it overrides {connections} +} + +# display_license {{{3 +sub display_license { + $clear_screen_sub->(); + + print $innotop_license; + + pause(); +} + +# Data-retrieval functions {{{2 +# get_status_info {{{3 +# Get SHOW STATUS and SHOW VARIABLES together. +sub get_status_info { + my @cxns = @_; + if ( !$info_gotten{status}++ ) { + foreach my $cxn ( @cxns ) { + $vars{$cxn}->{$clock} ||= {}; + my $vars = $vars{$cxn}->{$clock}; + + my $sth = do_stmt($cxn, 'SHOW_STATUS') or next; + my $res = $sth->fetchall_arrayref(); + map { $vars->{$_->[0]} = $_->[1] || 0 } @$res; + + # Calculate hi-res uptime and add cxn to the hash. This duplicates get_driver_status, + # but it's most important to have consistency. + $vars->{Uptime_hires} ||= get_uptime($cxn); + $vars->{cxn} = $cxn; + + # Add SHOW VARIABLES to the hash + $sth = do_stmt($cxn, 'SHOW_VARIABLES') or next; + $res = $sth->fetchall_arrayref(); + map { $vars->{$_->[0]} = $_->[1] || 0 } @$res; + } + } +} + +# Chooses a thread for explaining, killing, etc... +# First arg is a func that can be called in grep. +sub choose_thread { + my ( $grep_cond, $prompt ) = @_; + + # Narrow the list to queries that can be explained. + my %thread_for = map { + # Eliminate innotop's own threads. + $_ => $dbhs{$_}->{dbh} ? $dbhs{$_}->{dbh}->{mysql_thread_id} : 0 + } keys %connections; + + my @candidates = grep { + $_->{id} != $thread_for{$_->{cxn}} && $grep_cond->($_) + } @current_queries; + return unless @candidates; + + # Find out which server. + my @cxns = unique map { $_->{cxn} } @candidates; + my ( $cxn ) = select_cxn('On which server', @cxns); + return unless $cxn && exists($connections{$cxn}); + + # Re-filter the list of candidates to only those on this server + @candidates = grep { $_->{cxn} eq $cxn } @candidates; + + # Find out which thread to do. + my $info; + if ( @candidates > 1 ) { + + # Sort longest-active first, then longest-idle. + my $sort_func = sub { + my ( $a, $b ) = @_; + return $a->{query} && !$b->{query} ? 1 + : $b->{query} && !$a->{query} ? -1 + : ($a->{time} || 0) <=> ($b->{time} || 0); + }; + my @threads = map { $_->{id} } reverse sort { $sort_func->($a, $b) } @candidates; + + print "\n"; + my $thread = prompt_list($prompt, + $threads[0], + sub { return @threads }); + return unless $thread && $thread =~ m/$int_regex/; + + # Find the info hash of that query on that server. + ( $info ) = grep { $thread == $_->{id} } @candidates; + } + else { + $info = $candidates[0]; + } + return $info; +} + +# analyze_query {{{3 +# Allows the user to show fulltext, explain, show optimized... +sub analyze_query { + my ( $action ) = @_; + + my $info = choose_thread( + sub { $_[0]->{query} }, + 'Select a thread to analyze', + ); + return unless $info; + + my %actions = ( + e => \&display_explain, + f => \&show_full_query, + o => \&show_optimized_query, + ); + do { + $actions{$action}->($info); + print "\n"; + $action = pause('Press e to explain, f for full query, o for optimized query'); + } while ( exists($actions{$action}) ); +} + +# inc {{{3 +# Returns the difference between two sets of variables/status/innodb stuff. +sub inc { + my ( $offset, $cxn ) = @_; + my $vars = $vars{$cxn}; + if ( $offset < 0 ) { + return $vars->{$clock}; + } + elsif ( exists $vars{$clock - $offset} && !exists $vars->{$clock - $offset - 1} ) { + return $vars->{$clock - $offset}; + } + my $cur = $vars->{$clock - $offset}; + my $pre = $vars->{$clock - $offset - 1}; + return { + # Numeric variables get subtracted, non-numeric get passed straight through. + map { + $_ => + ( (defined $cur->{$_} && $cur->{$_} =~ m/$num_regex/) + ? $cur->{$_} - ($pre->{$_} || 0) + : $cur->{$_} ) + } keys %{$cur} + }; +} + +# extract_values {{{3 +# Arguments are a set of values (which may be incremental, derived from +# current and previous), current, and previous values. +# TODO: there are a few places that don't remember prev set so can't pass it. +sub extract_values { + my ( $set, $cur, $pre, $tbl ) = @_; + + # Hook in event listeners + foreach my $listener ( @{$event_listener_for{extract_values}} ) { + $listener->extract_values($set, $cur, $pre, $tbl); + } + + my $result = {}; + my $meta = $tbl_meta{$tbl}; + my $cols = $meta->{cols}; + foreach my $key ( keys %$cols ) { + my $info = $cols->{$key} + or die "Column '$key' doesn't exist in $tbl"; + die "No func defined for '$key' in $tbl" + unless $info->{func}; + eval { + $result->{$key} = $info->{func}->($set, $cur, $pre) + }; + if ( $EVAL_ERROR ) { + if ( $config{debug}->{val} ) { + die $EVAL_ERROR; + } + $result->{$key} = $info->{num} ? 0 : ''; + } + } + return $result; +} + +# get_full_processlist {{{3 +sub get_full_processlist { + my @cxns = @_; + my @result; + foreach my $cxn ( @cxns ) { + my $stmt = do_stmt($cxn, 'PROCESSLIST') or next; + my $arr = $stmt->fetchall_arrayref({}); + push @result, map { $_->{cxn} = $cxn; $_ } @$arr; + } + return @result; +} + +# get_open_tables {{{3 +sub get_open_tables { + my @cxns = @_; + my @result; + foreach my $cxn ( @cxns ) { + my $stmt = do_stmt($cxn, 'OPEN_TABLES') or next; + my $arr = $stmt->fetchall_arrayref({}); + push @result, map { $_->{cxn} = $cxn; $_ } @$arr; + } + return @result; +} + +# get_innodb_status {{{3 +sub get_innodb_status { + my ( $cxns, $addl_sections ) = @_; + if ( !$config{skip_innodb}->{val} && !$info_gotten{innodb_status}++ ) { + + # Determine which sections need to be parsed + my %sections_required = + map { $tbl_meta{$_}->{innodb} => 1 } + grep { $_ && $tbl_meta{$_}->{innodb} } + get_visible_tables(); + + # Add in any other sections the caller requested. + foreach my $sec ( @$addl_sections ) { + $sections_required{$sec} = 1; + } + + foreach my $cxn ( @$cxns ) { + my $innodb_status_text; + + if ( $file ) { # Try to fetch status text from the file. + my @stat = stat($file); + + # Initialize the file. + if ( !$file_mtime ) { + # Initialize to 130k from the end of the file (because the limit + # on the size of innodb status is 128k even with Google's patches) + # and try to grab the last status from the file. + sysseek($file, (-128 * 1_024), 2); + } + + # Read from the file. + my $buffer; + if ( !$file_mtime || $file_mtime != $stat[9] ) { + $file_data = ''; + while ( sysread($file, $buffer, 4096) ) { + $file_data .= $buffer; + } + $file_mtime = $stat[9]; + } + + # Delete everything but the last InnoDB status text from the file. + $file_data =~ s/\A.*(?=^=====================================\n...... ........ INNODB MONITOR OUTPUT)//ms; + $innodb_status_text = $file_data; + } + + else { + my $stmt = do_stmt($cxn, 'INNODB_STATUS') or next; + $innodb_status_text = $stmt->fetchrow_hashref()->{status}; + } + + next unless $innodb_status_text + && substr($innodb_status_text, 0, 100) =~ m/INNODB MONITOR OUTPUT/; + + # Parse and merge into %vars storage + my %innodb_status = ( + $innodb_parser->get_status_hash( + $innodb_status_text, + $config{debug}->{val}, + \%sections_required, + 0, # don't parse full lock information + ) + ); + if ( !$innodb_status{IB_got_all} && $config{auto_wipe_dl}->{val} ) { + clear_deadlock($cxn); + } + + # Merge using a hash slice, which is the fastest way + $vars{$cxn}->{$clock} ||= {}; + my $hash = $vars{$cxn}->{$clock}; + @{$hash}{ keys %innodb_status } = values %innodb_status; + $hash->{cxn} = $cxn; + $hash->{Uptime_hires} ||= get_uptime($cxn); + } + } +} + +# clear_deadlock {{{3 +sub clear_deadlock { + my ( $cxn ) = @_; + return if $clearing_deadlocks++; + my $tbl = $connections{$cxn}->{dl_table}; + return unless $tbl; + + eval { + # Set up the table for creating a deadlock. + my $engine = version_ge($dbhs{$cxn}->{dbh}, '4.1.2') ? 'engine' : 'type'; + return unless do_query($cxn, "drop table if exists $tbl"); + return unless do_query($cxn, "create table $tbl(a int) $engine=innodb"); + return unless do_query($cxn, "delete from $tbl"); + return unless do_query($cxn, "insert into $tbl(a) values(0), (1)"); + return unless do_query($cxn, "commit"); # Or the children will block against the parent + + # Fork off two children to deadlock against each other. + my %children; + foreach my $child ( 0..1 ) { + my $pid = fork(); + if ( defined($pid) && $pid == 0 ) { # I am a child + deadlock_thread( $child, $tbl, $cxn ); + } + elsif ( !defined($pid) ) { + die("Unable to fork for clearing deadlocks!\n"); + } + # I already exited if I'm a child, so I'm the parent. + $children{$child} = $pid; + } + + # Wait for the children to exit. + foreach my $child ( keys %children ) { + my $pid = waitpid($children{$child}, 0); + } + + # Clean up. + do_query($cxn, "drop table $tbl"); + }; + if ( $EVAL_ERROR ) { + print $EVAL_ERROR; + pause(); + } + + $clearing_deadlocks = 0; +} + +sub get_master_logs { + my @cxns = @_; + my @result; + if ( !$info_gotten{master_logs}++ ) { + foreach my $cxn ( @cxns ) { + my $stmt = do_stmt($cxn, 'SHOW_MASTER_LOGS') or next; + push @result, @{$stmt->fetchall_arrayref({})}; + } + } + return @result; +} + +# get_master_slave_status {{{3 +sub get_master_slave_status { + my @cxns = @_; + if ( !$info_gotten{replication_status}++ ) { + foreach my $cxn ( @cxns ) { + $vars{$cxn}->{$clock} ||= {}; + my $vars = $vars{$cxn}->{$clock}; + $vars->{cxn} = $cxn; + + my $stmt = do_stmt($cxn, 'SHOW_MASTER_STATUS') or next; + my $res = $stmt->fetchall_arrayref({})->[0]; + @{$vars}{ keys %$res } = values %$res; + $stmt = do_stmt($cxn, 'SHOW_SLAVE_STATUS') or next; + $res = $stmt->fetchall_arrayref({})->[0]; + @{$vars}{ keys %$res } = values %$res; + $vars->{Uptime_hires} ||= get_uptime($cxn); + } + } +} + +sub is_func { + my ( $word ) = @_; + return defined(&$word) + || eval "my \$x= sub { $word }; 1" + || $EVAL_ERROR !~ m/^Bareword/; +} + +# Documentation {{{1 +# ############################################################################ +# I put this last as per the Dog book. +# ############################################################################ +=pod + +=head1 NAME + +innotop - MySQL and InnoDB transaction/status monitor. + +=head1 SYNOPSIS + +To monitor servers normally: + + innotop + +To monitor InnoDB status information from a file: + + innotop /var/log/mysql/mysqld.err + +To run innotop non-interactively in a pipe-and-filter configuration: + + innotop --count 5 -d 1 -n + +=head1 DESCRIPTION + +innotop monitors MySQL servers. Each of its modes shows you a different aspect +of what's happening in the server. For example, there's a mode for monitoring +replication, one for queries, and one for transactions. innotop refreshes its +data periodically, so you see an updating view. + +innotop has lots of features for power users, but you can start and run it with +virtually no configuration. If you're just getting started, see +L<"QUICK-START">. Press '?' at any time while running innotop for +context-sensitive help. + +=head1 QUICK-START + +To start innotop, open a terminal or command prompt. If you have installed +innotop on your system, you should be able to just type "innotop" and press +Enter; otherwise, you will need to change to innotop's directory and type "perl +innotop". + +The first thing innotop needs to know is how to connect to a MySQL server. You +can just enter the hostname of the server, for example "localhost" or +"127.0.0.1" if the server is on the same machine as innotop. After this innotop +will prompt you for a DSN (data source name). You should be able to just accept +the defaults by pressing Enter. + +When innotop asks you about a table to use when resetting InnoDB deadlock +information, just accept the default for now. This is an advanced feature you +can configure later (see L<"D: InnoDB Deadlocks"> for more). + +If you have a .my.cnf file with your MySQL connection defaults, innotop can read +it, and you won't need to specify a username and password if it's in that file. +Otherwise, you should answer 'y' to the next couple of prompts. + +After this, you should be connected, and innotop should show you something like +the following: + + InnoDB Txns (? for help) localhost, 01:11:19, InnoDB 10s :-), 50 QPS, + + CXN History Versions Undo Dirty Buf Used Bufs Txns MaxTxn + localhost 7 2035 0 0 0.00% 92.19% 1 07:34 + + CXN ID User Host Txn Status Time Undo Query Tex + localhost 98379 user1 webserver ACTIVE 07:34 0 SELECT `c + localhost 98450 user1 webserver ACTIVE 01:06 0 INSERT IN + localhost 97750 user1 webserver not starte 00:00 0 + localhost 98375 user1 appserver not starte 00:00 0 + +(This sample is truncated at the right so it will fit on a terminal when running +'man innotop') + +This sample comes from a quiet server with few transactions active. If your +server is busy, you'll see more output. Notice the first line on the screen, +which tells you what mode you're in and what server you're connected to. You +can change to other modes with keystrokes; press 'Q' to switch to a list of +currently running queries. + +Press the '?' key to see what keys are active in the current mode. You can +press any of these keys and innotop will either take the requested action or +prompt you for more input. If your system has Term::ReadLine support, you can +use TAB and other keys to auto-complete and edit input. + +To quit innotop, press the 'q' key. + +=head1 OPTIONS + +innotop is mostly configured via its configuration file, but some of the +configuration options can come from the command line. You can also specify a +file to monitor for InnoDB status output; see L<"MONITORING A FILE"> for more +details. + +You can negate some options by prefixing the option name with --no. For +example, --noinc (or --no-inc) negates L<"--inc">. + +=over + +=item --help + +Print a summary of command-line usage and exit. + +=item --color + +Enable or disable terminal coloring. Corresponds to the L<"color"> config file +setting. + +=item --config + +Specifies a configuration file to read. This option is non-sticky, that is to +say it does not persist to the configuration file itself. + +=item --nonint + +Enable non-interactive operation. See L<"NON-INTERACTIVE OPERATION"> for more. + +=item --count + +Refresh only the specified number of times (ticks) before exiting. Each refresh +is a pause for L<"interval"> seconds, followed by requesting data from MySQL +connections and printing it to the terminal. + +=item --delay + +Specifies the amount of time to pause between ticks (refreshes). Corresponds to +the configuration option L<"interval">. + +=item --mode + +Specifies the mode in which innotop should start. Corresponds to the +configuration option L<"mode">. + +=item --inc + +Specifies whether innotop should display absolute numbers or relative numbers +(offsets from their previous values). Corresponds to the configuration option +L<"status_inc">. + +=item --version + +Output version information and exit. + +=back + +=head1 HOTKEYS + +innotop is interactive, and you control it with key-presses. + +=over + +=item * + +Uppercase keys switch between modes. + +=item * + +Lowercase keys initiate some action within the current mode. + +=item * + +Other keys do something special like change configuration or show the +innotop license. + +=back + +Press '?' at any time to see the currently active keys and what they do. + +=head1 MODES + +Each of innotop's modes retrieves and displays a particular type of data from +the servers you're monitoring. You switch between modes with uppercase keys. +The following is a brief description of each mode, in alphabetical order. To +switch to the mode, press the key listed in front of its heading in the +following list: + +=over + +=item B: InnoDB Buffers + +This mode displays information about the InnoDB buffer pool, page statistics, +insert buffer, and adaptive hash index. The data comes from SHOW INNODB STATUS. + +This mode contains the L<"buffer_pool">, L<"page_statistics">, +L<"insert_buffers">, and L<"adaptive_hash_index"> tables by default. + +=item C: Command Summary + +This mode is similar to mytop's Command Summary mode. It shows the +L<"cmd_summary"> table, which looks something like the following: + + Command Summary (? for help) localhost, 25+07:16:43, 2.45 QPS, 3 thd, 5.0.40 + _____________________ Command Summary _____________________ + Name Value Pct Last Incr Pct + Select_scan 3244858 69.89% 2 100.00% + Select_range 1354177 29.17% 0 0.00% + Select_full_join 39479 0.85% 0 0.00% + Select_full_range_join 4097 0.09% 0 0.00% + Select_range_check 0 0.00% 0 0.00% + +The command summary table is built by extracting variables from +L<"STATUS_VARIABLES">. The variables must be numeric and must match the prefix +given by the L<"cmd_filter"> configuration variable. The variables are then +sorted by value descending and compared to the last variable, as shown above. +The percentage columns are percentage of the total of all variables in the +table, so you can see the relative weight of the variables. + +The example shows what you see if the prefix is "Select_". The default +prefix is "Com_". You can choose a prefix with the 's' key. + +It's rather like running SHOW VARIABLES LIKE "prefix%" with memory and +nice formatting. + +Values are aggregated across all servers. The Pct columns are not correctly +aggregated across multiple servers. This is a known limitation of the grouping +algorithm that may be fixed in the future. + +=item D: InnoDB Deadlocks + +This mode shows the transactions involved in the last InnoDB deadlock. A second +table shows the locks each transaction held and waited for. A deadlock is +caused by a cycle in the waits-for graph, so there should be two locks held and +one waited for unless the deadlock information is truncated. + +InnoDB puts deadlock information before some other information in the SHOW +INNODB STATUS output. If there are a lot of locks, the deadlock information can +grow very large, and there is a limit on the size of the SHOW INNODB +STATUS output. A large deadlock can fill the entire output, or even be +truncated, and prevent you from seeing other information at all. If you are +running innotop in another mode, for example T mode, and suddenly you don't see +anything, you might want to check and see if a deadlock has wiped out the data +you need. + +If it has, you can create a small deadlock to replace the large one. Use the +'w' key to 'wipe' the large deadlock with a small one. This will not work +unless you have defined a deadlock table for the connection (see L<"SERVER +CONNECTIONS">). + +You can also configure innotop to automatically detect when a large deadlock +needs to be replaced with a small one (see L<"auto_wipe_dl">). + +This mode displays the L<"deadlock_transactions"> and L<"deadlock_locks"> tables +by default. + +=item F: InnoDB Foreign Key Errors + +This mode shows the last InnoDB foreign key error information, such as the +table where it happened, when and who and what query caused it, and so on. + +InnoDB has a huge variety of foreign key error messages, and many of them are +just hard to parse. innotop doesn't always do the best job here, but there's +so much code devoted to parsing this messy, unparseable output that innotop is +likely never to be perfect in this regard. If innotop doesn't show you what +you need to see, just look at the status text directly. + +This mode displays the L<"fk_error"> table by default. + +=item I: InnoDB I/O Info + +This mode shows InnoDB's I/O statistics, including the I/O threads, pending I/O, +file I/O miscellaneous, and log statistics. It displays the L<"io_threads">, +L<"pending_io">, L<"file_io_misc">, and L<"log_statistics"> tables by default. + +=item L: Locks + +This mode shows information about current locks. At the moment only InnoDB +locks are supported, and by default you'll only see locks for which transactions +are waiting. This information comes from the TRANSACTIONS section of the InnoDB +status text. If you have a very busy server, you may have frequent lock waits; +it helps to be able to see which tables and indexes are the "hot spot" for +locks. If your server is running pretty well, this mode should show nothing. + +You can configure MySQL and innotop to monitor not only locks for which a +transaction is waiting, but those currently held, too. You can do this with the +InnoDB Lock Monitor (L<http://dev.mysql.com/doc/en/innodb-monitor.html>). It's +not documented in the MySQL manual, but creating the lock monitor with the +following statement also affects the output of SHOW INNODB STATUS, which innotop +uses: + + CREATE TABLE innodb_lock_monitor(a int) ENGINE=INNODB; + +This causes InnoDB to print its output to the MySQL file every 16 seconds or so, +as stated in the manual, but it also makes the normal SHOW INNODB STATUS output +include lock information, which innotop can parse and display (that's the +undocumented feature). + +This means you can do what may have seemed impossible: to a limited extent +(InnoDB truncates some information in the output), you can see which transaction +holds the locks something else is waiting for. You can also enable and disable +the InnoDB Lock Monitor with the key mappings in this mode. + +This mode displays the L<"innodb_locks"> table by default. Here's a sample of +the screen when one connection is waiting for locks another connection holds: + + _________________________________ InnoDB Locks __________________________ + CXN ID Type Waiting Wait Active Mode DB Table Index + localhost 12 RECORD 1 00:10 00:10 X test t1 PRIMARY + localhost 12 TABLE 0 00:10 00:10 IX test t1 + localhost 12 RECORD 1 00:10 00:10 X test t1 PRIMARY + localhost 11 TABLE 0 00:00 00:25 IX test t1 + localhost 11 RECORD 0 00:00 00:25 X test t1 PRIMARY + +You can see the first connection, ID 12, is waiting for a lock on the PRIMARY +key on test.t1, and has been waiting for 10 seconds. The second connection +isn't waiting, because the Waiting column is 0, but it holds locks on the same +index. That tells you connection 11 is blocking connection 12. + +=item M: Master/Slave Replication Status + +This mode shows the output of SHOW SLAVE STATUS and SHOW MASTER STATUS in three +tables. The first two divide the slave's status into SQL and I/O thread status, +and the last shows master status. Filters are applied to eliminate non-slave +servers from the slave tables, and non-master servers from the master table. + +This mode displays the L<"slave_sql_status">, L<"slave_io_status">, and +L<"master_status"> tables by default. + +=item O: Open Tables + +This section comes from MySQL's SHOW OPEN TABLES command. By default it is +filtered to show tables which are in use by one or more queries, so you can +get a quick look at which tables are 'hot'. You can use this to guess which +tables might be locked implicitly. + +This mode displays the L<"open_tables"> mode by default. + +=item Q: Query List + +This mode displays the output from SHOW FULL PROCESSLIST, much like B<mytop>'s +query list mode. This mode does B<not> show InnoDB-related information. This +is probably one of the most useful modes for general usage. + +There is an informative header that shows general status information about +your server. You can toggle it on and off with the 'h' key. By default, +innotop hides inactive processes and its own process. You can toggle these on +and off with the 'i' and 'a' keys. + +You can EXPLAIN a query from this mode with the 'e' key. This displays the +query's full text, the results of EXPLAIN, and in newer MySQL versions, even +the optimized query resulting from EXPLAIN EXTENDED. innotop also tries to +rewrite certain queries to make them EXPLAIN-able. For example, INSERT/SELECT +statements are rewritable. + +This mode displays the L<"q_header"> and L<"processlist"> tables by default. + +=item R: InnoDB Row Operations and Semaphores + +This mode shows InnoDB row operations, row operation miscellaneous, semaphores, +and information from the wait array. It displays the L<"row_operations">, +L<"row_operation_misc">, L<"semaphores">, and L<"wait_array"> tables by default. + +=item S: Variables & Status + +This mode calculates statistics, such as queries per second, and prints them out +in several different styles. You can show absolute values, or incremental values +between ticks. + +You can switch between the views by pressing a key. The 's' key prints a +single line each time the screen updates, in the style of B<vmstat>. The 'g' +key changes the view to a graph of the same numbers, sort of like B<tload>. +The 'v' key changes the view to a pivoted table of variable names on the left, +with successive updates scrolling across the screen from left to right. You can +choose how many updates to put on the screen with the L<"num_status_sets"> +configuration variable. + +Headers may be abbreviated to fit on the screen in interactive operation. You +choose which variables to display with the 'c' key, which selects from +predefined sets, or lets you create your own sets. You can edit the current set +with the 'e' key. + +This mode doesn't really display any tables like other modes. Instead, it uses +a table definition to extract and format the data, but it then transforms the +result in special ways before outputting it. It uses the L<"var_status"> table +definition for this. + +=item T: InnoDB Transactions + +This mode shows transactions from the InnoDB monitor's output, in B<top>-like +format. This mode is the reason I wrote innotop. + +You can kill queries or processes with the 'k' and 'x' keys, and EXPLAIN a query +with the 'e' or 'f' keys. InnoDB doesn't print the full query in transactions, +so explaining may not work right if the query is truncated. + +The informational header can be toggled on and off with the 'h' key. By +default, innotop hides inactive transactions and its own transaction. You can +toggle this on and off with the 'i' and 'a' keys. + +This mode displays the L<"t_header"> and L<"innodb_transactions"> tables by +default. + +=back + +=head1 INNOTOP STATUS + +The first line innotop displays is a "status bar" of sorts. What it contains +depends on the mode you're in, and what servers you're monitoring. The first +few words are always the innotop mode, such as "InnoDB Txns" for T mode, +followed by a reminder to press '?' for help at any time. + +=head2 ONE SERVER + +The simplest case is when you're monitoring a single server. In this case, the +name of the connection is next on the status line. This is the name you gave +when you created the connection -- most likely the MySQL server's hostname. +This is followed by the server's uptime. + +If you're in an InnoDB mode, such as T or B, the next word is "InnoDB" followed +by some information about the SHOW INNODB STATUS output used to render the +screen. The first word is the number of seconds since the last SHOW INNODB +STATUS, which InnoDB uses to calculate some per-second statistics. The next is +a smiley face indicating whether the InnoDB output is truncated. If the smiley +face is a :-), all is well; there is no truncation. A :^| means the transaction +list is so long, InnoDB has only printed out some of the transactions. Finally, +a frown :-( means the output is incomplete, which is probably due to a deadlock +printing too much lock information (see L<"D: InnoDB Deadlocks">). + +The next two words indicate the server's queries per second (QPS) and how many +threads (connections) exist. Finally, the server's version number is the last +thing on the line. + +=head2 MULTIPLE SERVERS + +If you are monitoring multiple servers (see L<"SERVER CONNECTIONS">), the status +line does not show any details about individual servers. Instead, it shows the +names of the connections that are active. Again, these are connection names you +specified, which are likely to be the server's hostname. A connection that has +an error is prefixed with an exclamation point. + +If you are monitoring a group of servers (see L<"SERVER GROUPS">), the status +line shows the name of the group. If any connection in the group has an +error, the group's name is followed by the fraction of the connections that +don't have errors. + +See L<"ERROR HANDLING"> for more details about innotop's error handling. + +=head2 MONITORING A FILE + +If you give a filename on the command line, innotop will not connect to ANY +servers at all. It will watch the specified file for InnoDB status output and +use that as its data source. It will always show a single connection called +'file'. And since it can't connect to a server, it can't determine how long the +server it's monitoring has been up; so it calculates the server's uptime as time +since innotop started running. + +=head1 SERVER ADMINISTRATION + +While innotop is primarily a monitor that lets you watch and analyze your +servers, it can also send commands to servers. The most frequently useful +commands are killing queries and stopping or starting slaves. + +You can kill a connection, or in newer versions of MySQL kill a query but not a +connection, from L<"Q: Query List"> and L<"T: InnoDB Transactions"> modes. +Press 'k' to issue a KILL command, or 'x' to issue a KILL QUERY command. +innotop will prompt you for the server and/or connection ID to kill (innotop +does not prompt you if there is only one possible choice for any input). +innotop pre-selects the longest-running query, or the oldest connection. +Confirm the command with 'y'. + +In L<"M: Master/Slave Replication Status"> mode, you can start and stop slaves +with the 'a' and 'o' keys, respectively. You can send these commands to many +slaves at once. innotop fills in a default command of START SLAVE or STOP SLAVE +for you, but you can actually edit the command and send anything you wish, such +as SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1 to make the slave skip one binlog event +when it starts. + +You can also ask innotop to calculate the earliest binlog in use by any slave +and issue a PURGE MASTER LOGS on the master. Use the 'b' key for this. innotop +will prompt you for a master to run the command on, then prompt you for the +connection names of that master's slaves (there is no way for innotop to +determine this reliably itself). innotop will find the minimum binlog in use by +these slave connections and suggest it as the argument to PURGE MASTER LOGS. + +=head1 SERVER CONNECTIONS + +When you create a server connection, innotop asks you for a series of inputs, as +follows: + +=over + +=item DSN + +A DSN is a Data Source Name, which is the initial argument passed to the DBI +module for connecting to a server. It is usually of the form + + DBI:mysql:;mysql_read_default_group=mysql;host=HOSTNAME + +Since this DSN is passed to the DBD::mysql driver, you should read the driver's +documentation at L<"http://search.cpan.org/dist/DBD-mysql/lib/DBD/mysql.pm"> for +the exact details on all the options you can pass the driver in the DSN. You +can read more about DBI at L<http://dbi.perl.org/docs/>, and especially at +L<http://search.cpan.org/~timb/DBI/DBI.pm>. + +The mysql_read_default_group=mysql option lets the DBD driver read your MySQL +options files, such as ~/.my.cnf on UNIX-ish systems. You can use this to avoid +specifying a username or password for the connection. + +=item InnoDB Deadlock Table + +This optional item tells innotop a table name it can use to deliberately create +a small deadlock (see L<"D: InnoDB Deadlocks">). If you specify this option, +you just need to be sure the table doesn't exist, and that innotop can create +and drop the table with the InnoDB storage engine. You can safely omit or just +accept the default if you don't intend to use this. + +=item Username + +innotop will ask you if you want to specify a username. If you say 'y', it will +then prompt you for a user name. If you have a MySQL option file that specifies +your username, you don't have to specify a username. + +The username defaults to your login name on the system you're running innotop on. + +=item Password + +innotop will ask you if you want to specify a password. Like the username, the +password is optional, but there's an additional prompt that asks if you want to +save the password in the innotop configuration file. If you don't save it in +the configuration file, innotop will prompt you for a password each time it +starts. Passwords in the innotop configuration file are saved in plain text, +not encrypted in any way. + +=back + +Once you finish answering these questions, you should be connected to a server. +But innotop isn't limited to monitoring a single server; you can define many +server connections and switch between them by pressing the '@' key. See +L<"SWITCHING BETWEEN CONNECTIONS">. + +To create a new connection, press the '@' key and type the name of the new +connection, then follow the steps given above. + +=head1 SERVER GROUPS + +If you have multiple MySQL instances, you can put them into named groups, such +as 'all', 'masters', and 'slaves', which innotop can monitor all together. + +You can choose which group to monitor with the '#' key, and you can press the +TAB key to switch to the next group. If you're not currently monitoring a +group, pressing TAB selects the first group. + +To create a group, press the '#' key and type the name of your new group, then +type the names of the connections you want the group to contain. + +=head1 SWITCHING BETWEEN CONNECTIONS + +innotop lets you quickly switch which servers you're monitoring. The most basic +way is by pressing the '@' key and typing the name(s) of the connection(s) you +want to use. This setting is per-mode, so you can monitor different connections +in each mode, and innotop remembers which connections you choose. + +You can quickly switch to the 'next' connection in alphabetical order with the +'n' key. If you're monitoring a server group (see L<"SERVER GROUPS">) this will +switch to the first connection. + +You can also type many connection names, and innotop will fetch and display data +from them all. Just separate the connection names with spaces, for example +"server1 server2." Again, if you type the name of a connection that doesn't +exist, innotop will prompt you for connection information and create the +connection. + +Another way to monitor multiple connections at once is with server groups. You +can use the TAB key to switch to the 'next' group in alphabetical order, or if +you're not monitoring any groups, TAB will switch to the first group. + +innotop does not fetch data in parallel from connections, so if you are +monitoring a large group or many connections, you may notice increased delay +between ticks. + +When you monitor more than one connection, innotop's status bar changes. See +L<"INNOTOP STATUS">. + +=head1 ERROR HANDLING + +Error handling is not that important when monitoring a single connection, but is +crucial when you have many active connections. A crashed server or lost +connection should not crash innotop. As a result, innotop will continue to run +even when there is an error; it just won't display any information from the +connection that had an error. Because of this, innotop's behavior might confuse +you. It's a feature, not a bug! + +innotop does not continue to query connections that have errors, because they +may slow innotop and make it hard to use, especially if the error is a problem +connecting and causes a long time-out. Instead, innotop retries the connection +occasionally to see if the error still exists. If so, it will wait until some +point in the future. The wait time increases in ticks as the Fibonacci series, +so it tries less frequently as time passes. + +Since errors might only happen in certain modes because of the SQL commands +issued in those modes, innotop keeps track of which mode caused the error. If +you switch to a different mode, innotop will retry the connection instead of +waiting. + +By default innotop will display the problem in red text at the bottom of the +first table on the screen. You can disable this behavior with the +L<"show_cxn_errors_in_tbl"> configuration option, which is enabled by default. +If the L<"debug"> option is enabled, innotop will display the error at the +bottom of every table, not just the first. And if L<"show_cxn_errors"> is +enabled, innotop will print the error text to STDOUT as well. Error messages +might only display in the mode that caused the error, depending on the mode and +whether innotop is avoiding querying that connection. + +=head1 NON-INTERACTIVE OPERATION + +You can run innotop in non-interactive mode, in which case it is entirely +controlled from the configuration file and command-line options. To start +innotop in non-interactive mode, give the L"<--nonint"> command-line option. +This changes innotop's behavior in the following ways: + +=over + +=item * + +Certain Perl modules are not loaded. Term::Readline is not loaded, since +innotop doesn't prompt interactively. Term::ANSIColor and Win32::Console::ANSI +modules are not loaded. Term::ReadKey is still used, since innotop may have to +prompt for connection passwords when starting up. + +=item * + +innotop does not clear the screen after each tick. + +=item * + +innotop does not persist any changes to the configuration file. + +=item * + +If L<"--count"> is given and innotop is in incremental mode (see L<"status_inc"> +and L<"--inc">), innotop actually refreshes one more time than specified so it +can print incremental statistics. This suppresses output during the first +tick, so innotop may appear to hang. + +=item * + +innotop only displays the first table in each mode. This is so the output can +be easily processed with other command-line utilities such as awk and sed. To +change which tables display in each mode, see L<"TABLES">. Since L<"Q: Query +List"> mode is so important, innotop automatically disables the L<"q_header"> +table. This ensures you'll see the L<"processlist"> table, even if you have +innotop configured to show the q_header table during interactive operation. +Similarly, in L<"T: InnoDB Transactions"> mode, the L<"t_header"> table is +suppressed so you see only the L<"innodb_transactions"> table. + +=item * + +All output is tab-separated instead of being column-aligned with whitespace, and +innotop prints the full contents of each table instead of only printing one +screenful at a time. + +=item * + +innotop only prints column headers once instead of every tick (see +L<"hide_hdr">). innotop does not print table captions (see +L<"display_table_captions">). innotop ensures there are no empty lines in the +output. + +=item * + +innotop does not honor the L<"shorten"> transformation, which normally shortens +some numbers to human-readable formats. + +=item * + +innotop does not print a status line (see L<"INNOTOP STATUS">). + +=back + +=head1 CONFIGURING + +Nearly everything about innotop is configurable. Most things are possible to +change with built-in commands, but you can also edit the configuration file. + +While running innotop, press the '$' key to bring up the configuration editing +dialog. Press another key to select the type of data you want to edit: + +=over + +=item S: Statement Sleep Times + +Edits SQL statement sleep delays, which make innotop pause for the specified +amount of time after executing a statement. See L<"SQL STATEMENTS"> for a +definition of each statement and what it does. By default innotop does not +delay after any statements. + +This feature is included so you can customize the side-effects caused by +monitoring your server. You may not see any effects, but some innotop users +have noticed that certain MySQL versions under very high load with InnoDB +enabled take longer than usual to execute SHOW GLOBAL STATUS. If innotop calls +SHOW FULL PROCESSLIST immediately afterward, the processlist contains more +queries than the machine actually averages at any given moment. Configuring +innotop to pause briefly after calling SHOW GLOBAL STATUS alleviates this +effect. + +Sleep times are stored in the L<"stmt_sleep_times"> section of the configuration +file. Fractional-second sleeps are supported, subject to your hardware's +limitations. + +=item c: Edit Columns + +Starts the table editor on one of the displayed tables. See L<"TABLE EDITOR">. +An alternative way to start the table editor without entering the configuration +dialog is with the '^' key. + +=item g: General Configuration + +Starts the configuration editor to edit global and mode-specific configuration +variables (see L<"MODES">). innotop prompts you to choose a variable from among +the global and mode-specific ones depending on the current mode. + +=item k: Row-Coloring Rules + +Starts the row-coloring rules editor on one of the displayed table(s). See +L<"COLORS"> for details. + +=item p: Manage Plugins + +Starts the plugin configuration editor. See L<"PLUGINS"> for details. + +=item s: Server Groups + +Lets you create and edit server groups. See L<"SERVER GROUPS">. + +=item t: Choose Displayed Tables + +Lets you choose which tables to display in this mode. See L<"MODES"> and +L<"TABLES">. + +=back + +=head1 CONFIGURATION FILE + +innotop's default configuration file location is in $HOME/.innotop, but can be +overridden with the L<"--config"> command-line option. You can edit it by hand +safely. innotop reads the configuration file when it starts, and writes it out +again when it exits, so any changes you make while innotop is running will be +lost. + +innotop doesn't store its entire configuration in the configuration file. It +has a huge set of default configuration that it holds only in memory, and the +configuration file only overrides these defaults. When you customize a default +setting, innotop notices, and then stores the customizations into the file. +This keeps the file size down, makes it easier to edit, and makes upgrades +easier. + +A configuration file can be made read-only. See L<"readonly">. + +The configuration file is arranged into sections like an INI file. Each +section begins with [section-name] and ends with [/section-name]. Each +section's entries have a different syntax depending on the data they need to +store. You can put comments in the file; any line that begins with a # +character is a comment. innotop will not read the comments, so it won't write +them back out to the file when it exits. Comments in read-only configuration +files are still useful, though. + +The first line in the file is innotop's version number. This lets innotop +notice when the file format is not backwards-compatible, and upgrade smoothly +without destroying your customized configuration. + +The following list describes each section of the configuration file and the data +it contains: + +=over + +=item general + +The 'general' section contains global configuration variables and variables that +may be mode-specific, but don't belong in any other section. The syntax is a +simple key=value list. innotop writes a comment above each value to help you +edit the file by hand. + +=over + +=item S_func + +Controls S mode presentation (see L<"S: Variables & Status">). If g, values are +graphed; if s, values are like vmstat; if p, values are in a pivoted table. + +=item S_set + +Specifies which set of variables to display in L<"S: Variables & Status"> mode. +See L<"VARIABLE SETS">. + +=item auto_wipe_dl + +Instructs innotop to automatically wipe large deadlocks when it notices them. +When this happens you may notice a slight delay. At the next tick, you will +usually see the information that was being truncated by the large deadlock. + +=item charset + +Specifies what kind of characters to allow through the L<"no_ctrl_char"> +transformation. This keeps non-printable characters from confusing a +terminal when you monitor queries that contain binary data, such as images. + +The default is 'ascii', which considers anything outside normal ASCII to be a +control character. The other allowable values are 'unicode' and 'none'. 'none' +considers every character a control character, which can be useful for +collapsing ALL text fields in queries. + +=item cmd_filter + +This is the prefix that filters variables in L<"C: Command Summary"> mode. + +=item color + +Whether terminal coloring is permitted. + +=item cxn_timeout + +On MySQL versions 4.0.3 and newer, this variable is used to set the connection's +timeout, so MySQL doesn't close the connection if it is not used for a while. +This might happen because a connection isn't monitored in a particular mode, for +example. + +=item debug + +This option enables more verbose errors and makes innotop more strict in some +places. It can help in debugging filters and other user-defined code. It also +makes innotop write a lot of information to L<"debugfile"> when there is a +crash. + +=item debugfile + +A file to which innotop will write information when there is a crash. See +L<"FILES">. + +=item display_table_captions + +innotop displays a table caption above most tables. This variable suppresses or +shows captions on all tables globally. Some tables are configured with the +hide_caption property, which overrides this. + +=item global + +Whether to show GLOBAL variables and status. innotop only tries to do this on +servers which support the GLOBAL option to SHOW VARIABLES and SHOW STATUS. In +some MySQL versions, you need certain privileges to do this; if you don't have +them, innotop will not be able to fetch any variable and status data. This +configuration variable lets you run innotop and fetch what data you can even +without the elevated privileges. + +I can no longer find or reproduce the situation where GLOBAL wasn't allowed, but +I know there was one. + +=item graph_char + +Defines the character to use when drawing graphs in L<"S: Variables & Status"> +mode. + +=item header_highlight + +Defines how to highlight column headers. This only works if Term::ANSIColor is +available. Valid values are 'bold' and 'underline'. + +=item hide_hdr + +Hides column headers globally. + +=item interval + +The interval at which innotop will refresh its data (ticks). The interval is +implemented as a sleep time between ticks, so the true interval will vary +depending on how long it takes innotop to fetch and render data. + +This variable accepts fractions of a second. + +=item mode + +The mode in which innotop should start. Allowable arguments are the same as the +key presses that select a mode interactively. See L<"MODES">. + +=item num_digits + +How many digits to show in fractional numbers and percents. This variable's +range is between 0 and 9 and can be set directly from L<"S: Variables & Status"> +mode with the '+' and '-' keys. It is used in the L<"set_precision">, +L<"shorten">, and L<"percent"> transformations. + +=item num_status_sets + +Controls how many sets of status variables to display in pivoted L<"S: Variables +& Status"> mode. It also controls the number of old sets of variables innotop +keeps in its memory, so the larger this variable is, the more memory innotop +uses. + +=item plugin_dir + +Specifies where plugins can be found. By default, innotop stores plugins in the +'plugins' subdirectory of your innotop configuration directory. + +=item readonly + +Whether the configuration file is readonly. This cannot be set interactively, +because it would prevent itself from being written to the configuration file. + +=item show_cxn_errors + +Makes innotop print connection errors to STDOUT. See L<"ERROR HANDLING">. + +=item show_cxn_errors_in_tbl + +Makes innotop display connection errors as rows in the first table on screen. +See L<"ERROR HANDLING">. + +=item show_percent + +Adds a '%' character after the value returned by the L<"percent"> +transformation. + +=item show_statusbar + +Controls whether to show the status bar in the display. See L<"INNOTOP +STATUS">. + +=item skip_innodb + +Disables fetching SHOW INNODB STATUS, in case your server(s) do not have InnoDB +enabled and you don't want innotop to try to fetch it. This can also be useful +when you don't have the SUPER privilege, required to run SHOW INNODB STATUS. + +=item status_inc + +Whether to show absolute or incremental values for status variables. +Incremental values are calculated as an offset from the last value innotop saw +for that variable. This is a global setting, but will probably become +mode-specific at some point. Right now it is honored a bit inconsistently; some +modes don't pay attention to it. + +=back + +=item plugins + +This section holds a list of package names of active plugins. If the plugin +exists, innotop will activate it. See L<"PLUGINS"> for more information. + +=item filters + +This section holds user-defined filters (see L<"FILTERS">). Each line is in the +format filter_name=text='filter text' tbls='table list'. + +The filter text is the text of the subroutine's code. The table list is a list +of tables to which the filter can apply. By default, user-defined filters apply +to the table for which they were created, but you can manually override that by +editing the definition in the configuration file. + +=item active_filters + +This section stores which filters are active on each table. Each line is in the +format table_name=filter_list. + +=item tbl_meta + +This section stores user-defined or user-customized columns (see L<"COLUMNS">). +Each line is in the format col_name=properties, where the properties are a +name=quoted-value list. + +=item connections + +This section holds the server connections you have defined. Each line is in the +format name=properties, where the properties are a name=value list. The +properties are self-explanatory, and the only one that is treated specially is +'pass' which is only present if 'savepass' is set. See L<"SERVER CONNECTIONS">. + +=item active_connections + +This section holds a list of which connections are active in each mode. Each +line is in the format mode_name=connection_list. + +=item server_groups + +This section holds server groups. Each line is in the format +name=connection_list. See L<"SERVER GROUPS">. + +=item active_server_groups + +This section holds a list of which server group is active in each mode. Each +line is in the format mode_name=server_group. + +=item max_values_seen + +This section holds the maximum values seen for variables. This is used to scale +the graphs in L<"S: Variables & Status"> mode. Each line is in the format +name=value. + +=item active_columns + +This section holds table column lists. Each line is in the format +tbl_name=column_list. See L<"COLUMNS">. + +=item sort_cols + +This section holds the sort definition. Each line is in the format +tbl_name=column_list. If a column is prefixed with '-', that column sorts +descending. See L<"SORTING">. + +=item visible_tables + +This section defines which tables are visible in each mode. Each line is in the +format mode_name=table_list. See L<"TABLES">. + +=item varsets + +This section defines variable sets for use in L<"S: Status & Variables"> mode. +Each line is in the format name=variable_list. See L<"VARIABLE SETS">. + +=item colors + +This section defines colorization rules. Each line is in the format +tbl_name=property_list. See L<"COLORS">. + +=item stmt_sleep_times + +This section contains statement sleep times. Each line is in the format +statement_name=sleep_time. See L<"S: Statement Sleep Times">. + +=item group_by + +This section contains column lists for table group_by expressions. Each line is +in the format tbl_name=column_list. See L<"GROUPING">. + +=back + +=head1 CUSTOMIZING + +You can customize innotop a great deal. For example, you can: + +=over + +=item * + +Choose which tables to display, and in what order. + +=item * + +Choose which columns are in those tables, and create new columns. + +=item * + +Filter which rows display with built-in filters, user-defined filters, and +quick-filters. + +=item * + +Sort the rows to put important data first or group together related rows. + +=item * + +Highlight rows with color. + +=item * + +Customize the alignment, width, and formatting of columns, and apply +transformations to columns to extract parts of their values or format the values +as you wish (for example, shortening large numbers to familiar units). + +=item * + +Design your own expressions to extract and combine data as you need. This gives +you unlimited flexibility. + +=back + +All these and more are explained in the following sections. + +=head2 TABLES + +A table is what you'd expect: a collection of columns. It also has some other +properties, such as a caption. Filters, sorting rules, and colorization rules +belong to tables and are covered in later sections. + +Internally, table meta-data is defined in a data structure called %tbl_meta. +This hash holds all built-in table definitions, which contain a lot of default +instructions to innotop. The meta-data includes the caption, a list of columns +the user has customized, a list of columns, a list of visible columns, a list of +filters, color rules, a sort-column list, sort direction, and some information +about the table's data sources. Most of this is customizable via the table +editor (see L<"TABLE EDITOR">). + +You can choose which tables to show by pressing the '$' key. See L<"MODES"> and +L<"TABLES">. + +The table life-cycle is as follows: + +=over + +=item * + +Each table begins with a data source, which is an array of hashes. See below +for details on data sources. + +=item * + +Each element of the data source becomes a row in the final table. + +=item * + +For each element in the data source, innotop extracts values from the source and +creates a row. This row is another hash, which later steps will refer to as +$set. The values innotop extracts are determined by the table's columns. Each +column has an extraction subroutine, compiled from an expression (see +L<"EXPRESSIONS">). The resulting row is a hash whose keys are named the same as +the column name. + +=item * + +innotop filters the rows, removing those that don't need to be displayed. See +L<"FILTERS">. + +=item * + +innotop sorts the rows. See L<"SORTING">. + +=item * + +innotop groups the rows together, if specified. See L<"GROUPING">. + +=item * + +innotop colorizes the rows. See L<"COLORS">. + +=item * + +innotop transforms the column values in each row. See L<"TRANSFORMATIONS">. + +=item * + +innotop optionally pivots the rows (see L<"PIVOTING">), then filters and sorts +them. + +=item * + +innotop formats and justifies the rows as a table. During this step, innotop +applies further formatting to the column values, including alignment, maximum +and minimum widths. innotop also does final error checking to ensure there are +no crashes due to undefined values. innotop then adds a caption if specified, +and the table is ready to print. + +=back + +The lifecycle is slightly different if the table is pivoted, as noted above. To +clarify, if the table is pivoted, the process is extract, group, transform, +pivot, filter, sort, create. If it's not pivoted, the process is extract, +filter, sort, group, color, transform, create. This slightly convoluted process +doesn't map all that well to SQL, but pivoting complicates things pretty +thoroughly. Roughly speaking, filtering and sorting happen as late as needed to +effect the final result as you might expect, but as early as possible for +efficiency. + +Each built-in table is described below: + +=over + +=item adaptive_hash_index + +Displays data about InnoDB's adaptive hash index. Data source: +L<"STATUS_VARIABLES">. + +=item buffer_pool + +Displays data about InnoDB's buffer pool. Data source: L<"STATUS_VARIABLES">. + +=item cmd_summary + +Displays weighted status variables. Data source: L<"STATUS_VARIABLES">. + +=item deadlock_locks + +Shows which locks were held and waited for by the last detected deadlock. Data +source: L<"DEADLOCK_LOCKS">. + +=item deadlock_transactions + +Shows transactions involved in the last detected deadlock. Data source: +L<"DEADLOCK_TRANSACTIONS">. + +=item explain + +Shows the output of EXPLAIN. Data source: L<"EXPLAIN">. + +=item file_io_misc + +Displays data about InnoDB's file and I/O operations. Data source: +L<"STATUS_VARIABLES">. + +=item fk_error + +Displays various data about InnoDB's last foreign key error. Data source: +L<"STATUS_VARIABLES">. + +=item innodb_locks + +Displays InnoDB locks. Data source: L<"INNODB_LOCKS">. + +=item innodb_transactions + +Displays data about InnoDB's current transactions. Data source: +L<"INNODB_TRANSACTIONS">. + +=item insert_buffers + +Displays data about InnoDB's insert buffer. Data source: L<"STATUS_VARIABLES">. + +=item io_threads + +Displays data about InnoDB's I/O threads. Data source: L<"IO_THREADS">. + +=item log_statistics + +Displays data about InnoDB's logging system. Data source: L<"STATUS_VARIABLES">. + +=item master_status + +Displays replication master status. Data source: L<"STATUS_VARIABLES">. + +=item open_tables + +Displays open tables. Data source: L<"OPEN_TABLES">. + +=item page_statistics + +Displays InnoDB page statistics. Data source: L<"STATUS_VARIABLES">. + +=item pending_io + +Displays InnoDB pending I/O operations. Data source: L<"STATUS_VARIABLES">. + +=item processlist + +Displays current MySQL processes (threads/connections). Data source: +L<"PROCESSLIST">. + +=item q_header + +Displays various status values. Data source: L<"STATUS_VARIABLES">. + +=item row_operation_misc + +Displays data about InnoDB's row operations. Data source: +L<"STATUS_VARIABLES">. + +=item row_operations + +Displays data about InnoDB's row operations. Data source: +L<"STATUS_VARIABLES">. + +=item semaphores + +Displays data about InnoDB's semaphores and mutexes. Data source: +L<"STATUS_VARIABLES">. + +=item slave_io_status + +Displays data about the slave I/O thread. Data source: +L<"STATUS_VARIABLES">. + +=item slave_sql_status + +Displays data about the slave SQL thread. Data source: L<"STATUS_VARIABLES">. + +=item t_header + +Displays various InnoDB status values. Data source: L<"STATUS_VARIABLES">. + +=item var_status + +Displays user-configurable data. Data source: L<"STATUS_VARIABLES">. + +=item wait_array + +Displays data about InnoDB's OS wait array. Data source: L<"OS_WAIT_ARRAY">. + +=back + +=head2 COLUMNS + +Columns belong to tables. You can choose a table's columns by pressing the '^' +key, which starts the L<"TABLE EDITOR"> and lets you choose and edit columns. +Pressing 'e' from within the table editor lets you edit the column's properties: + +=over + +=item * + +hdr: a column header. This appears in the first row of the table. + +=item * + +just: justification. '-' means left-justified and '' means right-justified, +just as with printf formatting codes (not a coincidence). + +=item * + +dec: whether to further align the column on the decimal point. + +=item * + +num: whether the column is numeric. This affects how values are sorted +(lexically or numerically). + +=item * + +label: a small note about the column, which appears in dialogs that help the +user choose columns. + +=item * + +src: an expression that innotop uses to extract the column's data from its +source (see L<"DATA SOURCES">). See L<"EXPRESSIONS"> for more on expressions. + +=item * + +minw: specifies a minimum display width. This helps stabilize the display, +which makes it easier to read if the data is changing frequently. + +=item * + +maxw: similar to minw. + +=item * + +trans: a list of column transformations. See L<"TRANSFORMATIONS">. + +=item * + +agg: an aggregate function. See L<"GROUPING">. The default is L<"first">. + +=item * + +aggonly: controls whether the column only shows when grouping is enabled on the +table (see L<"GROUPING">). By default, this is disabled. This means columns +will always be shown by default, whether grouping is enabled or not. If a +column's aggonly is set true, the column will appear when you toggle grouping on +the table. Several columns are set this way, such as the count column on +L<"processlist"> and L<"innodb_transactions">, so you don't see a count when the +grouping isn't enabled, but you do when it is. + +=back + +=head2 FILTERS + +Filters remove rows from the display. They behave much like a WHERE clause in +SQL. innotop has several built-in filters, which remove irrelevant information +like inactive queries, but you can define your own as well. innotop also lets +you create quick-filters, which do not get saved to the configuration file, and +are just an easy way to quickly view only some rows. + +You can enable or disable a filter on any table. Press the '%' key (mnemonic: % +looks kind of like a line being filtered between two circles) and choose which +table you want to filter, if asked. You'll then see a list of possible filters +and a list of filters currently enabled for that table. Type the names of +filters you want to apply and press Enter. + +=head3 USER-DEFINED FILTERS + +If you type a name that doesn't exist, innotop will prompt you to create the +filter. Filters are easy to create if you know Perl, and not hard if you don't. +What you're doing is creating a subroutine that returns true if the row should +be displayed. The row is a hash reference passed to your subroutine as $set. + +For example, imagine you want to filter the processlist table so you only see +queries that have been running more than five minutes. Type a new name for your +filter, and when prompted for the subroutine body, press TAB to initiate your +terminal's auto-completion. You'll see the names of the columns in the +L<"processlist"> table (innotop generally tries to help you with auto-completion +lists). You want to filter on the 'time' column. Type the text "$set->{time} > +300" to return true when the query is more than five minutes old. That's all +you need to do. + +In other words, the code you're typing is surrounded by an implicit context, +which looks like this: + + sub filter { + my ( $set ) = @_; + # YOUR CODE HERE + } + +If your filter doesn't work, or if something else suddenly behaves differently, +you might have made an error in your filter, and innotop is silently catching +the error. Try enabling L<"debug"> to make innotop throw an error instead. + +=head3 QUICK-FILTERS + +innotop's quick-filters are a shortcut to create a temporary filter that doesn't +persist when you restart innotop. To create a quick-filter, press the '/' key. +innotop will prompt you for the column name and filter text. Again, you can use +auto-completion on column names. The filter text can be just the text you want +to "search for." For example, to filter the L<"processlist"> table on queries +that refer to the products table, type '/' and then 'info product'. + +The filter text can actually be any Perl regular expression, but of course a +literal string like 'product' works fine as a regular expression. + +Behind the scenes innotop compiles the quick-filter into a specially tagged +filter that is otherwise like any other filter. It just isn't saved to the +configuration file. + +To clear quick-filters, press the '\' key and innotop will clear them all at +once. + +=head2 SORTING + +innotop has sensible built-in defaults to sort the most important rows to the +top of the table. Like anything else in innotop, you can customize how any +table is sorted. + +To start the sort dialog, start the L<"TABLE EDITOR"> with the '^' key, choose a +table if necessary, and press the 's' key. You'll see a list of columns you can +use in the sort expression and the current sort expression, if any. Enter a +list of columns by which you want to sort and press Enter. If you want to +reverse sort, prefix the column name with a minus sign. For example, if you +want to sort by column a ascending, then column b descending, type 'a -b'. You +can also explicitly add a + in front of columns you want to sort ascending, but +it's not required. + +Some modes have keys mapped to open this dialog directly, and to quickly reverse +sort direction. Press '?' as usual to see which keys are mapped in any mode. + +=head2 GROUPING + +innotop can group, or aggregate, rows together (I use the terms +interchangeably). This is quite similar to an SQL GROUP BY clause. You can +specify to group on certain columns, or if you don't specify any, the entire set +of rows is treated as one group. This is quite like SQL so far, but unlike SQL, +you can also select un-grouped columns. innotop actually aggregates every +column. If you don't explicitly specify a grouping function, the default is +'first'. This is basically a convenience so you don't have to specify an +aggregate function for every column you want in the result. + +You can quickly toggle grouping on a table with the '=' key, which toggles its +aggregate property. This property doesn't persist to the config file. + +The columns by which the table is grouped are specified in its group_by +property. When you turn grouping on, innotop places the group_by columns at the +far left of the table, even if they're not supposed to be visible. The rest of +the visible columns appear in order after them. + +Two tables have default group_by lists and a count column built in: +L<"processlist"> and L<"innodb_transactions">. The grouping is by connection +and status, so you can quickly see how many queries or transactions are in a +given status on each server you're monitoring. The time columns are aggregated +as a sum; other columns are left at the default 'first' aggregation. + +By default, the table shown in L<"S: Variables & Status"> mode also uses +grouping so you can monitor variables and status across many servers. The +default aggregation function in this mode is 'avg'. + +Valid grouping functions are defined in the %agg_funcs hash. They include + +=over + +=item first + +Returns the first element in the group. + +=item count + +Returns the number of elements in the group, including undefined elements, much +like SQL's COUNT(*). + +=item avg + +Returns the average of defined elements in the group. + +=item sum + +Returns the sum of elements in the group. + +=back + +Here's an example of grouping at work. Suppose you have a very busy server with +hundreds of open connections, and you want to see how many connections are in +what status. Using the built-in grouping rules, you can press 'Q' to enter +L<"Q: Query List"> mode. Press '=' to toggle grouping (if necessary, select the +L<"processlist"> table when prompted). + +Your display might now look like the following: + + Query List (? for help) localhost, 32:33, 0.11 QPS, 1 thd, 5.0.38-log + + CXN Cmd Cnt ID User Host Time Query + localhost Query 49 12933 webusr localhost 19:38 SELECT * FROM + localhost Sending Da 23 2383 webusr localhost 12:43 SELECT col1, + localhost Sleep 120 140 webusr localhost 5:18:12 + localhost Statistics 12 19213 webusr localhost 01:19 SELECT * FROM + +That's actually quite a worrisome picture. You've got a lot of idle connections +(Sleep), and some connections executing queries (Query and Sending Data). +That's okay, but you also have a lot in Statistics status, collectively spending +over a minute. That means the query optimizer is having a really hard time +optimizing your statements. Something is wrong; it should normally take +milliseconds to optimize queries. You might not have seen this pattern if you +didn't look at your connections in aggregate. (This is a made-up example, but +it can happen in real life). + +=head2 PIVOTING + +innotop can pivot a table for more compact display, similar to a Pivot Table in +a spreadsheet (also known as a crosstab). Pivoting a table makes columns into +rows. Assume you start with this table: + + foo bar + === === + 1 3 + 2 4 + +After pivoting, the table will look like this: + + name set0 set1 + ==== ==== ==== + foo 1 2 + bar 3 4 + +To get reasonable results, you might need to group as well as pivoting. +innotop currently does this for L<"S: Variables & Status"> mode. + +=head2 COLORS + +By default, innotop highlights rows with color so you can see at a glance which +rows are more important. You can customize the colorization rules and add your +own to any table. Open the table editor with the '^' key, choose a table if +needed, and press 'o' to open the color editor dialog. + +The color editor dialog displays the rules applied to the table, in the order +they are evaluated. Each row is evaluated against each rule to see if the rule +matches the row; if it does, the row gets the specified color, and no further +rules are evaluated. The rules look like the following: + + state eq Locked black on_red + cmd eq Sleep white + user eq system user white + cmd eq Connect white + cmd eq Binlog Dump white + time > 600 red + time > 120 yellow + time > 60 green + time > 30 cyan + +This is the default rule set for the L<"processlist"> table. In order of +priority, these rules make locked queries black on a red background, "gray out" +connections from replication and sleeping queries, and make queries turn from +cyan to red as they run longer. + +(For some reason, the ANSI color code "white" is actually a light gray. Your +terminal's display may vary; experiment to find colors you like). + +You can use keystrokes to move the rules up and down, which re-orders their +priority. You can also delete rules and add new ones. If you add a new rule, +innotop prompts you for the column, an operator for the comparison, a value +against which to compare the column, and a color to assign if the rule matches. +There is auto-completion and prompting at each step. + +The value in the third step needs to be correctly quoted. innotop does not try +to quote the value because it doesn't know whether it should treat the value as +a string or a number. If you want to compare the column against a string, as +for example in the first rule above, you should enter 'Locked' surrounded by +quotes. If you get an error message about a bareword, you probably should have +quoted something. + +=head2 EXPRESSIONS + +Expressions are at the core of how innotop works, and are what enables you to +extend innotop as you wish. Recall the table lifecycle explained in +L<"TABLES">. Expressions are used in the earliest step, where it extracts +values from a data source to form rows. + +It does this by calling a subroutine for each column, passing it the source data +set, a set of current values, and a set of previous values. These are all +needed so the subroutine can calculate things like the difference between this +tick and the previous tick. + +The subroutines that extract the data from the set are compiled from +expressions. This gives significantly more power than just naming the values to +fill the columns, because it allows the column's value to be calculated from +whatever data is necessary, but avoids the need to write complicated and lengthy +Perl code. + +innotop begins with a string of text that can look as simple as a value's name +or as complicated as a full-fledged Perl expression. It looks at each +'bareword' token in the string and decides whether it's supposed to be a key +into the $set hash. A bareword is an unquoted value that isn't already +surrounded by code-ish things like dollar signs or curly brackets. If innotop +decides that the bareword isn't a function or other valid Perl code, it converts +it into a hash access. After the whole string is processed, innotop compiles a +subroutine, like this: + + sub compute_column_value { + my ( $set, $cur, $pre ) = @_; + my $val = # EXPANDED STRING GOES HERE + return $val; + } + +Here's a concrete example, taken from the header table L<"q_header"> in L<"Q: +Query List"> mode. This expression calculates the qps, or Queries Per Second, +column's values, from the values returned by SHOW STATUS: + + Questions/Uptime_hires + +innotop decides both words are barewords, and transforms this expression into +the following Perl code: + + $set->{Questions}/$set->{Uptime_hires} + +When surrounded by the rest of the subroutine's code, this is executable Perl +that calculates a high-resolution queries-per-second value. + +The arguments to the subroutine are named $set, $cur, and $pre. In most cases, +$set and $cur will be the same values. However, if L<"status_inc"> is set, $cur +will not be the same as $set, because $set will already contain values that are +the incremental difference between $cur and $pre. + +Every column in innotop is computed by subroutines compiled in the same fashion. +There is no difference between innotop's built-in columns and user-defined +columns. This keeps things consistent and predictable. + +=head2 TRANSFORMATIONS + +Transformations change how a value is rendered. For example, they can take a +number of seconds and display it in H:M:S format. The following transformations +are defined: + +=over + +=item commify + +Adds commas to large numbers every three decimal places. + +=item dulint_to_int + +Accepts two unsigned integers and converts them into a single longlong. This is +useful for certain operations with InnoDB, which uses two integers as +transaction identifiers, for example. + +=item no_ctrl_char + +Removes quoted control characters from the value. This is affected by the +L<"charset"> configuration variable. + +This transformation only operates within quoted strings, for example, values to +a SET clause in an UPDATE statement. It will not alter the UPDATE statement, +but will collapse the quoted string to [BINARY] or [TEXT], depending on the +charset. + +=item percent + +Converts a number to a percentage by multiplying it by two, formatting it with +L<"num_digits"> digits after the decimal point, and optionally adding a percent +sign (see L<"show_percent">). + +=item secs_to_time + +Formats a number of seconds as time in days+hours:minutes:seconds format. + +=item set_precision + +Formats numbers with L<"num_digits"> number of digits after the decimal point. + +=item shorten + +Formats a number as a unit of 1024 (k/M/G/T) and with L<"num_digits"> number of +digits after the decimal point. + +=back + +=head2 TABLE EDITOR + +The innotop table editor lets you customize tables with keystrokes. You start +the table editor with the '^' key. If there's more than one table on the +screen, it will prompt you to choose one of them. Once you do, innotop will +show you something like this: + + Editing table definition for Buffer Pool. Press ? for help, q to quit. + + name hdr label src + cxn CXN Connection from which cxn + buf_pool_size Size Buffer pool size IB_bp_buf_poo + buf_free Free Bufs Buffers free in the b IB_bp_buf_fre + pages_total Pages Pages total IB_bp_pages_t + pages_modified Dirty Pages Pages modified (dirty IB_bp_pages_m + buf_pool_hit_rate Hit Rate Buffer pool hit rate IB_bp_buf_poo + total_mem_alloc Memory Total memory allocate IB_bp_total_m + add_pool_alloc Add'l Pool Additonal pool alloca IB_bp_add_poo + +The first line shows which table you're editing, and reminds you again to press +'?' for a list of key mappings. The rest is a tabular representation of the +table's columns, because that's likely what you're trying to edit. However, you +can edit more than just the table's columns; this screen can start the filter +editor, color rule editor, and more. + +Each row in the display shows a single column in the table you're editing, along +with a couple of its properties such as its header and source expression (see +L<"EXPRESSIONS">). + +The key mappings are Vim-style, as in many other places. Pressing 'j' and 'k' +moves the highlight up or down. You can then (d)elete or (e)dit the highlighted +column. You can also (a)dd a column to the table. This actually just activates +one of the columns already defined for the table; it prompts you to choose from +among the columns available but not currently displayed. Finally, you can +re-order the columns with the '+' and '-' keys. + +You can do more than just edit the columns with the table editor, you can also +edit other properties, such as the table's sort expression and group-by +expression. Press '?' to see the full list, of course. + +If you want to really customize and create your own column, as opposed to just +activating a built-in one that's not currently displayed, press the (n)ew key, +and innotop will prompt you for the information it needs: + +=over + +=item * + +The column name: this needs to be a word without any funny characters, e.g. just +letters, numbers and underscores. + +=item * + +The column header: this is the label that appears at the top of the column, in +the table header. This can have spaces and funny characters, but be careful not +to make it too wide and waste space on-screen. + +=item * + +The column's data source: this is an expression that determines what data from +the source (see L<"TABLES">) innotop will put into the column. This can just be +the name of an item in the source, or it can be a more complex expression, as +described in L<"EXPRESSIONS">. + +=back + +Once you've entered the required data, your table has a new column. There is no +difference between this column and the built-in ones; it can have all the same +properties and behaviors. innotop will write the column's definition to the +configuration file, so it will persist across sessions. + +Here's an example: suppose you want to track how many times your slaves have +retried transactions. According to the MySQL manual, the +Slave_retried_transactions status variable gives you that data: "The total +number of times since startup that the replication slave SQL thread has retried +transactions. This variable was added in version 5.0.4." This is appropriate to +add to the L<"slave_sql_status"> table. + +To add the column, switch to the replication-monitoring mode with the 'M' key, +and press the '^' key to start the table editor. When prompted, choose +slave_sql_status as the table, then press 'n' to create the column. Type +'retries' as the column name, 'Retries' as the column header, and +'Slave_retried_transactions' as the source. Now the column is created, and you +see the table editor screen again. Press 'q' to exit the table editor, and +you'll see your column at the end of the table. + +=head1 VARIABLE SETS + +Variable sets are used in L<"S: Variables & Status"> mode to define more easily +what variables you want to monitor. Behind the scenes they are compiled to a +list of expressions, and then into a column list so they can be treated just +like columns in any other table, in terms of data extraction and +transformations. However, you're protected from the tedious details by a syntax +that ought to feel very natural to you: a SQL SELECT list. + +The data source for variable sets, and indeed the entire S mode, is the +combination of SHOW STATUS, SHOW VARIABLES, and SHOW INNODB STATUS. Imagine +that you had a huge table with one column per variable returned from those +statements. That's the data source for variable sets. You can now query this +data source just like you'd expect. For example: + + Questions, Uptime, Questions/Uptime as QPS + +Behind the scenes innotop will split that variable set into three expressions, +compile them and turn them into a table definition, then extract as usual. This +becomes a "variable set," or a "list of variables you want to monitor." + +innotop lets you name and save your variable sets, and writes them to the +configuration file. You can choose which variable set you want to see with the +'c' key, or activate the next and previous sets with the '>' and '<' keys. +There are many built-in variable sets as well, which should give you a good +start for creating your own. Press 'e' to edit the current variable set, or +just to see how it's defined. To create a new one, just press 'c' and type its +name. + +You may want to use some of the functions listed in L<"TRANSFORMATIONS"> to help +format the results. In particular, L<"set_precision"> is often useful to limit +the number of digits you see. Extending the above example, here's how: + + Questions, Uptime, set_precision(Questions/Uptime) as QPS + +Actually, this still needs a little more work. If your L<"interval"> is less +than one second, you might be dividing by zero because Uptime is incremental in +this mode by default. Instead, use Uptime_hires: + + Questions, Uptime, set_precision(Questions/Uptime_hires) as QPS + +This example is simple, but it shows how easy it is to choose which variables +you want to monitor. + +=head1 PLUGINS + +innotop has a simple but powerful plugin mechanism by which you can extend +or modify its existing functionality, and add new functionality. innotop's +plugin functionality is event-based: plugins register themselves to be called +when events happen. They then have a chance to influence the event. + +An innotop plugin is a Perl module placed in innotop's L<"plugin_dir"> +directory. On UNIX systems, you can place a symbolic link to the module instead +of putting the actual file there. innotop automatically discovers the file. If +there is a corresponding entry in the L<"plugins"> configuration file section, +innotop loads and activates the plugin. + +The module must conform to innotop's plugin interface. Additionally, the source +code of the module must be written in such a way that innotop can inspect the +file and determine the package name and description. + +=head2 Package Source Convention + +innotop inspects the plugin module's source to determine the Perl package name. +It looks for a line of the form "package Foo;" and if found, considers the +plugin's package name to be Foo. Of course the package name can be a valid Perl +package name, with double semicolons and so on. + +It also looks for a description in the source code, to make the plugin editor +more human-friendly. The description is a comment line of the form "# +description: Foo", where "Foo" is the text innotop will consider to be the +plugin's description. + +=head2 Plugin Interface + +The innotop plugin interface is quite simple: innotop expects the plugin to be +an object-oriented module it can call certain methods on. The methods are + +=over + +=item new(%variables) + +This is the plugin's constructor. It is passed a hash of innotop's variables, +which it can manipulate (see L<"Plugin Variables">). It must return a reference +to the newly created plugin object. + +At construction time, innotop has only loaded the general configuration and +created the default built-in variables with their default contents (which is +quite a lot). Therefore, the state of the program is exactly as in the innotop +source code, plus the configuration variables from the L<"general"> section in +the config file. + +If your plugin manipulates the variables, it is changing global data, which is +shared by innotop and all plugins. Plugins are loaded in the order they're +listed in the config file. Your plugin may load before or after another plugin, +so there is a potential for conflict or interaction between plugins if they +modify data other plugins use or modify. + +=item register_for_events() + +This method must return a list of events in which the plugin is interested, if +any. See L<"Plugin Events"> for the defined events. If the plugin returns an +event that's not defined, the event is ignored. + +=item event handlers + +The plugin must implement a method named the same as each event for which it has +registered. In other words, if the plugin returns qw(foo bar) from +register_for_events(), it must have foo() and bar() methods. These methods are +callbacks for the events. See L<"Plugin Events"> for more details about each +event. + +=back + +=head2 Plugin Variables + +The plugin's constructor is passed a hash of innotop's variables, which it can +manipulate. It is probably a good idea if the plugin object saves a copy of it +for later use. The variables are defined in the innotop variable +%pluggable_vars, and are as follows: + +=over + +=item action_for + +A hashref of key mappings. These are innotop's global hot-keys. + +=item agg_funcs + +A hashref of functions that can be used for grouping. See L<"GROUPING">. + +=item config + +The global configuration hash. + +=item connections + +A hashref of connection specifications. These are just specifications of how to +connect to a server. + +=item dbhs + +A hashref of innotop's database connections. These are actual DBI connection +objects. + +=item filters + +A hashref of filters applied to table rows. See L<"FILTERS"> for more. + +=item modes + +A hashref of modes. See L<"MODES"> for more. + +=item server_groups + +A hashref of server groups. See L<"SERVER GROUPS">. + +=item tbl_meta + +A hashref of innotop's table meta-data, with one entry per table (see +L<"TABLES"> for more information). + +=item trans_funcs + +A hashref of transformation functions. See L<"TRANSFORMATIONS">. + +=item var_sets + +A hashref of variable sets. See L<"VARIABLE SETS">. + +=back + +=head2 Plugin Events + +Each event is defined somewhere in the innotop source code. When innotop runs +that code, it executes the callback function for each plugin that expressed its +interest in the event. innotop passes some data for each event. The events are +defined in the %event_listener_for variable, and are as follows: + +=over + +=item extract_values($set, $cur, $pre, $tbl) + +This event occurs inside the function that extracts values from a data source. +The arguments are the set of values, the current values, the previous values, +and the table name. + +=item set_to_tbl + +Events are defined at many places in this subroutine, which is responsible for +turning an arrayref of hashrefs into an arrayref of lines that can be printed to +the screen. The events all pass the same data: an arrayref of rows and the name +of the table being created. The events are set_to_tbl_pre_filter, +set_to_tbl_pre_sort,set_to_tbl_pre_group, set_to_tbl_pre_colorize, +set_to_tbl_pre_transform, set_to_tbl_pre_pivot, set_to_tbl_pre_create, +set_to_tbl_post_create. + +=item draw_screen($lines) + +This event occurs inside the subroutine that prints the lines to the screen. +$lines is an arrayref of strings. + +=back + +=head2 Simple Plugin Example + +The easiest way to explain the plugin functionality is probably with a simple +example. The following module adds a column to the beginning of every table and +sets its value to 1. + + use strict; + use warnings FATAL => 'all'; + + package Innotop::Plugin::Example; + # description: Adds an 'example' column to every table + + sub new { + my ( $class, %vars ) = @_; + # Store reference to innotop's variables in $self + my $self = bless { %vars }, $class; + + # Design the example column + my $col = { + hdr => 'Example', + just => '', + dec => 0, + num => 1, + label => 'Example', + src => 'example', # Get data from this column in the data source + tbl => '', + trans => [], + }; + + # Add the column to every table. + my $tbl_meta = $vars{tbl_meta}; + foreach my $tbl ( values %$tbl_meta ) { + # Add the column to the list of defined columns + $tbl->{cols}->{example} = $col; + # Add the column to the list of visible columns + unshift @{$tbl->{visible}}, 'example'; + } + + # Be sure to return a reference to the object. + return $self; + } + + # I'd like to be called when a data set is being rendered into a table, please. + sub register_for_events { + my ( $self ) = @_; + return qw(set_to_tbl_pre_filter); + } + + # This method will be called when the event fires. + sub set_to_tbl_pre_filter { + my ( $self, $rows, $tbl ) = @_; + # Set the example column's data source to the value 1. + foreach my $row ( @$rows ) { + $row->{example} = 1; + } + } + + 1; + +=head2 Plugin Editor + +The plugin editor lets you view the plugins innotop discovered and activate or +deactivate them. Start the editor by pressing $ to start the configuration +editor from any mode. Press the 'p' key to start the plugin editor. You'll see +a list of plugins innotop discovered. You can use the 'j' and 'k' keys to move +the highlight to the desired one, then press the * key to toggle it active or +inactive. Exit the editor and restart innotop for the changes to take effect. + +=head1 SQL STATEMENTS + +innotop uses a limited set of SQL statements to retrieve data from MySQL for +display. The statements are customized depending on the server version against +which they are executed; for example, on MySQL 5 and newer, INNODB_STATUS +executes "SHOW ENGINE INNODB STATUS", while on earlier versions it executes +"SHOW INNODB STATUS". The statements are as follows: + + Statement SQL executed + =================== =============================== + INNODB_STATUS SHOW [ENGINE] INNODB STATUS + KILL_CONNECTION KILL + KILL_QUERY KILL QUERY + OPEN_TABLES SHOW OPEN TABLES + PROCESSLIST SHOW FULL PROCESSLIST + SHOW_MASTER_LOGS SHOW MASTER LOGS + SHOW_MASTER_STATUS SHOW MASTER STATUS + SHOW_SLAVE_STATUS SHOW SLAVE STATUS + SHOW_STATUS SHOW [GLOBAL] STATUS + SHOW_VARIABLES SHOW [GLOBAL] VARIABLES + +=head1 DATA SOURCES + +Each time innotop extracts values to create a table (see L<"EXPRESSIONS"> and +L<"TABLES">), it does so from a particular data source. Largely because of the +complex data extracted from SHOW INNODB STATUS, this is slightly messy. SHOW +INNODB STATUS contains a mixture of single values and repeated values that form +nested data sets. + +Whenever innotop fetches data from MySQL, it adds two extra bits to each set: +cxn and Uptime_hires. cxn is the name of the connection from which the data +came. Uptime_hires is a high-resolution version of the server's Uptime status +variable, which is important if your L<"interval"> setting is sub-second. + +Here are the kinds of data sources from which data is extracted: + +=over + +=item STATUS_VARIABLES + +This is the broadest category, into which the most kinds of data fall. It +begins with the combination of SHOW STATUS and SHOW VARIABLES, but other sources +may be included as needed, for example, SHOW MASTER STATUS and SHOW SLAVE +STATUS, as well as many of the non-repeated values from SHOW INNODB STATUS. + +=item DEADLOCK_LOCKS + +This data is extracted from the transaction list in the LATEST DETECTED DEADLOCK +section of SHOW INNODB STATUS. It is nested two levels deep: transactions, then +locks. + +=item DEADLOCK_TRANSACTIONS + +This data is from the transaction list in the LATEST DETECTED DEADLOCK +section of SHOW INNODB STATUS. It is nested one level deep. + +=item EXPLAIN + +This data is from the result set returned by EXPLAIN. + +=item INNODB_TRANSACTIONS + +This data is from the TRANSACTIONS section of SHOW INNODB STATUS. + +=item IO_THREADS + +This data is from the list of threads in the the FILE I/O section of SHOW INNODB +STATUS. + +=item INNODB_LOCKS + +This data is from the TRANSACTIONS section of SHOW INNODB STATUS and is nested +two levels deep. + +=item OPEN_TABLES + +This data is from SHOW OPEN TABLES. + +=item PROCESSLIST + +This data is from SHOW FULL PROCESSLIST. + +=item OS_WAIT_ARRAY + +This data is from the SEMAPHORES section of SHOW INNODB STATUS and is nested one +level deep. It comes from the lines that look like this: + + --Thread 1568861104 has waited at btr0cur.c line 424 .... + +=back + +=head1 MYSQL PRIVILEGES + +=over + +=item * + +You must connect to MySQL as a user who has the SUPER privilege for many of the +functions. + +=item * + +If you don't have the SUPER privilege, you can still run some functions, but you +won't necessarily see all the same data. + +=item * + +You need the PROCESS privilege to see the list of currently running queries in Q +mode. + +=item * + +You need special privileges to start and stop slave servers. + +=item * + +You need appropriate privileges to create and drop the deadlock tables if needed +(see L<"SERVER CONNECTIONS">). + +=back + +=head1 SYSTEM REQUIREMENTS + +You need Perl to run innotop, of course. You also need a few Perl modules: DBI, +DBD::mysql, Term::ReadKey, and Time::HiRes. These should be included with most +Perl distributions, but in case they are not, I recommend using versions +distributed with your operating system or Perl distribution, not from CPAN. +Term::ReadKey in particular has been known to cause problems if installed from +CPAN. + +If you have Term::ANSIColor, innotop will use it to format headers more readably +and compactly. (Under Microsoft Windows, you also need Win32::Console::ANSI for +terminal formatting codes to be honored). If you install Term::ReadLine, +preferably Term::ReadLine::Gnu, you'll get nice auto-completion support. + +I run innotop on Gentoo GNU/Linux, Debian and Ubuntu, and I've had feedback from +people successfully running it on Red Hat, CentOS, Solaris, and Mac OSX. I +don't see any reason why it won't work on other UNIX-ish operating systems, but +I don't know for sure. It also runs on Windows under ActivePerl without +problem. + +I use innotop on MySQL versions 3.23.58, 4.0.27, 4.1.0, 4.1.22, 5.0.26, 5.1.15, +and 5.2.3. If it doesn't run correctly for you, that is a bug and I hope you +report it. + +=head1 FILES + +$HOMEDIR/.innotop is used to store configuration information. Files include the +configuration file innotop.ini, the core_dump file which contains verbose error +messages if L<"debug"> is enabled, and the plugins/ subdirectory. + +=head1 GLOSSARY OF TERMS + +=over + +=item tick + +A tick is a refresh event, when innotop re-fetches data from connections and +displays it. + +=back + +=head1 ACKNOWLEDGEMENTS + +I'm grateful to the following people for various reasons, and hope I haven't +forgotten to include anyone: + +Allen K. Smith, +Aurimas Mikalauskas, +Bartosz Fenski, +Brian Miezejewski, +Christian Hammers, +Cyril Scetbon, +Dane Miller, +David Multer, +Dr. Frank Ullrich, +Giuseppe Maxia, +Google.com Site Reliability Engineers, +Jan Pieter Kunst, +Jari Aalto, +Jay Pipes, +Jeremy Zawodny, +Johan Idren, +Kristian Kohntopp, +Lenz Grimmer, +Maciej Dobrzanski, +Michiel Betel, +MySQL AB, +Paul McCullagh, +Sebastien Estienne, +Sourceforge.net, +Steven Kreuzer, +The Gentoo MySQL Team, +Trevor Price, +Yaar Schnitman, +and probably more people I've neglected to include. + +(If I misspelled your name, it's probably because I'm afraid of putting +international characters into this documentation; earlier versions of Perl might +not be able to compile it then). + +=head1 COPYRIGHT, LICENSE AND WARRANTY + +This program is copyright (c) 2006 Baron Schwartz. +Feedback and improvements are welcome. + +THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED +WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF +MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation, version 2; OR the Perl Artistic License. On UNIX and similar +systems, you can issue `man perlgpl' or `man perlartistic' to read these +licenses. + +You should have received a copy of the GNU General Public License along with +this program; if not, write to the Free Software Foundation, Inc., 59 Temple +Place, Suite 330, Boston, MA 02111-1307 USA. + +Execute innotop and press '!' to see this information at any time. + +=head1 AUTHOR + +Baron Schwartz. + +=head1 BUGS + +You can report bugs, ask for improvements, and get other help and support at +L<http://sourceforge.net/projects/innotop>. There are mailing lists, forums, +a bug tracker, etc. Please use these instead of contacting me directly, as it +makes my job easier and benefits others if the discussions are permanent and +public. Of course, if you need to contact me in private, please do. + +=cut === added file 'storage/xtradb/build/debian/additions/innotop/innotop.1' --- a/storage/xtradb/build/debian/additions/innotop/innotop.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/innotop/innotop.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,2086 @@ +.\" Automatically generated by Pod::Man v1.37, Pod::Parser v1.32 +.\" +.\" Standard preamble: +.\" ======================================================================== +.de Sh \" Subsection heading +.br +.if t .Sp +.ne 5 +.PP +\fB\\$1\fR +.PP +.. +.de Sp \" Vertical space (when we can't use .PP) +.if t .sp .5v +.if n .sp +.. +.de Vb \" Begin verbatim text +.ft CW +.nf +.ne \\$1 +.. +.de Ve \" End verbatim text +.ft R +.fi +.. +.\" Set up some character translations and predefined strings. \*(-- will +.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left +.\" double quote, and \*(R" will give a right double quote. \*(C+ will +.\" give a nicer C++. Capital omega is used to do unbreakable dashes and +.\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, +.\" nothing in troff, for use with C<>. +.tr \(*W- +.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' +.ie n \{\ +. ds -- \(*W- +. ds PI pi +. if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch +. if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch +. ds L" "" +. ds R" "" +. ds C` "" +. ds C' "" +'br\} +.el\{\ +. ds -- \|\(em\| +. ds PI \(*p +. ds L" `` +. ds R" '' +'br\} +.\" +.\" If the F register is turned on, we'll generate index entries on stderr for +.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index +.\" entries marked with X<> in POD. Of course, you'll have to process the +.\" output yourself in some meaningful fashion. +.if \nF \{\ +. de IX +. tm Index:\\$1\t\\n%\t"\\$2" +.. +. nr % 0 +. rr F +.\} +.\" +.\" For nroff, turn off justification. Always turn off hyphenation; it makes +.\" way too many mistakes in technical documents. +.hy 0 +.if n .na +.\" +.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). +.\" Fear. Run. Save yourself. No user-serviceable parts. +. \" fudge factors for nroff and troff +.if n \{\ +. ds #H 0 +. ds #V .8m +. ds #F .3m +. ds #[ \f1 +. ds #] \fP +.\} +.if t \{\ +. ds #H ((1u-(\\\\n(.fu%2u))*.13m) +. ds #V .6m +. ds #F 0 +. ds #[ \& +. ds #] \& +.\} +. \" simple accents for nroff and troff +.if n \{\ +. ds ' \& +. ds ` \& +. ds ^ \& +. ds , \& +. ds ~ ~ +. ds / +.\} +.if t \{\ +. ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" +. ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' +. ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' +. ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' +. ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' +. ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' +.\} +. \" troff and (daisy-wheel) nroff accents +.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' +.ds 8 \h'\*(#H'\(*b\h'-\*(#H' +.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] +.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' +.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' +.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] +.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] +.ds ae a\h'-(\w'a'u*4/10)'e +.ds Ae A\h'-(\w'A'u*4/10)'E +. \" corrections for vroff +.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' +.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' +. \" for low resolution devices (crt and lpr) +.if \n(.H>23 .if \n(.V>19 \ +\{\ +. ds : e +. ds 8 ss +. ds o a +. ds d- d\h'-1'\(ga +. ds D- D\h'-1'\(hy +. ds th \o'bp' +. ds Th \o'LP' +. ds ae ae +. ds Ae AE +.\} +.rm #[ #] #H #V #F C +.\" ======================================================================== +.\" +.IX Title "INNOTOP 1p" +.TH INNOTOP 1p "2007-11-09" "perl v5.8.8" "User Contributed Perl Documentation" +.SH "NAME" +innotop \- MySQL and InnoDB transaction/status monitor. +.SH "SYNOPSIS" +.IX Header "SYNOPSIS" +To monitor servers normally: +.PP +.Vb 1 +\& innotop +.Ve +.PP +To monitor InnoDB status information from a file: +.PP +.Vb 1 +\& innotop /var/log/mysql/mysqld.err +.Ve +.PP +To run innotop non-interactively in a pipe-and-filter configuration: +.PP +.Vb 1 +\& innotop \-\-count 5 \-d 1 \-n +.Ve +.SH "DESCRIPTION" +.IX Header "DESCRIPTION" +innotop monitors MySQL servers. Each of its modes shows you a different aspect +of what's happening in the server. For example, there's a mode for monitoring +replication, one for queries, and one for transactions. innotop refreshes its +data periodically, so you see an updating view. +.PP +innotop has lots of features for power users, but you can start and run it with +virtually no configuration. If you're just getting started, see +\&\*(L"\s-1QUICK\-START\s0\*(R". Press '?' at any time while running innotop for +context-sensitive help. +.SH "QUICK-START" +.IX Header "QUICK-START" +To start innotop, open a terminal or command prompt. If you have installed +innotop on your system, you should be able to just type \*(L"innotop\*(R" and press +Enter; otherwise, you will need to change to innotop's directory and type \*(L"perl +innotop\*(R". +.PP +The first thing innotop needs to know is how to connect to a MySQL server. You +can just enter the hostname of the server, for example \*(L"localhost\*(R" or +\&\*(L"127.0.0.1\*(R" if the server is on the same machine as innotop. After this innotop +will prompt you for a \s-1DSN\s0 (data source name). You should be able to just accept +the defaults by pressing Enter. +.PP +When innotop asks you about a table to use when resetting InnoDB deadlock +information, just accept the default for now. This is an advanced feature you +can configure later (see \*(L"D: InnoDB Deadlocks\*(R" for more). +.PP +If you have a .my.cnf file with your MySQL connection defaults, innotop can read +it, and you won't need to specify a username and password if it's in that file. +Otherwise, you should answer 'y' to the next couple of prompts. +.PP +After this, you should be connected, and innotop should show you something like +the following: +.PP +.Vb 1 +\& InnoDB Txns (? for help) localhost, 01:11:19, InnoDB 10s :\-), 50 QPS, +.Ve +.PP +.Vb 2 +\& CXN History Versions Undo Dirty Buf Used Bufs Txns MaxTxn +\& localhost 7 2035 0 0 0.00% 92.19% 1 07:34 +.Ve +.PP +.Vb 5 +\& CXN ID User Host Txn Status Time Undo Query Tex +\& localhost 98379 user1 webserver ACTIVE 07:34 0 SELECT `c +\& localhost 98450 user1 webserver ACTIVE 01:06 0 INSERT IN +\& localhost 97750 user1 webserver not starte 00:00 0 +\& localhost 98375 user1 appserver not starte 00:00 0 +.Ve +.PP +(This sample is truncated at the right so it will fit on a terminal when running +\&'man innotop') +.PP +This sample comes from a quiet server with few transactions active. If your +server is busy, you'll see more output. Notice the first line on the screen, +which tells you what mode you're in and what server you're connected to. You +can change to other modes with keystrokes; press 'Q' to switch to a list of +currently running queries. +.PP +Press the '?' key to see what keys are active in the current mode. You can +press any of these keys and innotop will either take the requested action or +prompt you for more input. If your system has Term::ReadLine support, you can +use \s-1TAB\s0 and other keys to auto-complete and edit input. +.PP +To quit innotop, press the 'q' key. +.SH "OPTIONS" +.IX Header "OPTIONS" +innotop is mostly configured via its configuration file, but some of the +configuration options can come from the command line. You can also specify a +file to monitor for InnoDB status output; see \*(L"\s-1MONITORING\s0 A \s-1FILE\s0\*(R" for more +details. +.PP +You can negate some options by prefixing the option name with \-\-no. For +example, \-\-noinc (or \-\-no\-inc) negates \*(L"\-\-inc\*(R". +.IP "\-\-help" 4 +.IX Item "--help" +Print a summary of command-line usage and exit. +.IP "\-\-color" 4 +.IX Item "--color" +Enable or disable terminal coloring. Corresponds to the \*(L"color\*(R" config file +setting. +.IP "\-\-config" 4 +.IX Item "--config" +Specifies a configuration file to read. This option is non\-sticky, that is to +say it does not persist to the configuration file itself. +.IP "\-\-nonint" 4 +.IX Item "--nonint" +Enable non-interactive operation. See \*(L"\s-1NON\-INTERACTIVE\s0 \s-1OPERATION\s0\*(R" for more. +.IP "\-\-count" 4 +.IX Item "--count" +Refresh only the specified number of times (ticks) before exiting. Each refresh +is a pause for \*(L"interval\*(R" seconds, followed by requesting data from MySQL +connections and printing it to the terminal. +.IP "\-\-delay" 4 +.IX Item "--delay" +Specifies the amount of time to pause between ticks (refreshes). Corresponds to +the configuration option \*(L"interval\*(R". +.IP "\-\-mode" 4 +.IX Item "--mode" +Specifies the mode in which innotop should start. Corresponds to the +configuration option \*(L"mode\*(R". +.IP "\-\-inc" 4 +.IX Item "--inc" +Specifies whether innotop should display absolute numbers or relative numbers +(offsets from their previous values). Corresponds to the configuration option +\&\*(L"status_inc\*(R". +.IP "\-\-version" 4 +.IX Item "--version" +Output version information and exit. +.SH "HOTKEYS" +.IX Header "HOTKEYS" +innotop is interactive, and you control it with key\-presses. +.IP "\(bu" 4 +Uppercase keys switch between modes. +.IP "\(bu" 4 +Lowercase keys initiate some action within the current mode. +.IP "\(bu" 4 +Other keys do something special like change configuration or show the +innotop license. +.PP +Press '?' at any time to see the currently active keys and what they do. +.SH "MODES" +.IX Header "MODES" +Each of innotop's modes retrieves and displays a particular type of data from +the servers you're monitoring. You switch between modes with uppercase keys. +The following is a brief description of each mode, in alphabetical order. To +switch to the mode, press the key listed in front of its heading in the +following list: +.IP "B: InnoDB Buffers" 4 +.IX Item "B: InnoDB Buffers" +This mode displays information about the InnoDB buffer pool, page statistics, +insert buffer, and adaptive hash index. The data comes from \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. +.Sp +This mode contains the \*(L"buffer_pool\*(R", \*(L"page_statistics\*(R", +\&\*(L"insert_buffers\*(R", and \*(L"adaptive_hash_index\*(R" tables by default. +.IP "C: Command Summary" 4 +.IX Item "C: Command Summary" +This mode is similar to mytop's Command Summary mode. It shows the +\&\*(L"cmd_summary\*(R" table, which looks something like the following: +.Sp +.Vb 8 +\& Command Summary (? for help) localhost, 25+07:16:43, 2.45 QPS, 3 thd, 5.0.40 +\& _____________________ Command Summary _____________________ +\& Name Value Pct Last Incr Pct +\& Select_scan 3244858 69.89% 2 100.00% +\& Select_range 1354177 29.17% 0 0.00% +\& Select_full_join 39479 0.85% 0 0.00% +\& Select_full_range_join 4097 0.09% 0 0.00% +\& Select_range_check 0 0.00% 0 0.00% +.Ve +.Sp +The command summary table is built by extracting variables from +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". The variables must be numeric and must match the prefix +given by the \*(L"cmd_filter\*(R" configuration variable. The variables are then +sorted by value descending and compared to the last variable, as shown above. +The percentage columns are percentage of the total of all variables in the +table, so you can see the relative weight of the variables. +.Sp +The example shows what you see if the prefix is \*(L"Select_\*(R". The default +prefix is \*(L"Com_\*(R". You can choose a prefix with the 's' key. +.Sp +It's rather like running \s-1SHOW\s0 \s-1VARIABLES\s0 \s-1LIKE\s0 \*(L"prefix%\*(R" with memory and +nice formatting. +.Sp +Values are aggregated across all servers. The Pct columns are not correctly +aggregated across multiple servers. This is a known limitation of the grouping +algorithm that may be fixed in the future. +.IP "D: InnoDB Deadlocks" 4 +.IX Item "D: InnoDB Deadlocks" +This mode shows the transactions involved in the last InnoDB deadlock. A second +table shows the locks each transaction held and waited for. A deadlock is +caused by a cycle in the waits-for graph, so there should be two locks held and +one waited for unless the deadlock information is truncated. +.Sp +InnoDB puts deadlock information before some other information in the \s-1SHOW\s0 +\&\s-1INNODB\s0 \s-1STATUS\s0 output. If there are a lot of locks, the deadlock information can +grow very large, and there is a limit on the size of the \s-1SHOW\s0 \s-1INNODB\s0 +\&\s-1STATUS\s0 output. A large deadlock can fill the entire output, or even be +truncated, and prevent you from seeing other information at all. If you are +running innotop in another mode, for example T mode, and suddenly you don't see +anything, you might want to check and see if a deadlock has wiped out the data +you need. +.Sp +If it has, you can create a small deadlock to replace the large one. Use the +\&'w' key to 'wipe' the large deadlock with a small one. This will not work +unless you have defined a deadlock table for the connection (see \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R"). +.Sp +You can also configure innotop to automatically detect when a large deadlock +needs to be replaced with a small one (see \*(L"auto_wipe_dl\*(R"). +.Sp +This mode displays the \*(L"deadlock_transactions\*(R" and \*(L"deadlock_locks\*(R" tables +by default. +.IP "F: InnoDB Foreign Key Errors" 4 +.IX Item "F: InnoDB Foreign Key Errors" +This mode shows the last InnoDB foreign key error information, such as the +table where it happened, when and who and what query caused it, and so on. +.Sp +InnoDB has a huge variety of foreign key error messages, and many of them are +just hard to parse. innotop doesn't always do the best job here, but there's +so much code devoted to parsing this messy, unparseable output that innotop is +likely never to be perfect in this regard. If innotop doesn't show you what +you need to see, just look at the status text directly. +.Sp +This mode displays the \*(L"fk_error\*(R" table by default. +.IP "I: InnoDB I/O Info" 4 +.IX Item "I: InnoDB I/O Info" +This mode shows InnoDB's I/O statistics, including the I/O threads, pending I/O, +file I/O miscellaneous, and log statistics. It displays the \*(L"io_threads\*(R", +\&\*(L"pending_io\*(R", \*(L"file_io_misc\*(R", and \*(L"log_statistics\*(R" tables by default. +.IP "L: Locks" 4 +.IX Item "L: Locks" +This mode shows information about current locks. At the moment only InnoDB +locks are supported, and by default you'll only see locks for which transactions +are waiting. This information comes from the \s-1TRANSACTIONS\s0 section of the InnoDB +status text. If you have a very busy server, you may have frequent lock waits; +it helps to be able to see which tables and indexes are the \*(L"hot spot\*(R" for +locks. If your server is running pretty well, this mode should show nothing. +.Sp +You can configure MySQL and innotop to monitor not only locks for which a +transaction is waiting, but those currently held, too. You can do this with the +InnoDB Lock Monitor (<http://dev.mysql.com/doc/en/innodb\-monitor.html>). It's +not documented in the MySQL manual, but creating the lock monitor with the +following statement also affects the output of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0, which innotop +uses: +.Sp +.Vb 1 +\& CREATE TABLE innodb_lock_monitor(a int) ENGINE=INNODB; +.Ve +.Sp +This causes InnoDB to print its output to the MySQL file every 16 seconds or so, +as stated in the manual, but it also makes the normal \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 output +include lock information, which innotop can parse and display (that's the +undocumented feature). +.Sp +This means you can do what may have seemed impossible: to a limited extent +(InnoDB truncates some information in the output), you can see which transaction +holds the locks something else is waiting for. You can also enable and disable +the InnoDB Lock Monitor with the key mappings in this mode. +.Sp +This mode displays the \*(L"innodb_locks\*(R" table by default. Here's a sample of +the screen when one connection is waiting for locks another connection holds: +.Sp +.Vb 7 +\& _________________________________ InnoDB Locks __________________________ +\& CXN ID Type Waiting Wait Active Mode DB Table Index +\& localhost 12 RECORD 1 00:10 00:10 X test t1 PRIMARY +\& localhost 12 TABLE 0 00:10 00:10 IX test t1 +\& localhost 12 RECORD 1 00:10 00:10 X test t1 PRIMARY +\& localhost 11 TABLE 0 00:00 00:25 IX test t1 +\& localhost 11 RECORD 0 00:00 00:25 X test t1 PRIMARY +.Ve +.Sp +You can see the first connection, \s-1ID\s0 12, is waiting for a lock on the \s-1PRIMARY\s0 +key on test.t1, and has been waiting for 10 seconds. The second connection +isn't waiting, because the Waiting column is 0, but it holds locks on the same +index. That tells you connection 11 is blocking connection 12. +.IP "M: Master/Slave Replication Status" 4 +.IX Item "M: Master/Slave Replication Status" +This mode shows the output of \s-1SHOW\s0 \s-1SLAVE\s0 \s-1STATUS\s0 and \s-1SHOW\s0 \s-1MASTER\s0 \s-1STATUS\s0 in three +tables. The first two divide the slave's status into \s-1SQL\s0 and I/O thread status, +and the last shows master status. Filters are applied to eliminate non-slave +servers from the slave tables, and non-master servers from the master table. +.Sp +This mode displays the \*(L"slave_sql_status\*(R", \*(L"slave_io_status\*(R", and +\&\*(L"master_status\*(R" tables by default. +.IP "O: Open Tables" 4 +.IX Item "O: Open Tables" +This section comes from MySQL's \s-1SHOW\s0 \s-1OPEN\s0 \s-1TABLES\s0 command. By default it is +filtered to show tables which are in use by one or more queries, so you can +get a quick look at which tables are 'hot'. You can use this to guess which +tables might be locked implicitly. +.Sp +This mode displays the \*(L"open_tables\*(R" mode by default. +.IP "Q: Query List" 4 +.IX Item "Q: Query List" +This mode displays the output from \s-1SHOW\s0 \s-1FULL\s0 \s-1PROCESSLIST\s0, much like \fBmytop\fR's +query list mode. This mode does \fBnot\fR show InnoDB-related information. This +is probably one of the most useful modes for general usage. +.Sp +There is an informative header that shows general status information about +your server. You can toggle it on and off with the 'h' key. By default, +innotop hides inactive processes and its own process. You can toggle these on +and off with the 'i' and 'a' keys. +.Sp +You can \s-1EXPLAIN\s0 a query from this mode with the 'e' key. This displays the +query's full text, the results of \s-1EXPLAIN\s0, and in newer MySQL versions, even +the optimized query resulting from \s-1EXPLAIN\s0 \s-1EXTENDED\s0. innotop also tries to +rewrite certain queries to make them EXPLAIN\-able. For example, \s-1INSERT/SELECT\s0 +statements are rewritable. +.Sp +This mode displays the \*(L"q_header\*(R" and \*(L"processlist\*(R" tables by default. +.IP "R: InnoDB Row Operations and Semaphores" 4 +.IX Item "R: InnoDB Row Operations and Semaphores" +This mode shows InnoDB row operations, row operation miscellaneous, semaphores, +and information from the wait array. It displays the \*(L"row_operations\*(R", +\&\*(L"row_operation_misc\*(R", \*(L"semaphores\*(R", and \*(L"wait_array\*(R" tables by default. +.IP "S: Variables & Status" 4 +.IX Item "S: Variables & Status" +This mode calculates statistics, such as queries per second, and prints them out +in several different styles. You can show absolute values, or incremental values +between ticks. +.Sp +You can switch between the views by pressing a key. The 's' key prints a +single line each time the screen updates, in the style of \fBvmstat\fR. The 'g' +key changes the view to a graph of the same numbers, sort of like \fBtload\fR. +The 'v' key changes the view to a pivoted table of variable names on the left, +with successive updates scrolling across the screen from left to right. You can +choose how many updates to put on the screen with the \*(L"num_status_sets\*(R" +configuration variable. +.Sp +Headers may be abbreviated to fit on the screen in interactive operation. You +choose which variables to display with the 'c' key, which selects from +predefined sets, or lets you create your own sets. You can edit the current set +with the 'e' key. +.Sp +This mode doesn't really display any tables like other modes. Instead, it uses +a table definition to extract and format the data, but it then transforms the +result in special ways before outputting it. It uses the \*(L"var_status\*(R" table +definition for this. +.IP "T: InnoDB Transactions" 4 +.IX Item "T: InnoDB Transactions" +This mode shows transactions from the InnoDB monitor's output, in \fBtop\fR\-like +format. This mode is the reason I wrote innotop. +.Sp +You can kill queries or processes with the 'k' and 'x' keys, and \s-1EXPLAIN\s0 a query +with the 'e' or 'f' keys. InnoDB doesn't print the full query in transactions, +so explaining may not work right if the query is truncated. +.Sp +The informational header can be toggled on and off with the 'h' key. By +default, innotop hides inactive transactions and its own transaction. You can +toggle this on and off with the 'i' and 'a' keys. +.Sp +This mode displays the \*(L"t_header\*(R" and \*(L"innodb_transactions\*(R" tables by +default. +.SH "INNOTOP STATUS" +.IX Header "INNOTOP STATUS" +The first line innotop displays is a \*(L"status bar\*(R" of sorts. What it contains +depends on the mode you're in, and what servers you're monitoring. The first +few words are always the innotop mode, such as \*(L"InnoDB Txns\*(R" for T mode, +followed by a reminder to press '?' for help at any time. +.Sh "\s-1ONE\s0 \s-1SERVER\s0" +.IX Subsection "ONE SERVER" +The simplest case is when you're monitoring a single server. In this case, the +name of the connection is next on the status line. This is the name you gave +when you created the connection \*(-- most likely the MySQL server's hostname. +This is followed by the server's uptime. +.PP +If you're in an InnoDB mode, such as T or B, the next word is \*(L"InnoDB\*(R" followed +by some information about the \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 output used to render the +screen. The first word is the number of seconds since the last \s-1SHOW\s0 \s-1INNODB\s0 +\&\s-1STATUS\s0, which InnoDB uses to calculate some per-second statistics. The next is +a smiley face indicating whether the InnoDB output is truncated. If the smiley +face is a :\-), all is well; there is no truncation. A :^| means the transaction +list is so long, InnoDB has only printed out some of the transactions. Finally, +a frown :\-( means the output is incomplete, which is probably due to a deadlock +printing too much lock information (see \*(L"D: InnoDB Deadlocks\*(R"). +.PP +The next two words indicate the server's queries per second (\s-1QPS\s0) and how many +threads (connections) exist. Finally, the server's version number is the last +thing on the line. +.Sh "\s-1MULTIPLE\s0 \s-1SERVERS\s0" +.IX Subsection "MULTIPLE SERVERS" +If you are monitoring multiple servers (see \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R"), the status +line does not show any details about individual servers. Instead, it shows the +names of the connections that are active. Again, these are connection names you +specified, which are likely to be the server's hostname. A connection that has +an error is prefixed with an exclamation point. +.PP +If you are monitoring a group of servers (see \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R"), the status +line shows the name of the group. If any connection in the group has an +error, the group's name is followed by the fraction of the connections that +don't have errors. +.PP +See \*(L"\s-1ERROR\s0 \s-1HANDLING\s0\*(R" for more details about innotop's error handling. +.Sh "\s-1MONITORING\s0 A \s-1FILE\s0" +.IX Subsection "MONITORING A FILE" +If you give a filename on the command line, innotop will not connect to \s-1ANY\s0 +servers at all. It will watch the specified file for InnoDB status output and +use that as its data source. It will always show a single connection called +\&'file'. And since it can't connect to a server, it can't determine how long the +server it's monitoring has been up; so it calculates the server's uptime as time +since innotop started running. +.SH "SERVER ADMINISTRATION" +.IX Header "SERVER ADMINISTRATION" +While innotop is primarily a monitor that lets you watch and analyze your +servers, it can also send commands to servers. The most frequently useful +commands are killing queries and stopping or starting slaves. +.PP +You can kill a connection, or in newer versions of MySQL kill a query but not a +connection, from \*(L"Q: Query List\*(R" and \*(L"T: InnoDB Transactions\*(R" modes. +Press 'k' to issue a \s-1KILL\s0 command, or 'x' to issue a \s-1KILL\s0 \s-1QUERY\s0 command. +innotop will prompt you for the server and/or connection \s-1ID\s0 to kill (innotop +does not prompt you if there is only one possible choice for any input). +innotop pre-selects the longest-running query, or the oldest connection. +Confirm the command with 'y'. +.PP +In \*(L"M: Master/Slave Replication Status\*(R" mode, you can start and stop slaves +with the 'a' and 'o' keys, respectively. You can send these commands to many +slaves at once. innotop fills in a default command of \s-1START\s0 \s-1SLAVE\s0 or \s-1STOP\s0 \s-1SLAVE\s0 +for you, but you can actually edit the command and send anything you wish, such +as \s-1SET\s0 \s-1GLOBAL\s0 SQL_SLAVE_SKIP_COUNTER=1 to make the slave skip one binlog event +when it starts. +.PP +You can also ask innotop to calculate the earliest binlog in use by any slave +and issue a \s-1PURGE\s0 \s-1MASTER\s0 \s-1LOGS\s0 on the master. Use the 'b' key for this. innotop +will prompt you for a master to run the command on, then prompt you for the +connection names of that master's slaves (there is no way for innotop to +determine this reliably itself). innotop will find the minimum binlog in use by +these slave connections and suggest it as the argument to \s-1PURGE\s0 \s-1MASTER\s0 \s-1LOGS\s0. +.SH "SERVER CONNECTIONS" +.IX Header "SERVER CONNECTIONS" +When you create a server connection, innotop asks you for a series of inputs, as +follows: +.IP "\s-1DSN\s0" 4 +.IX Item "DSN" +A \s-1DSN\s0 is a Data Source Name, which is the initial argument passed to the \s-1DBI\s0 +module for connecting to a server. It is usually of the form +.Sp +.Vb 1 +\& DBI:mysql:;mysql_read_default_group=mysql;host=HOSTNAME +.Ve +.Sp +Since this \s-1DSN\s0 is passed to the DBD::mysql driver, you should read the driver's +documentation at \*(L"http://search.cpan.org/dist/DBD\-mysql/lib/DBD/mysql.pm\*(R" for +the exact details on all the options you can pass the driver in the \s-1DSN\s0. You +can read more about \s-1DBI\s0 at <http://dbi.perl.org/docs/>, and especially at +<http://search.cpan.org/~timb/DBI/DBI.pm>. +.Sp +The mysql_read_default_group=mysql option lets the \s-1DBD\s0 driver read your MySQL +options files, such as ~/.my.cnf on UNIX-ish systems. You can use this to avoid +specifying a username or password for the connection. +.IP "InnoDB Deadlock Table" 4 +.IX Item "InnoDB Deadlock Table" +This optional item tells innotop a table name it can use to deliberately create +a small deadlock (see \*(L"D: InnoDB Deadlocks\*(R"). If you specify this option, +you just need to be sure the table doesn't exist, and that innotop can create +and drop the table with the InnoDB storage engine. You can safely omit or just +accept the default if you don't intend to use this. +.IP "Username" 4 +.IX Item "Username" +innotop will ask you if you want to specify a username. If you say 'y', it will +then prompt you for a user name. If you have a MySQL option file that specifies +your username, you don't have to specify a username. +.Sp +The username defaults to your login name on the system you're running innotop on. +.IP "Password" 4 +.IX Item "Password" +innotop will ask you if you want to specify a password. Like the username, the +password is optional, but there's an additional prompt that asks if you want to +save the password in the innotop configuration file. If you don't save it in +the configuration file, innotop will prompt you for a password each time it +starts. Passwords in the innotop configuration file are saved in plain text, +not encrypted in any way. +.PP +Once you finish answering these questions, you should be connected to a server. +But innotop isn't limited to monitoring a single server; you can define many +server connections and switch between them by pressing the '@' key. See +\&\*(L"\s-1SWITCHING\s0 \s-1BETWEEN\s0 \s-1CONNECTIONS\s0\*(R". +.PP +To create a new connection, press the '@' key and type the name of the new +connection, then follow the steps given above. +.SH "SERVER GROUPS" +.IX Header "SERVER GROUPS" +If you have multiple MySQL instances, you can put them into named groups, such +as 'all', 'masters', and 'slaves', which innotop can monitor all together. +.PP +You can choose which group to monitor with the '#' key, and you can press the +\&\s-1TAB\s0 key to switch to the next group. If you're not currently monitoring a +group, pressing \s-1TAB\s0 selects the first group. +.PP +To create a group, press the '#' key and type the name of your new group, then +type the names of the connections you want the group to contain. +.SH "SWITCHING BETWEEN CONNECTIONS" +.IX Header "SWITCHING BETWEEN CONNECTIONS" +innotop lets you quickly switch which servers you're monitoring. The most basic +way is by pressing the '@' key and typing the name(s) of the connection(s) you +want to use. This setting is per\-mode, so you can monitor different connections +in each mode, and innotop remembers which connections you choose. +.PP +You can quickly switch to the 'next' connection in alphabetical order with the +\&'n' key. If you're monitoring a server group (see \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R") this will +switch to the first connection. +.PP +You can also type many connection names, and innotop will fetch and display data +from them all. Just separate the connection names with spaces, for example +\&\*(L"server1 server2.\*(R" Again, if you type the name of a connection that doesn't +exist, innotop will prompt you for connection information and create the +connection. +.PP +Another way to monitor multiple connections at once is with server groups. You +can use the \s-1TAB\s0 key to switch to the 'next' group in alphabetical order, or if +you're not monitoring any groups, \s-1TAB\s0 will switch to the first group. +.PP +innotop does not fetch data in parallel from connections, so if you are +monitoring a large group or many connections, you may notice increased delay +between ticks. +.PP +When you monitor more than one connection, innotop's status bar changes. See +\&\*(L"\s-1INNOTOP\s0 \s-1STATUS\s0\*(R". +.SH "ERROR HANDLING" +.IX Header "ERROR HANDLING" +Error handling is not that important when monitoring a single connection, but is +crucial when you have many active connections. A crashed server or lost +connection should not crash innotop. As a result, innotop will continue to run +even when there is an error; it just won't display any information from the +connection that had an error. Because of this, innotop's behavior might confuse +you. It's a feature, not a bug! +.PP +innotop does not continue to query connections that have errors, because they +may slow innotop and make it hard to use, especially if the error is a problem +connecting and causes a long time\-out. Instead, innotop retries the connection +occasionally to see if the error still exists. If so, it will wait until some +point in the future. The wait time increases in ticks as the Fibonacci series, +so it tries less frequently as time passes. +.PP +Since errors might only happen in certain modes because of the \s-1SQL\s0 commands +issued in those modes, innotop keeps track of which mode caused the error. If +you switch to a different mode, innotop will retry the connection instead of +waiting. +.PP +By default innotop will display the problem in red text at the bottom of the +first table on the screen. You can disable this behavior with the +\&\*(L"show_cxn_errors_in_tbl\*(R" configuration option, which is enabled by default. +If the \*(L"debug\*(R" option is enabled, innotop will display the error at the +bottom of every table, not just the first. And if \*(L"show_cxn_errors\*(R" is +enabled, innotop will print the error text to \s-1STDOUT\s0 as well. Error messages +might only display in the mode that caused the error, depending on the mode and +whether innotop is avoiding querying that connection. +.SH "NON-INTERACTIVE OPERATION" +.IX Header "NON-INTERACTIVE OPERATION" +You can run innotop in non-interactive mode, in which case it is entirely +controlled from the configuration file and command-line options. To start +innotop in non-interactive mode, give the L\*(L"<\-\-nonint\*(R"> command-line option. +This changes innotop's behavior in the following ways: +.IP "\(bu" 4 +Certain Perl modules are not loaded. Term::Readline is not loaded, since +innotop doesn't prompt interactively. Term::ANSIColor and Win32::Console::ANSI +modules are not loaded. Term::ReadKey is still used, since innotop may have to +prompt for connection passwords when starting up. +.IP "\(bu" 4 +innotop does not clear the screen after each tick. +.IP "\(bu" 4 +innotop does not persist any changes to the configuration file. +.IP "\(bu" 4 +If \*(L"\-\-count\*(R" is given and innotop is in incremental mode (see \*(L"status_inc\*(R" +and \*(L"\-\-inc\*(R"), innotop actually refreshes one more time than specified so it +can print incremental statistics. This suppresses output during the first +tick, so innotop may appear to hang. +.IP "\(bu" 4 +innotop only displays the first table in each mode. This is so the output can +be easily processed with other command-line utilities such as awk and sed. To +change which tables display in each mode, see \*(L"\s-1TABLES\s0\*(R". Since \*(L"Q: Query List\*(R" mode is so important, innotop automatically disables the \*(L"q_header\*(R" +table. This ensures you'll see the \*(L"processlist\*(R" table, even if you have +innotop configured to show the q_header table during interactive operation. +Similarly, in \*(L"T: InnoDB Transactions\*(R" mode, the \*(L"t_header\*(R" table is +suppressed so you see only the \*(L"innodb_transactions\*(R" table. +.IP "\(bu" 4 +All output is tab-separated instead of being column-aligned with whitespace, and +innotop prints the full contents of each table instead of only printing one +screenful at a time. +.IP "\(bu" 4 +innotop only prints column headers once instead of every tick (see +\&\*(L"hide_hdr\*(R"). innotop does not print table captions (see +\&\*(L"display_table_captions\*(R"). innotop ensures there are no empty lines in the +output. +.IP "\(bu" 4 +innotop does not honor the \*(L"shorten\*(R" transformation, which normally shortens +some numbers to human-readable formats. +.IP "\(bu" 4 +innotop does not print a status line (see \*(L"\s-1INNOTOP\s0 \s-1STATUS\s0\*(R"). +.SH "CONFIGURING" +.IX Header "CONFIGURING" +Nearly everything about innotop is configurable. Most things are possible to +change with built-in commands, but you can also edit the configuration file. +.PP +While running innotop, press the '$' key to bring up the configuration editing +dialog. Press another key to select the type of data you want to edit: +.IP "S: Statement Sleep Times" 4 +.IX Item "S: Statement Sleep Times" +Edits \s-1SQL\s0 statement sleep delays, which make innotop pause for the specified +amount of time after executing a statement. See \*(L"\s-1SQL\s0 \s-1STATEMENTS\s0\*(R" for a +definition of each statement and what it does. By default innotop does not +delay after any statements. +.Sp +This feature is included so you can customize the side-effects caused by +monitoring your server. You may not see any effects, but some innotop users +have noticed that certain MySQL versions under very high load with InnoDB +enabled take longer than usual to execute \s-1SHOW\s0 \s-1GLOBAL\s0 \s-1STATUS\s0. If innotop calls +\&\s-1SHOW\s0 \s-1FULL\s0 \s-1PROCESSLIST\s0 immediately afterward, the processlist contains more +queries than the machine actually averages at any given moment. Configuring +innotop to pause briefly after calling \s-1SHOW\s0 \s-1GLOBAL\s0 \s-1STATUS\s0 alleviates this +effect. +.Sp +Sleep times are stored in the \*(L"stmt_sleep_times\*(R" section of the configuration +file. Fractional-second sleeps are supported, subject to your hardware's +limitations. +.IP "c: Edit Columns" 4 +.IX Item "c: Edit Columns" +Starts the table editor on one of the displayed tables. See \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R". +An alternative way to start the table editor without entering the configuration +dialog is with the '^' key. +.IP "g: General Configuration" 4 +.IX Item "g: General Configuration" +Starts the configuration editor to edit global and mode-specific configuration +variables (see \*(L"\s-1MODES\s0\*(R"). innotop prompts you to choose a variable from among +the global and mode-specific ones depending on the current mode. +.IP "k: Row-Coloring Rules" 4 +.IX Item "k: Row-Coloring Rules" +Starts the row-coloring rules editor on one of the displayed table(s). See +\&\*(L"\s-1COLORS\s0\*(R" for details. +.IP "p: Manage Plugins" 4 +.IX Item "p: Manage Plugins" +Starts the plugin configuration editor. See \*(L"\s-1PLUGINS\s0\*(R" for details. +.IP "s: Server Groups" 4 +.IX Item "s: Server Groups" +Lets you create and edit server groups. See \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R". +.IP "t: Choose Displayed Tables" 4 +.IX Item "t: Choose Displayed Tables" +Lets you choose which tables to display in this mode. See \*(L"\s-1MODES\s0\*(R" and +\&\*(L"\s-1TABLES\s0\*(R". +.SH "CONFIGURATION FILE" +.IX Header "CONFIGURATION FILE" +innotop's default configuration file location is in \f(CW$HOME\fR/.innotop, but can be +overridden with the \*(L"\-\-config\*(R" command-line option. You can edit it by hand +safely. innotop reads the configuration file when it starts, and writes it out +again when it exits, so any changes you make while innotop is running will be +lost. +.PP +innotop doesn't store its entire configuration in the configuration file. It +has a huge set of default configuration that it holds only in memory, and the +configuration file only overrides these defaults. When you customize a default +setting, innotop notices, and then stores the customizations into the file. +This keeps the file size down, makes it easier to edit, and makes upgrades +easier. +.PP +A configuration file can be made read\-only. See \*(L"readonly\*(R". +.PP +The configuration file is arranged into sections like an \s-1INI\s0 file. Each +section begins with [section\-name] and ends with [/section\-name]. Each +section's entries have a different syntax depending on the data they need to +store. You can put comments in the file; any line that begins with a # +character is a comment. innotop will not read the comments, so it won't write +them back out to the file when it exits. Comments in read-only configuration +files are still useful, though. +.PP +The first line in the file is innotop's version number. This lets innotop +notice when the file format is not backwards\-compatible, and upgrade smoothly +without destroying your customized configuration. +.PP +The following list describes each section of the configuration file and the data +it contains: +.IP "general" 4 +.IX Item "general" +The 'general' section contains global configuration variables and variables that +may be mode\-specific, but don't belong in any other section. The syntax is a +simple key=value list. innotop writes a comment above each value to help you +edit the file by hand. +.RS 4 +.IP "S_func" 4 +.IX Item "S_func" +Controls S mode presentation (see \*(L"S: Variables & Status\*(R"). If g, values are +graphed; if s, values are like vmstat; if p, values are in a pivoted table. +.IP "S_set" 4 +.IX Item "S_set" +Specifies which set of variables to display in \*(L"S: Variables & Status\*(R" mode. +See \*(L"\s-1VARIABLE\s0 \s-1SETS\s0\*(R". +.IP "auto_wipe_dl" 4 +.IX Item "auto_wipe_dl" +Instructs innotop to automatically wipe large deadlocks when it notices them. +When this happens you may notice a slight delay. At the next tick, you will +usually see the information that was being truncated by the large deadlock. +.IP "charset" 4 +.IX Item "charset" +Specifies what kind of characters to allow through the \*(L"no_ctrl_char\*(R" +transformation. This keeps non-printable characters from confusing a +terminal when you monitor queries that contain binary data, such as images. +.Sp +The default is 'ascii', which considers anything outside normal \s-1ASCII\s0 to be a +control character. The other allowable values are 'unicode' and 'none'. 'none' +considers every character a control character, which can be useful for +collapsing \s-1ALL\s0 text fields in queries. +.IP "cmd_filter" 4 +.IX Item "cmd_filter" +This is the prefix that filters variables in \*(L"C: Command Summary\*(R" mode. +.IP "color" 4 +.IX Item "color" +Whether terminal coloring is permitted. +.IP "cxn_timeout" 4 +.IX Item "cxn_timeout" +On MySQL versions 4.0.3 and newer, this variable is used to set the connection's +timeout, so MySQL doesn't close the connection if it is not used for a while. +This might happen because a connection isn't monitored in a particular mode, for +example. +.IP "debug" 4 +.IX Item "debug" +This option enables more verbose errors and makes innotop more strict in some +places. It can help in debugging filters and other user-defined code. It also +makes innotop write a lot of information to \*(L"debugfile\*(R" when there is a +crash. +.IP "debugfile" 4 +.IX Item "debugfile" +A file to which innotop will write information when there is a crash. See +\&\*(L"\s-1FILES\s0\*(R". +.IP "display_table_captions" 4 +.IX Item "display_table_captions" +innotop displays a table caption above most tables. This variable suppresses or +shows captions on all tables globally. Some tables are configured with the +hide_caption property, which overrides this. +.IP "global" 4 +.IX Item "global" +Whether to show \s-1GLOBAL\s0 variables and status. innotop only tries to do this on +servers which support the \s-1GLOBAL\s0 option to \s-1SHOW\s0 \s-1VARIABLES\s0 and \s-1SHOW\s0 \s-1STATUS\s0. In +some MySQL versions, you need certain privileges to do this; if you don't have +them, innotop will not be able to fetch any variable and status data. This +configuration variable lets you run innotop and fetch what data you can even +without the elevated privileges. +.Sp +I can no longer find or reproduce the situation where \s-1GLOBAL\s0 wasn't allowed, but +I know there was one. +.IP "graph_char" 4 +.IX Item "graph_char" +Defines the character to use when drawing graphs in \*(L"S: Variables & Status\*(R" +mode. +.IP "header_highlight" 4 +.IX Item "header_highlight" +Defines how to highlight column headers. This only works if Term::ANSIColor is +available. Valid values are 'bold' and 'underline'. +.IP "hide_hdr" 4 +.IX Item "hide_hdr" +Hides column headers globally. +.IP "interval" 4 +.IX Item "interval" +The interval at which innotop will refresh its data (ticks). The interval is +implemented as a sleep time between ticks, so the true interval will vary +depending on how long it takes innotop to fetch and render data. +.Sp +This variable accepts fractions of a second. +.IP "mode" 4 +.IX Item "mode" +The mode in which innotop should start. Allowable arguments are the same as the +key presses that select a mode interactively. See \*(L"\s-1MODES\s0\*(R". +.IP "num_digits" 4 +.IX Item "num_digits" +How many digits to show in fractional numbers and percents. This variable's +range is between 0 and 9 and can be set directly from \*(L"S: Variables & Status\*(R" +mode with the '+' and '\-' keys. It is used in the \*(L"set_precision\*(R", +\&\*(L"shorten\*(R", and \*(L"percent\*(R" transformations. +.IP "num_status_sets" 4 +.IX Item "num_status_sets" +Controls how many sets of status variables to display in pivoted \*(L"S: Variables & Status\*(R" mode. It also controls the number of old sets of variables innotop +keeps in its memory, so the larger this variable is, the more memory innotop +uses. +.IP "plugin_dir" 4 +.IX Item "plugin_dir" +Specifies where plugins can be found. By default, innotop stores plugins in the +\&'plugins' subdirectory of your innotop configuration directory. +.IP "readonly" 4 +.IX Item "readonly" +Whether the configuration file is readonly. This cannot be set interactively, +because it would prevent itself from being written to the configuration file. +.IP "show_cxn_errors" 4 +.IX Item "show_cxn_errors" +Makes innotop print connection errors to \s-1STDOUT\s0. See \*(L"\s-1ERROR\s0 \s-1HANDLING\s0\*(R". +.IP "show_cxn_errors_in_tbl" 4 +.IX Item "show_cxn_errors_in_tbl" +Makes innotop display connection errors as rows in the first table on screen. +See \*(L"\s-1ERROR\s0 \s-1HANDLING\s0\*(R". +.IP "show_percent" 4 +.IX Item "show_percent" +Adds a '%' character after the value returned by the \*(L"percent\*(R" +transformation. +.IP "show_statusbar" 4 +.IX Item "show_statusbar" +Controls whether to show the status bar in the display. See \*(L"\s-1INNOTOP\s0 \s-1STATUS\s0\*(R". +.IP "skip_innodb" 4 +.IX Item "skip_innodb" +Disables fetching \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0, in case your server(s) do not have InnoDB +enabled and you don't want innotop to try to fetch it. This can also be useful +when you don't have the \s-1SUPER\s0 privilege, required to run \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. +.IP "status_inc" 4 +.IX Item "status_inc" +Whether to show absolute or incremental values for status variables. +Incremental values are calculated as an offset from the last value innotop saw +for that variable. This is a global setting, but will probably become +mode-specific at some point. Right now it is honored a bit inconsistently; some +modes don't pay attention to it. +.RE +.RS 4 +.RE +.IP "plugins" 4 +.IX Item "plugins" +This section holds a list of package names of active plugins. If the plugin +exists, innotop will activate it. See \*(L"\s-1PLUGINS\s0\*(R" for more information. +.IP "filters" 4 +.IX Item "filters" +This section holds user-defined filters (see \*(L"\s-1FILTERS\s0\*(R"). Each line is in the +format filter_name=text='filter text' tbls='table list'. +.Sp +The filter text is the text of the subroutine's code. The table list is a list +of tables to which the filter can apply. By default, user-defined filters apply +to the table for which they were created, but you can manually override that by +editing the definition in the configuration file. +.IP "active_filters" 4 +.IX Item "active_filters" +This section stores which filters are active on each table. Each line is in the +format table_name=filter_list. +.IP "tbl_meta" 4 +.IX Item "tbl_meta" +This section stores user-defined or user-customized columns (see \*(L"\s-1COLUMNS\s0\*(R"). +Each line is in the format col_name=properties, where the properties are a +name=quoted\-value list. +.IP "connections" 4 +.IX Item "connections" +This section holds the server connections you have defined. Each line is in the +format name=properties, where the properties are a name=value list. The +properties are self\-explanatory, and the only one that is treated specially is +\&'pass' which is only present if 'savepass' is set. See \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R". +.IP "active_connections" 4 +.IX Item "active_connections" +This section holds a list of which connections are active in each mode. Each +line is in the format mode_name=connection_list. +.IP "server_groups" 4 +.IX Item "server_groups" +This section holds server groups. Each line is in the format +name=connection_list. See \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R". +.IP "active_server_groups" 4 +.IX Item "active_server_groups" +This section holds a list of which server group is active in each mode. Each +line is in the format mode_name=server_group. +.IP "max_values_seen" 4 +.IX Item "max_values_seen" +This section holds the maximum values seen for variables. This is used to scale +the graphs in \*(L"S: Variables & Status\*(R" mode. Each line is in the format +name=value. +.IP "active_columns" 4 +.IX Item "active_columns" +This section holds table column lists. Each line is in the format +tbl_name=column_list. See \*(L"\s-1COLUMNS\s0\*(R". +.IP "sort_cols" 4 +.IX Item "sort_cols" +This section holds the sort definition. Each line is in the format +tbl_name=column_list. If a column is prefixed with '\-', that column sorts +descending. See \*(L"\s-1SORTING\s0\*(R". +.IP "visible_tables" 4 +.IX Item "visible_tables" +This section defines which tables are visible in each mode. Each line is in the +format mode_name=table_list. See \*(L"\s-1TABLES\s0\*(R". +.IP "varsets" 4 +.IX Item "varsets" +This section defines variable sets for use in \*(L"S: Status & Variables\*(R" mode. +Each line is in the format name=variable_list. See \*(L"\s-1VARIABLE\s0 \s-1SETS\s0\*(R". +.IP "colors" 4 +.IX Item "colors" +This section defines colorization rules. Each line is in the format +tbl_name=property_list. See \*(L"\s-1COLORS\s0\*(R". +.IP "stmt_sleep_times" 4 +.IX Item "stmt_sleep_times" +This section contains statement sleep times. Each line is in the format +statement_name=sleep_time. See \*(L"S: Statement Sleep Times\*(R". +.IP "group_by" 4 +.IX Item "group_by" +This section contains column lists for table group_by expressions. Each line is +in the format tbl_name=column_list. See \*(L"\s-1GROUPING\s0\*(R". +.SH "CUSTOMIZING" +.IX Header "CUSTOMIZING" +You can customize innotop a great deal. For example, you can: +.IP "\(bu" 4 +Choose which tables to display, and in what order. +.IP "\(bu" 4 +Choose which columns are in those tables, and create new columns. +.IP "\(bu" 4 +Filter which rows display with built-in filters, user-defined filters, and +quick\-filters. +.IP "\(bu" 4 +Sort the rows to put important data first or group together related rows. +.IP "\(bu" 4 +Highlight rows with color. +.IP "\(bu" 4 +Customize the alignment, width, and formatting of columns, and apply +transformations to columns to extract parts of their values or format the values +as you wish (for example, shortening large numbers to familiar units). +.IP "\(bu" 4 +Design your own expressions to extract and combine data as you need. This gives +you unlimited flexibility. +.PP +All these and more are explained in the following sections. +.Sh "\s-1TABLES\s0" +.IX Subsection "TABLES" +A table is what you'd expect: a collection of columns. It also has some other +properties, such as a caption. Filters, sorting rules, and colorization rules +belong to tables and are covered in later sections. +.PP +Internally, table meta-data is defined in a data structure called \f(CW%tbl_meta\fR. +This hash holds all built-in table definitions, which contain a lot of default +instructions to innotop. The meta-data includes the caption, a list of columns +the user has customized, a list of columns, a list of visible columns, a list of +filters, color rules, a sort-column list, sort direction, and some information +about the table's data sources. Most of this is customizable via the table +editor (see \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R"). +.PP +You can choose which tables to show by pressing the '$' key. See \*(L"\s-1MODES\s0\*(R" and +\&\*(L"\s-1TABLES\s0\*(R". +.PP +The table life-cycle is as follows: +.IP "\(bu" 4 +Each table begins with a data source, which is an array of hashes. See below +for details on data sources. +.IP "\(bu" 4 +Each element of the data source becomes a row in the final table. +.IP "\(bu" 4 +For each element in the data source, innotop extracts values from the source and +creates a row. This row is another hash, which later steps will refer to as +\&\f(CW$set\fR. The values innotop extracts are determined by the table's columns. Each +column has an extraction subroutine, compiled from an expression (see +\&\*(L"\s-1EXPRESSIONS\s0\*(R"). The resulting row is a hash whose keys are named the same as +the column name. +.IP "\(bu" 4 +innotop filters the rows, removing those that don't need to be displayed. See +\&\*(L"\s-1FILTERS\s0\*(R". +.IP "\(bu" 4 +innotop sorts the rows. See \*(L"\s-1SORTING\s0\*(R". +.IP "\(bu" 4 +innotop groups the rows together, if specified. See \*(L"\s-1GROUPING\s0\*(R". +.IP "\(bu" 4 +innotop colorizes the rows. See \*(L"\s-1COLORS\s0\*(R". +.IP "\(bu" 4 +innotop transforms the column values in each row. See \*(L"\s-1TRANSFORMATIONS\s0\*(R". +.IP "\(bu" 4 +innotop optionally pivots the rows (see \*(L"\s-1PIVOTING\s0\*(R"), then filters and sorts +them. +.IP "\(bu" 4 +innotop formats and justifies the rows as a table. During this step, innotop +applies further formatting to the column values, including alignment, maximum +and minimum widths. innotop also does final error checking to ensure there are +no crashes due to undefined values. innotop then adds a caption if specified, +and the table is ready to print. +.PP +The lifecycle is slightly different if the table is pivoted, as noted above. To +clarify, if the table is pivoted, the process is extract, group, transform, +pivot, filter, sort, create. If it's not pivoted, the process is extract, +filter, sort, group, color, transform, create. This slightly convoluted process +doesn't map all that well to \s-1SQL\s0, but pivoting complicates things pretty +thoroughly. Roughly speaking, filtering and sorting happen as late as needed to +effect the final result as you might expect, but as early as possible for +efficiency. +.PP +Each built-in table is described below: +.IP "adaptive_hash_index" 4 +.IX Item "adaptive_hash_index" +Displays data about InnoDB's adaptive hash index. Data source: +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "buffer_pool" 4 +.IX Item "buffer_pool" +Displays data about InnoDB's buffer pool. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "cmd_summary" 4 +.IX Item "cmd_summary" +Displays weighted status variables. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "deadlock_locks" 4 +.IX Item "deadlock_locks" +Shows which locks were held and waited for by the last detected deadlock. Data +source: \*(L"\s-1DEADLOCK_LOCKS\s0\*(R". +.IP "deadlock_transactions" 4 +.IX Item "deadlock_transactions" +Shows transactions involved in the last detected deadlock. Data source: +\&\*(L"\s-1DEADLOCK_TRANSACTIONS\s0\*(R". +.IP "explain" 4 +.IX Item "explain" +Shows the output of \s-1EXPLAIN\s0. Data source: \*(L"\s-1EXPLAIN\s0\*(R". +.IP "file_io_misc" 4 +.IX Item "file_io_misc" +Displays data about InnoDB's file and I/O operations. Data source: +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "fk_error" 4 +.IX Item "fk_error" +Displays various data about InnoDB's last foreign key error. Data source: +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "innodb_locks" 4 +.IX Item "innodb_locks" +Displays InnoDB locks. Data source: \*(L"\s-1INNODB_LOCKS\s0\*(R". +.IP "innodb_transactions" 4 +.IX Item "innodb_transactions" +Displays data about InnoDB's current transactions. Data source: +\&\*(L"\s-1INNODB_TRANSACTIONS\s0\*(R". +.IP "insert_buffers" 4 +.IX Item "insert_buffers" +Displays data about InnoDB's insert buffer. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "io_threads" 4 +.IX Item "io_threads" +Displays data about InnoDB's I/O threads. Data source: \*(L"\s-1IO_THREADS\s0\*(R". +.IP "log_statistics" 4 +.IX Item "log_statistics" +Displays data about InnoDB's logging system. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "master_status" 4 +.IX Item "master_status" +Displays replication master status. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "open_tables" 4 +.IX Item "open_tables" +Displays open tables. Data source: \*(L"\s-1OPEN_TABLES\s0\*(R". +.IP "page_statistics" 4 +.IX Item "page_statistics" +Displays InnoDB page statistics. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "pending_io" 4 +.IX Item "pending_io" +Displays InnoDB pending I/O operations. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "processlist" 4 +.IX Item "processlist" +Displays current MySQL processes (threads/connections). Data source: +\&\*(L"\s-1PROCESSLIST\s0\*(R". +.IP "q_header" 4 +.IX Item "q_header" +Displays various status values. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "row_operation_misc" 4 +.IX Item "row_operation_misc" +Displays data about InnoDB's row operations. Data source: +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "row_operations" 4 +.IX Item "row_operations" +Displays data about InnoDB's row operations. Data source: +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "semaphores" 4 +.IX Item "semaphores" +Displays data about InnoDB's semaphores and mutexes. Data source: +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "slave_io_status" 4 +.IX Item "slave_io_status" +Displays data about the slave I/O thread. Data source: +\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "slave_sql_status" 4 +.IX Item "slave_sql_status" +Displays data about the slave \s-1SQL\s0 thread. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "t_header" 4 +.IX Item "t_header" +Displays various InnoDB status values. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "var_status" 4 +.IX Item "var_status" +Displays user-configurable data. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R". +.IP "wait_array" 4 +.IX Item "wait_array" +Displays data about InnoDB's \s-1OS\s0 wait array. Data source: \*(L"\s-1OS_WAIT_ARRAY\s0\*(R". +.Sh "\s-1COLUMNS\s0" +.IX Subsection "COLUMNS" +Columns belong to tables. You can choose a table's columns by pressing the '^' +key, which starts the \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R" and lets you choose and edit columns. +Pressing 'e' from within the table editor lets you edit the column's properties: +.IP "\(bu" 4 +hdr: a column header. This appears in the first row of the table. +.IP "\(bu" 4 +just: justification. '\-' means left-justified and '' means right\-justified, +just as with printf formatting codes (not a coincidence). +.IP "\(bu" 4 +dec: whether to further align the column on the decimal point. +.IP "\(bu" 4 +num: whether the column is numeric. This affects how values are sorted +(lexically or numerically). +.IP "\(bu" 4 +label: a small note about the column, which appears in dialogs that help the +user choose columns. +.IP "\(bu" 4 +src: an expression that innotop uses to extract the column's data from its +source (see \*(L"\s-1DATA\s0 \s-1SOURCES\s0\*(R"). See \*(L"\s-1EXPRESSIONS\s0\*(R" for more on expressions. +.IP "\(bu" 4 +minw: specifies a minimum display width. This helps stabilize the display, +which makes it easier to read if the data is changing frequently. +.IP "\(bu" 4 +maxw: similar to minw. +.IP "\(bu" 4 +trans: a list of column transformations. See \*(L"\s-1TRANSFORMATIONS\s0\*(R". +.IP "\(bu" 4 +agg: an aggregate function. See \*(L"\s-1GROUPING\s0\*(R". The default is \*(L"first\*(R". +.IP "\(bu" 4 +aggonly: controls whether the column only shows when grouping is enabled on the +table (see \*(L"\s-1GROUPING\s0\*(R"). By default, this is disabled. This means columns +will always be shown by default, whether grouping is enabled or not. If a +column's aggonly is set true, the column will appear when you toggle grouping on +the table. Several columns are set this way, such as the count column on +\&\*(L"processlist\*(R" and \*(L"innodb_transactions\*(R", so you don't see a count when the +grouping isn't enabled, but you do when it is. +.Sh "\s-1FILTERS\s0" +.IX Subsection "FILTERS" +Filters remove rows from the display. They behave much like a \s-1WHERE\s0 clause in +\&\s-1SQL\s0. innotop has several built-in filters, which remove irrelevant information +like inactive queries, but you can define your own as well. innotop also lets +you create quick\-filters, which do not get saved to the configuration file, and +are just an easy way to quickly view only some rows. +.PP +You can enable or disable a filter on any table. Press the '%' key (mnemonic: % +looks kind of like a line being filtered between two circles) and choose which +table you want to filter, if asked. You'll then see a list of possible filters +and a list of filters currently enabled for that table. Type the names of +filters you want to apply and press Enter. +.PP +\fIUSER-DEFINED \s-1FILTERS\s0\fR +.IX Subsection "USER-DEFINED FILTERS" +.PP +If you type a name that doesn't exist, innotop will prompt you to create the +filter. Filters are easy to create if you know Perl, and not hard if you don't. +What you're doing is creating a subroutine that returns true if the row should +be displayed. The row is a hash reference passed to your subroutine as \f(CW$set\fR. +.PP +For example, imagine you want to filter the processlist table so you only see +queries that have been running more than five minutes. Type a new name for your +filter, and when prompted for the subroutine body, press \s-1TAB\s0 to initiate your +terminal's auto\-completion. You'll see the names of the columns in the +\&\*(L"processlist\*(R" table (innotop generally tries to help you with auto-completion +lists). You want to filter on the 'time' column. Type the text \*(L"$set\->{time} > +300\*(R" to return true when the query is more than five minutes old. That's all +you need to do. +.PP +In other words, the code you're typing is surrounded by an implicit context, +which looks like this: +.PP +.Vb 4 +\& sub filter { +\& my ( $set ) = @_; +\& # YOUR CODE HERE +\& } +.Ve +.PP +If your filter doesn't work, or if something else suddenly behaves differently, +you might have made an error in your filter, and innotop is silently catching +the error. Try enabling \*(L"debug\*(R" to make innotop throw an error instead. +.PP +\fIQUICK-FILTERS\fR +.IX Subsection "QUICK-FILTERS" +.PP +innotop's quick-filters are a shortcut to create a temporary filter that doesn't +persist when you restart innotop. To create a quick\-filter, press the '/' key. +innotop will prompt you for the column name and filter text. Again, you can use +auto-completion on column names. The filter text can be just the text you want +to \*(L"search for.\*(R" For example, to filter the \*(L"processlist\*(R" table on queries +that refer to the products table, type '/' and then 'info product'. +.PP +The filter text can actually be any Perl regular expression, but of course a +literal string like 'product' works fine as a regular expression. +.PP +Behind the scenes innotop compiles the quick-filter into a specially tagged +filter that is otherwise like any other filter. It just isn't saved to the +configuration file. +.PP +To clear quick\-filters, press the '\e' key and innotop will clear them all at +once. +.Sh "\s-1SORTING\s0" +.IX Subsection "SORTING" +innotop has sensible built-in defaults to sort the most important rows to the +top of the table. Like anything else in innotop, you can customize how any +table is sorted. +.PP +To start the sort dialog, start the \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R" with the '^' key, choose a +table if necessary, and press the 's' key. You'll see a list of columns you can +use in the sort expression and the current sort expression, if any. Enter a +list of columns by which you want to sort and press Enter. If you want to +reverse sort, prefix the column name with a minus sign. For example, if you +want to sort by column a ascending, then column b descending, type 'a \-b'. You +can also explicitly add a + in front of columns you want to sort ascending, but +it's not required. +.PP +Some modes have keys mapped to open this dialog directly, and to quickly reverse +sort direction. Press '?' as usual to see which keys are mapped in any mode. +.Sh "\s-1GROUPING\s0" +.IX Subsection "GROUPING" +innotop can group, or aggregate, rows together (I use the terms +interchangeably). This is quite similar to an \s-1SQL\s0 \s-1GROUP\s0 \s-1BY\s0 clause. You can +specify to group on certain columns, or if you don't specify any, the entire set +of rows is treated as one group. This is quite like \s-1SQL\s0 so far, but unlike \s-1SQL\s0, +you can also select un-grouped columns. innotop actually aggregates every +column. If you don't explicitly specify a grouping function, the default is +\&'first'. This is basically a convenience so you don't have to specify an +aggregate function for every column you want in the result. +.PP +You can quickly toggle grouping on a table with the '=' key, which toggles its +aggregate property. This property doesn't persist to the config file. +.PP +The columns by which the table is grouped are specified in its group_by +property. When you turn grouping on, innotop places the group_by columns at the +far left of the table, even if they're not supposed to be visible. The rest of +the visible columns appear in order after them. +.PP +Two tables have default group_by lists and a count column built in: +\&\*(L"processlist\*(R" and \*(L"innodb_transactions\*(R". The grouping is by connection +and status, so you can quickly see how many queries or transactions are in a +given status on each server you're monitoring. The time columns are aggregated +as a sum; other columns are left at the default 'first' aggregation. +.PP +By default, the table shown in \*(L"S: Variables & Status\*(R" mode also uses +grouping so you can monitor variables and status across many servers. The +default aggregation function in this mode is 'avg'. +.PP +Valid grouping functions are defined in the \f(CW%agg_funcs\fR hash. They include +.IP "first" 4 +.IX Item "first" +Returns the first element in the group. +.IP "count" 4 +.IX Item "count" +Returns the number of elements in the group, including undefined elements, much +like \s-1SQL\s0's \s-1COUNT\s0(*). +.IP "avg" 4 +.IX Item "avg" +Returns the average of defined elements in the group. +.IP "sum" 4 +.IX Item "sum" +Returns the sum of elements in the group. +.PP +Here's an example of grouping at work. Suppose you have a very busy server with +hundreds of open connections, and you want to see how many connections are in +what status. Using the built-in grouping rules, you can press 'Q' to enter +\&\*(L"Q: Query List\*(R" mode. Press '=' to toggle grouping (if necessary, select the +\&\*(L"processlist\*(R" table when prompted). +.PP +Your display might now look like the following: +.PP +.Vb 1 +\& Query List (? for help) localhost, 32:33, 0.11 QPS, 1 thd, 5.0.38\-log +.Ve +.PP +.Vb 5 +\& CXN Cmd Cnt ID User Host Time Query +\& localhost Query 49 12933 webusr localhost 19:38 SELECT * FROM +\& localhost Sending Da 23 2383 webusr localhost 12:43 SELECT col1, +\& localhost Sleep 120 140 webusr localhost 5:18:12 +\& localhost Statistics 12 19213 webusr localhost 01:19 SELECT * FROM +.Ve +.PP +That's actually quite a worrisome picture. You've got a lot of idle connections +(Sleep), and some connections executing queries (Query and Sending Data). +That's okay, but you also have a lot in Statistics status, collectively spending +over a minute. That means the query optimizer is having a really hard time +optimizing your statements. Something is wrong; it should normally take +milliseconds to optimize queries. You might not have seen this pattern if you +didn't look at your connections in aggregate. (This is a made-up example, but +it can happen in real life). +.Sh "\s-1PIVOTING\s0" +.IX Subsection "PIVOTING" +innotop can pivot a table for more compact display, similar to a Pivot Table in +a spreadsheet (also known as a crosstab). Pivoting a table makes columns into +rows. Assume you start with this table: +.PP +.Vb 4 +\& foo bar +\& === === +\& 1 3 +\& 2 4 +.Ve +.PP +After pivoting, the table will look like this: +.PP +.Vb 4 +\& name set0 set1 +\& ==== ==== ==== +\& foo 1 2 +\& bar 3 4 +.Ve +.PP +To get reasonable results, you might need to group as well as pivoting. +innotop currently does this for \*(L"S: Variables & Status\*(R" mode. +.Sh "\s-1COLORS\s0" +.IX Subsection "COLORS" +By default, innotop highlights rows with color so you can see at a glance which +rows are more important. You can customize the colorization rules and add your +own to any table. Open the table editor with the '^' key, choose a table if +needed, and press 'o' to open the color editor dialog. +.PP +The color editor dialog displays the rules applied to the table, in the order +they are evaluated. Each row is evaluated against each rule to see if the rule +matches the row; if it does, the row gets the specified color, and no further +rules are evaluated. The rules look like the following: +.PP +.Vb 9 +\& state eq Locked black on_red +\& cmd eq Sleep white +\& user eq system user white +\& cmd eq Connect white +\& cmd eq Binlog Dump white +\& time > 600 red +\& time > 120 yellow +\& time > 60 green +\& time > 30 cyan +.Ve +.PP +This is the default rule set for the \*(L"processlist\*(R" table. In order of +priority, these rules make locked queries black on a red background, \*(L"gray out\*(R" +connections from replication and sleeping queries, and make queries turn from +cyan to red as they run longer. +.PP +(For some reason, the \s-1ANSI\s0 color code \*(L"white\*(R" is actually a light gray. Your +terminal's display may vary; experiment to find colors you like). +.PP +You can use keystrokes to move the rules up and down, which re-orders their +priority. You can also delete rules and add new ones. If you add a new rule, +innotop prompts you for the column, an operator for the comparison, a value +against which to compare the column, and a color to assign if the rule matches. +There is auto-completion and prompting at each step. +.PP +The value in the third step needs to be correctly quoted. innotop does not try +to quote the value because it doesn't know whether it should treat the value as +a string or a number. If you want to compare the column against a string, as +for example in the first rule above, you should enter 'Locked' surrounded by +quotes. If you get an error message about a bareword, you probably should have +quoted something. +.Sh "\s-1EXPRESSIONS\s0" +.IX Subsection "EXPRESSIONS" +Expressions are at the core of how innotop works, and are what enables you to +extend innotop as you wish. Recall the table lifecycle explained in +\&\*(L"\s-1TABLES\s0\*(R". Expressions are used in the earliest step, where it extracts +values from a data source to form rows. +.PP +It does this by calling a subroutine for each column, passing it the source data +set, a set of current values, and a set of previous values. These are all +needed so the subroutine can calculate things like the difference between this +tick and the previous tick. +.PP +The subroutines that extract the data from the set are compiled from +expressions. This gives significantly more power than just naming the values to +fill the columns, because it allows the column's value to be calculated from +whatever data is necessary, but avoids the need to write complicated and lengthy +Perl code. +.PP +innotop begins with a string of text that can look as simple as a value's name +or as complicated as a full-fledged Perl expression. It looks at each +\&'bareword' token in the string and decides whether it's supposed to be a key +into the \f(CW$set\fR hash. A bareword is an unquoted value that isn't already +surrounded by code-ish things like dollar signs or curly brackets. If innotop +decides that the bareword isn't a function or other valid Perl code, it converts +it into a hash access. After the whole string is processed, innotop compiles a +subroutine, like this: +.PP +.Vb 5 +\& sub compute_column_value { +\& my ( $set, $cur, $pre ) = @_; +\& my $val = # EXPANDED STRING GOES HERE +\& return $val; +\& } +.Ve +.PP +Here's a concrete example, taken from the header table \*(L"q_header\*(R" in \*(L"Q: Query List\*(R" mode. This expression calculates the qps, or Queries Per Second, +column's values, from the values returned by \s-1SHOW\s0 \s-1STATUS:\s0 +.PP +.Vb 1 +\& Questions/Uptime_hires +.Ve +.PP +innotop decides both words are barewords, and transforms this expression into +the following Perl code: +.PP +.Vb 1 +\& $set\->{Questions}/$set\->{Uptime_hires} +.Ve +.PP +When surrounded by the rest of the subroutine's code, this is executable Perl +that calculates a high-resolution queries-per-second value. +.PP +The arguments to the subroutine are named \f(CW$set\fR, \f(CW$cur\fR, and \f(CW$pre\fR. In most cases, +\&\f(CW$set\fR and \f(CW$cur\fR will be the same values. However, if \*(L"status_inc\*(R" is set, \f(CW$cur\fR +will not be the same as \f(CW$set\fR, because \f(CW$set\fR will already contain values that are +the incremental difference between \f(CW$cur\fR and \f(CW$pre\fR. +.PP +Every column in innotop is computed by subroutines compiled in the same fashion. +There is no difference between innotop's built-in columns and user-defined +columns. This keeps things consistent and predictable. +.Sh "\s-1TRANSFORMATIONS\s0" +.IX Subsection "TRANSFORMATIONS" +Transformations change how a value is rendered. For example, they can take a +number of seconds and display it in H:M:S format. The following transformations +are defined: +.IP "commify" 4 +.IX Item "commify" +Adds commas to large numbers every three decimal places. +.IP "dulint_to_int" 4 +.IX Item "dulint_to_int" +Accepts two unsigned integers and converts them into a single longlong. This is +useful for certain operations with InnoDB, which uses two integers as +transaction identifiers, for example. +.IP "no_ctrl_char" 4 +.IX Item "no_ctrl_char" +Removes quoted control characters from the value. This is affected by the +\&\*(L"charset\*(R" configuration variable. +.Sp +This transformation only operates within quoted strings, for example, values to +a \s-1SET\s0 clause in an \s-1UPDATE\s0 statement. It will not alter the \s-1UPDATE\s0 statement, +but will collapse the quoted string to [\s-1BINARY\s0] or [\s-1TEXT\s0], depending on the +charset. +.IP "percent" 4 +.IX Item "percent" +Converts a number to a percentage by multiplying it by two, formatting it with +\&\*(L"num_digits\*(R" digits after the decimal point, and optionally adding a percent +sign (see \*(L"show_percent\*(R"). +.IP "secs_to_time" 4 +.IX Item "secs_to_time" +Formats a number of seconds as time in days+hours:minutes:seconds format. +.IP "set_precision" 4 +.IX Item "set_precision" +Formats numbers with \*(L"num_digits\*(R" number of digits after the decimal point. +.IP "shorten" 4 +.IX Item "shorten" +Formats a number as a unit of 1024 (k/M/G/T) and with \*(L"num_digits\*(R" number of +digits after the decimal point. +.Sh "\s-1TABLE\s0 \s-1EDITOR\s0" +.IX Subsection "TABLE EDITOR" +The innotop table editor lets you customize tables with keystrokes. You start +the table editor with the '^' key. If there's more than one table on the +screen, it will prompt you to choose one of them. Once you do, innotop will +show you something like this: +.PP +.Vb 1 +\& Editing table definition for Buffer Pool. Press ? for help, q to quit. +.Ve +.PP +.Vb 9 +\& name hdr label src +\& cxn CXN Connection from which cxn +\& buf_pool_size Size Buffer pool size IB_bp_buf_poo +\& buf_free Free Bufs Buffers free in the b IB_bp_buf_fre +\& pages_total Pages Pages total IB_bp_pages_t +\& pages_modified Dirty Pages Pages modified (dirty IB_bp_pages_m +\& buf_pool_hit_rate Hit Rate Buffer pool hit rate IB_bp_buf_poo +\& total_mem_alloc Memory Total memory allocate IB_bp_total_m +\& add_pool_alloc Add\(aql Pool Additonal pool alloca IB_bp_add_poo +.Ve +.PP +The first line shows which table you're editing, and reminds you again to press +\&'?' for a list of key mappings. The rest is a tabular representation of the +table's columns, because that's likely what you're trying to edit. However, you +can edit more than just the table's columns; this screen can start the filter +editor, color rule editor, and more. +.PP +Each row in the display shows a single column in the table you're editing, along +with a couple of its properties such as its header and source expression (see +\&\*(L"\s-1EXPRESSIONS\s0\*(R"). +.PP +The key mappings are Vim\-style, as in many other places. Pressing 'j' and 'k' +moves the highlight up or down. You can then (d)elete or (e)dit the highlighted +column. You can also (a)dd a column to the table. This actually just activates +one of the columns already defined for the table; it prompts you to choose from +among the columns available but not currently displayed. Finally, you can +re-order the columns with the '+' and '\-' keys. +.PP +You can do more than just edit the columns with the table editor, you can also +edit other properties, such as the table's sort expression and group-by +expression. Press '?' to see the full list, of course. +.PP +If you want to really customize and create your own column, as opposed to just +activating a built-in one that's not currently displayed, press the (n)ew key, +and innotop will prompt you for the information it needs: +.IP "\(bu" 4 +The column name: this needs to be a word without any funny characters, e.g. just +letters, numbers and underscores. +.IP "\(bu" 4 +The column header: this is the label that appears at the top of the column, in +the table header. This can have spaces and funny characters, but be careful not +to make it too wide and waste space on\-screen. +.IP "\(bu" 4 +The column's data source: this is an expression that determines what data from +the source (see \*(L"\s-1TABLES\s0\*(R") innotop will put into the column. This can just be +the name of an item in the source, or it can be a more complex expression, as +described in \*(L"\s-1EXPRESSIONS\s0\*(R". +.PP +Once you've entered the required data, your table has a new column. There is no +difference between this column and the built-in ones; it can have all the same +properties and behaviors. innotop will write the column's definition to the +configuration file, so it will persist across sessions. +.PP +Here's an example: suppose you want to track how many times your slaves have +retried transactions. According to the MySQL manual, the +Slave_retried_transactions status variable gives you that data: \*(L"The total +number of times since startup that the replication slave \s-1SQL\s0 thread has retried +transactions. This variable was added in version 5.0.4.\*(R" This is appropriate to +add to the \*(L"slave_sql_status\*(R" table. +.PP +To add the column, switch to the replication-monitoring mode with the 'M' key, +and press the '^' key to start the table editor. When prompted, choose +slave_sql_status as the table, then press 'n' to create the column. Type +\&'retries' as the column name, 'Retries' as the column header, and +\&'Slave_retried_transactions' as the source. Now the column is created, and you +see the table editor screen again. Press 'q' to exit the table editor, and +you'll see your column at the end of the table. +.SH "VARIABLE SETS" +.IX Header "VARIABLE SETS" +Variable sets are used in \*(L"S: Variables & Status\*(R" mode to define more easily +what variables you want to monitor. Behind the scenes they are compiled to a +list of expressions, and then into a column list so they can be treated just +like columns in any other table, in terms of data extraction and +transformations. However, you're protected from the tedious details by a syntax +that ought to feel very natural to you: a \s-1SQL\s0 \s-1SELECT\s0 list. +.PP +The data source for variable sets, and indeed the entire S mode, is the +combination of \s-1SHOW\s0 \s-1STATUS\s0, \s-1SHOW\s0 \s-1VARIABLES\s0, and \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. Imagine +that you had a huge table with one column per variable returned from those +statements. That's the data source for variable sets. You can now query this +data source just like you'd expect. For example: +.PP +.Vb 1 +\& Questions, Uptime, Questions/Uptime as QPS +.Ve +.PP +Behind the scenes innotop will split that variable set into three expressions, +compile them and turn them into a table definition, then extract as usual. This +becomes a \*(L"variable set,\*(R" or a \*(L"list of variables you want to monitor.\*(R" +.PP +innotop lets you name and save your variable sets, and writes them to the +configuration file. You can choose which variable set you want to see with the +\&'c' key, or activate the next and previous sets with the '>' and '<' keys. +There are many built-in variable sets as well, which should give you a good +start for creating your own. Press 'e' to edit the current variable set, or +just to see how it's defined. To create a new one, just press 'c' and type its +name. +.PP +You may want to use some of the functions listed in \*(L"\s-1TRANSFORMATIONS\s0\*(R" to help +format the results. In particular, \*(L"set_precision\*(R" is often useful to limit +the number of digits you see. Extending the above example, here's how: +.PP +.Vb 1 +\& Questions, Uptime, set_precision(Questions/Uptime) as QPS +.Ve +.PP +Actually, this still needs a little more work. If your \*(L"interval\*(R" is less +than one second, you might be dividing by zero because Uptime is incremental in +this mode by default. Instead, use Uptime_hires: +.PP +.Vb 1 +\& Questions, Uptime, set_precision(Questions/Uptime_hires) as QPS +.Ve +.PP +This example is simple, but it shows how easy it is to choose which variables +you want to monitor. +.SH "PLUGINS" +.IX Header "PLUGINS" +innotop has a simple but powerful plugin mechanism by which you can extend +or modify its existing functionality, and add new functionality. innotop's +plugin functionality is event\-based: plugins register themselves to be called +when events happen. They then have a chance to influence the event. +.PP +An innotop plugin is a Perl module placed in innotop's \*(L"plugin_dir\*(R" +directory. On \s-1UNIX\s0 systems, you can place a symbolic link to the module instead +of putting the actual file there. innotop automatically discovers the file. If +there is a corresponding entry in the \*(L"plugins\*(R" configuration file section, +innotop loads and activates the plugin. +.PP +The module must conform to innotop's plugin interface. Additionally, the source +code of the module must be written in such a way that innotop can inspect the +file and determine the package name and description. +.Sh "Package Source Convention" +.IX Subsection "Package Source Convention" +innotop inspects the plugin module's source to determine the Perl package name. +It looks for a line of the form \*(L"package Foo;\*(R" and if found, considers the +plugin's package name to be Foo. Of course the package name can be a valid Perl +package name, with double semicolons and so on. +.PP +It also looks for a description in the source code, to make the plugin editor +more human\-friendly. The description is a comment line of the form \*(L"# +description: Foo\*(R", where \*(L"Foo\*(R" is the text innotop will consider to be the +plugin's description. +.Sh "Plugin Interface" +.IX Subsection "Plugin Interface" +The innotop plugin interface is quite simple: innotop expects the plugin to be +an object-oriented module it can call certain methods on. The methods are +.IP "new(%variables)" 4 +.IX Item "new(%variables)" +This is the plugin's constructor. It is passed a hash of innotop's variables, +which it can manipulate (see \*(L"Plugin Variables\*(R"). It must return a reference +to the newly created plugin object. +.Sp +At construction time, innotop has only loaded the general configuration and +created the default built-in variables with their default contents (which is +quite a lot). Therefore, the state of the program is exactly as in the innotop +source code, plus the configuration variables from the \*(L"general\*(R" section in +the config file. +.Sp +If your plugin manipulates the variables, it is changing global data, which is +shared by innotop and all plugins. Plugins are loaded in the order they're +listed in the config file. Your plugin may load before or after another plugin, +so there is a potential for conflict or interaction between plugins if they +modify data other plugins use or modify. +.IP "\fIregister_for_events()\fR" 4 +.IX Item "register_for_events()" +This method must return a list of events in which the plugin is interested, if +any. See \*(L"Plugin Events\*(R" for the defined events. If the plugin returns an +event that's not defined, the event is ignored. +.IP "event handlers" 4 +.IX Item "event handlers" +The plugin must implement a method named the same as each event for which it has +registered. In other words, if the plugin returns qw(foo bar) from +\&\fIregister_for_events()\fR, it must have \fIfoo()\fR and \fIbar()\fR methods. These methods are +callbacks for the events. See \*(L"Plugin Events\*(R" for more details about each +event. +.Sh "Plugin Variables" +.IX Subsection "Plugin Variables" +The plugin's constructor is passed a hash of innotop's variables, which it can +manipulate. It is probably a good idea if the plugin object saves a copy of it +for later use. The variables are defined in the innotop variable +\&\f(CW%pluggable_vars\fR, and are as follows: +.IP "action_for" 4 +.IX Item "action_for" +A hashref of key mappings. These are innotop's global hot\-keys. +.IP "agg_funcs" 4 +.IX Item "agg_funcs" +A hashref of functions that can be used for grouping. See \*(L"\s-1GROUPING\s0\*(R". +.IP "config" 4 +.IX Item "config" +The global configuration hash. +.IP "connections" 4 +.IX Item "connections" +A hashref of connection specifications. These are just specifications of how to +connect to a server. +.IP "dbhs" 4 +.IX Item "dbhs" +A hashref of innotop's database connections. These are actual \s-1DBI\s0 connection +objects. +.IP "filters" 4 +.IX Item "filters" +A hashref of filters applied to table rows. See \*(L"\s-1FILTERS\s0\*(R" for more. +.IP "modes" 4 +.IX Item "modes" +A hashref of modes. See \*(L"\s-1MODES\s0\*(R" for more. +.IP "server_groups" 4 +.IX Item "server_groups" +A hashref of server groups. See \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R". +.IP "tbl_meta" 4 +.IX Item "tbl_meta" +A hashref of innotop's table meta\-data, with one entry per table (see +\&\*(L"\s-1TABLES\s0\*(R" for more information). +.IP "trans_funcs" 4 +.IX Item "trans_funcs" +A hashref of transformation functions. See \*(L"\s-1TRANSFORMATIONS\s0\*(R". +.IP "var_sets" 4 +.IX Item "var_sets" +A hashref of variable sets. See \*(L"\s-1VARIABLE\s0 \s-1SETS\s0\*(R". +.Sh "Plugin Events" +.IX Subsection "Plugin Events" +Each event is defined somewhere in the innotop source code. When innotop runs +that code, it executes the callback function for each plugin that expressed its +interest in the event. innotop passes some data for each event. The events are +defined in the \f(CW%event_listener_for\fR variable, and are as follows: +.ie n .IP "extract_values($set, $cur\fR, \f(CW$pre\fR, \f(CW$tbl)" 4 +.el .IP "extract_values($set, \f(CW$cur\fR, \f(CW$pre\fR, \f(CW$tbl\fR)" 4 +.IX Item "extract_values($set, $cur, $pre, $tbl)" +This event occurs inside the function that extracts values from a data source. +The arguments are the set of values, the current values, the previous values, +and the table name. +.IP "set_to_tbl" 4 +.IX Item "set_to_tbl" +Events are defined at many places in this subroutine, which is responsible for +turning an arrayref of hashrefs into an arrayref of lines that can be printed to +the screen. The events all pass the same data: an arrayref of rows and the name +of the table being created. The events are set_to_tbl_pre_filter, +set_to_tbl_pre_sort,set_to_tbl_pre_group, set_to_tbl_pre_colorize, +set_to_tbl_pre_transform, set_to_tbl_pre_pivot, set_to_tbl_pre_create, +set_to_tbl_post_create. +.IP "draw_screen($lines)" 4 +.IX Item "draw_screen($lines)" +This event occurs inside the subroutine that prints the lines to the screen. +\&\f(CW$lines\fR is an arrayref of strings. +.Sh "Simple Plugin Example" +.IX Subsection "Simple Plugin Example" +The easiest way to explain the plugin functionality is probably with a simple +example. The following module adds a column to the beginning of every table and +sets its value to 1. +.PP +.Vb 2 +\& use strict; +\& use warnings FATAL => \(aqall\(aq; +.Ve +.PP +.Vb 2 +\& package Innotop::Plugin::Example; +\& # description: Adds an \(aqexample\(aq column to every table +.Ve +.PP +.Vb 4 +\& sub new { +\& my ( $class, %vars ) = @_; +\& # Store reference to innotop\(aqs variables in $self +\& my $self = bless { %vars }, $class; +.Ve +.PP +.Vb 11 +\& # Design the example column +\& my $col = { +\& hdr => \(aqExample\(aq, +\& just => \(aq\(aq, +\& dec => 0, +\& num => 1, +\& label => \(aqExample\(aq, +\& src => \(aqexample\(aq, # Get data from this column in the data source +\& tbl => \(aq\(aq, +\& trans => [], +\& }; +.Ve +.PP +.Vb 8 +\& # Add the column to every table. +\& my $tbl_meta = $vars{tbl_meta}; +\& foreach my $tbl ( values %$tbl_meta ) { +\& # Add the column to the list of defined columns +\& $tbl\->{cols}\->{example} = $col; +\& # Add the column to the list of visible columns +\& unshift @{$tbl\->{visible}}, \(aqexample\(aq; +\& } +.Ve +.PP +.Vb 3 +\& # Be sure to return a reference to the object. +\& return $self; +\& } +.Ve +.PP +.Vb 5 +\& # I\(aqd like to be called when a data set is being rendered into a table, please. +\& sub register_for_events { +\& my ( $self ) = @_; +\& return qw(set_to_tbl_pre_filter); +\& } +.Ve +.PP +.Vb 8 +\& # This method will be called when the event fires. +\& sub set_to_tbl_pre_filter { +\& my ( $self, $rows, $tbl ) = @_; +\& # Set the example column\(aqs data source to the value 1. +\& foreach my $row ( @$rows ) { +\& $row\->{example} = 1; +\& } +\& } +.Ve +.PP +.Vb 1 +\& 1; +.Ve +.Sh "Plugin Editor" +.IX Subsection "Plugin Editor" +The plugin editor lets you view the plugins innotop discovered and activate or +deactivate them. Start the editor by pressing $ to start the configuration +editor from any mode. Press the 'p' key to start the plugin editor. You'll see +a list of plugins innotop discovered. You can use the 'j' and 'k' keys to move +the highlight to the desired one, then press the * key to toggle it active or +inactive. Exit the editor and restart innotop for the changes to take effect. +.SH "SQL STATEMENTS" +.IX Header "SQL STATEMENTS" +innotop uses a limited set of \s-1SQL\s0 statements to retrieve data from MySQL for +display. The statements are customized depending on the server version against +which they are executed; for example, on MySQL 5 and newer, \s-1INNODB_STATUS\s0 +executes \*(L"\s-1SHOW\s0 \s-1ENGINE\s0 \s-1INNODB\s0 \s-1STATUS\s0\*(R", while on earlier versions it executes +\&\*(L"\s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0\*(R". The statements are as follows: +.PP +.Vb 12 +\& Statement SQL executed +\& =================== =============================== +\& INNODB_STATUS SHOW [ENGINE] INNODB STATUS +\& KILL_CONNECTION KILL +\& KILL_QUERY KILL QUERY +\& OPEN_TABLES SHOW OPEN TABLES +\& PROCESSLIST SHOW FULL PROCESSLIST +\& SHOW_MASTER_LOGS SHOW MASTER LOGS +\& SHOW_MASTER_STATUS SHOW MASTER STATUS +\& SHOW_SLAVE_STATUS SHOW SLAVE STATUS +\& SHOW_STATUS SHOW [GLOBAL] STATUS +\& SHOW_VARIABLES SHOW [GLOBAL] VARIABLES +.Ve +.SH "DATA SOURCES" +.IX Header "DATA SOURCES" +Each time innotop extracts values to create a table (see \*(L"\s-1EXPRESSIONS\s0\*(R" and +\&\*(L"\s-1TABLES\s0\*(R"), it does so from a particular data source. Largely because of the +complex data extracted from \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0, this is slightly messy. \s-1SHOW\s0 +\&\s-1INNODB\s0 \s-1STATUS\s0 contains a mixture of single values and repeated values that form +nested data sets. +.PP +Whenever innotop fetches data from MySQL, it adds two extra bits to each set: +cxn and Uptime_hires. cxn is the name of the connection from which the data +came. Uptime_hires is a high-resolution version of the server's Uptime status +variable, which is important if your \*(L"interval\*(R" setting is sub\-second. +.PP +Here are the kinds of data sources from which data is extracted: +.IP "\s-1STATUS_VARIABLES\s0" 4 +.IX Item "STATUS_VARIABLES" +This is the broadest category, into which the most kinds of data fall. It +begins with the combination of \s-1SHOW\s0 \s-1STATUS\s0 and \s-1SHOW\s0 \s-1VARIABLES\s0, but other sources +may be included as needed, for example, \s-1SHOW\s0 \s-1MASTER\s0 \s-1STATUS\s0 and \s-1SHOW\s0 \s-1SLAVE\s0 +\&\s-1STATUS\s0, as well as many of the non-repeated values from \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. +.IP "\s-1DEADLOCK_LOCKS\s0" 4 +.IX Item "DEADLOCK_LOCKS" +This data is extracted from the transaction list in the \s-1LATEST\s0 \s-1DETECTED\s0 \s-1DEADLOCK\s0 +section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. It is nested two levels deep: transactions, then +locks. +.IP "\s-1DEADLOCK_TRANSACTIONS\s0" 4 +.IX Item "DEADLOCK_TRANSACTIONS" +This data is from the transaction list in the \s-1LATEST\s0 \s-1DETECTED\s0 \s-1DEADLOCK\s0 +section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. It is nested one level deep. +.IP "\s-1EXPLAIN\s0" 4 +.IX Item "EXPLAIN" +This data is from the result set returned by \s-1EXPLAIN\s0. +.IP "\s-1INNODB_TRANSACTIONS\s0" 4 +.IX Item "INNODB_TRANSACTIONS" +This data is from the \s-1TRANSACTIONS\s0 section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. +.IP "\s-1IO_THREADS\s0" 4 +.IX Item "IO_THREADS" +This data is from the list of threads in the the \s-1FILE\s0 I/O section of \s-1SHOW\s0 \s-1INNODB\s0 +\&\s-1STATUS\s0. +.IP "\s-1INNODB_LOCKS\s0" 4 +.IX Item "INNODB_LOCKS" +This data is from the \s-1TRANSACTIONS\s0 section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 and is nested +two levels deep. +.IP "\s-1OPEN_TABLES\s0" 4 +.IX Item "OPEN_TABLES" +This data is from \s-1SHOW\s0 \s-1OPEN\s0 \s-1TABLES\s0. +.IP "\s-1PROCESSLIST\s0" 4 +.IX Item "PROCESSLIST" +This data is from \s-1SHOW\s0 \s-1FULL\s0 \s-1PROCESSLIST\s0. +.IP "\s-1OS_WAIT_ARRAY\s0" 4 +.IX Item "OS_WAIT_ARRAY" +This data is from the \s-1SEMAPHORES\s0 section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 and is nested one +level deep. It comes from the lines that look like this: +.Sp +.Vb 1 +\& \-\-Thread 1568861104 has waited at btr0cur.c line 424 .... +.Ve +.SH "MYSQL PRIVILEGES" +.IX Header "MYSQL PRIVILEGES" +.IP "\(bu" 4 +You must connect to MySQL as a user who has the \s-1SUPER\s0 privilege for many of the +functions. +.IP "\(bu" 4 +If you don't have the \s-1SUPER\s0 privilege, you can still run some functions, but you +won't necessarily see all the same data. +.IP "\(bu" 4 +You need the \s-1PROCESS\s0 privilege to see the list of currently running queries in Q +mode. +.IP "\(bu" 4 +You need special privileges to start and stop slave servers. +.IP "\(bu" 4 +You need appropriate privileges to create and drop the deadlock tables if needed +(see \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R"). +.SH "SYSTEM REQUIREMENTS" +.IX Header "SYSTEM REQUIREMENTS" +You need Perl to run innotop, of course. You also need a few Perl modules: \s-1DBI\s0, +DBD::mysql, Term::ReadKey, and Time::HiRes. These should be included with most +Perl distributions, but in case they are not, I recommend using versions +distributed with your operating system or Perl distribution, not from \s-1CPAN\s0. +Term::ReadKey in particular has been known to cause problems if installed from +\&\s-1CPAN\s0. +.PP +If you have Term::ANSIColor, innotop will use it to format headers more readably +and compactly. (Under Microsoft Windows, you also need Win32::Console::ANSI for +terminal formatting codes to be honored). If you install Term::ReadLine, +preferably Term::ReadLine::Gnu, you'll get nice auto-completion support. +.PP +I run innotop on Gentoo GNU/Linux, Debian and Ubuntu, and I've had feedback from +people successfully running it on Red Hat, CentOS, Solaris, and Mac \s-1OSX\s0. I +don't see any reason why it won't work on other UNIX-ish operating systems, but +I don't know for sure. It also runs on Windows under ActivePerl without +problem. +.PP +I use innotop on MySQL versions 3.23.58, 4.0.27, 4.1.0, 4.1.22, 5.0.26, 5.1.15, +and 5.2.3. If it doesn't run correctly for you, that is a bug and I hope you +report it. +.SH "FILES" +.IX Header "FILES" +$HOMEDIR/.innotop is used to store configuration information. Files include the +configuration file innotop.ini, the core_dump file which contains verbose error +messages if \*(L"debug\*(R" is enabled, and the plugins/ subdirectory. +.SH "GLOSSARY OF TERMS" +.IX Header "GLOSSARY OF TERMS" +.IP "tick" 4 +.IX Item "tick" +A tick is a refresh event, when innotop re-fetches data from connections and +displays it. +.SH "ACKNOWLEDGEMENTS" +.IX Header "ACKNOWLEDGEMENTS" +I'm grateful to the following people for various reasons, and hope I haven't +forgotten to include anyone: +.PP +Allen K. Smith, +Aurimas Mikalauskas, +Bartosz Fenski, +Brian Miezejewski, +Christian Hammers, +Cyril Scetbon, +Dane Miller, +David Multer, +Dr. Frank Ullrich, +Giuseppe Maxia, +Google.com Site Reliability Engineers, +Jan Pieter Kunst, +Jari Aalto, +Jay Pipes, +Jeremy Zawodny, +Johan Idren, +Kristian Kohntopp, +Lenz Grimmer, +Maciej Dobrzanski, +Michiel Betel, +MySQL \s-1AB\s0, +Paul McCullagh, +Sebastien Estienne, +Sourceforge.net, +Steven Kreuzer, +The Gentoo MySQL Team, +Trevor Price, +Yaar Schnitman, +and probably more people I've neglected to include. +.PP +(If I misspelled your name, it's probably because I'm afraid of putting +international characters into this documentation; earlier versions of Perl might +not be able to compile it then). +.SH "COPYRIGHT, LICENSE AND WARRANTY" +.IX Header "COPYRIGHT, LICENSE AND WARRANTY" +This program is copyright (c) 2006 Baron Schwartz. +Feedback and improvements are welcome. +.PP +\&\s-1THIS\s0 \s-1PROGRAM\s0 \s-1IS\s0 \s-1PROVIDED\s0 \*(L"\s-1AS\s0 \s-1IS\s0\*(R" \s-1AND\s0 \s-1WITHOUT\s0 \s-1ANY\s0 \s-1EXPRESS\s0 \s-1OR\s0 \s-1IMPLIED\s0 +\&\s-1WARRANTIES\s0, \s-1INCLUDING\s0, \s-1WITHOUT\s0 \s-1LIMITATION\s0, \s-1THE\s0 \s-1IMPLIED\s0 \s-1WARRANTIES\s0 \s-1OF\s0 +\&\s-1MERCHANTIBILITY\s0 \s-1AND\s0 \s-1FITNESS\s0 \s-1FOR\s0 A \s-1PARTICULAR\s0 \s-1PURPOSE\s0. +.PP +This program is free software; you can redistribute it and/or modify it under +the terms of the \s-1GNU\s0 General Public License as published by the Free Software +Foundation, version 2; \s-1OR\s0 the Perl Artistic License. On \s-1UNIX\s0 and similar +systems, you can issue `man perlgpl' or `man perlartistic' to read these +licenses. +.PP +You should have received a copy of the \s-1GNU\s0 General Public License along with +this program; if not, write to the Free Software Foundation, Inc., 59 Temple +Place, Suite 330, Boston, \s-1MA\s0 02111\-1307 \s-1USA\s0. +.PP +Execute innotop and press '!' to see this information at any time. +.SH "AUTHOR" +.IX Header "AUTHOR" +Baron Schwartz. +.SH "BUGS" +.IX Header "BUGS" +You can report bugs, ask for improvements, and get other help and support at +<http://sourceforge.net/projects/innotop>. There are mailing lists, forums, +a bug tracker, etc. Please use these instead of contacting me directly, as it +makes my job easier and benefits others if the discussions are permanent and +public. Of course, if you need to contact me in private, please do. === added file 'storage/xtradb/build/debian/additions/msql2mysql.1' --- a/storage/xtradb/build/debian/additions/msql2mysql.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/msql2mysql.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +msql2mysql \- MySQL importer for msql style data. +.SH SYNOPSIS +msql2mysql [options] +.SH DESCRIPTION +This program imports old msql database files. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/my.cnf' --- a/storage/xtradb/build/debian/additions/my.cnf 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/my.cnf 2010-01-26 18:02:46 +0000 @@ -0,0 +1,129 @@ +# +# The MySQL database server configuration file. +# +# You can copy this to one of: +# - "/etc/mysql/my.cnf" to set global options, +# - "~/.my.cnf" to set user-specific options. +# +# One can use all long options that the program supports. +# Run program with --help to get a list of available options and with +# --print-defaults to see which it would actually understand and use. +# +# For explanations see +# http://dev.mysql.com/doc/mysql/en/server-system-variables.html + +# This will be passed to all mysql clients +# It has been reported that passwords should be enclosed with ticks/quotes +# escpecially if they contain "#" chars... +# Remember to edit /etc/mysql/debian.cnf when changing the socket location. +[client] +port = 3306 +socket = /var/run/mysqld/mysqld.sock + +# Here is entries for some specific programs +# The following values assume you have at least 32M ram + +# This was formally known as [safe_mysqld]. Both versions are currently parsed. +[mysqld_safe] +socket = /var/run/mysqld/mysqld.sock +nice = 0 + +[mysqld] +# +# * Basic Settings +# +user = mysql +pid-file = /var/run/mysqld/mysqld.pid +socket = /var/run/mysqld/mysqld.sock +port = 3306 +basedir = /usr +datadir = /var/lib/mysql +tmpdir = /tmp +language = /usr/share/mysql/english +skip-external-locking +# +# For compatibility to other Debian packages that still use +# libmysqlclient10 and libmysqlclient12. +old_passwords = 1 +# +# Instead of skip-networking the default is now to listen only on +# localhost which is more compatible and is not less secure. +bind-address = 127.0.0.1 +# +# * Fine Tuning +# +key_buffer = 16M +max_allowed_packet = 16M +thread_stack = 128K +thread_cache_size = 8 +# This replaces the startup script and checks MyISAM tables if needed +# the first time they are touched +myisam-recover = BACKUP +#max_connections = 100 +#table_cache = 64 +#thread_concurrency = 10 +# +# * Query Cache Configuration +# +query_cache_limit = 1M +query_cache_size = 16M +# +# * Logging and Replication +# +# Both location gets rotated by the cronjob. +# Be aware that this log type is a performance killer. +# As of 5.1 you can enable the at runtime! +#log_type = FILE +#general_log = /var/log/mysql/mysql.log +# +# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf. +# +# Here you can see queries with especially long duration +#log_slow_queries = /var/log/mysql/mysql-slow.log +#long_query_time = 2 +#log-queries-not-using-indexes +# +# The following can be used as easy to replay backup logs or for replication. +# note: if you are setting up a replication slave, see README.Debian about +# other settings you may need to change. +#server-id = 1 +#log_bin = /var/log/mysql/mysql-bin.log +expire_logs_days = 10 +max_binlog_size = 100M +#binlog_do_db = include_database_name +#binlog_ignore_db = include_database_name +# +# * InnoDB +# +# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/. +# Read the manual for more InnoDB related options. There are many! +# +# * Security Features +# +# Read the manual, too, if you want chroot! +# chroot = /var/lib/mysql/ +# +# For generating SSL certificates I recommend the OpenSSL GUI "tinyca". +# +# ssl-ca=/etc/mysql/cacert.pem +# ssl-cert=/etc/mysql/server-cert.pem +# ssl-key=/etc/mysql/server-key.pem + + + +[mysqldump] +quick +quote-names +max_allowed_packet = 16M + +[mysql] +#no-auto-rehash # faster start of mysql but no tab completition + +[isamchk] +key_buffer = 16M + +# +# * IMPORTANT: Additional settings that can override those from this file! +# The files must end with '.cnf', otherwise they'll be ignored. +# +!includedir /etc/mysql/conf.d/ === added file 'storage/xtradb/build/debian/additions/my_print_defaults.1' --- a/storage/xtradb/build/debian/additions/my_print_defaults.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/my_print_defaults.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +my_print_defaults \- MySQL helper script that prints defaults. +.SH SYNOPSIS +my_print_defaults [options] +.SH DESCRIPTION +Prints all arguments that is give to some program using the default files. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/myisam_ftdump.1' --- a/storage/xtradb/build/debian/additions/myisam_ftdump.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/myisam_ftdump.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +myisam_ftdump \- Dumps full text tables. +.SH SYNOPSIS +myisam_ftdump [options] +.SH DESCRIPTION +Dumps information and contents of full text tables. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/myisamchk.1' --- a/storage/xtradb/build/debian/additions/myisamchk.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/myisamchk.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,17 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +myisamchk \- Checks MySQL myisam type databases. +.SH SYNOPSIS +myisamchk [options] +.SH DESCRIPTION +Description, check and repair of ISAM tables. +Used without options all tables on the command will be checked for errors + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/myisamlog.1' --- a/storage/xtradb/build/debian/additions/myisamlog.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/myisamlog.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +myisamlog \- MySQL helper script. +.SH SYNOPSIS +myisamlog [options] +.SH DESCRIPTION +Function unknown. Mail to ch(a)debian.org. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/myisampack.1' --- a/storage/xtradb/build/debian/additions/myisampack.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/myisampack.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,19 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +myisampack \- Compresses MySQL database files. +.SH SYNOPSIS +myisampack [options] +.SH DESCRIPTION +Pack a MyISAM-table to take much less space. +Keys are not updated, you must run myisamchk -rq on the datafile +afterwards to update the keys. +You should give the .MYI file as the filename argument. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql-server.lintian-overrides' --- a/storage/xtradb/build/debian/additions/mysql-server.lintian-overrides 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql-server.lintian-overrides 2010-02-18 23:51:40 +0000 @@ -0,0 +1,2 @@ +W: mysql-dfsg source: maintainer-script-lacks-debhelper-token debian/percona-xtradb-server.postinst +W: percona-xtradb-server: possible-bashism-in-maintainer-script postinst:68 'p{("a".."z","A".."Z",0..9)[int(rand(62))]}' === added file 'storage/xtradb/build/debian/additions/mysql_config.1' --- a/storage/xtradb/build/debian/additions/mysql_config.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_config.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,17 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysqlconfig \- MySQL compile settings. +.SH SYNOPSIS +mysqlconfig [options] +.SH DESCRIPTION +This program is only useful for people who want to compile agains +libmysqlclient. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql_convert_table_format.1' --- a/storage/xtradb/build/debian/additions/mysql_convert_table_format.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_convert_table_format.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,17 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysql_convert_table_format \- MySQL table converter. +.SH SYNOPSIS +mysql_convert_table_format [options] +.SH DESCRIPTION +Conversion of a MySQL tables to other table types. +If no tables has been specifed, all tables in the database will be converted. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql_find_rows.1' --- a/storage/xtradb/build/debian/additions/mysql_find_rows.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_find_rows.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,18 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysql_find_rows \- MySQL shell skript for searching in update logs. +.SH SYNOPSIS +mysql_find_rows [options] +.SH DESCRIPTION +Prints all SQL queries that matches a regexp or contains a 'use +database' or 'set ..' command to stdout. A SQL query may contain +newlines. This is useful to find things in a MySQL update log. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql_fix_extensions.1' --- a/storage/xtradb/build/debian/additions/mysql_fix_extensions.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_fix_extensions.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,18 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysql_fix_extensions \- Corrects MySQL database file names. +.SH SYNOPSIS +mysql_fix_extensions <datadir> +.SH DESCRIPTION +Makes .frm lowercase and .MYI/MYD/ISM/ISD uppercase +useful when datafiles are copied from windows. +Does not work with RAID, with InnoDB or BDB tables. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (8) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql_install_db.1' --- a/storage/xtradb/build/debian/additions/mysql_install_db.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_install_db.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysql_install_db \- MySQL helper program. +.SH SYNOPSIS +mysql_install_db [options] +.SH DESCRIPTION +This program is normally not needed by any user. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql_secure_installation.1' --- a/storage/xtradb/build/debian/additions/mysql_secure_installation.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_secure_installation.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,17 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysql_secure_installation \- Secures the MySQL access control lists. +.SH SYNOPSIS +mysql_secure_installation [options] +.SH DESCRIPTION +This interactive programm suggests changes like removing anonymous users that +are supposed to make your installation more secure. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (8) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql_setpermission.1' --- a/storage/xtradb/build/debian/additions/mysql_setpermission.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_setpermission.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,23 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysql_setpermission \- Adds MySQL users or changes passwords. +.SH SYNOPSIS +mysql_setpermission [options] +.SH DESCRIPTION +The permission setter is a little program which can help you add users +or databases or change passwords in MySQL. Keep in mind that we don't +check permissions which already been set in MySQL. So if you can't +connect to MySQL using the permission you just added, take a look at +the permissions which have already been set in MySQL. + +The permission setter first reads your .my.cnf file in your Home +directory if it exists. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysql_tableinfo.1' --- a/storage/xtradb/build/debian/additions/mysql_tableinfo.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_tableinfo.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,322 @@ +.\" Automatically generated by Pod::Man v1.34, Pod::Parser v1.13 +.\" +.\" Standard preamble: +.\" ======================================================================== +.de Sh \" Subsection heading +.br +.if t .Sp +.ne 5 +.PP +\fB\\$1\fR +.PP +.. +.de Sp \" Vertical space (when we can't use .PP) +.if t .sp .5v +.if n .sp +.. +.de Vb \" Begin verbatim text +.ft CW +.nf +.ne \\$1 +.. +.de Ve \" End verbatim text +.ft R +.fi +.. +.\" Set up some character translations and predefined strings. \*(-- will +.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left +.\" double quote, and \*(R" will give a right double quote. | will give a +.\" real vertical bar. \*(C+ will give a nicer C++. Capital omega is used to +.\" do unbreakable dashes and therefore won't be available. \*(C` and \*(C' +.\" expand to `' in nroff, nothing in troff, for use with C<>. +.tr \(*W-|\(bv\*(Tr +.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' +.ie n \{\ +. ds -- \(*W- +. ds PI pi +. if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch +. if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch +. ds L" "" +. ds R" "" +. ds C` "" +. ds C' "" +'br\} +.el\{\ +. ds -- \|\(em\| +. ds PI \(*p +. ds L" `` +. ds R" '' +'br\} +.\" +.\" If the F register is turned on, we'll generate index entries on stderr for +.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index +.\" entries marked with X<> in POD. Of course, you'll have to process the +.\" output yourself in some meaningful fashion. +.if \nF \{\ +. de IX +. tm Index:\\$1\t\\n%\t"\\$2" +.. +. nr % 0 +. rr F +.\} +.\" +.\" For nroff, turn off justification. Always turn off hyphenation; it makes +.\" way too many mistakes in technical documents. +.hy 0 +.if n .na +.\" +.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). +.\" Fear. Run. Save yourself. No user-serviceable parts. +. \" fudge factors for nroff and troff +.if n \{\ +. ds #H 0 +. ds #V .8m +. ds #F .3m +. ds #[ \f1 +. ds #] \fP +.\} +.if t \{\ +. ds #H ((1u-(\\\\n(.fu%2u))*.13m) +. ds #V .6m +. ds #F 0 +. ds #[ \& +. ds #] \& +.\} +. \" simple accents for nroff and troff +.if n \{\ +. ds ' \& +. ds ` \& +. ds ^ \& +. ds , \& +. ds ~ ~ +. ds / +.\} +.if t \{\ +. ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" +. ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' +. ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' +. ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' +. ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' +. ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' +.\} +. \" troff and (daisy-wheel) nroff accents +.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' +.ds 8 \h'\*(#H'\(*b\h'-\*(#H' +.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] +.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' +.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' +.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] +.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] +.ds ae a\h'-(\w'a'u*4/10)'e +.ds Ae A\h'-(\w'A'u*4/10)'E +. \" corrections for vroff +.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' +.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' +. \" for low resolution devices (crt and lpr) +.if \n(.H>23 .if \n(.V>19 \ +\{\ +. ds : e +. ds 8 ss +. ds o a +. ds d- d\h'-1'\(ga +. ds D- D\h'-1'\(hy +. ds th \o'bp' +. ds Th \o'LP' +. ds ae ae +. ds Ae AE +.\} +.rm #[ #] #H #V #F C +.\" ======================================================================== +.\" +.IX Title "MYSQL_TABLEINFO 1" +.TH MYSQL_TABLEINFO 1 "2003-04-05" "perl v5.8.0" "User Contributed Perl Documentation" +.SH "NAME" +mysql_tableinfo \- creates and populates information tables with +the output of SHOW DATABASES, SHOW TABLES (or SHOW TABLE STATUS), +SHOW COLUMNS and SHOW INDEX. +.PP +This is version 1.1. +.SH "SYNOPSIS" +.IX Header "SYNOPSIS" +.Vb 1 +\& mysql_tableinfo [OPTIONS] database_to_write [database_like_wild] [table_like_wild] +.Ve +.PP +.Vb 2 +\& Do not backquote (``) database_to_write, +\& and do not quote ('') database_like_wild or table_like_wild +.Ve +.PP +.Vb 1 +\& Examples: +.Ve +.PP +.Vb 1 +\& mysql_tableinfo info +.Ve +.PP +.Vb 1 +\& mysql_tableinfo info this_db +.Ve +.PP +.Vb 1 +\& mysql_tableinfo info %a% b% +.Ve +.PP +.Vb 1 +\& mysql_tableinfo info --clear-only +.Ve +.PP +.Vb 1 +\& mysql_tableinfo info --col --idx --table-status +.Ve +.SH "DESCRIPTION" +.IX Header "DESCRIPTION" +mysql_tableinfo asks a MySQL server information about its +databases, tables, table columns and index, and stores this +in tables called `db`, `tbl` (or `tbl_status`), `col`, `idx` +(with an optional prefix specified with \-\-prefix). +After that, you can query these information tables, for example +to build your admin scripts with \s-1SQL\s0 queries, like +.PP +\&\s-1SELECT\s0 \s-1CONCAT\s0(\*(L"\s-1CHECK\s0 \s-1TABLE\s0 \*(R",`database`,\*(L".\*(R",`table`,\*(L" \s-1EXTENDED\s0;\*(R") +\&\s-1FROM\s0 info.tbl \s-1WHERE\s0 ... ; +.PP +as people usually do with some other \s-1RDBMS\s0 +(note: to increase the speed of your queries on the info tables, +you may add some index on them). +.PP +The database_like_wild and table_like_wild instructs the program +to gather information only about databases and tables +whose names match these patterns. If the info +tables already exist, their rows matching the patterns are simply +deleted and replaced by the new ones. That is, +old rows not matching the patterns are not touched. +If the database_like_wild and table_like_wild arguments +are not specified on the command-line they default to \*(L"%\*(R". +.PP +The program : +.PP +\&\- does \s-1CREATE\s0 \s-1DATABASE\s0 \s-1IF\s0 \s-1NOT\s0 \s-1EXISTS\s0 database_to_write +where database_to_write is the database name specified on the command\-line. +.PP +\&\- does \s-1CREATE\s0 \s-1TABLE\s0 \s-1IF\s0 \s-1NOT\s0 \s-1EXISTS\s0 database_to_write.`db` +.PP +\&\- fills database_to_write.`db` with the output of +\&\s-1SHOW\s0 \s-1DATABASES\s0 \s-1LIKE\s0 database_like_wild +.PP +\&\- does \s-1CREATE\s0 \s-1TABLE\s0 \s-1IF\s0 \s-1NOT\s0 \s-1EXISTS\s0 database_to_write.`tbl` +(respectively database_to_write.`tbl_status` +if the \-\-tbl\-status option is on) +.PP +\&\- for every found database, +fills database_to_write.`tbl` (respectively database_to_write.`tbl_status`) +with the output of +\&\s-1SHOW\s0 \s-1TABLES\s0 \s-1FROM\s0 found_db \s-1LIKE\s0 table_like_wild +(respectively \s-1SHOW\s0 \s-1TABLE\s0 \s-1STATUS\s0 \s-1FROM\s0 found_db \s-1LIKE\s0 table_like_wild) +.PP +\&\- if the \-\-col option is on, + * does \s-1CREATE\s0 \s-1TABLE\s0 \s-1IF\s0 \s-1NOT\s0 \s-1EXISTS\s0 database_to_write.`col` + * for every found table, + fills database_to_write.`col` with the output of + \s-1SHOW\s0 \s-1COLUMNS\s0 \s-1FROM\s0 found_tbl \s-1FROM\s0 found_db +.PP +\&\- if the \-\-idx option is on, + * does \s-1CREATE\s0 \s-1TABLE\s0 \s-1IF\s0 \s-1NOT\s0 \s-1EXISTS\s0 database_to_write.`idx` + * for every found table, + fills database_to_write.`idx` with the output of + \s-1SHOW\s0 \s-1INDEX\s0 \s-1FROM\s0 found_tbl \s-1FROM\s0 found_db +.PP +Some options may modify this general scheme (see below). +.PP +As mentioned, the contents of the info tables are the output of +\&\s-1SHOW\s0 commands. In fact the contents are slightly more complete : +.PP +\&\- the `tbl` (or `tbl_status`) info table + has an extra column which contains the database name, +.PP +\&\- the `col` info table + has an extra column which contains the table name, + and an extra column which contains, for each described column, + the number of this column in the table owning it (this extra column + is called `Seq_in_table`). `Seq_in_table` makes it possible for you + to retrieve your columns in sorted order, when you are querying + the `col` table. +.PP +\&\- the `index` info table + has an extra column which contains the database name. +.PP +Caution: info tables contain certain columns (e.g. +Database, Table, Null...) whose names, as they are MySQL reserved words, +need to be backquoted (`...`) when used in \s-1SQL\s0 statements. +.PP +Caution: as information fetching and info tables filling happen at the +same time, info tables may contain inaccurate information about +themselves. +.SH "OPTIONS" +.IX Header "OPTIONS" +.IP "\-\-clear" 4 +.IX Item "--clear" +Does \s-1DROP\s0 \s-1TABLE\s0 on the info tables (only those that the program is +going to fill, for example if you do not use \-\-col it won't drop +the `col` table) and processes normally. Does not drop database_to_write. +.IP "\-\-clear\-only" 4 +.IX Item "--clear-only" +Same as \-\-clear but exits after the DROPs. +.IP "\-\-col" 4 +.IX Item "--col" +Adds columns information (into table `col`). +.IP "\-\-idx" 4 +.IX Item "--idx" +Adds index information (into table `idx`). +.IP "\-\-prefix prefix" 4 +.IX Item "--prefix prefix" +The info tables are named from the concatenation of prefix and, +respectively, db, tbl (or tbl_status), col, idx. Do not quote ('') +or backquote (``) prefix. +.IP "\-q, \-\-quiet" 4 +.IX Item "-q, --quiet" +Does not warn you about what the script is going to do (\s-1DROP\s0 \s-1TABLE\s0 etc) +and does not ask for a confirmation before starting. +.IP "\-\-tbl\-status" 4 +.IX Item "--tbl-status" +Instead of using \s-1SHOW\s0 \s-1TABLES\s0, uses \s-1SHOW\s0 \s-1TABLE\s0 \s-1STATUS\s0 +(much more complete information, but slower). +.IP "\-\-help" 4 +.IX Item "--help" +Display helpscreen and exit +.IP "\-u, \-\-user=#" 4 +.IX Item "-u, --user=#" +user for database login if not current user. Give a user +who has sufficient privileges (\s-1CREATE\s0, ...). +.IP "\-p, \-\-password=# (INSECURE)" 4 +.IX Item "-p, --password=# (INSECURE)" +password to use when connecting to server. +WARNING: Providing a password on command line is insecure as it is visible through /proc to anyone for a short time. +.IP "\-h, \-\-host=#" 4 +.IX Item "-h, --host=#" +host to connect to +.IP "\-P, \-\-port=#" 4 +.IX Item "-P, --port=#" +port to use when connecting to server +.IP "\-S, \-\-socket=#" 4 +.IX Item "-S, --socket=#" +\&\s-1UNIX\s0 domain socket to use when connecting to server +.SH "WARRANTY" +.IX Header "WARRANTY" +This software is free and comes without warranty of any kind. You +should never trust backup software without studying the code yourself. +Study the code inside this script and only rely on it if \fIyou\fR believe +that it does the right thing for you. +.Sp +Patches adding bug fixes, documentation and new features are welcome. +.SH "TO DO" +.IX Header "TO DO" +Use extended inserts to be faster (for servers with many databases +or tables). But to do that, must care about net\-buffer\-length. +.SH "AUTHOR" +.IX Header "AUTHOR" +2002\-06\-18 Guilhem Bichot (guilhem.bichot(a)mines\-paris.org) +.Sp +And all the authors of mysqlhotcopy, which served as a model for +the structure of the program. === added file 'storage/xtradb/build/debian/additions/mysql_waitpid.1' --- a/storage/xtradb/build/debian/additions/mysql_waitpid.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysql_waitpid.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,20 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysql_waitpid \- Waits a specified amount of seconds for a PID to terminate. +.SH SYNOPSIS +mysql_waitpid [options] <pid> <seconds> +.SH DESCRIPTION +Description: Waits for a program, which program id is #pid, to +terminate within #time seconds. If the program terminates within +this time, or if the #pid no longer exists, value 0 is returned. +Otherwise 1 is returned. Both #pid and #time must be positive +integer arguments. + +See mysql_waitpid for options. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysqlbinlog.1' --- a/storage/xtradb/build/debian/additions/mysqlbinlog.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqlbinlog.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,17 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysqlbinlog \- Dumps MySQL binary logs. +.SH SYNOPSIS +mysqlbinlog [options] +.SH DESCRIPTION +Dumps a MySQL binary log in a format usable for viewing or for pipeing to +the mysql command line client + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysqlbug.1' --- a/storage/xtradb/build/debian/additions/mysqlbug.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqlbug.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,14 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysqlbug \- MySQL bug reporting tool. +.SH SYNOPSIS +mysqlbug [options] +.SH DESCRIPTION +Interactive bug reporting tool. Use reportbug on Debian systems. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysqlcheck.1' --- a/storage/xtradb/build/debian/additions/mysqlcheck.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqlcheck.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,28 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysqlcheck \- MySQL program for repairing, checking and optimizing tables. +.SH SYNOPSIS +mysqlcheck | mysqlanalyze | mysqloptimize [options] +.SH DESCRIPTION +This program can be used to CHECK (-c,-m,-C), REPAIR (-r), ANALYZE (-a) +or OPTIMIZE (-o) tables. Some of the options (like -e or -q) can be +used same time. It works on MyISAM and in some cases on BDB tables. +Please consult the MySQL manual for latest information about the +above. The options -c,-r,-a and -o are exclusive to each other, which +means that the last option will be used, if several was specified. + +The option -c will be used by default, if none was specified. You +can change the default behavior by making a symbolic link, or +copying this file somewhere with another name, the alternatives are: +mysqlrepair: The default option will be -r +mysqlanalyze: The default option will be -a +mysqloptimize: The default option will be -o + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (8) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysqld_safe_syslog.cnf' --- a/storage/xtradb/build/debian/additions/mysqld_safe_syslog.cnf 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqld_safe_syslog.cnf 2010-01-26 18:02:46 +0000 @@ -0,0 +1,2 @@ +[mysqld_safe] +syslog === added file 'storage/xtradb/build/debian/additions/mysqldumpslow.1' --- a/storage/xtradb/build/debian/additions/mysqldumpslow.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqldumpslow.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,50 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysqldumpslow \- Parse and summarize the MySQL slow query log. +.SH SYNOPSIS +mysqldumpslow [options] +.SH DESCRIPTION +This program parses and summarizes a 'slow query log'. + +.TP +\fB\-v\fR +verbose +.TP +\fB\-d\fR +debug +.TP +\fB\-s=WORD\fR +what to sort by (t, at, l, al, r, ar etc) +.TP +\fB\-r\fR +reverse the sort order (largest last instead of first) +.TP +\fB\-t=NUMBER\fR +just show the top n queries +.TP +\fB\-a\fR +don't abstract all numbers to N and strings to 'S' +.TP +\fB\-n=NUMBER\fR +abstract numbers with at least n digits within names +.TP +\fB\-g=WORD\fR +grep: only consider stmts that include this string +.TP +\fB\-h=WORD\fR +hostname of db server for *-slow.log filename (can be wildcard) +.TP +\fB\-i=WORD\fR +name of server instance (if using mysql.server startup script) +.TP +\fB\-l\fR +don't subtract lock time from total time + +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org> based on +the commends in the code. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysqlimport.1' --- a/storage/xtradb/build/debian/additions/mysqlimport.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqlimport.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,20 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysqlimport \- Imports text files with MySQL database queries. +.SH SYNOPSIS +mysqlimport [options] +.SH DESCRIPTION +Loads tables from text files in various formats. The base name of the +text file must be the name of the table that should be used. +If one uses sockets to connect to the MySQL server, the server will open and +read the text file directly. In other cases the client will open the text +file. The SQL command 'LOAD DATA INFILE' is used to import the rows. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/mysqlmanager.1' --- a/storage/xtradb/build/debian/additions/mysqlmanager.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqlmanager.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,49 @@ +.TH mysql 1 "March 2005" "MySQL 4.1" "MySQL database" +.SH NAME +mysqlmanager \- Manages instances of MySQL server. +.SH SYNOPSIS +.B mysqlmanager +[\fIOPTIONS\fR] +.SH DESCRIPTION +Manages instances of MySQL server. +.TP +\-?, \fB\-\-help\fR +Display this help and exit. +.TP +\fB\-P\fR, \fB\-\-port=\fR# +Port number to listen on. +.TP +\fB\-l\fR, \fB\-\-log\fR=\fIname\fR +Path to log file. +.TP +\fB\-b\fR, \fB\-\-bind\-address=\fR# +Address to listen on. +.HP +\fB\-B\fR, \fB\-\-tcp\-backlog=\fR# Size of TCP/IP listen queue. +.HP +\fB\-g\fR, \fB\-\-greeting\fR=\fIname\fR Set greeting on connect. +.TP +\fB\-m\fR, \fB\-\-max\-command\-len=\fR# +Maximum command length. +.TP +\fB\-d\fR, \fB\-\-one\-thread\fR +Use one thread ( for debugging). +.TP +\fB\-C\fR, \fB\-\-connect\-retries=\fR# +Number of attempts to establish MySQL connection. +.TP +\fB\-p\fR, \fB\-\-password\-file\fR=\fIname\fR +Password file for manager. +.HP +\fB\-f\fR, \fB\-\-pid\-file\fR=\fIname\fR Pid file to use. +.TP +\fB\-V\fR, \fB\-\-version\fR +Output version information and exit. +.SH "SEE ALSO" +The full documentation for +.B mysqlmanager +is available in the package mysql-doc-4.1 or on the MySQL +homepage www.mysql.com. +.SH AUTHOR +This manpage was created by Christian Hammers <ch(a)debian.org> +using help2man. === added file 'storage/xtradb/build/debian/additions/mysqlreport' --- a/storage/xtradb/build/debian/additions/mysqlreport 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqlreport 2010-01-26 18:02:46 +0000 @@ -0,0 +1,1298 @@ +#!/usr/bin/perl -w + +# mysqlreport v3.5 Apr 16 2008 +# http://hackmysql.com/mysqlreport + +# mysqlreport makes an easy-to-read report of important MySQL status values. +# Copyright 2006-2008 Daniel Nichter +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# The GNU General Public License is available at: +# http://www.gnu.org/copyleft/gpl.html + +use strict; +use File::Temp qw(tempfile); +use DBI; +use Getopt::Long; +eval { require Term::ReadKey; }; +my $RK = ($@ ? 0 : 1); + +sub have_op; + +my $WIN = ($^O eq 'MSWin32' ? 1 : 0); +my %op; +my %mycnf; # ~/.my.cnf +my ($tmpfile_fh, $tmpfile); +my ($stat_name, $stat_val, $stat_label); +my $MySQL_version; +my (%stats, %vars); # SHOW STATUS, SHOW VARIABLES +my (%DMS_vals, %Com_vals, %ib_vals); +my ($dbh, $query); +my ($questions, $key_read_ratio, $key_write_ratio, $dms, $slow_query_t); +my ($key_cache_block_size, $key_buffer_used, $key_buffer_usage); +my ($qc_mem_used, $qc_hi_r, $qc_ip_r); # Query Cache +my $have_innodb_vals; +my ($ib_bp_used, $ib_bp_total, $ib_bp_read_ratio); +my ($relative_live, $relative_infiles); +my $real_uptime; +my (%stats_present, %stats_past); # For relative reports + +GetOptions ( + \%op, + "user=s", + "password:s", + "host=s", + "port=s", + "socket=s", + "no-mycnf", + "infile|in=s", + "outfile=s", + "flush-status", + "email=s", + "r|relative:i", + "c|report-count=i", + "detach", + "help|?", + "debug" +); + +show_help_and_exit() if $op{'help'}; + +get_user_mycnf() unless $op{'no-mycnf'}; + +# Command line options override ~/.my.cnf +$mycnf{'host'} = $op{'host'} if have_op 'host'; +$mycnf{'port'} = $op{'port'} if have_op 'port'; +$mycnf{'socket'} = $op{'socket'} if have_op 'socket'; +$mycnf{'user'} = $op{'user'} if have_op 'user'; + +$mycnf{'user'} ||= $ENV{'USER'}; + +if(exists $op{'password'}) +{ + if($op{'password'} eq '') # Prompt for password + { + Term::ReadKey::ReadMode(2) if $RK; + print "Password for database user $mycnf{'user'}: "; + chomp($mycnf{'pass'} = <STDIN>); + Term::ReadKey::ReadMode(0), print "\n" if $RK; + } + else { $mycnf{'pass'} = $op{'password'}; } # Use password given on command line +} + +$op{'com'} ||= 3; +$op{'c'} ||= 1; # Used in collect_reports() if --r given integer value + +$relative_live = 0; +$relative_infiles = 0; + +if(defined $op{'r'}) +{ + if($op{r}) { $relative_live = 1; } # if -r was given an integer value + else { $relative_infiles = 1; } +} + +# The report is written to a tmp file first. +# Later it will be moved to $op{'outfile'} or emailed $op{'email'} if needed. +($tmpfile_fh, $tmpfile) = tempfile() or die "Cannot open temporary file for writing: $!\n"; + +if($op{'detach'}) +{ + $SIG{'TERM'} = 'sig_handler'; + + if(fork()) + { + print "mysqlreport has forked and detached.\n"; + print "While running detached, mysqlreport writes reports to '$tmpfile'.\n"; + + exit; + } + + open(STDIN, "</dev/null"); + open(STDOUT, "> $tmpfile") or die "Cannot dup STDOUT: $!\n"; + open(STDERR, "> $tmpfile") or die "Cannot dup STDERR: $!\n"; +} + +select $tmpfile_fh; +$| = 1 if ($op{'detach'} || $relative_live); + +print "tmp file: $tmpfile\n" if $op{debug}; + +# Connect to MySQL +if(!$op{'infile'} && !$relative_infiles) +{ + connect_to_MySQL(); +} + +$have_innodb_vals = 1; # This might be set to 0 later in get_MySQL_version() + +if(defined $op{'r'}) +{ + if($relative_live) + { + print STDERR "mysqlreport is writing relative reports to '$tmpfile'.\n" unless $op{'detach'}; + get_MySQL_version(); + collect_reports(); + } + + if($relative_infiles) { read_relative_infiles(); } +} +else +{ + if(!$op{'infile'}) + { + get_MySQL_version(); + get_vals(); + get_vars(); + } + else + { + read_infile($op{'infile'}); + } + + get_Com_values(); + + set_myisam_vals(); + set_ib_vals() if $have_innodb_vals; + + write_report(); +} + +exit_tasks_and_cleanup(); + +exit; + +# +# Subroutines +# +sub show_help_and_exit +{ + print <<"HELP"; +mysqlreport v3.5 Apr 16 2008 +mysqlreport makes an easy-to-read report of important MySQL status values. + +Command line options (abbreviations work): + --user USER Connect to MySQL as USER + --password PASS Use PASS or prompt for MySQL user's password + --host ADDRESS Connect to MySQL at ADDRESS + --port PORT Connect to MySQL at PORT + --socket SOCKET Connect to MySQL at SOCKET + --no-mycnf Don't read ~/.my.cnf + --infile FILE Read status values from FILE instead of MySQL + --outfile FILE Write report to FILE + --email ADDRESS Email report to ADDRESS (doesn't work on Windows) + --flush-status Issue FLUSH STATUS; after getting current values + --relative X Generate relative reports. If X is an integer, + reports are live from the MySQL server X seconds apart. + If X is a list of infiles (file1 file2 etc.), + reports are generated from the infiles in the order + that they are given. + --report-count N Collect N number of live relative reports (default 1) + --detach Fork and detach from terminal (run in background) + --help Prints this + --debug Print debugging information + +Visit http://hackmysql.com/mysqlreport for more information. +HELP + + exit; +} + +sub get_user_mycnf +{ + print "get_user_mycnf\n" if $op{debug}; + + return if $WIN; + open MYCNF, "$ENV{HOME}/.my.cnf" or return; + while(<MYCNF>) + { + if(/^(.+?)\s*=\s*"?(.+?)"?\s*$/) + { + $mycnf{$1} = $2; + print "get_user_mycnf: read '$1 = $2'\n" if $op{debug}; + } + } + $mycnf{'pass'} ||= $mycnf{'password'} if exists $mycnf{'password'}; + close MYCNF; +} + +sub connect_to_MySQL +{ + print "connect_to_MySQL\n" if $op{debug}; + + my $dsn; + + if($mycnf{'socket'} && -S $mycnf{'socket'}) + { + $dsn = "DBI:mysql:mysql_socket=$mycnf{socket}"; + } + elsif($mycnf{'host'}) + { + $dsn = "DBI:mysql:host=$mycnf{host}" . ($mycnf{port} ? ";port=$mycnf{port}" : ""); + } + else + { + $dsn = "DBI:mysql:host=localhost"; + } + + print "connect_to_MySQL: DBI DSN: $dsn\n" if $op{debug}; + + $dbh = DBI->connect($dsn, $mycnf{'user'}, $mycnf{'pass'}) or die; +} + +sub collect_reports +{ + print "collect_reports\n" if $op{debug}; + + my $i; + + get_vals(); + get_vars(); + + get_Com_values(); + + %stats_past = %stats; + + set_myisam_vals(); + set_ib_vals() if $have_innodb_vals; + + print "#\n# Beginning report, 0 0:0:0\n#\n"; + + write_report(); + + for($i = 0; $i < $op{'c'}; $i++) + { + $dbh->disconnect(); + + sleep($op{'r'}); + + connect_to_MySQL(); + + print "\n#\n# Interval report " , $i + 1 , ", +", sec_to_dhms(($i + 1) * $op{'r'}), "\n#\n"; + + get_vals(); + + write_relative_report(); + } +} + +sub read_relative_infiles +{ + print "read_relative_infiles\n" if $op{debug}; + + my $slurp; # Used to check infiles for multiple sets of status values + my $n_stats; # Number of multiple sets of status values in an infile + my $infile; + my $report_n; # Report number + + $report_n = 1; + + foreach $infile (@ARGV) + { + # Read all of infile into $slurp + open INFILE, "< $infile" or warn and next; + $slurp = do { local $/; <INFILE> }; + close INFILE; + + $n_stats = 0; + + # Count number of status value sets + $n_stats++ while $slurp =~ /Aborted_clients/g; + + print "read_relative_infiles: found $n_stats sets of status values in file '$infile'\n" + if $op{debug}; + + if($n_stats == 1) + { + read_infile($infile); + relative_infile_report($report_n++); + } + + if($n_stats > 1) + { + my @tmpfile_fh; + my @tmpfile_name; + my $i; + my $stat_n; # Status value set number + + # Create a tmp file for each set of status values + for($i = 0; $i < $n_stats; $i++) + { + my ($fh, $name) = tempfile() + or die "read_relative_infiles: cannot open temporary file for writing: $!\n"; + + push(@tmpfile_fh, $fh); + push(@tmpfile_name, $name); + + print "read_relative_infiles: created tmp file '$name' for set $i\n" if $op{debug}; + } + + $i = 0; + $stat_n = 0; + + select $tmpfile_fh[$i]; + + # Read infile again and copy each set of status values to seperate tmp files + open INFILE, "< $infile" or warn and next; + while(<INFILE>) + { + next if /^\+/; + next if /^$/; + + # The infile must begin with the system variable values. + # Therefore, the first occurance of Aborted_clients indicates the beginning + # of the first set of status values if no sets have occured yet ($stat_n == 0). + # In this case, the following status values are printed to the current fh, + # along with the system variable values read thus far, until Aborted_clients + # occurs again. Then begins the second and subsequent sets of status values. + + if(/Aborted_clients/) + { + print and next if $stat_n++ == 0; + select $tmpfile_fh[++$i]; + } + + print; + } + close INFILE; + + # Re-select the main tmp file into which the reports are being written. + select $tmpfile_fh; + + for($i = 0; $i < $n_stats; $i++) + { + close $tmpfile_fh[$i]; + + print "read_relative_infiles: reading set $i tmp file '$tmpfile_name[$i]'\n" + if $op{debug}; + + read_infile($tmpfile_name[$i]); + relative_infile_report($report_n++); + + if($WIN) { `del $tmpfile_name[$i]`; } + else { `rm -f $tmpfile_name[$i]`; } + + print "read_relative_infiles: deleted set $i tmp file '$tmpfile_name[$i]'\n" + if $op{debug}; + } + + } # if($n_stats > 1) + } # foreach $infile (@files) +} + +sub relative_infile_report +{ + print "relative_infile_report\n" if $op{debug}; + + my $report_n = shift; + + if($report_n == 1) + { + get_Com_values(); + + %stats_past = %stats; + + set_myisam_vals(); + set_ib_vals() if $have_innodb_vals; + + print "#\n# Beginning report, 0 0:0:0\n#\n"; + + write_report(); + } + else + { + print "\n#\n# Interval report ", $report_n - 1, ", +", + sec_to_dhms($stats{Uptime} - $stats_past{Uptime}), + "\n#\n"; + + write_relative_report(); + } +} + +sub get_vals +{ + print "get_vals\n" if $op{debug}; + + my @row; + + # Get status values + if($MySQL_version >= 50002) + { + $query = $dbh->prepare("SHOW GLOBAL STATUS;"); + } + else + { + $query = $dbh->prepare("SHOW STATUS;"); + } + $query->execute(); + while(@row = $query->fetchrow_array()) { $stats{$row[0]} = $row[1]; } + + $real_uptime = $stats{'Uptime'}; +} + +sub get_vars +{ + print "get_vars\n" if $op{debug}; + + my @row; + + # Get server system variables + $query = $dbh->prepare("SHOW VARIABLES;"); + $query->execute(); + while(@row = $query->fetchrow_array()) { $vars{$row[0]} = $row[1]; } + + # table_cache was renamed to table_open_cache in MySQL 5.1.3 + if($MySQL_version >= 50103) + { + $vars{'table_cache'} = $vars{'table_open_cache'}; + } +} + +sub read_infile +{ + print "read_infile\n" if $op{debug}; + + my $infile = shift; + + # Default required system variable values if not set in INFILE. + # As of mysqlreport v3.5 the direct output from SHOW VARIABLES; + # can be put into INFILE instead. See http://hackmysql.com/mysqlreportdoc + # for details. + $vars{'version'} = "0.0.0" if !exists $vars{'version'}; + $vars{'table_cache'} = 64 if !exists $vars{'table_cache'}; + $vars{'max_connections'} = 100 if !exists $vars{'max_connections'}; + $vars{'key_buffer_size'} = 8388600 if !exists $vars{'key_buffer_size'}; # 8M + $vars{'thread_cache_size'} = 0 if !exists $vars{'thread_cache_size'}; + $vars{'tmp_table_size'} = 0 if !exists $vars{'tmp_table_size'}; + $vars{'long_query_time'} = '?' if !exists $vars{'long_query_time'}; + $vars{'log_slow_queries'} = '?' if !exists $vars{'log_slow_queries'}; + + # One should also add: + # key_cache_block_size + # query_cache_size + # to INFILE if needed. + + open INFILE, "< $infile" or die "Cannot open INFILE '$infile': $!\n"; + + while(<INFILE>) + { + last if !defined $_; + + next if /^\+/; # skip divider lines + next if /^$/; # skip blank lines + + next until /(Aborted_clients|back_log|=)/; + + if($1 eq 'Aborted_clients') # status values + { + print "read_infile: start stats\n" if $op{debug}; + + while($_) + { + chomp; + if(/([A-Za-z_]+)[\s\t|]+(\d+)/) + { + $stats{$1} = $2; + print "read_infile: save $1 = $2\n" if $op{debug}; + } + else { print "read_infile: ignore '$_'\n" if $op{debug}; } + + last if $1 eq 'Uptime'; # exit while() if end of status values + $_ = <INFILE>; # otherwise, read next line of status values + } + } + elsif($1 eq 'back_log') # system variable values + { + print "read_infile: start vars\n" if $op{debug}; + + while($_) + { + chomp; + if(/([A-Za-z_]+)[\s\t|]+([\w\.\-]+)/) # This will exclude some vars + { # like pid_file which we don't need + $vars{$1} = $2; + print "read_infile: save $1 = $2\n" if $op{debug}; + } + else { print "read_infile: ignore '$_'\n" if $op{debug}; } + + last if $1 eq 'wait_timeout'; # exit while() if end of vars + $_ = <INFILE>; # otherwise, read next line of vars + } + } + elsif($1 eq '=') # old style, manually added system variable values + { + print "read_infile: start old vars\n" if $op{debug}; + + while($_ && $_ =~ /=/) + { + chomp; + if(/^\s*(\w+)\s*=\s*([0-9.]+)(M*)\s*$/) # e.g.: key_buffer_size = 128M + { + $vars{$1} = ($3 ? $2 * 1024 * 1024 : $2); + print "read_infile: read '$_' as $1 = $vars{$1}\n" if $op{debug}; + } + else { print "read_infile: ignore '$_'\n" if $op{debug}; } + + $_ = <INFILE>; # otherwise, read next line of old vars + } + + redo; + } + else + { + print "read_infile: unrecognized line: '$_'\n" if $op{debug}; + } + } + + close INFILE; + + $real_uptime = $stats{'Uptime'}; + + $vars{'table_cache'} = $vars{'table_open_cache'} if exists $vars{'table_open_cache'}; + + get_MySQL_version(); +} + +sub get_MySQL_version +{ + print "get_MySQL_version\n" if $op{debug}; + + return if $MySQL_version; + + my ($major, $minor, $patch); + + if($op{'infile'} || $relative_infiles) + { + ($major, $minor, $patch) = ($vars{'version'} =~ /(\d{1,2})\.(\d{1,2})\.(\d{1,2})/); + } + else + { + my @row; + + $query = $dbh->prepare("SHOW VARIABLES LIKE 'version';"); + $query->execute(); + @row = $query->fetchrow_array(); + ($major, $minor, $patch) = ($row[1] =~ /(\d{1,2})\.(\d{1,2})\.(\d{1,2})/); + } + + $MySQL_version = sprintf("%d%02d%02d", $major, $minor, $patch); + + # Innodb_ status values were added in 5.0.2 + if($MySQL_version < 50002) + { + $have_innodb_vals = 0; + print "get_MySQL_version: no InnoDB reports because MySQL version is older than 5.0.2\n" if $op{debug}; + } +} + +sub set_myisam_vals +{ + print "set_myisam_vals\n" if $op{debug}; + + $questions = $stats{'Questions'}; + + $key_read_ratio = sprintf "%.2f", + ($stats{'Key_read_requests'} ? + 100 - ($stats{'Key_reads'} / $stats{'Key_read_requests'}) * 100 : + 0); + + $key_write_ratio = sprintf "%.2f", + ($stats{'Key_write_requests'} ? + 100 - ($stats{'Key_writes'} / $stats{'Key_write_requests'}) * 100 : + 0); + + $key_cache_block_size = (defined $vars{'key_cache_block_size'} ? + $vars{'key_cache_block_size'} : + 1024); + + $key_buffer_used = $stats{'Key_blocks_used'} * $key_cache_block_size; + + if(defined $stats{'Key_blocks_unused'}) # MySQL 4.1.2+ + { + $key_buffer_usage = $vars{'key_buffer_size'} - + ($stats{'Key_blocks_unused'} * $key_cache_block_size); + } + else { $key_buffer_usage = -1; } + + # Data Manipulation Statements: http://dev.mysql.com/doc/refman/5.0/en/data-manipulation.html + %DMS_vals = + ( + SELECT => $stats{'Com_select'}, + INSERT => $stats{'Com_insert'} + $stats{'Com_insert_select'}, + REPLACE => $stats{'Com_replace'} + $stats{'Com_replace_select'}, + UPDATE => $stats{'Com_update'} + + (exists $stats{'Com_update_multi'} ? $stats{'Com_update_multi'} : 0), + DELETE => $stats{'Com_delete'} + + (exists $stats{'Com_delete_multi'} ? $stats{'Com_delete_multi'} : 0) + ); + + $dms = $DMS_vals{SELECT} + $DMS_vals{INSERT} + $DMS_vals{REPLACE} + $DMS_vals{UPDATE} + $DMS_vals{DELETE}; + + $slow_query_t = format_u_time($vars{long_query_time}); + +} + +sub set_ib_vals +{ + print "set_ib_vals\n" if $op{debug}; + + $ib_bp_used = ($stats{'Innodb_buffer_pool_pages_total'} - + $stats{'Innodb_buffer_pool_pages_free'}) * + $stats{'Innodb_page_size'}; + + $ib_bp_total = $stats{'Innodb_buffer_pool_pages_total'} * $stats{'Innodb_page_size'}; + + $ib_bp_read_ratio = sprintf "%.2f", + ($stats{'Innodb_buffer_pool_read_requests'} ? + 100 - ($stats{'Innodb_buffer_pool_reads'} / + $stats{'Innodb_buffer_pool_read_requests'}) * 100 : + 0); +} + +sub write_relative_report +{ + print "write_relative_report\n" if $op{debug}; + + %stats_present = %stats; + + for(keys %stats) + { + if($stats_past{$_} =~ /\d+/) + { + if($stats_present{$_} >= $stats_past{$_}) # Avoid negative values + { + $stats{$_} = $stats_present{$_} - $stats_past{$_}; + } + } + } + + # These values are either "at present" or "high water marks". + # Therefore, it is more logical to not relativize these values. + # Doing otherwise causes strange and misleading values. + $stats{'Key_blocks_used'} = $stats_present{'Key_blocks_used'}; + $stats{'Open_tables'} = $stats_present{'Open_tables'}; + $stats{'Max_used_connections'} = $stats_present{'Max_used_connections'}; + $stats{'Threads_running'} = $stats_present{'Threads_running'}; + $stats{'Threads_connected'} = $stats_present{'Threads_connected'}; + $stats{'Threads_cached'} = $stats_present{'Threads_cached'}; + $stats{'Qcache_free_blocks'} = $stats_present{'Qcache_free_blocks'}; + $stats{'Qcache_total_blocks'} = $stats_present{'Qcache_total_blocks'}; + $stats{'Qcache_free_memory'} = $stats_present{'Qcache_free_memory'}; + if($have_innodb_vals) + { + $stats{'Innodb_page_size'} = $stats_present{'Innodb_page_size'}; + $stats{'Innodb_buffer_pool_pages_data'} = $stats_present{'Innodb_buffer_pool_pages_data'}; + $stats{'Innodb_buffer_pool_pages_dirty'} = $stats_present{'Innodb_buffer_pool_pages_dirty'}; + $stats{'Innodb_buffer_pool_pages_free'} = $stats_present{'Innodb_buffer_pool_pages_free'}; + $stats{'Innodb_buffer_pool_pages_latched'} = $stats_present{'Innodb_buffer_pool_pages_latched'}; + $stats{'Innodb_buffer_pool_pages_misc'} = $stats_present{'Innodb_buffer_pool_pages_misc'}; + $stats{'Innodb_buffer_pool_pages_total'} = $stats_present{'Innodb_buffer_pool_pages_total'}; + $stats{'Innodb_data_pending_fsyncs'} = $stats_present{'Innodb_data_pending_fsyncs'}; + $stats{'Innodb_data_pending_reads'} = $stats_present{'Innodb_data_pending_reads'}; + $stats{'Innodb_data_pending_writes'} = $stats_present{'Innodb_data_pending_writes'}; + + # Innodb_row_lock_ values were added in MySQL 5.0.3 + if($MySQL_version >= 50003) + { + $stats{'Innodb_row_lock_current_waits'} = $stats_present{'Innodb_row_lock_current_waits'}; + $stats{'Innodb_row_lock_time_avg'} = $stats_present{'Innodb_row_lock_time_avg'}; + $stats{'Innodb_row_lock_time_max'} = $stats_present{'Innodb_row_lock_time_max'}; + } + } + + get_Com_values(); + + %stats_past = %stats_present; + + set_myisam_vals(); + set_ib_vals() if $have_innodb_vals; + + write_report(); +} + +sub write_report +{ + print "write_report\n" if $op{debug}; + + $~ = 'MYSQL_TIME', write; + $~ = 'KEY_BUFF_MAX', write; + if($key_buffer_usage != -1) { $~ = 'KEY_BUFF_USAGE', write } + $~ = 'KEY_RATIOS', write; + write_DTQ(); + $~ = 'SLOW_DMS', write; + write_DMS(); + write_Com(); + $~ = 'SAS', write; + write_qcache(); + $~ = 'REPORT_END', write; + $~ = 'TAB', write; + + write_InnoDB() if $have_innodb_vals; +} + +sub sec_to_dhms # Seconds to days hours:minutes:seconds +{ + my $s = shift; + my ($d, $h, $m) = (0, 0, 0); + + return '0 0:0:0' if $s <= 0; + + if($s >= 86400) + { + $d = int $s / 86400; + $s -= $d * 86400; + } + + if($s >= 3600) + { + $h = int $s / 3600; + $s -= $h * 3600; + } + + $m = int $s / 60; + $s -= $m * 60; + + return "$d $h:$m:$s"; +} + +sub make_short +{ + my ($number, $kb, $d) = @_; + my $n = 0; + my $short; + + $d ||= 2; + + if($kb) { while ($number > 1023) { $number /= 1024; $n++; }; } + else { while ($number > 999) { $number /= 1000; $n++; }; } + + $short = sprintf "%.${d}f%s", $number, ('','k','M','G','T')[$n]; + if($short =~ /^(.+)\.(00)$/) { return $1; } # 12.00 -> 12 but not 12.00k -> 12k + + return $short; +} + +# What began as a simple but great idea has become the new standard: +# long_query_time in microseconds. For MySQL 5.1.21+ and 6.0.4+ this +# is now standard. For 4.1 and 5.0 patches, the architects of this +# idea provide: http://www.mysqlperformanceblog.com/mysql-patches/ +# Relevant notes in MySQL manual: +# http://dev.mysql.com/doc/refman/5.1/en/slow-query-log.html +# http://dev.mysql.com/doc/refman/6.0/en/slow-query-log.html +# +# The format_u_time sub simply beautifies long_query_time. + +sub format_u_time # format microsecond (�) time value +{ + # 0.000000 - 0.000999 = 0 - 999 � + # 0.001000 - 0.999999 = 1 ms - 999.999 ms + # 1.000000 - n.nnnnnn = 1 s - n.nnnnn s + + my $t = shift; + my $f; # formatted � time + my $u = chr(($WIN ? 230 : 181)); + + $t = 0 if $t < 0; + + if($t > 0 && $t <= 0.000999) + { + $f = ($t * 1000000) . " $u"; + } + elsif($t >= 0.001000 && $t <= 0.999999) + { + $f = ($t * 1000) . ' ms'; + } + elsif($t >= 1) + { + $f = ($t * 1) . ' s'; # * 1 to remove insignificant zeros + } + else + { + $f = 0; # $t should = 0 at this point + } + + return $f; +} + +sub perc # Percentage +{ + my($is, $of) = @_; + $is = 0 if (not defined $is); + return sprintf "%.2f", ($is * 100) / ($of ||= 1); +} + +sub t # Time average per second +{ + my $val = shift; + return 0 if !$val; + return(make_short($val / $stats{'Uptime'}, 0, 1)); +} + +sub email_report # Email given report to $op{'email'} +{ + print "email_report\n" if $op{debug}; + + return if $WIN; + + my $report = shift; + + open SENDMAIL, "|/usr/sbin/sendmail -t"; + print SENDMAIL "From: mysqlreport\n"; + print SENDMAIL "To: $op{email}\n"; + print SENDMAIL "Subject: MySQL status report on " . ($mycnf{'host'} || 'localhost') . "\n\n"; + print SENDMAIL `cat $report`; + close SENDMAIL; +} + +sub cat_report # Print given report to screen +{ + print "cat_report\n" if $op{debug}; + + my $report = shift; + my @report; + + open REPORT, "< $report"; + @report = <REPORT>; + close REPORT; + print @report; +} + +sub get_Com_values +{ + print "get_Com_values\n" if $op{debug}; + + %Com_vals = (); + + # Make copy of just the Com_ values + for(keys %stats) + { + if(grep /^Com_/, $_ and $stats{$_} > 0) + { + /^Com_(.*)/; + $Com_vals{$1} = $stats{$_}; + } + } + + # Remove DMS values + delete $Com_vals{'select'}; + delete $Com_vals{'insert'}; + delete $Com_vals{'insert_select'}; + delete $Com_vals{'replace'}; + delete $Com_vals{'replace_select'}; + delete $Com_vals{'update'}; + delete $Com_vals{'update_multi'} if exists $Com_vals{'update_multi'}; + delete $Com_vals{'delete'}; + delete $Com_vals{'delete_multi'} if exists $Com_vals{'delete_multi'}; +} + +sub write_DTQ # Write DTQ report in descending order by values +{ + print "write_DTQ\n" if $op{debug}; + + $~ = 'DTQ'; + + my %DTQ; + my $first = 1; + + # Total Com values + $stat_val = 0; + for(values %Com_vals) { $stat_val += $_; } + $DTQ{'Com_'} = $stat_val; + + $DTQ{'DMS'} = $dms; + $DTQ{'QC Hits'} = $stats{'Qcache_hits'} if $stats{'Qcache_hits'} != 0; + $DTQ{'COM_QUIT'} = int (($stats{'Connections'} - 2) - ($stats{'Aborted_clients'} / 2)); + + $stat_val = 0; + for(values %DTQ) { $stat_val += $_; } + if($questions != $stat_val) + { + $DTQ{($questions > $stat_val ? '+Unknown' : '-Unknown')} = abs $questions - $stat_val; + } + + for(sort { $DTQ{$b} <=> $DTQ{$a} } keys(%DTQ)) + { + if($first) { $stat_label = '%Total:'; $first = 0; } + else { $stat_label = ''; } + + $stat_name = $_; + $stat_val = $DTQ{$_}; + write; + } +} + +sub write_DMS # Write DMS report in descending order by values +{ + print "write_DMS\n" if $op{debug}; + + $~ = 'DMS'; + + for(sort { $DMS_vals{$b} <=> $DMS_vals{$a} } keys(%DMS_vals)) + { + $stat_name = $_; + $stat_val = $DMS_vals{$_}; + write; + } +} + +sub write_Com # Write COM report in descending order by values +{ + print "write_Com\n" if $op{debug}; + + my $i = $op{'com'}; + + $~ = 'COM_1'; + + # Total Com values and write first line of COM report + $stat_label = '%Total:' unless $op{'dtq'}; + $stat_val = 0; + for(values %Com_vals) { $stat_val += $_; } + write; + + $~ = 'COM_2'; + + # Sort remaining Com values, print only the top $op{'com'} number of values + for(sort { $Com_vals{$b} <=> $Com_vals{$a} } keys(%Com_vals)) + { + $stat_name = $_; + $stat_val = $Com_vals{$_}; + write; + + last if !(--$i); + } +} + +sub write_qcache +{ + print "write_qcache\n" if $op{debug}; + + # Query cache was added in 4.0.1, but have_query_cache was added in 4.0.2, + # ergo this method is slightly more reliable + return if not exists $vars{'query_cache_size'}; + return if $vars{'query_cache_size'} == 0; + + $qc_mem_used = $vars{'query_cache_size'} - $stats{'Qcache_free_memory'}; + $qc_hi_r = sprintf "%.2f", $stats{'Qcache_hits'} / ($stats{'Qcache_inserts'} ||= 1); + $qc_ip_r = sprintf "%.2f", $stats{'Qcache_inserts'} / ($stats{'Qcache_lowmem_prunes'} ||= 1); + + $~ = 'QCACHE'; + write; +} + +sub write_InnoDB +{ + print "write_InnoDB\n" if $op{debug}; + + return if not defined $stats{'Innodb_page_size'}; + + $stats{'Innodb_buffer_pool_pages_latched'} = 0 if not defined $stats{'Innodb_buffer_pool_pages_latched'}; + + $~ = 'IB'; + write; + + # Innodb_row_lock_ values were added in MySQL 5.0.3 + if($MySQL_version >= 50003) + { + $~ = 'IB_LOCK'; + write; + } + + # Data, Pages, Rows + $~ = 'IB_DPR'; + write; +} + +sub have_op +{ + my $key = shift; + return 1 if (exists $op{$key} && $op{$key} ne ''); + return 0; +} + +sub sig_handler +{ + print "\nReceived signal at " , scalar localtime , "\n"; + exit_tasks_and_cleanup(); + exit; +} + +sub exit_tasks_and_cleanup +{ + print "exit_tasks_and_cleanup\n" if $op{debug}; + + close $tmpfile_fh; + select STDOUT unless $op{'detach'}; + + email_report($tmpfile) if $op{'email'}; + + cat_report($tmpfile) unless $op{'detach'}; + + if($op{'outfile'}) + { + if($WIN) { `move $tmpfile $op{outfile}`; } + else { `mv $tmpfile $op{outfile}`; } + } + else + { + if($WIN) { `del $tmpfile`; } + else { `rm -f $tmpfile`; } + } + + if(!$op{'infile'} && !$relative_infiles) + { + if($op{'flush-status'}) + { + $query = $dbh->prepare("FLUSH STATUS;"); + $query->execute(); + } + + $query->finish(); + $dbh->disconnect(); + } +} + +# +# Formats +# + +format MYSQL_TIME = +MySQL @<<<<<<<<<<<<<<<< uptime @<<<<<<<<<<< @>>>>>>>>>>>>>>>>>>>>>>>> +$vars{'version'}, sec_to_dhms($real_uptime), (($op{infile} || $relative_infiles) ? '' : scalar localtime) +. + +format KEY_BUFF_MAX = + +__ Key _________________________________________________________________ +Buffer used @>>>>>> of @>>>>>> %Used: @>>>>> +make_short($key_buffer_used, 1), make_short($vars{'key_buffer_size'}, 1), perc($key_buffer_used, $vars{'key_buffer_size'}) +. + +format KEY_BUFF_USAGE = + Current @>>>>>> %Usage: @>>>>> +make_short($key_buffer_usage, 1), perc($key_buffer_usage, $vars{'key_buffer_size'}) +. + +format KEY_RATIOS = +Write hit @>>>>>% +$key_write_ratio +Read hit @>>>>>% +$key_read_ratio + +__ Questions ___________________________________________________________ +Total @>>>>>>>> @>>>>>/s +make_short($questions), t($questions) +. + +format DTQ = + @<<<<<<< @>>>>>>>> @>>>>>/s @>>>>>> @>>>>> +$stat_name, make_short($stat_val), t($stat_val), $stat_label, perc($stat_val, $questions) +. + +format SLOW_DMS = +Slow @<<<<<<< @>>>>>> @>>>>>/s @>>>>> %DMS: @>>>>> Log: @>> +$slow_query_t, make_short($stats{'Slow_queries'}), t($stats{'Slow_queries'}), perc($stats{'Slow_queries'}, $questions), perc($stats{'Slow_queries'}, $dms), $vars{'log_slow_queries'} +DMS @>>>>>>>> @>>>>>/s @>>>>> +make_short($dms), t($dms), perc($dms, $questions) +. + +format DMS = + @<<<<<<< @>>>>>>>> @>>>>>/s @>>>>> @>>>>> +$stat_name, make_short($stat_val), t($stat_val), perc($stat_val, $questions), perc($stat_val, $dms) +. + +format COM_1 = +Com_ @>>>>>>>> @>>>>>/s @>>>>> +make_short($stat_val), t($stat_val), perc($stat_val, $questions) +. + +format COM_2 = + @<<<<<<<<<< @>>>>>> @>>>>>/s @>>>>> +$stat_name, make_short($stat_val), t($stat_val), perc($stat_val, $questions) +. + +format SAS = + +__ SELECT and Sort _____________________________________________________ +Scan @>>>>>> @>>>>/s %SELECT: @>>>>> +make_short($stats{'Select_scan'}), t($stats{'Select_scan'}), perc($stats{'Select_scan'}, $stats{'Com_select'}) +Range @>>>>>> @>>>>/s @>>>>> +make_short($stats{'Select_range'}), t($stats{'Select_range'}), perc($stats{'Select_range'}, $stats{'Com_select'}) +Full join @>>>>>> @>>>>/s @>>>>> +make_short($stats{'Select_full_join'}), t($stats{'Select_full_join'}), perc($stats{'Select_full_join'}, $stats{'Com_select'}) +Range check @>>>>>> @>>>>/s @>>>>> +make_short($stats{'Select_range_check'}), t($stats{'Select_range_check'}), perc($stats{'Select_range_check'}, $stats{'Com_select'}) +Full rng join @>>>>>> @>>>>/s @>>>>> +make_short($stats{'Select_full_range_join'}), t($stats{'Select_full_range_join'}), perc($stats{'Select_full_range_join'}, $stats{'Com_select'}) +Sort scan @>>>>>> @>>>>/s +make_short($stats{'Sort_scan'}), t($stats{'Sort_scan'}) +Sort range @>>>>>> @>>>>/s +make_short($stats{'Sort_range'}), t($stats{'Sort_range'}) +Sort mrg pass @>>>>>> @>>>>/s +make_short($stats{'Sort_merge_passes'}), t($stats{'Sort_merge_passes'}) +. + +format QCACHE = + +__ Query Cache _________________________________________________________ +Memory usage @>>>>>> of @>>>>>> %Used: @>>>>> +make_short($qc_mem_used, 1), make_short($vars{'query_cache_size'}, 1), perc($qc_mem_used, $vars{'query_cache_size'}) +Block Fragmnt @>>>>>% +perc($stats{'Qcache_free_blocks'}, $stats{'Qcache_total_blocks'}) +Hits @>>>>>> @>>>>/s +make_short($stats{'Qcache_hits'}), t($stats{'Qcache_hits'}) +Inserts @>>>>>> @>>>>/s +make_short($stats{'Qcache_inserts'}), t($stats{'Qcache_inserts'}) +Insrt:Prune @>>>>>>:1 @>>>>/s +make_short($qc_ip_r), t($stats{'Qcache_inserts'} - $stats{'Qcache_lowmem_prunes'}) +Hit:Insert @>>>>>>:1 +$qc_hi_r, t($qc_hi_r) +. + +# Not really the end... +format REPORT_END = + +__ Table Locks _________________________________________________________ +Waited @>>>>>>>> @>>>>>/s %Total: @>>>>> +make_short($stats{'Table_locks_waited'}), t($stats{'Table_locks_waited'}), perc($stats{'Table_locks_waited'}, $stats{'Table_locks_waited'} + $stats{'Table_locks_immediate'}); +Immediate @>>>>>>>> @>>>>>/s +make_short($stats{'Table_locks_immediate'}), t($stats{'Table_locks_immediate'}) + +__ Tables ______________________________________________________________ +Open @>>>>>>>> of @>>> %Cache: @>>>>> +$stats{'Open_tables'}, $vars{'table_cache'}, perc($stats{'Open_tables'}, $vars{'table_cache'}) +Opened @>>>>>>>> @>>>>>/s +make_short($stats{'Opened_tables'}), t($stats{'Opened_tables'}) + +__ Connections _________________________________________________________ +Max used @>>>>>>>> of @>>> %Max: @>>>>> +$stats{'Max_used_connections'}, $vars{'max_connections'}, perc($stats{'Max_used_connections'}, $vars{'max_connections'}) +Total @>>>>>>>> @>>>>>/s +make_short($stats{'Connections'}), t($stats{'Connections'}) + +__ Created Temp ________________________________________________________ +Disk table @>>>>>>>> @>>>>>/s +make_short($stats{'Created_tmp_disk_tables'}), t($stats{'Created_tmp_disk_tables'}) +Table @>>>>>>>> @>>>>>/s Size: @>>>>> +make_short($stats{'Created_tmp_tables'}), t($stats{'Created_tmp_tables'}), make_short($vars{'tmp_table_size'}, 1, 1) +File @>>>>>>>> @>>>>>/s +make_short($stats{'Created_tmp_files'}), t($stats{'Created_tmp_files'}) +. + +format TAB = + +__ Threads _____________________________________________________________ +Running @>>>>>>>> of @>>> +$stats{'Threads_running'}, $stats{'Threads_connected'} +Cached @>>>>>>>> of @>>> %Hit: @>>>>> +$stats{'Threads_cached'}, $vars{'thread_cache_size'}, make_short(100 - perc($stats{'Threads_created'}, $stats{'Connections'})) +Created @>>>>>>>> @>>>>>/s +make_short($stats{'Threads_created'}), t($stats{'Threads_created'}) +Slow @>>>>>>>> @>>>>>/s +$stats{'Slow_launch_threads'}, t($stats{'Slow_launch_threads'}) + +__ Aborted _____________________________________________________________ +Clients @>>>>>>>> @>>>>>/s +make_short($stats{'Aborted_clients'}), t($stats{'Aborted_clients'}) +Connects @>>>>>>>> @>>>>>/s +make_short($stats{'Aborted_connects'}), t($stats{'Aborted_connects'}) + +__ Bytes _______________________________________________________________ +Sent @>>>>>>>> @>>>>>/s +make_short($stats{'Bytes_sent'}), t($stats{'Bytes_sent'}) +Received @>>>>>>>> @>>>>>/s +make_short($stats{'Bytes_received'}), t($stats{'Bytes_received'}) +. + +format IB = + +__ InnoDB Buffer Pool __________________________________________________ +Usage @>>>>>> of @>>>>>> %Used: @>>>>> +make_short($ib_bp_used, 1), make_short($ib_bp_total, 1), perc($ib_bp_used, $ib_bp_total) +Read hit @>>>>>% +$ib_bp_read_ratio; +Pages + Free @>>>>>>>> %Total: @>>>>> +make_short($stats{'Innodb_buffer_pool_pages_free'}), perc($stats{'Innodb_buffer_pool_pages_free'}, $stats{'Innodb_buffer_pool_pages_total'}) + Data @>>>>>>>> @>>>>> %Drty: @>>>>> +make_short($stats{'Innodb_buffer_pool_pages_data'}), perc($stats{'Innodb_buffer_pool_pages_data'}, $stats{'Innodb_buffer_pool_pages_total'}), perc($stats{'Innodb_buffer_pool_pages_dirty'}, $stats{'Innodb_buffer_pool_pages_data'}) + Misc @>>>>>>>> @>>>>> + $stats{'Innodb_buffer_pool_pages_misc'}, perc($stats{'Innodb_buffer_pool_pages_misc'}, $stats{'Innodb_buffer_pool_pages_total'}) + Latched @>>>>>>>> @>>>>> +$stats{'Innodb_buffer_pool_pages_latched'}, perc($stats{'Innodb_buffer_pool_pages_latched'}, $stats{'Innodb_buffer_pool_pages_total'}) +Reads @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_buffer_pool_read_requests'}), t($stats{'Innodb_buffer_pool_read_requests'}) + From file @>>>>>>>> @>>>>>/s @>>>>> +make_short($stats{'Innodb_buffer_pool_reads'}), t($stats{'Innodb_buffer_pool_reads'}), perc($stats{'Innodb_buffer_pool_reads'}, $stats{'Innodb_buffer_pool_read_requests'}) + Ahead Rnd @>>>>>>>> @>>>>>/s +$stats{'Innodb_buffer_pool_read_ahead_rnd'}, t($stats{'Innodb_buffer_pool_read_ahead_rnd'}) + Ahead Sql @>>>>>>>> @>>>>>/s +$stats{'Innodb_buffer_pool_read_ahead_seq'}, t($stats{'Innodb_buffer_pool_read_ahead_seq'}) +Writes @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_buffer_pool_write_requests'}), t($stats{'Innodb_buffer_pool_write_requests'}) +Flushes @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_buffer_pool_pages_flushed'}), t($stats{'Innodb_buffer_pool_pages_flushed'}) +Wait Free @>>>>>>>> @>>>>>/s +$stats{'Innodb_buffer_pool_wait_free'}, t($stats{'Innodb_buffer_pool_wait_free'}) +. + +format IB_LOCK = + +__ InnoDB Lock _________________________________________________________ +Waits @>>>>>>>> @>>>>>/s +$stats{'Innodb_row_lock_waits'}, t($stats{'Innodb_row_lock_waits'}) +Current @>>>>>>>> +$stats{'Innodb_row_lock_current_waits'} +Time acquiring + Total @>>>>>>>> ms +$stats{'Innodb_row_lock_time'} + Average @>>>>>>>> ms +$stats{'Innodb_row_lock_time_avg'} + Max @>>>>>>>> ms +$stats{'Innodb_row_lock_time_max'} +. + +format IB_DPR = + +__ InnoDB Data, Pages, Rows ____________________________________________ +Data + Reads @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_data_reads'}), t($stats{'Innodb_data_reads'}) + Writes @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_data_writes'}), t($stats{'Innodb_data_writes'}) + fsync @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_data_fsyncs'}), t($stats{'Innodb_data_fsyncs'}) + Pending + Reads @>>>>>>>> +$stats{'Innodb_data_pending_reads'}, t($stats{'Innodb_data_pending_reads'}) + Writes @>>>>>>>> +$stats{'Innodb_data_pending_writes'}, t($stats{'Innodb_data_pending_writes'}) + fsync @>>>>>>>> +$stats{'Innodb_data_pending_fsyncs'}, t($stats{'Innodb_data_pending_fsyncs'}) + +Pages + Created @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_pages_created'}), t($stats{'Innodb_pages_created'}) + Read @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_pages_read'}), t($stats{'Innodb_pages_read'}) + Written @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_pages_written'}), t($stats{'Innodb_pages_written'}) + +Rows + Deleted @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_rows_deleted'}), t($stats{'Innodb_rows_deleted'}) + Inserted @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_rows_inserted'}), t($stats{'Innodb_rows_inserted'}) + Read @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_rows_read'}), t($stats{'Innodb_rows_read'}) + Updated @>>>>>>>> @>>>>>/s +make_short($stats{'Innodb_rows_updated'}), t($stats{'Innodb_rows_updated'}) +. === added file 'storage/xtradb/build/debian/additions/mysqlreport.1' --- a/storage/xtradb/build/debian/additions/mysqlreport.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqlreport.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,180 @@ +.TH "mysqlreport" "1" "2.5 2006-09-01 (docrev 2006-05-19)" "Daniel Nichter" "MYSQL" +.SH "NAME" +.LP +mysqlreport \- Makes a friendly report of important MySQL status values +.SH "SYNTAX" +.LP +mysqlreport [\fIoptions\fP] +.SH "DESCRIPTION" +.LP +mysqlreport makes a friendly report of important MySQL status values. Actually, +it makes a friendly report of nearly every status value from SHOW STATUS. +Unlike SHOW STATUS which simply dumps over 100 values to screen in one long +list, mysqlreport interprets and formats the values and presents the basic +values and many more inferred values in a human\-readable format. Numerous +example reports are available at the mysqlreport web page at +http://hackmysql.com/mysqlreport. + +The benefit of mysqlreport is that it allows you to very quickly see a wide +array of performance indicators for your MySQL server which would otherwise +need to be calculated by hand from all the various SHOW STATUS values. For +example, the Index Read Ratio is an important value but it's not present in +SHOW STATUS; it's an inferred value (the ratio of Key_reads to +Key_read_requests). + +This documentation outlines all the command line options in mysqlreport, most +of which control which reports are printed. This document does not address +how to interpret these reports; that topic is covered in the document Guide +To Understanding mysqlreport at http://hackmysql.com/mysqlreportguide. + +.SH "OPTIONS" +Technically, command line options are in the form \-\-option, but \-option works +too. All options can be abbreviated if the abbreviation is unique. For example, +option \-\-host can be abbreviated \-\-ho but not \-\-h because \-\-h is ambiguous: it +could mean \-\-host or \-\-help. + +.LP + +.TP +\fB\-\-help\fR +Output help information and exit. + +.TP +\fB\-\-user USER\fR + +.TP +\fB\-\-password\fR +As of version 2.3 \-\-password can take the password on the +command line like "\-\-password FOO". Using \-\-password +alone without giving a password on the command line +causes mysqlreport to prompt for a password. + +.TP +\fB\-\-host ADDRESS\fR + +.TP +\fB\-\-port PORT\fR + +.TP +\fB\-\-socket SOCKET\fR + +.TP +\fB\-\-no\-mycnf\fR +\-\-no\-mycnf makes mysqlreport not read ~/.my.cnf which it does by default +otherwise. \-\-user and \-\-password always override values from ~/.my.cnf. + +.TP +\fB\-\-dtq\fR +Print Distribution of Total Queries (DTQ) report (under +Total in Questions report). Queries (or Questions) can +be divided into four main areas: DMS (see \-\-dms below), +Com_ (see \-\-com below), COM_QUIT (see COM_QUIT and +Questions at http://hackmysql.com/com_quit) and +Unknown. \-\-dtq lists the number of queries in each of +these areas in descending order. + +.TP +\fB\-\-dms\fR +Print Data Manipulation Statements (DMS) report (under +DMS in Questions report). DMS are those from the MySQL +manual section 13.2. Data Manipulation Statements. +(Currently, mysqlreport considers only SELECT, INSERT, +REPLACE, UPDATE, and DELETE.) Each DMS is listed in +descending order by count. + +.TP +\fB\-\-com N\fR +Print top N number of non\-DMS Com_ status values in +descending order (after DMS in Questions report). If N +is not given, default is 3. Such non\-DMS Com_ values +include Com_change_db, Com_show_tables, Com_rollback, +etc. + +.TP +\fB\-\-sas\fR +Print report for Select_ and Sort_ status values (after +Questions report). See MySQL Select and Sort Status +Variables at http://hackmysql.com/selectandsort. + +.TP +\fB\-\-tab\fR +Print Threads, Aborted, and Bytes status reports (after +Created temp report). As of mysqlreport v2.3 the +Threads report reports on all Threads_ status values. + +.TP +\fB\-\-qcache\fR +Print Query Cache report. +.TP +\fB\-\-all\fR +Equivalent to "\-\-dtq \-\-dms \-\-com 3 \-\-sas \-\-qcache". +(Notice \-\-tab is not invoked by \-\-all.) + +.TP +\fB\-\-infile FILE\fR +Instead of getting SHOW STATUS values from MySQL, read +values from FILE. FILE is often a copy of the output of +SHOW STATUS including formatting characters (|, +, \-). +mysqlreport expects FILE to have the format +" value number " where value is only alpha and +underscore characters (A\-Z and _) and number is a +positive integer. Anything before, between, or after +value and number is ignored. mysqlreport also needs +the following MySQL server variables: version, +table_cache, max_connections, key_buffer_size, +query_cache_size. These values can be specified in +INFILE in the format "name = value" where name is one +of the aforementioned server variables and value is a +positive integer with or without a trailing M and +possible periods (for version). For example, to specify +an 18M key_buffer_size: key_buffer_size = 18M. Or, a +256 table_cache: table_cache = 256. The M implies +Megabytes not million, so 18M means 18,874,368 not +18,000,000. If these server variables are not specified +the following defaults are used (respectively) which +may cause strange values to be reported: 0.0.0, 64, +100, 8M, 0. + +.TP +\fB\-\-outfile FILE\fR +After printing the report to screen, print the report +to FILE too. Internally, mysqlreport always writes the +report to a temp file first: /tmp/mysqlreport.PID on +*nix, c:\mysqlreport.PID on Windows (PID is the +script's process ID). Then it prints the temp file to +screen. Then if \-\-outfile is specified, the temp file +is copied to OUTFILE. After \-\-email (below), the temp +file is deleted. + +.TP +\fB\-\-email ADDRESS\fR +After printing the report to screen, email the report +to ADDRESS. This option requires sendmail in +/usr/sbin/, therefore it does not work on Windows. +/usr/sbin/sendmail can be a sym link to qmail, for +example, or any MTA that emulates sendmail's \-t +command line option and operation. The FROM: field is +"mysqlreport", SUBJECT: is "MySQL status report". + +.TP +\fB\-\-flush\-status\fR +Execute a "FLUSH STATUS;" after generating the reports. +If you do not have permissions in MySQL to do this an +error from DBD::mysql::st will be printed after the +reports. + +.SH "AUTHORS" +.LP +Daniel Nichter + +If mysqlreport breaks, send me a message from +http://hackmysql.com/feedback +with the error. + +.SH "SEE ALSO" +.LP +mytop(1) +.LP +The comprehensive Guide To Understanding mysqlreport at +http://hackmysql.com/mysqlreportguide. + === added file 'storage/xtradb/build/debian/additions/mysqltest.1' --- a/storage/xtradb/build/debian/additions/mysqltest.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/mysqltest.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +mysqltest \- Regressiontest program for MySQL. +.SH SYNOPSIS +mysqltest [options] +.SH DESCRIPTION +Runs a test against the mysql server and compares output with a results file. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/pack_isam.1' --- a/storage/xtradb/build/debian/additions/pack_isam.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/pack_isam.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,19 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +myisampack \- Compresses MySQL database files. +.SH SYNOPSIS +myisampack [options] +.SH DESCRIPTION +Pack a ISAM-table to take much smaller space +Keys are not updated, so you must run isamchk -rq on any table +that has keys after you have compressed it +You should give the .ISM file as the filename argument + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/resolve_stack_dump.1' --- a/storage/xtradb/build/debian/additions/resolve_stack_dump.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/resolve_stack_dump.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +resolve_stack_dump \- MySQL helper program for reporting bugs. +.SH SYNOPSIS +resolve_stack_dump [options] +.SH DESCRIPTION +Resolve numeric stack strace dump into symbols. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/additions/resolveip.1' --- a/storage/xtradb/build/debian/additions/resolveip.1 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/additions/resolveip.1 2010-01-26 18:02:46 +0000 @@ -0,0 +1,16 @@ +.TH mysql 1 "17 March 2003" "MySQL 3.23" "MySQL database" +.SH NAME +resolveip \- MySQL helper program to retrive IP addresses. +.SH SYNOPSIS +resolveip [options] +.SH DESCRIPTION +Get hostname based on IP-address or IP-address based on hostname. + +For more information start the program with '--help'. +.SH "SEE ALSO" +mysql (1), mysqld (1) +.SH AUTHOR +This manpage was written by Christian Hammers <ch(a)debian.org>. + +MySQL is available at http://www.mysql.com/. +.\" end of man page === added file 'storage/xtradb/build/debian/changelog' --- a/storage/xtradb/build/debian/changelog 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/changelog 2010-02-18 23:51:40 +0000 @@ -0,0 +1,4186 @@ +percona-xtradb-dfsg-5.1 (5.1.36-1) experimental; urgency=low + + [TODO] + * Link libmysqlclient.so to libmysqlclient_r.so to help applications + like Apache where some modules, like libaprutil, want to use the thread + safe library and some, like PHP, do not. As the client library just copies + data between client and server, we do not expect significant performance + losses. (thanks to Stefan Fritsch). Closes: #450535 + + Add the following to libmysqlclient16.links: + usr/lib/libmysqlclient_r.so.16.0.0 usr/lib/libmysqlclient.so.16.0.0 + + * Ex-maintainer upload :) + * New upstream release. + * SECURITY: Upstream fix for "mysql client does not escape strings in + --html mode." (CVE-2008-4456) Closes: #526254 + * Upstream fixes REPEAT() function. Closes: #447028 + * Upstream fixes problems when mixing ORDER and GROUP BY. Closes: #470854 + * There were many innodb fixes in the last two years, probably + also for this unreproducible crash. CLoses: #447713 + * Removed amd64 specific -fPIC compiler option that was introduced + especially for building the NDB cluster module which is no longer + part of this package (thanks to Modestas Vainius). Closes: #508406 + * Put /etc/mysql/conf.d to mysql-server-5.1.dirs (thanks to Alexander + Gerasiov). Closes: #515145 + * Fixed mysql-test suite by adding 50_mysql-test__db_test.dpatch. + It now passes 100% of the tests again. Also Closes: #533999 + * Preinst now prevents Installation if NDB configuration is detected. + * Applied Ubuntu patch that fixes privilege bootstrapping in postinst + (thanks to Mathias Gug). Closes: #535492 + * Applied Ubuntu patch that sets the debconf prio for the root password + question to high and prevents it from being asked on 5.0 -> 5.1 upgrades + (thanks to Mathias Gug). Closes: #535500 + * Removed the check for ISAM tables as the only supported upgrade path is + from lenny's MySQL-5.0. + * Added /etc/mysql/conf.d/mysqld_safe_syslog.cnf which enables mysqld_safe + to pipe all mysqld output into the syslog. The reason for not letting dpkg + handle it via a normal config file change was that my.cnf is usually + heavily tuned by the admin so the setting would go lost too easily. + * Updated mysqlreport to version 3.5 (including two minor patches by me). + + -- Christian Hammers <ch(a)debian.org> Wed, 01 Jul 2009 20:54:58 +0200 + +mysql-dfsg-5.1 (5.1.34-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Mon, 20 Apr 2009 20:23:10 +0200 + +mysql-dfsg-5.1 (5.1.33-2) experimental; urgency=low + + * Remove no longer active developers from uploaders field. + * Drop workaround for upgrades from MySQL 3.23, not necessary any more. + + -- Norbert Tretkowski <nobse(a)debian.org> Tue, 07 Apr 2009 11:23:25 +0200 + +mysql-dfsg-5.1 (5.1.33-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Thu, 02 Apr 2009 21:12:23 +0200 + +mysql-dfsg-5.1 (5.1.32-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Fri, 06 Mar 2009 18:48:23 +0100 + +mysql-dfsg-5.1 (5.1.31-2) experimental; urgency=low + + * Update SSL certificates, and re-enable SSL related tests when running + the testsuite. + + -- Norbert Tretkowski <nobse(a)debian.org> Tue, 10 Feb 2009 16:08:42 +0100 + +mysql-dfsg-5.1 (5.1.31-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 08 Feb 2009 17:07:11 +0100 + +mysql-dfsg-5.1 (5.1.30-2) experimental; urgency=low + + * Drop MySQL Cluster support, it's deprecated since 5.1.24-RC. + * Fix FTBFS if build twice in a row. (closes: #487091) + + -- Norbert Tretkowski <nobse(a)debian.org> Fri, 05 Dec 2008 21:04:55 +0100 + +mysql-dfsg-5.1 (5.1.30-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Thu, 27 Nov 2008 09:09:55 +0100 + +mysql-dfsg-5.1 (5.1.29rc-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Mon, 27 Oct 2008 20:00:43 +0100 + +mysql-dfsg-5.1 (5.1.26rc-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Mon, 14 Jul 2008 21:46:59 +0200 + +mysql-dfsg-5.1 (5.1.25rc-1) experimental; urgency=low + + * New upstream release. + + -- Norbert Tretkowski <nobse(a)debian.org> Sat, 21 Jun 2008 13:55:02 +0200 + +mysql-dfsg-5.1 (5.1.24rc-1) experimental; urgency=low + + * New upstream release. + * Ignore errors in testsuite on ia64 and s390. + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 16 Apr 2008 22:03:44 +0200 + +mysql-dfsg-5.1 (5.1.23rc-1) experimental; urgency=low + + * New upstream release. + + [ Christian Hammers ] + * Add PIC support for NDB libraries on amd64 (thanks to Monty Taylor). + * Add extra information when aborting due to a detected downgrade (thanks to + Raphael Pinson). + * Move libndbclient.so.3 to its own package as it now has a version != 0 + (thanks to Raphael Pinson for reminding me). + + [ Monty Taylor ] + * Remove 85_ndb__staticlib.dpatch since we have a libndbclient package now. + * Add myself to the uploaders so that I don't get complaints about package + signing. + * Add libndbclient-dev package to go with libndbclient3. + + [ Norbert Tretkowski ] + * Update patches: + + 41_scripts__mysql_install_db.sh__no_test.dpatch + * Drop patches: + + 70_upstream_debian__configure.dpatch + + 71_upstream_debian__Makefile.in.dpatch + + 99_TEMP_minmax.dpatch + * Remove Adam Conrad from uploaders on his request. Thanks for your work in + the past! + * Ignore errors in testsuite on amd64 and i386. + + -- Norbert Tretkowski <nobse(a)debian.org> Fri, 29 Feb 2008 10:38:27 +0100 + +mysql-dfsg-5.1 (5.1.22rc-1) experimental; urgency=low + + * New upstream version. + * Let mysql-server-5.1 pre-depend on debconf as it uses it in the preinst. + * Fixed mysql-client-5.1 menu entry for upcoming menu policy 1.4. + + -- Christian Hammers <ch(a)debian.org> Tue, 02 Oct 2007 22:45:37 +0200 + +mysql-dfsg-5.1 (5.1.21beta-1) experimental; urgency=low + + * My "Greetings from FrOSCon!" release. + * New upstream version. + * libmysqlclient.so.15 has been superseded by libmysqlclient.so.16. + * Renamed libmysqlclient15-dev to libmysqlclient-dev but added an empty + package libmysqlclient15-dev to ease the transition for packages with + a versioned build-dep to libmysqlclient15-dev which is something that + currently does not work with "Provides:". + * Synced with 5.0 branch up to subversion release r909. + * Commented out most of the compile conditionals in the hope that + all architectures can be build the same way. + * Added a lot of new binaries and manpages. + * Switched to plugin based engines. + + -- Christian Hammers <ch(a)debian.org> Sat, 25 Aug 2007 14:24:40 +0200 + +mysql-dfsg-5.1 (5.1.19beta-1) experimental; urgency=low + + * New upstream release. + + -- Christian Hammers <ch(a)debian.org> Mon, 11 Jun 2007 23:18:35 +0200 + +mysql-dfsg-5.1 (5.1.16beta-4) experimental; urgency=high + + * Merged with 5.0 r850: + * SECURITY: + In some previous versions mysql_install_db was not idempotent and did + always create passwordless root accounts although it should only on + initial installs (thanks to Olaf van der Spek). Closes: #418672 + * Added check for passwordless root accounts to debian-start. + * As MySQL-5.0 is, at least currently, incompatible with Kernel 2.4 the + installation is aborted for such old kernels. Debian Etch does not + support them anyway according to the release notes but this might be + unexpected and many production servers still have self build ones + installed (thanks to Marc-Christian Petersen). See: #416841 + * Adjusted TeX build-deps to texlive. + * Added innotop. + * Changed maintainer email address to + pkg-mysql-commits(a)lists.alioth.debian.org + + -- Christian Hammers <ch(a)debian.org> Thu, 19 Apr 2007 19:29:29 +0200 + +mysql-dfsg-5.1 (5.1.16beta-3) experimental; urgency=low + + * Merged with 5.0 r837: + * Activated the blackhole engine as it's needed for replicating partition + designs (thanks to Cyril SCETBON). + * Fixed segfault on i486 systems without cpuid instruction (thanks to + Lennart Sorensen). Closes: #410474 + * Only use of the non-essential debconf package in postrm if it is + still installed (thanks to Michael Ablassmeier). Closes: #416838 + + -- Christian Hammers <ch(a)debian.org> Sun, 18 Mar 2007 21:48:11 +0100 + +mysql-dfsg-5.1 (5.1.16beta-2) experimental; urgency=low + + * Merged with 5.0 r818: + * Fixed FTBFS on Sparc introduced with the "make -j" trick in + 5.0.32-8 (thanks to Frank Lichtenheld). Closes: #415026 + + -- Christian Hammers <ch(a)debian.org> Sun, 18 Mar 2007 21:20:11 +0100 + +mysql-dfsg-5.1 (5.1.16beta-1) experimental; urgency=low + + * New upstream release. + * SECURITY: Using an INFORMATION_SCHEMA table with ORDER BY in a subquery + could cause a server crash (CVE-2007-1420). + * Added temporary patch 90_TEMP_sqlparse-ifdef to avoid build problems. + * Merged with 5.0 r809: + * Updated mysqlreport to latest upstream (and patched --help usage + message and "return if qcache_size==0"). + * Merged with 5.0 r798: + * Adapt MAKE_J to use the -j option with the number of available + processors. (thanks to Raphael Pinson). + * Merged with 5.0 r758: + * Changed minimum required version in dh_makeshlibs to 5.0.27-1 as + 5.0.26 had an ABI breakage in it! + This is the cause for Perl programs crashing with the following error: + Transactions not supported by database at /usr/lib/perl5/DBI.pm line 672 + * Added some more comments to the default my.cnf. + * Added support for /etc/mysql/conf.d/. + * The debian-start script that runs on every server start now first upgrades + the system tables (if neccessary) and then check them as it sometimes did + not work the other way around (e.g. for MediaWiki). The script now uses + mysql_update instead of mysql_update_script as recommended. See: 409780 + + -- Christian Hammers <ch(a)debian.org> Fri, 2 Mar 2007 01:00:55 +0100 + +mysql-dfsg-5.1 (5.1.15beta-1) experimental; urgency=low + + * New upstream release. + [Monty Taylor] + * Removed patches/25_mysys__default.c - fixed upstream. + * Removed patches/26_client__mysql_upgrade.c - fixed upstream. + * Removed patches/29_scripts__mysqlbug.sh - fixed upstream. + * Removed patches/39_scripts__mysqld_safe.sh__port_dir - fixed upstream. + * Removed patches/42_scripts__mysqldumpslow__slowdir - fixed upstream. + * Removed patches/45_warn-CLI-passwords - fixed upstream. + * Removed patches/89_ndb__records.dpatch - fixed upstream. + * Removed patches/86_ndbapi_tc_selection.dpatch - fixed upstream. + [Christian Hammers] + * Synced with 5.0.32-4. + * mysql-server-5.0 pre-depends on adduser now and has --disabled-login + explicitly added to be on the safe side (thanks to the puiparts team). + Closes: #408362 + * Corrections the terminology regarding NDB in the comments of all config + files and init scripts (thanks to Geert Vanderkelen of MySQL). + + -- Christian Hammers <ch(a)debian.org> Wed, 7 Feb 2007 11:34:52 -0200 + +mysql-dfsg-5.1 (5.1.14beta-2) experimental; urgency=low + + [Christian Hammers] + * Readded 85_ndb__staticlib.dpatch with slight modifications. + * Backported debian-start scripts from 5.0. + [Monty Taylor] + * Now build-depends on bison. + * Updated to standards 3.7.2. + * Removed references to comp_err. + * build-depend on automake1.9 to match upstream + * Merged runlevel changes from 5.0. + * Added 26_client__mysql_upgrade.c.dpatch to fix a segfault in mysql_upgrade + when using a password. It's been fixed upstream in 5.1.15. + * Moved BDB check to sanity_checks() and added a note about deprecation. + * Use my_print_defaults instead of mysqld --print-defaults + * Changed NDB Data and Management node startup seqence. Prevented both + from restarting on upgrade to address rolling upgrade issues. + * Added a "start-initial" option to the Data Node init script to support + initial node starts. + * Added 86_ndbapi_tc_selection.dpatch to fix a bug that causes a segfault + when using the NdbApi. http://bugs.mysql.com/bug.php?id=24914 + Fixed in 5.1.15 + * Added 89_ndb__records.dpatch to fix + http://bugs.mysql.com/bug.php?id=25567, which causes a table scan per + table per query. + + -- Christian Hammers <ch(a)debian.org> Wed, 31 Jan 2007 01:17:35 +0100 + +mysql-dfsg-5.1 (5.1.14beta-1) experimental; urgency=low + + * New upstream. + * Removed references to mysql_explain_log + * Changed context for patch to mysqld_multi.1 + * Removed 70_kfreebsd.dpatch - applied to upstream + * Removed 87_ps_Hurd - applied to upstream + * Replaced --without-readline to --with-libedit to configure options, as + --without-readline doesn't seem to do the right thing anymore. + + -- Monty Taylor <mordred(a)inaugust.com> Wed, 10 Jan 2007 12:59:55 -0800 + +mysql-dfsg-5.1 (5.1.11beta-1) experimental; urgency=low + + * Starting new 5.1 branch! + * FIXME: Following patch couldn't be applied: + ## 85_ndb__staticlib.dpatch by <ch(a)debian.org> + * FIXME: Following patch couldn't be applied: + ## 86_PATH_MAX.dpatch + + -- Christian Hammers <ch(a)debian.org> Sat, 29 Jul 2006 11:35:42 +0200 + +mysql-dfsg-5.0 (5.0.51a-19) UNRELEASED; urgency=low + + * New patch 50_fix_mysqldump2.dpatch from 5.0.60 to fix dumping databases + from mysql 4.0 server. (closes: #507789) + * Don't create a guest account during bootstrap. (closes: #463704) + + -- Norbert Tretkowski <nobse(a)debian.org> Thu, 04 Dec 2008 23:07:19 +0100 + +mysql-dfsg-5.0 (5.0.51a-18) testing-proposed-updates; urgency=high + + * SECURITY: + Fix for CVE-2008-4098: Inadequate validation of paths used in DATA + DIRECTORY and INDEX DIRECTORY clauses of CREATE TABLE statements enabled + attackers to write to tables in other databases to which they could not + ordinarily have access. + + -- Devin Carraway <devin(a)debian.org> Tue, 25 Nov 2008 05:38:45 +0000 + +mysql-dfsg-5.0 (5.0.51a-17) testing-proposed-updates; urgency=low + + * Don't use commented out passwords from debian.cnf. (closes: #453820) + * Update watch file to recognize releases > 5.0.45. + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 02 Nov 2008 13:31:32 +0100 + +mysql-dfsg-5.0 (5.0.51a-16) unstable; urgency=low + + * New patch 60_rpl_test_failure.dpatch from 5.0.54 to fix a race condition + with the rpl_packet test in some cases. (closes: #501413) + + -- Norbert Tretkowski <nobse(a)debian.org> Thu, 09 Oct 2008 08:50:43 +0200 + +mysql-dfsg-5.0 (5.0.51a-15) unstable; urgency=high + + * SECURITY: + Fix for CVE-2008-3963: An empty bit-string literal (b'') caused a server + crash. Now the value is parsed as an empty bit value (which is treated as + an empty string in string context or 0 in numeric context). + (closes: #498362) + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 14 Sep 2008 18:27:46 +0200 + +mysql-dfsg-5.0 (5.0.51a-14) unstable; urgency=low + + * Update debconf translations: + - Swedish, from Martin Bagge. (closes: #491688) + - Netherlands, from Thijs Kinkhorst. (closes: #492723) + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 07 Sep 2008 20:18:31 +0200 + +mysql-dfsg-5.0 (5.0.51a-13) unstable; urgency=medium + + * New patch 59_fix_relay_logs_corruption.dpatch from 5.0.56 to fix + corruption in relay logs. (closes: #463515) + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 03 Sep 2008 09:13:46 +0200 + +mysql-dfsg-5.0 (5.0.51a-12) unstable; urgency=low + + * Disable rpl_ndb_innodb_trans test when running the testsuite, fails + randomly on i386. (closes: #494238) + + -- Norbert Tretkowski <nobse(a)debian.org> Sat, 09 Aug 2008 15:56:45 +0200 + +mysql-dfsg-5.0 (5.0.51a-11) unstable; urgency=low + + * Disable innodb_handler test when running the testsuite, fails randomly + on s390. (closes: #491363) + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 23 Jul 2008 08:34:51 +0200 + +mysql-dfsg-5.0 (5.0.51a-10) unstable; urgency=high + + * Merge testing-security upload to finally fix CVE-2008-2079, thanks to + Devin Carraway and Steffen Joeris. (closes: #480292) + * New patch 58_disable-ndb-backup-print.dpatch from 5.0.54 to disable + ndb_backup_print, ndb_alter_table and ndb_replace tests when running the + testsuite. (closes: #474893) + * Reenable error handling in testsuite on i386, disabling it was just a + workaround for the problem which is now fixed with the above patch. + * Update debconf translations: + - Vietnamese, from Clytie Siddall. (closes: #486443) + - Spanish, from Javier Fernández-Sanguino Peña. (closes: #488740) + - Slovak, from helix84. (closes: #489266) + * Make lintian happy: + - Fix build-dependency on -1 revision. + - Fix deprecated chown usage. + - Fix spelling error in description. + + -- Norbert Tretkowski <nobse(a)debian.org> Tue, 15 Jul 2008 19:37:35 +0200 + +mysql-dfsg-5.0 (5.0.51a-9+lenny2) testing-security; urgency=high + + * Non-maintainer upload by the security team. + * Correct error number in symlink.test to avoid FTBFS on some archs. + + -- Steffen Joeris <white(a)debian.org> Sun, 13 Jul 2008 11:44:57 +0000 + +mysql-dfsg-5.0 (5.0.51a-9+lenny1) testing-security; urgency=high + + * Non-maintainer upload by the security team. + * Correct and expand 92_SECURITY_CVE-2008-2079.dpatch to cover all symlinks + and check the output of fn_format(). (closes: #480292) + Fixes: CVE-2008-2079 + + -- Steffen Joeris <white(a)debian.org> Sat, 12 Jul 2008 05:30:39 +0000 + +mysql-dfsg-5.0 (5.0.51a-9) unstable; urgency=low + + * Ignore errors in testsuite on i386. (workaround for #474893) + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 25 Jun 2008 15:07:03 +0200 + +mysql-dfsg-5.0 (5.0.51a-8) unstable; urgency=low + + * New patch 80_fix_user_setup_on_localhost.dpatch from Daniel Hahler to fix + a duplicate key error when install MySQL server on a host with hostname + localhost. (closes: #478319) + * Really fix build on non-linux systems, this time without producing a build + error on some architectures. (closes: #485971) + * Update debconf translations: + - French, from Christian Perrier. (closes: #478553) + - German, from Alwin Meschede. (closes: #478672) + - Italian, from Luca Monducci. (closes: #479363) + - Czech, from Miroslav Kure. (closes: #480924) + - Galician, from Jacobo Tarrio. (closes: #480965) + - Basque, from Piarres Beobide. (closes: #481840) + - Swedish, from Martin Bagge. (closes: #482466, #486307) + - Turkish, from Mert Dirik. (closes: #484704) + - Russian, from Yuri Kozlov. (closes: #486149) + - Finnish, from Esko Arajärvi. (closes: #486554) + - Portuguese, from Miguel Figueiredo. (closes: #486709) + - Romanian, from Eddy Petrișor. (closes: #486944) + - Japanese, from Hideki Yamane. (closes: #487270) + + -- Norbert Tretkowski <nobse(a)debian.org> Sat, 21 Jun 2008 19:20:48 +0200 + +mysql-dfsg-5.0 (5.0.51a-7) unstable; urgency=high + + [ Norbert Tretkowski ] + * SECURITY: + Fix for CVE-2008-2079: It was possible to circumvent privileges through + the creation of MyISAM tables employing the DATA DIRECTORY and INDEX + DIRECTORY options to overwrite existing table files in the MySQL data + directory. Use of the MySQL data directory in DATA DIRECTORY and INDEX + DIRECTORY is now disallowed. Patch from openSUSE 11.0, thanks to Michal + Marek. (closes: #480292) + * Fix build on non-linux systems, like hurd-i386. (closes: #480362) + * Include symlinks for mysqlcheck. (closes: #480647) + + [ Monty Taylor ] + * Remove ndb_cpcd, as it is only for the NDB test suite and not useful as a + public program. + * Fix debian-start.inc.sh for table names with characters needing quotes. + Thanks Felix Rublack! (closes: #480525, #481154, #481303, #484012) + * Delete mysql-common.README.Debian. Nothing in it was relevant, and the + useful information is in mysql-server anyway. (closes: #480940) + * Remove a spurious HOME= in logrotate script. + + -- Norbert Tretkowski <nobse(a)debian.org> Thu, 05 Jun 2008 11:49:45 +0200 + +mysql-dfsg-5.0 (5.0.51a-6) unstable; urgency=low + + * Fix debian-start.inc.sh to not print the row counts of the tables + queried. (closes: #478256, #479697) + + -- Monty Taylor <mordred(a)inaugust.com> Wed, 14 May 2008 00:47:46 -0700 + +mysql-dfsg-5.0 (5.0.51a-5) unstable; urgency=medium + + * New patch 57_fix_mysql_replication.dpatch from 5.0.54 to fix directory for + relay logs when using replication. + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 27 Apr 2008 13:55:04 +0200 + +mysql-dfsg-5.0 (5.0.51a-4) unstable; urgency=low + + [ Monty Taylor ] + * Remove build of ndb docs, since they are not installed. Removed build deps + on TeX and doxygen since that's all they were there for. + * Replace script in check_for_crashed_tables with a myisam-recover option + and a script to trigger a check of those tables. (thanks HarrisonF and + kolbe) + * Replace direct calls to test suite with calls to the make targets used by + the MySQL build and qa teams for releases. + * Add --skip-ndbcluster to the postinst bootstrap command. It's really a + workaround for a bug in 5.1, but it's probably a good idea anyway since we + certainly don't need cluster to spin up, and if people have enabled + cluster in their my.cnf file, there could be postinst issues if cluster + isn't running. + * Remove reference to configure options that no longer exist. + * Add myself to uploaders. + + [ Norbert Tretkowski ] + * New patch 56_fix_order_by.dpatch from Ubuntu to fix ORDER BY not working + with GROUP BY. (closes: #471737) + * Add note about filename extensions in the /etc/mysql/conf.d/ directory in + my.cnf. (closes: #461759) + * Confirm password on install, patch from Nicolas Valcárcel. + (closes: #471887) + * Remove Adam Conrad from uploaders on his request. Thanks for your work in + the past! + * Use lsb_release to detect distribution. + + -- Norbert Tretkowski <nobse(a)debian.org> Sat, 05 Apr 2008 21:51:43 +0200 + +mysql-dfsg-5.0 (5.0.51a-3) unstable; urgency=low + + * Disable patch 60_raise-max-keylength.dpatch in default build, but still + ship it in the source package. + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 17 Feb 2008 18:54:42 +0100 + +mysql-dfsg-5.0 (5.0.51a-2) unstable; urgency=low + + * Replace 54_ssl-client-support.dpatch added in 5.0.51-2 with patch from + upstream. + * Ignore errors in testsuite on powerpc. + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 17 Feb 2008 12:42:58 +0100 + +mysql-dfsg-5.0 (5.0.51a-1) unstable; urgency=low + + [ Norbert Tretkowski ] + * New upstream security hotfix release. Low priority upload anyway because + 5.0.51-3 already contained all security fixes. + * Remove patches: + + debian/patches/51_mysqlcheck-result.dpatch + + debian/patches/92_SECURITY_CVE-2007-6303.dpatch + + debian/patches/93_SECURITY_CVE-2007-6304.dpatch + + debian/patches/94_SECURITY_CVE-2008-0226+0227.dpatch + * Add recommendation on libhtml-template-perl to -server package, used by + ndb_size. (closes: #462265) + * New patch 60_raise-max-keylength.dpatch to raise the maximum key length to + 4005 bytes or 1335 UTF-8 characters. (closes: #463137) + * New patch 51_sort-order.dpatch from 5.0.52 to fix incorrect order when + using range conditions on 2 tables or more. + * Support DEB_BUILD_OPTIONS option 'nocheck' to skip tests. + * Update mysqlreport to 3.4a release. + + [ Luk Claes ] + * Updated Japanese debconf translation. (closes: #462158) + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 06 Feb 2008 11:57:45 +0100 + +mysql-dfsg-5.0 (5.0.51-3) unstable; urgency=high + + * SECURITY: + Fix for CVE-2008-0226 and CVE-2008-0227: Three vulnerabilities in yaSSL + versions 1.7.5 and earlier were discovered that could lead to a server + crash or execution of unauthorized code. The exploit requires a server + with yaSSL enabled and TCP/IP connections enabled, but does not require + valid MySQL account credentials. The exploit does not apply to OpenSSL. + (closes: #460873) + * Fix LSB header in init scripts (patch from Petter Reinholdtsen). + (closes: #458798) + * Run testsuite on all archs, but ignore errors on alpha, arm, armel, hppa, + mipsel and sparc. (closes: #460402) + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 23 Jan 2008 11:37:11 +0100 + +mysql-dfsg-5.0 (5.0.51-2) unstable; urgency=low + + [ Monty Taylor ] + * Added --with-system-type to set the version_compile_os field. + * Cleaned up some lintian warnings. + * Removed 43_scripts__mysql_update__password.dpatch since we don't use + mysql_upgrade_shell anymore and use mysql_upgrade instead. + * Removed 88_mctype_attrib.dpatch, http://bugs.mysql.com/bug.php?id=25118 is + closed with http://lists.mysql.com/commits/24337 + * Added mysql-community/mysql-enterprise virtual packages in provides and + conflicts to ease transitions between versions. + + [ Norbert Tretkowski ] + * Add -fPIC to CFLAGS to allow other packages to be built against + libmysqld.a on amd64. (closes: #457915) + * New patch 55_testsuite-2008.dpatch to fix FTBFS in testsuite. + (closes: #458695) + * New patch 54_ssl-client-support.dpatch to fix SSL client support. + * Don't run testsuite on alpha, arm, hppa, mipsel and sparc. + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 02 Jan 2008 18:40:04 +0100 + +mysql-dfsg-5.0 (5.0.51-1) unstable; urgency=low + + * New upstream release. + + Fix a crash in mysql_client_test due to gcc 4.x optimizations. + (closes: #452558) + * Update patches: + + debian/patches/41_scripts__mysql_install_db.sh__no_test.dpatch + + debian/patches/89_ndb__staticlib.dpatch + * Run testsuite after build. + * Re-add manpages, they are licensed under GPL now and redistribution is + permitted. + * Drop linux-libc-dev build-dependency, it's now being pulled by libc-dev + which is build-essential. (closes: #431018) + * Remove old optimizations for MySQL 3.23.x, they are no longer required. + (closes: #436552) + * Don't fail when upgrading mysql-common if $datadir is empty or not defined + (patch from Edward Allcutt). (closes: #453127) + * New patch from 5.0.52 to fix mysqldump because 'null' is shown as type of + fields for view with bad definer. (closes: #454227) + * New patch from 5.0.52 to fix mysqlcheck test result. + * New patch from 5.0.52 to fix wrong optimization in ndb code when building + with gcc 4.2.x. + * New patch from 5.0.54 to fix wrong number output due to integer overflow + when building with gcc 4.2.x. + * New Finnish debconf translation from Esko Arajärvi. (closes: #448776) + * Update Basque debconf translation from Aitor Ibañez. (closes: #456193) + * Add Vcs-* and Homepage fields to source stanza in control file. + * Update mysqlreport to 3.2 release. + * Let mysql-server-5.0 pre-depend on debconf, because it's preinst is using + it. + * Drop menu item for innotop. + + -- Norbert Tretkowski <nobse(a)debian.org> Fri, 14 Dec 2007 09:59:36 +0100 + +mysql-dfsg-5.0 (5.0.45-5) unstable; urgency=high + + * SECURITY: + Fix for CVE-2007-6303: ALTER VIEW retained the original DEFINER value, + even when altered by another user, which could allow that user to gain the + access rights of the view. Now ALTER VIEW is allowed only to the original + definer or users with the SUPER privilege. (closes: #455737) + * SECURITY: + Fix for CVE-2007-6304: When using a FEDERATED table, the local server can + be forced to crash if the remote server returns a result with fewer columns + than expected. + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 12 Dec 2007 20:23:43 +0100 + +mysql-dfsg-5.0 (5.0.45-4) unstable; urgency=high + + * SECURITY: + Fix for CVE-2007-5969: Using RENAME TABLE against a table with explicit + DATA DIRECTORY and INDEX DIRECTORY options can be used to overwrite system + table information by replacing the file to which the symlink points. + (closes: #455010) + + -- Norbert Tretkowski <nobse(a)debian.org> Sun, 09 Dec 2007 12:29:54 +0100 + +mysql-dfsg-5.0 (5.0.45-3) unstable; urgency=high + + * SECURITY: + Fix for CVE-2007-5925: The convert_search_mode_to_innobase function in + ha_innodb.cc in the InnoDB engine in MySQL 5.1.23-BK and earlier allows + remote authenticated users to cause a denial of service (database crash) + via a certain CONTAINS operation on an indexed column, which triggers an + assertion error. (closes: #451235) + + -- Norbert Tretkowski <nobse(a)debian.org> Thu, 15 Nov 2007 18:40:11 +0100 + +mysql-dfsg-5.0 (5.0.45-2) unstable; urgency=low + + * Package is now team-maintained. (closes: #421026) + + [ Sean Finney ] + * New/updated debconf translations: + - Spanish, from Javier Fernández-Sanguino Peña (closes: #426442). + - German, from Alwin Meschede (closes: #426545). + - Danish, from Claus Hindsgaul (closes: #426783). + - French, from Christian Perrier (closes: #430944). + * Add Recommends on libterm-readkey-perl for mysql-client-5.0 package, used + by mysqlreport add-on to mask password entry (closes: #438375). + + [ Norbert Tretkowski ] + * Add myself to uploaders. + * Suggest usage of an update statement on the user table to change the mysql + root user password instead using mysqladmin, to catch all root users from + all hosts. (closes: #435744) + * Remove informations about a crash in the server during flush-logs when + having expire_logs_days enabled but log-bin not, this bug was fixed in + 5.0.32 already. (closes: #368547) + * Disable log_bin option in default config file and add a note to the NEWS + file. (closes: #349661) + * Fix FTBFS if build twice in a row. (closes: #442684) + * Remove check for buggy options from init script. + * Update innotop to 1.6.0 release. + * Add mysqlreport and innotop to mysql-client description. + * Use shorter server version string. + + -- Norbert Tretkowski <nobse(a)debian.org> Wed, 14 Nov 2007 20:00:06 +0100 + +mysql-dfsg-5.0 (5.0.45-1) unstable; urgency=low + + * New upstream release. + + [sean finney] + * removed patches that are incorporated into the latest release: + - 70_cpuid_on_i486.dpatch + - 91_SECURITY_CVE-2007-2691_alter-drop + * new patch 90_upstreamdebiandir.dpatch to keep a few lingering references + to the upstream ./debian dir out of the build, at least until we find + a nice way to collaborate on sharing the directory. + * updated CRUFT list to fix double-build breakage (closes: #424590). + * add conditional build-deps for linux-libc-dev to fix FTBFS for + non-linux arch's (closes: #431018). + * added notes to my.cnf and README.Debian about setting tmpdir when + configuring a replication slave. thanks to Rudy Gevaert for pointing + this out (closes: #431825). + + -- sean finney <seanius(a)debian.org> Tue, 17 Jul 2007 23:50:33 +0200 + +mysql-dfsg-5.0 (5.0.41a-1) unstable; urgency=high + + [sean finney] + * SECURITY: + Fix for CVE-2007-2691: DROP/RENAME TABLE statements (closes: #424778). + [Christian Hammers] + * Removed all manpages from the source (therefore the "41a") as they + are not licensed under the GPL and redistribution is not permitted + (thanks to Mathias Gug). Closes: #430018 + * Added linux-libc-dev to the build-depends as else an illegal dependency to + asm/atomic.h is generated in /usr/include/mysql/my_global.h. Closes: 424276 + [Christian Perrier] + * Debconf templates and debian/control reviewed by the debian-l10n- + english team as part of the Smith review project. Closes: #419974 + * Debconf translation updates: + - French. Closes: #422187 + - Galician. Closes: #420118 + - Italian. Closes: #421349 + - Brazilian Portuguese. Closes: #421516 + - Arabic. Closes: #421751 + - Czech. Closes: #421766 + - Portuguese. Closes: #422428 + + -- Christian Hammers <ch(a)debian.org> Sun, 24 Jun 2007 21:12:42 +0200 + +mysql-dfsg-5.0 (5.0.41-2) unstable; urgency=low + + * the previous "translation changes" inadvertently introduced unrelated + changes in the package control file. + + -- sean finney <seanius(a)debian.org> Sun, 13 May 2007 12:32:45 +0200 + +mysql-dfsg-5.0 (5.0.41-1) unstable; urgency=low + + * New upstream release + [sean finney] + * Bump the priority of the debconf prompt for the root password to high, to + ensure the question shows up in a default installation (closes: #418672). + * Debconf templates and debian/control reviewed by the debian-l10n- + english team as part of the Smith review project. Closes: #419974 + * Debconf translation updates: + - French. Closes: #422187 + - Galician. Closes: #420118 + - Italian. Closes: #421349 + - Brazilian Portuguese. Closes: #421516 + - Arabic. Closes: #421751 + - Czech. Closes: #421766 + - Portuguese. Closes: #422428 + * massaged the local PATH_MAX patch. + * removed temp sql parsing patch which has been incorporated upstream + * upstream no longer includes the mysql_create_system_tables command, + so removed our local patches for it. + * the following issues may have been fixed in a previous version of + mysql-server-5.0, but the exact version is not clear so they will be + marked as fixed in this version. + * lots of NDB-related fixes, including those related to problems with + AUTO_INCREMENT (closes: #310878). + * fix for "connections remaining in sleep state" (closes: #318011). + * fix for "denies queries randomly" (closes: #399602). + * problems indexing on char() binary fields were ISAM specific, which is + no longer supported (closes: #326698). + * fix for problems with "complicated joins" (closes: 348682). + * fix for problems with "flushing logs, server crash" (closes: #348682). + * fix for AUTO_INCREMENT and duplicate keys (closes: #416145). + * fix for "DROP FUNCTIONS doesn't work" (closes: #290670). + + -- sean finney <seanius(a)debian.org> Sat, 12 May 2007 12:10:20 +0200 + +mysql-dfsg-5.0 (5.0.38-3) unstable; urgency=low + + * Added innotop. + * Changed maintainer email address to + pkg-mysql-commits(a)lists.alioth.debian.org + + -- Christian Hammers <ch(a)debian.org> Thu, 19 Apr 2007 19:21:15 +0200 + +mysql-dfsg-5.0 (5.0.38-2) unstable; urgency=high + + * SECURITY: + In some previous versions mysql_install_db was not idempotent and did + always create passwordless root accounts although it should only on + initial installs (thanks to Olaf van der Spek). Closes: #418672 + * Added check for passwordless root accounts to debian-start. + * As MySQL-5.0 is, at least currently, incompatible with Kernel 2.4 the + installation is aborted for such old kernels. Debian Etch does not support + them anyway according to the release notes but this might be unexpected + and many production servers still have self build ones installed (thanks + to Marc-Christian Petersen). See: #416841 + * Adjusted TeX build-deps to texlive. + + -- Christian Hammers <ch(a)debian.org> Tue, 17 Apr 2007 01:00:41 +0200 + +mysql-dfsg-5.0 (5.0.38-1) unstable; urgency=low + + * New upstream release. + * Activated the blackhole engine as it's needed for replicating partition + designs (thanks to Cyril SCETBON). + * Fixed segfault on i486 systems without cpuid instruction (thanks to + Lennart Sorensen). Closes: #410474 + * Only use of the non-essential debconf package in postrm if it is still + installed (thanks to Michael Ablassmeier). Closes: #416838 + + -- Christian Hammers <ch(a)debian.org> Thu, 5 Apr 2007 22:43:41 +0200 + +mysql-dfsg-5.0 (5.0.36-1) unstable; urgency=low + + * New upstream release. + Closes: #400460, #408159, #408533 + + -- Christian Hammers <ch(a)debian.org> Thu, 22 Mar 2007 22:16:31 +0100 + +mysql-dfsg-5.0 (5.0.32-10) unstable; urgency=high + + * Really fixed FTBFS on Sparc introduced with the "make -j" trick in + 5.0.32-8 (thanks to Frank Lichtenheld). Closes: #415026 + + -- Christian Hammers <ch(a)debian.org> Sun, 18 Mar 2007 20:52:33 +0100 + +mysql-dfsg-5.0 (5.0.32-9) unstable; urgency=high + + * Fixed FTBFS on Sparc introduced with the "make -j" trick in 5.0.32-8 + (thanks to Frank Lichtenheld). Closes: #415026 + + -- Christian Hammers <ch(a)debian.org> Tue, 15 Mar 2007 18:55:42 +0100 + +mysql-dfsg-5.0 (5.0.32-8) unstable; urgency=high + + [Sean Finney] + * SECURITY: + - CVE-2007-1420: Single Row Subselect DoS. Specially crafted subselect + queries could crash the mysql server. Patch backported from upstream + changeset 19685 (46_CVE-2007-1420_subselect_dos.dpatch) + closes: #414790. + [Christian Hammers] + * Adapt MAKE_J to use the -j option with the number of available processors. + (thanks to Raphael Pinson). + * Updated mysqlreport to latest upstream (and patched --help usage message + and "return if qcache_size==0"). + + -- sean finney <seanius(a)debian.org> Wed, 14 Mar 2007 20:19:08 +0100 + +mysql-dfsg-5.0 (5.0.32-7) unstable; urgency=low + + * Updated French Debconf translation (thanks to Christian Perrier). + Closes: #411330 + * Updated Danish Debconf translation (thanks to Claus Hindsgaul). + Closes: #411328 + * Updated Portuguese Debconf translation (thanks to "Traduz"). + Closes: #411339 + * Updated Czech Debconf translation (thanks to Miroslav Kure). + Closes: #411341 + * Added Norwegian Debconf translation (thanks to Bjorn Steensrud). + Closes: #411345 + * Updated Spanish Debconf translation (thanks to Javier Fernandez-Sanguino + Pena). Closes: #411347 + * Updated Japanese Debconf translation (thanks to Hideki Yamane). + Closes: #411368 + * Updated Swedish Debconf translation (thanks to Andreas Henriksson). + Closes: #411370 + * Updated Italian Debconf translation (thanks to Luca Monducci). + Closes: #411377 + * Updated Galician Debconf translation (thanks to Jacobo Tarrio). + Closes: #411379 + * Updated Russian Debconf translation (thanks to Yuriy Talakan). + Closes: #411442 + * Updated Basque Debconf translation (thanks to Piarres Beobide). + Closes: #411457 + * Updated German Debconf translation (thanks to Alwin Meschede). + Closes: #411480 + * Updated Dutch Debconf translation (thanks to Thijs Kinkhorst). + * Updated Brazilian Portuguese translation (thanks to Andre Luis Lopes). + Closes: #411536 + * Updated Romanian Debconf translation (thanks to Stan Ioan-Eugen). + Closes: #411764 + + -- Christian Hammers <ch(a)debian.org> Fri, 16 Feb 2007 23:20:42 +0100 + +mysql-dfsg-5.0 (5.0.32-6) unstable; urgency=low + + * Changed wording in Debconf templates to better fit to the graphical + interface (thanks to Frank Kuester). Closes: #411165 + * Lintian suggested style changes to some other Debconf questions. + * Removed accidently stdout output from init script. + + -- Christian Hammers <ch(a)debian.org> Fri, 16 Feb 2007 20:29:18 +0100 + +mysql-dfsg-5.0 (5.0.32-5) unstable; urgency=medium + + * Backported upstream patch for a bug that crashed the server when using + certain join/group/limit combinations. + Users of the Joomla CMS seemed to be affected by this. Closes: #403721 + * The debian-start script that runs on every server start now first upgrades + the system tables (if neccessary) and then check them as it sometimes did + not work the other way around (e.g. for MediaWiki). The script now uses + mysql_update instead of mysql_update_script as recommended. Closes: 409780 + * Remove the Debconf generated config file in postrm. + + -- Christian Hammers <ch(a)debian.org> Thu, 15 Feb 2007 04:47:04 +0100 + +mysql-dfsg-5.0 (5.0.32-4) unstable; urgency=high + + [Christian Hammers] + * Changed minimum required version in dh_makeshlibs to 5.0.27-1 as + 5.0.26 had an ABI breakage in it! + This is the cause for Perl programs crashing with the following error: + "Transactions not supported by database at /usr/lib/perl5/DBI.pm line 672" + * The old_passwords setting that is set according to a Debconf question is + now written to /etc/mysql/conf.d/old_passwords.cnf instead directly to the + conffile /etc/mysql/my.cnf which would be fobidden by policy (thanks to + Robert Bihlmeyer). Closes: #409750 + * Added some more comments to the default my.cnf. + [Monty Taylor] + * Added bison to build dependencies. + * Added a "start-initial" option to the Data Node init script to support + initial node starts. + * Changed NDB Data and Management node startup seqence. Prevented both from + restarting on upgrade to address rolling upgrade issues. + * Updated build-depends to depend on automake1.9 instead of automake1.8 + to match what upstream uses. + + -- Christian Hammers <ch(a)debian.org> Wed, 31 Jan 2007 01:14:09 +0100 + +mysql-dfsg-5.0 (5.0.32-3) unstable; urgency=high + + * mysql-server-5.0 pre-depends on adduser now and has --disabled-login + explicitly added to be on the safe side (thanks to the puiparts team). + Closes: #408362 + * Corrections the terminology regarding NDB in the comments of all config + files and init scripts (thanks to Geert Vanderkelen of MySQL). + * Updated Swedish Debconf translation (thanks to Andreas Henriksson). + Closes: #407859 + * Updated Czech Debconf translation (thanks to Miroslav Kure). + Closes: #407809 + + -- Christian Hammers <ch(a)debian.org> Thu, 11 Jan 2007 11:18:47 +0100 + +mysql-dfsg-5.0 (5.0.32-2) unstable; urgency=high + + * The last upload suffered from a regression that made NDB totally + unusable and caused a dependency to libmysqlclient15-dev in the + mysql-server-5.0 package. The relevant 85_* patch was re-added again. + Closes: #406435 + * Added lintian-overrides for an error that does not affect our packages. + There are now only warnings and not errors left. + + -- Christian Hammers <ch(a)debian.org> Tue, 9 Jan 2007 23:55:10 +0100 + +mysql-dfsg-5.0 (5.0.32-1) unstable; urgency=high + + * New upstream version. + * SECURITY: mysql_fix_privilege_tables.sql altered the + table_privs.table_priv column to contain too few privileges, causing + loss of the CREATE VIEW and SHOW VIEW privileges. (MySQL Bug#20589) + * SECURITY (DoS): ALTER TABLE statements that performed both RENAME TO + and {ENABLE|DISABLE} KEYS operations caused a server crash. (MySQL + Bug#24089) + * SECURITY (DoS): LAST_DAY('0000-00-00') could cause a server crash. + (MySQL Bug#23653) + * SECURITY (DoS): Using EXPLAIN caused a server crash for queries that + selected from INFORMATION_SCHEMA in a subquery in the FROM clause. + (MySQL Bug#22413) + * SECURITY (DoS): Invalidating the query cache (e.g. when using stored procedures) + caused a server crash for INSERT INTO ... SELECT statements that + selected from a view. (MySQL Bug#20045) + * Using mysql_upgrade with a password crashed the server. Closes: #406229 + * yaSSL crashed on pre-Pentium Intel and Cyrix CPUs. (MySQL Bug#21765) + Closes: #383759 + * Lots of small fixes to the NDB cluster storage engine. + * Updated Japanese Debconf template (thanks to Hideki Yamane). + Closes: #405793 + * Fixed comment regarding "mycheck" in debian-start (thanks to + Enrico Zini). Closes: #405787 + + -- Christian Hammers <ch(a)debian.org> Sat, 6 Jan 2007 14:26:20 +0100 + +mysql-dfsg-5.0 (5.0.30-3) unstable; urgency=low + + * Updated Brazilian Debconf translation (thanks to Andre Luis Lopes). + Closes: #403821 + * Added Romanian Debconf translation (thanks to Stan Ioan-Eugen). + Closes: #403943 + * Updated Spanish Debconf translation (thanks to Javier Fernandez-Sanguino + Pena). Closes: #404084 + * Updated Galician Debconf translation (thanks to Jacobo Tarrio). + Closes: #404318 + * Updated Dutch Debconf translation (thanks to Vincent Zweije). + Closes: #404566 + * Updated Danish Debconf translation (thanks to Claus Hindsgaul). + Closes: #405018 + + -- Christian Hammers <ch(a)debian.org> Thu, 21 Dec 2006 21:35:09 +0100 + +mysql-dfsg-5.0 (5.0.30-2) unstable; urgency=high + + * Fixed upstream regression in header files that lead to FTBFS for + mysql-admin, mysql-query-browser and probably other pacakges. + (thanks to Andreas Henriksson). Closes: #403081, #403082 + * Fixed some upstream scripts by replacing /etc by /etc/mysql (thanks to + Julien Antony). Closes: #401083 + * Updated French Debconf translation (thanks to Christian Perrier). + Closes: #401434 + * Added Spanish Debconf translation (thanks to Javier Fernandez-Sanguino + Pena). Closes: #401953 + * Marked a Debconf question that is just a dummy and only internally + used as not-needing-translation. Closes: #403163 + * Fixed mysqlslowdump patch to not remove the usage() function (thanks + to Monty Tailor). + + -- Christian Hammers <ch(a)debian.org> Sun, 3 Dec 2006 19:20:10 +0100 + +mysql-dfsg-5.0 (5.0.30-1) unstable; urgency=low + + * New upstream version (switch to the MySQL Enterprise branch). + * Upstream bugfix for the Innodb performance bug: + "Very poor performance with multiple queries running + concurrently (Bug#15815)". + * Upstream bugfix for a possible server crash: + "Selecting from a MERGE table could result in a server crash if the + underlying tables had fewer indexes than the MERGE table itself + (Bug#22937)" + * Upstream bugfies for *lot* of NDB problems. + * Upstream bugfix for Innodb optimizer bug. Closes: #397597 + * Updated Italian Debconf translation (thanks to Luca Monducci). + Closes: #401305 + * Updated debian/watch file to MySQL Enterprise branch. + + -- Christian Hammers <ch(a)debian.org> Sat, 2 Dec 2006 16:36:38 +0100 + +mysql-dfsg-5.0 (5.0.27-2) unstable; urgency=medium + + * Disabled YaSSL x86 assembler as it was reported to crash applications + like pam-mysql or proftpd-mysql which are linked against libmysqlclient + on i486 and Cyrix (i586) CPUs. Closes: #385147 + * Adjusted mysql-server-4.1 priority to extra and section to oldlibs + according to the ftp masters overrides. + * Updated German Debconf translation (thanks to Alwin Meschede). + Closes: #400809 + + -- Christian Hammers <ch(a)debian.org> Wed, 22 Nov 2006 13:36:31 +0100 + +mysql-dfsg-5.0 (5.0.27-1) unstable; urgency=medium + + * New upstream version (but no codechange, the only difference to 5.0.26 + was a patch to the ABI change which Debian already included. + * When dist-upgrading from mysql-server-4.1/sarge dpkg does not longer + ask unnecessary "config file has changed" questions regarding + /etc/init.d/mysql, /etc/logrotate.d/mysql-server and + /etc/mysql/debian-start just because these files previously belonged + to mysql-server-4.1 and not to mysql-server-5.0. + To archive this mysql-server-5.0 now pre-depends on mysql-common which + provides current versions of those files. + * The automatic run mysql_upgrade now works with non-standard datadir + settings, too (thanks to Benjami Villoslada). Closes: #394607 + * Debconf now asks if the old_passwords option is really needed. + * Improved explanations of the old_passwords variable in my.cnf. + * Removed possibly leftover cron script from MySQL-4.1 (thanks to + Mario Oyorzabal Salgado). Closes: #390889 + * Postrm ignores failed "userdel mysql". + * Updated Danish Debconf translation (thanks to Claus Hindsgaul). + Closes: #398784 + * Added Euskarian Debconf translation (thanks to Piarres Beobide). + Closes: #399045 + * Updated Japanese Debconf translation (thanks to Hideki Yamane). + Closes: #399074 + * Updated German Debconf translation (thanks to Alwin Meschede). + Closes: #399087 + * New Portuguese debconf translations from Miguel Figueiredo. + Closes: #398186 + + -- Christian Hammers <ch(a)debian.org> Tue, 7 Nov 2006 21:26:25 +0100 + +mysql-dfsg-5.0 (5.0.26-3) unstable; urgency=high + + [sean finney] + * Fix for the deadly ISAM trap. Now during upgrades we will do our + very best to convert pre-existing ISAM format tables using the + binaries from the previous package. Success is not guaranteed, but + this is probably as good as it gets. Note that this also necessitates + re-introducing an (empty transitional) mysql-server-4.1 package. + Closes: #354544, #354850 + * Remove a couple spurious and wrongly placed WARNING statements from + 45_warn-CLI-passwords.dpatch. thanks to Dan Jacobsen for pointing these + out. Closes: #394262 + + -- sean finney <seanius(a)debian.org> Fri, 03 Nov 2006 18:34:46 +0100 + +mysql-dfsg-5.0 (5.0.26-2) unstable; urgency=high + + * Fixed FTBFS for Alpha by applying an upstream patch (thanks to Falk + Hueffner). Closes: #395921 + + -- Christian Hammers <ch(a)debian.org> Sat, 28 Oct 2006 20:13:46 +0200 + +mysql-dfsg-5.0 (5.0.26-1) unstable; urgency=high + + * SECURITY: + This combined release of 5.0.25 and 5.0.26 fixes lot of possible server + crashs so it should get into Etch. Quoting the changelog (bug numbers are + bugs.mysql.com ones): + - character_set_results can be NULL to signify no conversion, but some + code did not check for NULL, resulting in a server crash. (Bug#21913) + - Using cursors with READ COMMITTED isolation level could cause InnoDB to + crash. (Bug#19834) + - Some prepared statements caused a server crash when executed a second + time. (Bug#21166) + - When DROP DATABASE or SHOW OPEN TABLES was issued while concurrently + issuing DROP TABLE (or RENAME TABLE, CREATE TABLE LIKE or any other + statement that required a name lock) in another connection, the server + crashed. (Bug#21216) + - Use of zero-length variable names caused a server crash. (Bug#20908) + - For InnoDB tables, the server could crash when executing NOT IN () + subqueries. (Bug#21077) + - Repeated DROP TABLE statements in a stored procedure could sometimes + cause the server to crash. (Bug#19399) + - Performing an INSERT on a view that was defined using a SELECT that + specified a collation and a column alias caused the server to crash + (Bug#21086). + - A query of the form shown here caused the server to crash. (Bug#21007) + - NDB Cluster: Some queries involving joins on very large NDB tables could + crash the MySQL server. (Bug#21059) + - The character set was not being properly initialized for CAST() with a + type like CHAR(2) BINARY, which resulted in incorrect results or even a + server crash. (Bug#17903) + - For certain queries, the server incorrectly resolved a reference to an + aggregate function and crashed. (Bug#20868) + - The server crashed when using the range access method to execut a + subquery with a ORDER BY DESC clause. (Bug#20869) + - Triggers on tables in the mysql database caused a server crash. Triggers + for tables in this database now are disallowed. (Bug#18361) + - Using SELECT on a corrupt MyISAM table using the dynamic record format + could cause a server crash. (Bug#19835) + - Use of MIN() or MAX() with GROUP BY on a ucs2 column could cause a + server crash. (Bug#20076) + - Selecting from a MERGE table could result in a server crash if the + underlying tables had fewer indexes than the MERGE table itself. + (Bug#21617, Bug#22937) + + * New upstream release. + - This bug would cause trouble for Sarge->Etch upgrades, it was supposed to + have been fixed in 5.0.16 but that apparently did not fix the whole + problem: + Using tables from MySQL 4.x in MySQL 5.x, in particular those with VARCHAR + fields and using INSERT DELAYED to update data in the table would result in + either data corruption or a server crash. (Bug#16611, Bug#16218, Bug#17294) + Closes: #386337 + - Fixes data corruption as an automatic client reconnect used to set + the wrong character set. Closes: #365050 + - Fixes an undefined ulong type in an include file. Closes: #389102 + - Fixes wrong output format when using Unicode characters. Closes: #355302 + - Fixes mysql_upgrade when using a password. Closes: #371841 + + [Christian Hammers] + * Removed --sysconfdir from debian/rules as it puts /etc/mysql/ at the + end of the my.cnf search patch thus overriding $HOME/my.cnf + (thanks to Christoph Biedl). Closes: #394992 + * The provided patch from bug #385947 was wrong, the variable is called + BLOCKSIZE not BLOCK_SIZE according to "strings `which df`" (thanks to + Bruno Muller). Closes: #385947 + + [sean finney] + * new dutch debconf translations from Vincent Zweije (closes: #392809). + * new japanese debconf translations from Hideki Yamane (closes: #391625). + * new italian debconf translations from Luca Monducci (closes: #391741). + * new french debconf translations from Christian Perrier (closes: #393334). + * ran debconf-updatepo to merge the fuzzies into svn. + * massage the following patches so they continue to apply cleanly: + - 44_scripts__mysql_config__libs.dpatch to cleanly apply. + - 45_warn-CLI-passwords.dpatch + - 96_TEMP__libmysqlclient_ssl_symbols.dpatch (note, this patch might + no longer be needed, but is retained "just in case" after massaging it) + * the following patches have been incorporated upstream: + - 70_kfreebsd.dpatch + - 80_hurd_mach.dpatch + - 87_ps_Hurd.dpatch + - 90_TEMP__client__mysql_upgrade__O_EXEC.dpatch + - 91_TEMP__client__mysql_upgrade__password.dpatch + - 92_TEMP__client__mysql_upgrade__defaultgroups.dpatch + - 94_TEMP__CVE-2006-4227.dpatch + - 95_TEMP__CVE-2006-4226.dpatch + * the udf_example.cc has disappeared from the source code, but there's + a udf_example.c which seems to be a good example to use instead :) + * update documentation in the configuration to no longer reference + using my.cnf in the DATADIR, as it's never been the recommended + method for debian systems and hasn't worked since 5.0 was released + anyway (closes: #393868). + + -- Christian Hammers <ch(a)debian.org> Wed, 25 Oct 2006 19:54:04 +0200 + +mysql-dfsg-5.0 (5.0.24a-9) unstable; urgency=medium + + * Having expire_logs_days enabled but log-bin not crashes the server. Using + both or none of those options is safe. To prevent this happening during the + nightly log rotation via /etc/logrotate.d/mysql the initscript checks for + malicious combination of options. See: #368547 + * The Sarge package "mysql-server" which used to include the mysqld daemon + may still be in unselected-configured state (i.e. after a remove but not + purge) in which case its now obsolete cronscript has to be moved away + (thanks to Charles Lepple). Closes: #385669 + * Updated Danish Debconf translation (thanks to Claus Hindsgaul). + Closes: #390315 + * Updated Frensh Debconf translation (thanks to Christian Perrier). + Closes: #390980 + + -- Christian Hammers <ch(a)debian.org> Tue, 3 Oct 2006 14:55:31 +0200 + +mysql-dfsg-5.0 (5.0.24a-8) unstable; urgency=low + + * (broken upload) + + -- Christian Hammers <ch(a)debian.org> Tue, 3 Oct 2006 14:55:31 +0200 + +mysql-dfsg-5.0 (5.0.24a-7) unstable; urgency=low + + * Stopped mysql_config from announcing unnecessary library dependencies + which until now cause "NEEDED" dependencies in the "readelf -d" output + of libraries who only depend on libmysqlclient.so (thanks to Michal + Cihar). Closes: #390692 + + -- Christian Hammers <ch(a)debian.org> Sun, 1 Oct 2006 23:59:43 +0200 + +mysql-dfsg-5.0 (5.0.24a-6) unstable; urgency=low + + [sean finney] + * finally add support for setting a root password at install. + while this is not a random password as requested in one bug + report, we believe it is the best solution and provides a + means to set a random password via preseeding if it's really + desired (Closes: #316127, #298295). + + -- sean finney <seanius(a)debian.org> Sun, 01 Oct 2006 23:34:30 +0200 + +mysql-dfsg-5.0 (5.0.24a-5) unstable; urgency=low + + * Added ${shlibs:Depends} to debian/control section libmysqlclient-dev as it + contains the experimental /usr/lib/mysql/libndbclient.so.0.0.0. + * Bumped standards version to 3.7.2. + * Added LSB info section to init scripts. + * Rephrased Debconf templates as suggested by lintian. + * Added benchmark suite in /usr/share/mysql/sql-bench/. + * The mysql.timezone* tables are now filled by the postinst script (thanks + to Mark Sheppard). Closes: #388491 + * Moved Debconf install notes to README.Debian. Displaying them with + medium priority was a bug anyway. Closes: #388941 + * Replaced /usr/bin/mysql_upgrade by /usr/bin/mysql_upgrade_shell in + /etc/mysql/debian-start.sh as it works without errors (thanks to Javier + Kohen). Closes: #389443 + + -- Christian Hammers <ch(a)debian.org> Wed, 20 Sep 2006 15:01:42 +0200 + +mysql-dfsg-5.0 (5.0.24a-4) unstable; urgency=high + + * libmysqlclient.so.15 from 5.0.24 accidentaly exports some symbols that are + historically exported by OpenSSL's libcrypto.so. This bug was supposed to + be fixed in 5.0.24a bug according to the mysql bug tracking system will + only be fixed in 5.0.25 so I backported the patch. People already reported + crashing apps due to this (thanks to Duncan Simpson). See also: #385348 + Closes: #388262 + * Fixed BLOCKSIZE to BLOCK_SIZE in initscript (thanks to Bruno Muller). + Closes: #385947 + * Added hint to "--extended-insert=0" to mysqldump manpage (thanks to Martin + Schulze). + * Documented the meaning of "NDB" in README.Debian (thanks to Dan Jacobson). + Closes: #386274 + * Added patch to build on hurd-i386 (thanks to Cyril Brulebois). Closes: #387369 + * Fixed debian-start script to work together with the recend LSB modifications in + the initscript (thanks to wens). Closes: #387481 + * Reverted tmpdir change in my.cnf back to /tmp to comply with FHS (thanks + to Alessandro Valente). Closes: #382778 + * Added logcheck filter rule (thanks to Paul Wise). Closes: #381043 + * I will definetly not disable InnoDB but added a note to the default my.cnf + that disabling it saves about 100MB virtual memory (thanks to Olivier + Berger). Closes: #384399 + * Added thread_cache_size=8 to default my.cnf as this variable seems to have + a negligible memory footprint but can improve performance when lots of + threads connect simultaneously as often seen on web servers. + + -- Christian Hammers <ch(a)debian.org> Mon, 4 Sep 2006 00:21:50 +0200 + +mysql-dfsg-5.0 (5.0.24a-3) unstable; urgency=low + + * Fixed potential tempfile problem in the newly added mysqlreport script. + + -- Christian Hammers <ch(a)debian.org> Sun, 3 Sep 2006 23:17:24 +0200 + +mysql-dfsg-5.0 (5.0.24a-2) unstable; urgency=low + + * Added "mysqlreport" (GPL'ed) from hackmysql.com. + * Temporarily disabled expire_days option as it causes the server + to crash. See #368547 + * Made output of init scripts LSB compliant (thanks to David Haerdeman). + Closes: #385874 + + -- Christian Hammers <ch(a)debian.org> Sun, 3 Sep 2006 19:06:53 +0200 + +mysql-dfsg-5.0 (5.0.24a-1) unstable; urgency=high + + * New upstream version. + * The shared library in the 5.0.24 upstream release accidently exported + some symbols that are also exported by the OpenSSL libraries (notably + BN_bin2bn) causing unexpected behaviour in applications using these + functions (thanks to Peter Cernak). Closes: #385348 + * Added note about possible crash on certain i486 clone CPUs. + * Made recipient address of startup mysqlcheck output configurable + (thanks to Mattias Guns). Closes: #385119 + + -- Christian Hammers <ch(a)debian.org> Mon, 28 Aug 2006 01:22:12 +0200 + +mysql-dfsg-5.0 (5.0.24-3) unstable; urgency=high + + * SECURITY: + CVE-2006-4226: + When run on case-sensitive filesystems, MySQL allows remote + authenticated users to create or access a database when the database + name differs only in case from a database for which they have + permissions. + CVE-2006-4227: + MySQL evaluates arguments of suid routines in the security context of + the routine's definer instead of the routine's caller, which allows + remote authenticated users to gain privileges through a routine that + has been made available using GRANT EXECUTE. + Thanks to Stefan Fritsch for reporting. Closes: #384798 + + -- Christian Hammers <ch(a)debian.org> Sat, 26 Aug 2006 04:55:17 +0200 + +mysql-dfsg-5.0 (5.0.24-2) unstable; urgency=high + + * 5.0.24-1 introduced an ABI incompatibility, which this patch reverts. + Programs compiled against 5.0.24-1 are not compatible with any other + version and needs a rebuild. + This bug already caused a lot of segfaults and crashes in various + programs. Thanks to Chad MILLER from MySQL for quickly providing a patch. + The shlibdeps version has been increased to 5.0.24-2. + Closes: #384047, #384221, #383700 + + -- Christian Hammers <ch(a)debian.org> Fri, 25 Aug 2006 21:47:35 +0200 + +mysql-dfsg-5.0 (5.0.24-1) unstable; urgency=high + + * SECURITY: Upstream fixes a security bug which allows a user to continue + accessing a table using a MERGE TABLE after the right to direct access to + the database has been revoked (CVE-2006-4031, MySQL bug #15195). + (Well they did not exactly fixed it, they documented the behaviour and + allow the admin to disable merge table alltogether...). Closes: #380271 + * SECURITY: Applied patch that fixes a possibly insecure filehandling + in the recently added mysql_upgrade binary file (MySQL bug #10320). + * New upstream version. + - Fixes nasty MySQL bug #19618 that leads to crashes when using + "SELECT ... WHERE ... not in (1, -1)" (e.g. vbulletin was affected). + - Fixes upstream bug #16803 so that linking ~/.mysql_history to /dev/null + now has the desired effect of having no history. + * Really fixed the runlevels. Closes: #377651 + * Added patch for broken upstream handling of "host=" to mysql_upgrade.c. + * Adjusted /etc/mysql/debian-start to new mysql_upgrade.c + + -- Christian Hammers <ch(a)debian.org> Tue, 8 Aug 2006 00:44:13 +0200 + +mysql-dfsg-5.0 (5.0.22-5) unstable; urgency=low + + * Added further line to the logcheck ignore files (thanks to Paul Wise). + Closes: #381038 + + -- Christian Hammers <ch(a)debian.org> Wed, 2 Aug 2006 00:28:50 +0200 + +mysql-dfsg-5.0 (5.0.22-4) unstable; urgency=low + + * Upstream fixes a bug in the (never released) version 5.0.23 which could + maybe used to crash the server if the mysqlmanager daemon is in use + which is not yet the default in Debian. (CVE-2006-3486 *DISPUTED*) + * Changed runlevel priority of mysqld from 20 to 19 so that it gets started + before apache and proftpd etc. which might depend on an already running + database server (thanks to Martin Gruner). Closes: #377651 + * Added patch which sets PATH_MAX in ndb (thanks to Cyril Brulebois). + Closes: #378949 + * Activated YaSSL as licence issues are settled according to: + http://bugs.mysql.com/?id=16755. This also closes the FTBFS bug + regarding OpenSSL as it is discouraged to use now. Closes: #368639 + * Removed SSL-MINI-HOWTO as the official documentation is good enough now. + * mysql_upgrade no longer gives --password on the commandline which would + be insecure (thanks to Dean Gaudet). Closes: #379199 + * Adjusted debian/patches/45* to make consecutive builds in the same source + tree possible (thanks to Bob Tanner). Closes: #368661 + * mysql-server-5.0 is now suggesting tinyca as yaSSL is enabled and tinyca + was found to be really cool :) + * Moved tempdir from /tmp to /var/tmp as it will more likely have enough + free space as /tmp is often on the root partition and /var or at least + /var/tmp is on a bigger one. + + -- Christian Hammers <ch(a)debian.org> Mon, 10 Jul 2006 23:30:26 +0200 + +mysql-dfsg-5.0 (5.0.22-3) unstable; urgency=low + + * Added patch for MySQL bug #19618: "select x from x + where x not in(1,-1)" may crash the server" (thanks to + Ruben Puettmann). + + -- Christian Hammers <ch(a)debian.org> Fri, 9 Jun 2006 01:41:44 +0200 + +mysql-dfsg-5.0 (5.0.22-2) unstable; urgency=high + + * Fixed debian-sys-maint related bug in postinst (thanks to + Jean-Christophe Dubacq). Closes: #369970 + * The last upload was a security patch (which I did not know as I + uploaded before the announcement came). I now added the CVE id for + reference and set urgency to high as the last entry did not. + + -- Christian Hammers <ch(a)debian.org> Wed, 31 May 2006 01:04:11 +0200 + +mysql-dfsg-5.0 (5.0.22-1) unstable; urgency=low + + * SECURITY: This upstream release fixes an SQL-injection with multibyte + encoding problem. (CVE-2006-2753) + * New upstream release. + * Upstream fixes REPAIR TABLE problem. Closes: #354300 + * Upstream fixes problem that empty strings in varchar and text columns + are displayed as NULL. Closes: #368663 + + -- Christian Hammers <ch(a)debian.org> Tue, 30 May 2006 23:43:24 +0200 + +mysql-dfsg-5.0 (5.0.21-4) unstable; urgency=low + + * Added "BLOCKSIZE=" to the diskfree check (thanks to Farzad FARID). + Closes: #367027, #367083 + * Further fixed mysql_upgrade upstream script (thanks to Andreas Pakulat) + Closes: #366155 + * Adjusted the /proc test in debian/rules from /proc/1 to /proc/self + to make building on grsec systems possible (thanks to K. Rosenegger). + Closes: #366824 + * Updated Russion Debconf translation (thanks to Yuriy Talakan). + Closes: #367141 + * Updated Czech Debconf translation (thanks to Kiroslav Kure). + Closes: #367160 + * Updated Galician Debconf translation (thanks to Jacobo Tarrio). + Closes: #367384 + * Updated Swedish Debconf translation (thanks to Daniel Nylander). + Closes: #368186 + + -- Christian Hammers <ch(a)debian.org> Wed, 10 May 2006 08:45:42 +0200 + +mysql-dfsg-5.0 (5.0.21-3) unstable; urgency=low + + * Fixed FTBFS problem which was caused by a patch that modifies Makefile.am + as well as Makefile.in and was not deteced because my desktop was fast + enough to patch both files within the same second and so fooled automake. + (thanks to Blars Blarson for notifying me). Closes: #366534 + + -- Christian Hammers <ch(a)debian.org> Sat, 6 May 2006 19:03:58 +0200 + +mysql-dfsg-5.0 (5.0.21-2) unstable; urgency=low + + * Fixed bug in postinst that did not correctly rewrite + /etc/mysql/debian.cnf (thanks to Daniel Leidert). + Closes: #365433, #366155 + + -- Christian Hammers <ch(a)debian.org> Thu, 4 May 2006 02:37:03 +0200 + +mysql-dfsg-5.0 (5.0.21-1) unstable; urgency=high + + * SECURITY: New upstream release with some security relevant bugfixes: + * "Buffer over-read in check_connection with usernames lacking a + trailing null byte" (CVE-2006-1516) + * "Anonymous Login Handshake - Information Leakage" (CVE-2006-1517) + * "COM_TABLE_DUMP Information Leakage and Arbitrary command execution" + (CVE-2006-1518) + Closes: #365938, #365939 + * Added diskfree check to the init script (thanks to Tim Baverstock). + Closes: #365460 + * First amd64 upload! + + -- Christian Hammers <ch(a)debian.org> Sat, 29 Apr 2006 04:31:27 +0200 + +mysql-dfsg-5.0 (5.0.20a-2) unstable; urgency=low + + * The new mysql-upgrade which is started from /etc/mysql/debian-start + does now use the debian-sys-maint user for authentication (thanks to + Philipp). Closes: #364991 + * Wrote patch debian/patches/43* which adds a password option to + mysql_update. See MySQL bug #19400. + * Added "Provides: libmysqlclient-dev" to libmysqlclient15-dev as I saw no + obvious reasons against it (problems should be documented in + debian/README.Maintainer!) (thanks to Olaf van der Spek). Closes: #364899 + * Updated Netherlands debconf translation (thanks to Vincent Zweije) + Closes: #364464 + * Updated French debconf translation (thanks to Christian Perrier) + Closes: #364401 + * Updated Danish debconf translation (thanks to Claus Hindsgaul) + Closes: #365135 + + -- Christian Hammers <ch(a)debian.org> Wed, 26 Apr 2006 01:14:53 +0200 + +mysql-dfsg-5.0 (5.0.20a-1) unstable; urgency=low + + * New upstream release. + * Added the new mysql_upgrade script and added it to + /etc/mysql/debian-start (thanks to Alessandro Polverini). + The script is currently very noise that is a known bug and will be + fixed in the next release! + Closes: #363458 + * No longer creates the "test" database. This actuallay had been tried + to archive before (at least patches) exists but apparently was not the + case in the last versions (thanks to Olaf van der Spek). Closes: #362126 + * Reformatted libmysqlclient15off.NEWS.Debian to changelog format + (thanks to Peter Palfrader). Closes: #363062 + + -- Christian Hammers <ch(a)debian.org> Sat, 15 Apr 2006 13:05:22 +0200 + +mysql-dfsg-5.0 (5.0.20-1) unstable; urgency=high + + * Upstream contains a fix for a nasty bug (MySQL#18153) that users + already experienced and that caused corrupted triggers after + REPAIR/OPTIMIZE/ALTER TABLE statements. + (thanks to Jerome Despatis for pointing out) + * Added patch for the "updates on multiple tables is buggy after + upgrading from 4.1 to 5.0" problem which MySQL has been committed + for the upcoming 5.0.21 release. Closes #352704 + * Added Netherlands debconf translation (thanks to Vincent Zweije). + Closes: #360443 + * Added Galician debconf translation (thanks to Jacobo Tarrio). + Closes: #361257 + + -- Christian Hammers <ch(a)debian.org> Fri, 7 Apr 2006 00:00:43 +0200 + +mysql-dfsg-5.0 (5.0.19-3) unstable; urgency=high + + [ Christian Hammers ] + * Fixed libmysqlclient15.README.Debian regarding package name changes + (thanks to Leppo). + * Moved libheap.a etc. back to /usr/lib/mysql/ as their names are just + too generic. Closes: #353924 + [ Sean Finney ] + * updated danish debconf translation, thanks to Claus Hindsgaul + (closes: #357424). + [ Adam Conrad ] + * Send stderr from 'find' in preinst to /dev/null to tidy up chatter. + * Backport patch for CVE-2006-0903 from the upcoming release to resolve + a log bypass vulnerability when using non-binary logs (closes: #359701) + + -- Adam Conrad <adconrad(a)0c3.net> Tue, 4 Apr 2006 15:23:18 +1000 + +mysql-dfsg-5.0 (5.0.19-2) unstable; urgency=medium + + * New upstream release. + * Renamed package libmysqlclient15 to libmysqlclient15off due to + binary incompatible changes. + See /usr/share/doc/libmysqlclient15off/README.Debian + * Updated Czech debconf translation (thanks to Miroslav Kure). + Closes: #356503 + * Updated French debconf translation (thanks to Christian Perrier). + Closes: #356332 + * Improved README.Debian (thanks to Olaf van der Spek). Closes: #355702 + * Fixed 5.0.18-8 changelog by saying in which package the NEWS.Debian + file is (thanks to Ross Boylan). Closes: #355978 + + -- Christian Hammers <ch(a)debian.org> Fri, 17 Mar 2006 02:32:19 +0100 + +mysql-dfsg-5.0 (5.0.19-1) experimental; urgency=medium + + * New upstream release. + * SECURITY: CVE-2006-3081: A bug where str_to_date(1,NULL) lead to a + server crash has been fixed. + (this note has been added subsequently for reference) + * Renamed package libmysqlclient15 to libmysqlclient15off. + See /usr/share/doc/libmysqlclient15off/NEWS.Debian + * Updated Czech debconf translation (thanks to Miroslav Kure). + Closes: #356503 + * Updated French debconf translation (thanks to Christian Perrier). + Closes: #356332 + * Improved README.Debian (thanks to Olaf van der Spek). Closes: #355702 + * Fixed 5.0.18-8 changelog by saying in which package the NEWS.Debian + file is (thanks to Ross Boylan). Closes: #355978 + + -- Christian Hammers <ch(a)debian.org> Tue, 14 Mar 2006 22:56:13 +0100 + +mysql-dfsg-5.0 (5.0.18-9) unstable; urgency=medium + + [ Christian Hammers ] + * When using apt-get the check for left-over ISAM tables can abort the + installation of mysql-server-5.0 but not prevent the mysql-server-4.1 + package from getting removed. The only thing I can do is reflect this + in the Debconf notice that is shown and suggest to reinstall + mysql-server-4.1 for converting. See: #354850 + * Suggests removing of /etc/cron.daily/mysql-server in last NEWS message + (thanks to Mourad De Clerck). Closes: #354111 + * Added versioned symbols for kfreebsd and Hurd, too (thanks to Aurelien + Jarno and Michael Bank). Closes: #353971 + * Added versioned symbols for kfreebsd, too (thanks to Aurelien Jarno). + Closes: #353971 + [ Adam Conrad ] + * Add 39_scripts__mysqld_safe.sh__port_dir.dpatch to ensure that the + permissions on /var/run/mysqld are always correct, even on a tmpfs. + + -- Christian Hammers <ch(a)debian.org> Mon, 6 Mar 2006 21:42:13 +0100 + +mysql-dfsg-5.0 (5.0.18-8) unstable; urgency=low + + * The rotation of the binary logs is now configured via + expire-logs-days in /etc/mysql/my.cnf and handled completely + by the server and no longer in configured in debian-log-rotate.conf + and handled by a cron job. Thanks to David Johnson. + See /usr/share/doc/mysql-server-5.0/NEWS.Debian + * Ran aspell over some files in debian/ and learned a lot :) + * debian/rules: Added check if versioned symbols are really there. + * Updated SSL-MINI-HOWTO. + * Updated copyright (removed the parts regarding the now removed + BerkeleyDB table handler and mysql-doc package). + * Relocated a variable in preinst (thanks to Michael Heldebrant). + Closes: #349258, #352587, #351216 + * Updated Danish debconf translation (thanks to Claus Hindsgaul). + Closes: #349013 + * Updated Swedish debconf translation (thanks to Daniel Nylander). + Closes: #349522 + * Updated French debconf translation (thanks to Christian Perrier). + Closes: #349592 + * Fixed typo in README.Debian (thanks to Vincent Ricard). + * Prolonged waiting time for mysqld in the init script. Closes: #352070 + + -- Christian Hammers <ch(a)debian.org> Mon, 23 Jan 2006 23:13:46 +0100 + +mysql-dfsg-5.0 (5.0.18-7) unstable; urgency=low + + * Made mailx in debian-start.inc.sh optional and changed the dependency on it + on it to a mere recommendation. Closes: #316297 + * the previous FTBFS patches for GNU/Hurd inadvertently led to configure + being regenerating, losing a couple trivial things like our versioned + symbols patch, causing many nasty problems (closes: #348854). + + -- sean finney <seanius(a)debian.org> Fri, 20 Jan 2006 20:59:27 +0100 + +mysql-dfsg-5.0 (5.0.18-6) unstable; urgency=low + + * Added version comment (thanks to Daniel van Eeden). + * Added two patches to build on GNU/Hurd (thanks to Michael Bank). + Closes: #348182 + * Abort upgrade if old and now unsupported ISAM tables are present + (thanks to David Coe). Closes: #345895 + + -- Christian Hammers <ch(a)debian.org> Tue, 17 Jan 2006 19:25:59 +0100 + +mysql-dfsg-5.0 (5.0.18-5) unstable; urgency=low + + * Bump shlibdeps for libmysqlclient15 to (>= 5.0.15-1), which was + the first non-beta release from upstream, as well as being shortly + after we broke the ABI in Debian by introducing versioned symbols. + + -- Adam Conrad <adconrad(a)0c3.net> Fri, 13 Jan 2006 13:18:03 +1100 + +mysql-dfsg-5.0 (5.0.18-4) unstable; urgency=low + + * Munge our dependencies further to smooth upgrades even more, noting + that we really need 5.0 to conflict with 4.1, and stealing a page from + the book of mysql-common, it doesn't hurt to hint package managers in + the direction of "hey, this stuff is a complete replacement for 4.1" + * Change the description of mysql-server and mysql-client to remove the + references to it being "transition", and instead point out that it's + the way to get the "current best version" of each package installed. + + -- Adam Conrad <adconrad(a)0c3.net> Wed, 11 Jan 2006 11:39:45 +1100 + +mysql-dfsg-5.0 (5.0.18-3) unstable; urgency=low + + * Make the mysql-{client,server}-5.0 conflict against mysql-{client,server} + versioned, so they can be installed side-by-side and upgrade properly. + * Add myself to Uploaders; since I have access to the alioth repository. + + -- Adam Conrad <adconrad(a)0c3.net> Tue, 10 Jan 2006 19:15:48 +1100 + +mysql-dfsg-5.0 (5.0.18-2) unstable; urgency=low + + * Removed the transitional package that forced an upgrade from + mysql-server-4.1 to mysql-server-5.0 as I was convinced that + having a general "mysql-server" package with adjusted dependencies + is enough (thanks to Adam Conrad). + * Updated logcheck.ignore files (thanks to Jamie McCarthy). Closes: #340193 + + -- Christian Hammers <ch(a)debian.org> Mon, 9 Jan 2006 21:54:53 +0100 + +mysql-dfsg-5.0 (5.0.18-1) unstable; urgency=low + + * New upstream version. + * Added empty transitional packages that force an upgrade from the + server and client packages that have been present in Sarge. + * Fixed SSL-MINI-HOWTO (thanks to Jonas Smedegaard). Closes: #340589 + + -- Christian Hammers <ch(a)debian.org> Mon, 2 Jan 2006 21:17:51 +0100 + +mysql-dfsg-5.0 (5.0.17-1) unstable; urgency=low + + * Never released as Debian package. + + -- Christian Hammers <ch(a)debian.org> Thu, 22 Dec 2005 07:49:52 +0100 + +mysql-dfsg-5.0 (5.0.16-1) unstable; urgency=low + + * New upstream version. + * Removed the error logs from the logrotate script as Debian does + not use them anymore. Closes: #339628 + + -- Christian Hammers <ch(a)debian.org> Tue, 22 Nov 2005 01:19:11 +0100 + +mysql-dfsg-5.0 (5.0.15-2) unstable; urgency=medium + + * Added 14_configure__gcc-atomic.h.diff to fix FTBFS on m68k + (thanks to Stephen R Marenka). Closes: #337082 + * Removed dynamic linking against libstdc++ as it was not really + needed (thanks to Adam Conrad). Closes: #328613 + * Fixed the "/var/lib/mysql is a symlink" workaround that accidently + left a stalled symlink (thanks to Thomas Lamy). Closes: #336759 + * As the init script cannot distinguish between a broken startup and + one that just takes very long the "failed" message now says + "or took more than 6s" (thanks to Olaf van der Spek). Closes: #335547 + + -- Christian Hammers <ch(a)debian.org> Thu, 3 Nov 2005 22:00:15 +0100 + +mysql-dfsg-5.0 (5.0.15-1) unstable; urgency=low + + * New upstream version. 5.0 has finally been declared STABLE! + * Added small patch to debian/rules that fixed sporadic build errors + where stdout and stderr were piped together, got mixed up and broke + * Added --with-big-tables to ./configure (thanks to tj.trevelyan). + Closes: #333090 + * Added capability to parse "-rc" to debian/watch. + * Fixed cronscript (thanks to Andrew Deason). Closes: #335244 + * Added Swedish debconf translation (thanks to Daniel Nylander). + Closes: #333670 + * Added comment to README.Debian regarding applications that manually + set new-style passwords... Closes: #334444 + * Sean Finney: + - Fix duplicate reference to [-e|--extended-insert]. Closes: #334957 + - Fix default behavior for mysqldumpslow. Closes: #334517 + - Reference documentation issue in mysql manpage. Closes: #335219 + + -- Christian Hammers <ch(a)debian.org> Fri, 30 Sep 2005 00:10:39 +0200 + +mysql-dfsg-5.0 (5.0.13rc-1) unstable; urgency=low + + * New upstream release. Now "release-candidate"! + * Removed any dynamic link dependencies to libndbclient.so.0 which + is due to its version only distributed as a static library. + * Sean Finney: + - FTBFS fix related to stripping rpath in debian/rules + + -- Christian Hammers <ch(a)debian.org> Mon, 26 Sep 2005 22:09:26 +0200 + +mysql-dfsg-5.0 (5.0.12beta-5) unstable; urgency=low + + * The recent FTBFS were probably result of a timing bug in the + debian/patches/75_*.dpatch file where Makefile.in got patched just + before the Makefile.shared which it depended on. For that reason + only some of the autobuilders failed. Closes: #330149 + * Fixed chrpath removal (option -k had to be added). + * Corrected debconf dependency as requested by Joey Hess. + + -- Christian Hammers <ch(a)debian.org> Mon, 26 Sep 2005 18:37:07 +0200 + +mysql-dfsg-5.0 (5.0.12beta-4) unstable; urgency=low + + * Removed experimental shared library libndbclient.so.0.0.0 as it + is doomed to cause trouble as long as it is present in both MySQL 4.1 + and 5.0 without real soname and its own package. We still have + libndbclient.a for developers. (thanks to Adam Conrad and + mediaforest.net) Closes: #329772 + + -- Christian Hammers <ch(a)debian.org> Fri, 23 Sep 2005 12:36:48 +0200 + +mysql-dfsg-5.0 (5.0.12beta-3) unstable; urgency=medium + + * Symbol versioning support! wooooohoooooo! + (thanks to Steve Langasek) Closes: #236288 + * Moved libndbcclient.so.0 to the -dev package as it is provided by + libmysqlclient14 and -15 which must be installable simultaneously. + * Removed mysql-*-doc suggestions. + + -- Christian Hammers <ch(a)debian.org> Tue, 20 Sep 2005 00:07:03 +0200 + +mysql-dfsg-5.0 (5.0.12beta-2) unstable; urgency=low + + * Added patch to build on GNU/kFreeBSD (thanks to Aurelien Jarno). + Closes: #327702 + * Added patch that was already been present on the 4.1 branch which + makes the "status" command of the init script more sensible + (thanks to Stephen Gildea). Closes: #311836 + * Added Vietnamese Debconf translation (thanks to Clytie Siddal). + Closes: #313006 + * Updated German Debconf translation (thanks to Jens Seidel). + Closes: #313957 + * Corrected commends in example debian-log-rotate.conf. The default is + unlike the mysql-sever-4.1 package which needed to stay backwards + compatible now 2 to avoid filling up the disk endlessly. + * Fixed watch file to be "-beta" aware. + + -- Christian Hammers <ch(a)debian.org> Thu, 15 Sep 2005 20:50:19 +0200 + +mysql-dfsg-5.0 (5.0.12beta-1) unstable; urgency=medium + + * Christian Hammers: + - New upstream release. + - Changed build-dep to libreadline5-dev as requested by Matthias Klose. + Closes: #326316 + - Applied fix for changed output format of SHOW MASTER LOGS for + binary log rotation (thanks to Martin Krueger). Closes: #326427, #326427 + - Removed explicit setting of $PATH as I saw no sense in it and + it introduced a bug (thanks to Quim Calpe). Closes: #326769 + - Removed PID file creation from /etc/init.d/mysql-ndb as it does + not work with this daemon (thanks to Quim Calpe). + - Updated French Debconf translation (thanks to Christian Perrier). + Closes: #324805 + - Moved conflicts line in debian/control from libmysqlclient15 to + libmysqlclient15-dev and removed some pre-sarge conflicts as + suggested by Adam Majer. Closes: #324623 + * Sean Finney: + - For posterity, CAN-2005-2558 has been fixed since 5.0.7beta. + + -- Christian Hammers <ch(a)debian.org> Thu, 15 Sep 2005 19:58:22 +0200 + +mysql-dfsg-5.0 (5.0.11beta-3) unstable; urgency=low + + * Temporarily build only with -O2 to circumvent gcc internal errors + (thanks to Matthias Klose). Related to: #321165 + + -- Christian Hammers <ch(a)debian.org> Thu, 18 Aug 2005 15:44:04 +0200 + +mysql-dfsg-5.0 (5.0.11beta-2) unstable; urgency=low + + * Fixed README.Debian regarding the status of mysql-doc. + * Added "set +e" around chgrp in mysql-server-5.0.preinst to + not fail on .journal files (thanks to Christophe Nowicki). + Closes: #318435 + + -- Christian Hammers <ch(a)debian.org> Sun, 14 Aug 2005 18:02:08 +0200 + +mysql-dfsg-5.0 (5.0.11beta-1) unstable; urgency=low + + * New upstream version. + * Added Danish Debconf translations (thanks to Claus Hindsgaul). + Closes: #322384 + * Updated Czech Debconf translations (thanks to Miroslav Kure). + Closes: #321765 + + -- Christian Hammers <ch(a)debian.org> Sat, 13 Aug 2005 11:56:15 +0000 + +mysql-dfsg-5.0 (5.0.10beta-1) unstable; urgency=low + + * New upstream release. + * Christian Hammers: + - Added check for mounted /proc to debian/rules. + * Sean Finney: + - fix for fix_mysql_privilege_tables/mysql_fix_privilege_tables typo + in mysql-server-5.0's README.Debian (see #319838). + + -- Christian Hammers <ch(a)debian.org> Sun, 31 Jul 2005 00:30:45 +0200 + +mysql-dfsg-5.0 (5.0.7beta-1) unstable; urgency=low + + * Second try for new upstream release. + * Renamed mysql-common-5.0 to mysql-common as future libmysqlclient16 + from e.g. MySQL-5.1 would else introduce mysql-common-5.1 which makes + a simultanous installation of libmysqlclient14 impossible as that + depends on either mysql-common or mysql-common-5.0 but not on future + versions. Thus we decided to always let the newest MySQL version + provide mysql-common. + * Added ${misc:Depends} as suggested by debhelper manpage. + * Raised standard in control file to 3.6.2. + * Removed DH_COMPAT from rules in faviour of debian/compat. + * Checkes for presence of init script before executing it in preinst. + Referres: 315959 + * Added 60_includes_mysys.h__gcc40.dpatch for GCC-4.0 compatibility. + + -- Christian Hammers <ch(a)debian.org> Wed, 29 Jun 2005 00:39:05 +0200 + +mysql-dfsg-5.0 (5.0.5beta-1) unstable; urgency=low + + * New major release! Still beta so be carefull... + * Added federated storage engine. + + -- Christian Hammers <ch(a)debian.org> Wed, 8 Jun 2005 19:29:45 +0200 + +mysql-dfsg-4.1 (4.1.12-1) unstable; urgency=low + + * Christian Hammers: + - New upstream release. + - Disabled BerkeleyDB finally. It has been obsoleted by InnoDB. + * Sean Finney: + - Updated French translation from Christian Perrier (Closes: #310526). + - Updated Japanese translation from Hideki Yamane (Closes: #310263). + - Updated Russian translation from Yuriy Talakan (Closes: #310197). + + -- Christian Hammers <ch(a)debian.org> Sat, 4 Jun 2005 05:49:11 +0200 + +mysql-dfsg-4.1 (4.1.11a-4) unstable; urgency=high + + * Fixed FTBFS problem which was caused due to the fact that last uploads + BerkeleyDB patch was tried to applied on all architectures and not only + on those where BerkeleyDB is actually beeing built. Closes: #310296 + + -- Christian Hammers <ch(a)debian.org> Mon, 23 May 2005 00:54:51 +0200 + +mysql-dfsg-4.1 (4.1.11a-3) unstable; urgency=high + + * Added patch from Piotr Roszatycki to compile the bundled db3 library + that is needed for the BerkeleyDB support with versioned symbols so + that mysqld no longer crashes when it gets linked together with the + Debian db3 version which happens when e.g. using libnss-db. + Closes: #308966 + + -- Christian Hammers <ch(a)debian.org> Thu, 19 May 2005 01:41:14 +0200 + +mysql-dfsg-4.1 (4.1.11a-2) unstable; urgency=high + + * Okay, the hackery with /var/lib/dpkg/info/mysql-server.list will not + stand and is removed from the preinst of mysql-server. + * New workaround for the symlink problem that does not involve mucking + with dpkg's file lists is storing the symlinks in a temporary location + across upgrades. + As this sometimes fails since apt-get does not always call new.preinst + before old.postrm, some remarks were added to README.Debian and the + Debconf installation notes to minimize the inconvinience this causes. + + -- sean finney <seanius(a)debian.org> Sun, 15 May 2005 10:25:31 -0400 + +mysql-dfsg-4.1 (4.1.11a-1) unstable; urgency=high + + * Added the "a" to the version number to be able to upload a new + .orig.tar.gz file which now has the non-free Docs/ directory removed + as this has been forgotten in the 4.1.11 release (thanks to Goeran + Weinholt). Closes: #308691 + * The Woody package listed /var/lib/mysql and /var/log/mysql in its + /var/lib/dpkg/info/mysql-server.list. These directories are often + replaced by symlinks to data partitions which triggers a dpkg bug + that causes these symlinks to be removed on upgrades. The new preinst + prevents this by removing the two lines from the .list file + (thanks to Andreas Barth and Jamin W. Collins). See dpkg bug #287978. + * Updated French Debconf translation (thanks to Christian Perrier). + Closes: #308353 + + -- Christian Hammers <ch(a)debian.org> Thu, 12 May 2005 21:52:46 +0200 + +mysql-dfsg-4.1 (4.1.11-3) unstable; urgency=high + + * The "do you want to remove /var/lib/mysql when purging the package" flag + from old versions is removed once this package is beeing installed so + that purging an old Woody mysql-server package while having a + mysql-server-4.1 package installed can no longer lead to the removal of + all databases. Additionaly clarified the wording of this versions Debconf + template and added a check that skips this purge in the postrm script + if another mysql-server* package has /usr/sbin/mysqld installed. + (thanks to Adrian Bunk for spotting that problem) Closes: #307473 + * Cronfile was not beeing installed as the filename was not in the + correct format for "dh_installcron --name" (thanks to Tomislav + Gountchev). Closes: #302712 + + -- Christian Hammers <ch(a)debian.org> Sat, 23 Apr 2005 22:55:15 +0200 + +mysql-dfsg-4.1 (4.1.11-2) unstable; urgency=low + + * Sean Finney: + - don't freak out if we can't remove /etc/mysql during purge. + - debian/rules clean works again. + * Christian Hammers: + - Fixed typo in README.Debian (thanks to Joerg Rieger). Closes: #304897 + - Completely removed the passwordless test user as it was not only + insecure but also lead to irritations as MySQL checks first the + permissions of this user and then those of a password having one. + See bug report from Hilko Bengen for details. Closes: #301741 + + -- Christian Hammers <ch(a)debian.org> Sat, 16 Apr 2005 15:55:00 +0200 + +mysql-dfsg-4.1 (4.1.11-1) unstable; urgency=low + + * New upstream version. + * Upstream fix for charset/collation problem. Closes: #282256 + * Upstream fix for subselect crash. Closes: #297687 + * Corrected minor issue in Debconf template regarding skip-networking + (thanks to Isaac Clerencia). Closes: #303417 + * Made dependency to gawk unnecessary (thanks to Zoran Dzelajlija). + Closes: #302284 + * Removed obsolete 50_innodb_mixlen.dpatch. + * Removed obsolete 51_CAN-2004-0957_db_grant_underscore.dpatch. + + -- Christian Hammers <ch(a)debian.org> Fri, 8 Apr 2005 00:23:53 +0200 + +mysql-dfsg-4.1 (4.1.10a-7) unstable; urgency=low + + * Sean Finney: + - fix for the mysteriously disappeared cronjob. thanks to + Peter Palfrader <weasel(a)debian.org> for pointing out this omission. + (closes: #302712). + + -- sean finney <seanius(a)debian.org> Sat, 02 Apr 2005 16:54:13 -0500 + +mysql-dfsg-4.1 (4.1.10a-6) unstable; urgency=high + + * Sean Finney: + - the previous upload did not completely address the issue. this one + should do so. d'oh. + + -- sean finney <seanius(a)debian.org> Thu, 31 Mar 2005 03:35:50 +0000 + +mysql-dfsg-4.1 (4.1.10a-5) unstable; urgency=high + + * Sean Finney: + - the following security issue is addressed in this upload: + CAN-2004-0957 (grant privilege escalation on tables with underscores) + thanks to sergei at mysql for all his help with this. + + -- sean finney <seanius(a)debian.org> Wed, 30 Mar 2005 21:19:26 -0500 + +mysql-dfsg-4.1 (4.1.10a-4) unstable; urgency=low + + * Sean Finney: + - FTBFS fix for amd64/gcc-4.0. Thanks to Andreas Jochens <aj(a)andaco.de> + for reporting this (closes: #301807). + - ANSI-compatible quoting fix in daily cron job. thanks to + Karl Hammar <karl(a)aspodata.se> for pointing out the problem in + the 4.0 branch. + - Added myself as a co-maintainer in the control file (closes: #295312). + + -- sean finney <seanius(a)debian.org> Tue, 29 Mar 2005 18:54:42 -0500 + +mysql-dfsg-4.1 (4.1.10a-3) unstable; urgency=low + + * BerkeleyDB is now disabled by default as its use is discouraged by MySQL. + * Added embedded server libraries as they finally do compile. + They are currently in libmysqlclient-dev as they are still + experimental and only available as .a library (thanks to Keith Packard). + Closes: #297062 + * Fixed obsolete "tail" syntax (thanks to Sven Mueller). Closes: #301413 + * Added CAN numbers for the latest security bugfix upload. + * Updated manpage of mysqlmanager (thanks to Justin Pryzby). Closes: #299844 + * Added comments to default configuration. + + -- Christian Hammers <ch(a)debian.org> Sun, 20 Mar 2005 17:40:18 +0100 + +mysql-dfsg-4.1 (4.1.10a-2) unstable; urgency=low + + * Disabled "--with-mysqld-ldflags=-all-static" as it causes sig11 crashes + if LDAP is used for groups in /etc/nsswitch.conf. Confirmed by Sean Finney + and Daniel Dehennin. Closes: #299382 + + -- Christian Hammers <ch(a)debian.org> Mon, 14 Mar 2005 03:01:03 +0100 + +mysql-dfsg-4.1 (4.1.10a-1) unstable; urgency=high + + * SECURITY: + - The following security related updates are addressed: + CAN-2005-0711 (temporary file creation with "CREATE TEMPORARY TABLE") + CAN-2005-0709 (arbitrary library injection in udf_init()) + CAN-2005-0710 (arbitrary code execution via "CREATE FUNCTION") + Closes: #299029, #299031, #299065 + * New Upstream Release. + - Fixes some server crash conditions. + - Upstream includes fix for TMPDIR overriding my.cnf tmpdir setting + Closes: #294347 + - Fixes InnoDB error message. Closes: #298875 + - Fixes resouce limiting. Closes: #285044 + * Improved checking whether or not the server is alive in the init script + which should make it possible to run several mysqld instances in + different chroot environments. Closes: #297772 + * Fixed cron script name as dots are not allowed (thanks to Michel + v/d Ven). Closes: #298447 + * Added -O3 and --with-mysqld-ldflags=-all-static as MySQL recommends to + build the server binary statically in order to gain about 13% more + performance (thanks to Marcin Kowalski). + * Added patch to let mysqld_safe react to signals (thanks to Erich + Schubert). Closes: #208364 + * (Thanks to Sean Finney for doing a great share of work for this release!) + + -- Christian Hammers <ch(a)debian.org> Thu, 3 Mar 2005 02:36:39 +0100 + +mysql-dfsg-4.1 (4.1.10-4) unstable; urgency=medium + + * Fixed bug that prevented MySQL from starting after upgrades. + Closes: #297198, #296403 + * Added comment about logging to syslog to the default my.cnf + and the logrotate script (thanks to Ryszard Lach). Closes: #295507 + + -- Christian Hammers <ch(a)debian.org> Thu, 3 Mar 2005 00:28:02 +0100 + +mysql-dfsg-4.1 (4.1.10-3) unstable; urgency=low + + * Sean Finney: Cronjobs now exit silently when the server package + has been removed but not purged (thanks to Vineet Kumar). + Closes: #297404 + * Fixed comments of /etc/mysql/debian-log-rotate.conf (thanks to + Philip Ross). Closes: #297467 + * Made mysqld_safe reacting sane on signals (thanks to Erich Schubert). + Closes: #208364 + + -- Christian Hammers <ch(a)debian.org> Tue, 1 Mar 2005 19:44:34 +0100 + +mysql-dfsg-4.1 (4.1.10-2) unstable; urgency=low + + * Converted to dpatch. + * debian/ is now maintained via Subversion on svn.debian.org. + + -- Christian Hammers <ch(a)debian.org> Tue, 1 Mar 2005 02:16:36 +0100 + +mysql-dfsg-4.1 (4.1.10-1) unstable; urgency=low + + * New upstream version. + * Upstream fixed memleak bug. Closes: #205587 + * Added debian/copyright.more for personal reference. + * Lowered default query cache size as suggested by Arjen from MySQL. + * Switched from log to log-bin as suggested by Arjen from MySQL. + * Fixed typo in my.cnf (thanks to Sebastian Feltel). Closes: #295247 + * Replaced --defaults-extra-file by --defaults-file in Debian scripts + as former lets password/host etc be overwriteable by /root/.my.cnf. + Added socket to /etc/mysql/debian.cnf to let it work. (thanks to + SATOH Fumiyasu). Closes: #295170 + + -- Christian Hammers <ch(a)debian.org> Tue, 15 Feb 2005 23:47:02 +0100 + +mysql-dfsg-4.1 (4.1.9-4) unstable; urgency=low + + * Improved the way mysqld is started and registered with update-rc.d + in cases where the admin modifies the runlevel configuration. + Most notably removed the debconf question whether or not mysql should + start on when booting. Closes: #274264 + * Renamed configuration option old-passwords to the more preferred + naming convention old_passwords. Same for some others (thanks to + Patrice Pawlak). Closes: #293983 + + -- Christian Hammers <ch(a)debian.org> Tue, 8 Feb 2005 02:21:18 +0100 + +mysql-dfsg-4.1 (4.1.9-3) unstable; urgency=low + + * Renamed ca_ES.po to ca.po to reach a broader audience (thanks to + Christian Perrier). Closes: #293786 + * Expicitly disabled mysqlfs support as it has never been enabled by + configure during the autodetection but fails due to broken upstream + code when users try to build the package theirselves while having + liborbit-dev installed which triggers the mysqlfs autodetection + (thanks to Max Kellermann). Closes: #293431 + * Added dependencies to gawk as one script does not work with original-awk + (thanks to Petr Ferschmann). Closes: #291634 + + -- Christian Hammers <ch(a)debian.org> Sun, 6 Feb 2005 23:33:11 +0100 + +mysql-dfsg-4.1 (4.1.9-2) unstable; urgency=high + + * SECURITY: + For historical reasons /usr/share/mysql/ was owned and writable by + the user "mysql". This is a security problem as some scripts that + are run by root are in this directory and could be modified and used + by a malicious user who already has mysql privileges to gain full root + rights (thanks to Matt Brubeck). Closes: #293345 + * Changed "skip-networking" to "bind-address 127.0.0.1" which is more + compatible and not less secure but maybe even more, as less people enable + networking for all interfaces (thanks to Arjen Lentz). + * Enabled InnoDB by default as recommended by Arjen Lentz from MySQL. + * Added remarks about hosts.allow to README.Debian (thanks to David + Chappell). Closes: #291300 + * mysql-server-4.1 now provides mysql-server (thanks to Paul van den Berg). + Closes: #287735 + + -- Christian Hammers <ch(a)debian.org> Wed, 2 Feb 2005 23:31:55 +0100 + +mysql-dfsg-4.1 (4.1.9-1) unstable; urgency=low + + * New upstream version. + * mysql-client-4.1 now provides "mysql-client" so that packages depending + on mysql-client (ca. 40) can now be used with MySQL-4.1, too. + + -- Christian Hammers <ch(a)debian.org> Sun, 23 Jan 2005 22:52:48 +0100 + +mysql-dfsg-4.1 (4.1.8a-6) unstable; urgency=high + + * SECURITY: + Javier Fernandez-Sanguino Pena from the Debian Security Audit Project + discovered a temporary file vulnerability in the mysqlaccess script of + MySQL that could allow an unprivileged user to let root overwrite + arbitrary files via a symlink attack and could also could unveil the + contents of a temporary file which might contain sensitive information. + (CAN-2005-0004, http://lists.mysql.com/internals/20600) Closes: #291122 + + -- Christian Hammers <ch(a)debian.org> Tue, 18 Jan 2005 23:11:48 +0100 + +mysql-dfsg-4.1 (4.1.8a-5) unstable; urgency=medium + + * Fixed important upstream bug that causes from_unixtime(0) to return + NULL instead of "1970-01-01 00:00:00" which fails on NOT NULL columns. + Closes: #287792 + * Fixes upstream bug in mysql_list_fields() . Closes: #282486 + * Fixes bug that lead to double rotated logfiles when mysql-server 4.0 + was previously installed (thanks to Olaf van der Spek). Closes: #289851 + * Fixed typo in README.Debian (thanks to Mark Nipper). Closes: #289131 + * Changed max_allowed_packet in my.cnf to 16M as in 4.0.x (thanks to + Olaf van der Spek). Closes: #289840 + * Updated French debconf translation (thanks to Christian Perrier). + Closes: #287955 + + -- Christian Hammers <ch(a)debian.org> Thu, 13 Jan 2005 01:29:05 +0100 + +mysql-dfsg-4.1 (4.1.8a-4) unstable; urgency=low + + * Broken patch again :-( + + -- Christian Hammers <ch(a)debian.org> Sun, 9 Jan 2005 23:47:55 +0100 + +mysql-dfsg-4.1 (4.1.8a-3) unstable; urgency=low + + * The mutex patch was a bit too x86 centric. This broke the alpha build. + + -- Christian Hammers <ch(a)debian.org> Sun, 9 Jan 2005 14:18:49 +0100 + +mysql-dfsg-4.1 (4.1.8a-2) unstable; urgency=medium + + * Some Makefiles that were patched by me got overwritten by the GNU + autotools, probably because I also patched ./configure. Fixed now, + the critical mutex patch is now back in again. Closes: #286961 + * Added patch to make MySQL compile on ARM (thanks to Adam Majer). + Closes: #285071 + + -- Christian Hammers <ch(a)debian.org> Thu, 6 Jan 2005 09:30:13 +0100 + +mysql-dfsg-4.1 (4.1.8a-1) unstable; urgency=medium + + * Upstream 4.1.8 had some problems in their GNU Autotools files so they + released 4.1.8a. Debian's 4.1.8 was fixed by running autoreconf but this + again overwrote MySQL changes to ltmain.sh which are supposed to fix some + problems on uncommon architectures (maybe the FTBFS on alpha, arm, m68k + and sparc?). + * libmysqlclient_r.so.14 from 4.1.8-3 also missed a link dependency to + libz which lead to unresolved symbols visible with "ldd -r" (thanks + to Laurent Bonnaud). Closes: #287573 + + -- Christian Hammers <ch(a)debian.org> Wed, 29 Dec 2004 14:26:33 +0100 + +mysql-dfsg-4.1 (4.1.8-3) unstable; urgency=low + + * Fixed checking for error messages by forcing english language + output by adding LC_ALL=C to debian-start (thanks to Rene + Konasz) Closes: #285709 + * Fixed bashisms in Debian scripts. Closes: #286863 + * Updated Japanese Debconf translation (thanks to Hideki Yamane). + Closes: #287003 + * Improved 4.0 to 4.1 upgrade if /var/lib/mysql is a symlink + (thanks to Thomas Lamy). Closes: #286560 + * Added patch for FTBFS problem where no LinuxThreads can be found. + I don't know if this still applies but it should not hurt. + The patch is debian/patches/configure__AMD64-LinuxThreads-vs-NPTL.diff + + -- Christian Hammers <ch(a)debian.org> Sun, 26 Dec 2004 14:04:20 +0100 + +mysql-dfsg-4.1 (4.1.8-2) unstable; urgency=low + + * If /var/lib/mysql is a symlink then it is kept as such. + * Added the old-passwords option to the default my.cnf to stay + compatible to clients that are still compiled to libmysqlclient10 + and libmysqlclient12 for licence reasons. + * Adjusted tetex build-deps to ease backporting (thanks to Norbert + Tretkowski from backports.org) + + -- Christian Hammers <ch(a)debian.org> Tue, 21 Dec 2004 01:00:27 +0100 + +mysql-dfsg-4.1 (4.1.8-1) unstable; urgency=medium + + * New upstream version. Closes: #286175 + * Added conflict to libmysqlclient-dev (thanks to Adam Majer). + Closes: #286538 + * Added debconf-updatepo to debian/rules:clean. + * Updated Japanese Debconf translation (thanks to Hideki Yamane). + Closes: #285107 + * Updated French Debconf translation (thanks to Christian Perrier). + Closes: #285977 + * Renamed cz.po to cs.po (thanks to Miroslav Kure). Closes: #285438 + * Aplied patch for changed server notice to debian-start (thanks to + Adam Majer). Closes: #286035 + * Changed nice value in default my.cnf as nohup changed its behaviour + (thanks to Dariush Pietrzak). Closes: #285446 + * Increased verbosity of preinst script in cases where it cannot stop + a running server (thanks to Jan Minar). Closes: #285982 + * Splitted the code parts of /etc/mysql/debian-start to + /usr/share/mysql/debian-start.inc.sh (thanks to Jan Minar). + Closes: #285988 + + -- Christian Hammers <ch(a)debian.org> Mon, 20 Dec 2004 00:33:21 +0100 + +mysql-dfsg-4.1 (4.1.7-4) unstable; urgency=medium + + * Removed OpenSSL support. + After a short discussion with MySQL, I decided to drop OpenSSL support as + 1. MySQL started shipping their binaries without it, too and do not + seem to support it in favour of using a different library somewhen. + 2. MySQL did not adjust their licence to grant permission to link + against OpenSSL. + 3. Even if they did, third parties who use libmysqlclient.so often + do not realise licencing problems or even do not want OpenSSL. + (thanks to Jordi Mallach and the responders to MySQL bug #6924) + Closes: #283786 + * debian/control: Improved depends and conflicts to mysql-4.0. + + -- Christian Hammers <ch(a)debian.org> Thu, 2 Dec 2004 22:02:28 +0100 + +mysql-dfsg-4.1 (4.1.7-3) unstable; urgency=low + + * Raised version to make it higher as the one in experimental. + + -- Christian Hammers <ch(a)debian.org> Wed, 1 Dec 2004 21:09:20 +0100 + +mysql-dfsg-4.1 (4.1.7-2) unstable; urgency=low + + * Patched scripts/mysql_install_db so that it no longer creates a + passwordless test database during installation (thanks to Patrick + Schnorbus). Closes: #281158 + * Added Czech debconf translation (thanks to Miroslav Kure). + Closes: #283222 + + -- Christian Hammers <ch(a)debian.org> Wed, 1 Dec 2004 01:29:31 +0100 + +mysql-dfsg-4.1 (4.1.7-1) unstable; urgency=low + + * New upstream branch! + * Adjusted debian/control to make this package suitable to get parallel + to version 4.0.x into unstable and sarge. The package names are + different so that "mysql-server" still defaults to the rock-stable + 4.0 instead to this announced-to-be-stable 4.1. + * Added --with-mutex=i86/gcc-assemler to the Berkeley-DB configure + to prevent the use of NPLT threads when compiling under kernel 2.6 + because the binaries are else not runable on kernel 2.4 hosts. + Closes: #278638, #274598 + + -- Christian Hammers <ch(a)debian.org> Sun, 31 Oct 2004 20:15:03 +0100 + +mysql-dfsg (4.1.6-1) experimental; urgency=low + + * New upstream version. + * Fixed symlinks in libmysqlclient-dev package. Closes: #277028 + * This time I did not update the libtool files as they were pretty + up to date and I want to have a shorter diff file. + + -- Christian Hammers <ch(a)debian.org> Wed, 20 Oct 2004 00:07:58 +0200 + +mysql-dfsg (4.1.5-3) experimental; urgency=low + + * debian/postinst: mysql_install_db changed parameter from --IN-RPM + to --rpm which caused problems during installs. Closes: #276320 + + -- Christian Hammers <ch(a)debian.org> Sat, 16 Oct 2004 20:36:46 +0200 + +mysql-dfsg (4.1.5-2) experimental; urgency=low + + * Activated support for ndb clustering (thanks to Kevin M. Rosenberg). + Closes: #275109 + + -- Christian Hammers <ch(a)debian.org> Wed, 6 Oct 2004 01:58:00 +0200 + +mysql-dfsg (4.1.5-1) experimental; urgency=low + + * WARNING: + The upstream branch 4.1 is still considered BETA. + The Debian packages for 4.1 were done without big testing. If you miss + a new functionality or binary, contact me and I check add the relevant + configure option or include the program. + * New MAJOR upstream version. + Thanks to the great demand here's now the first MySQL 4.1 experimental + release. FEEDBACK IS WELCOME. + * 4.0->4.1 notes: + - debian/patches/alpha.diff could not be applied, I fix that later + - debian/patches/scripts__mysql_install_db.sh.diff was obsolete + - debian/patches/scripts__Makefile.in was neccessary due to a dependency + to the removed non-free Docs/ directory. Upstream has been contacted. + - Build-Deps: += automake1.7 + - debian/rules: embedded servers examples did not compile, removed + + -- Christian Hammers <ch(a)debian.org> Sun, 26 Sep 2004 19:46:47 +0200 + +mysql-dfsg (4.0.21-3) unstable; urgency=low + + * Upstream tried to fix a security bug in mysqlhotcopy and broke it :-) + Applied a patch (see debian/patches) from Martin Pitt. Closes: #271632 + * Between 4.0.20 and 4.0.21 the Debian specific changes in + /usr/bin/mysqld_safe that piped the error log to syslog got lost + and are now back again. + * Fixed capitalization in debconf headings. + * Changed wording of the initscript status message to make heartbeat + happier. Closes: #271591 + + -- Christian Hammers <ch(a)debian.org> Fri, 17 Sep 2004 18:42:25 +0200 + +mysql-dfsg (4.0.21-2) unstable; urgency=medium + + * The dependencies between mysql-client and libmysqlclient12 were + too loose, when upgrading only the client this can lead to non working + binaries due to relocation errors (thanks to Dominic Cleal). + Closes: #271803 + * Fixed typo in mysqldump.1 manpage (thanks to Nicolas Francois). + Closes: #271334 + + -- Christian Hammers <ch(a)debian.org> Wed, 15 Sep 2004 15:38:11 +0200 + +mysql-dfsg (4.0.21-1) unstable; urgency=high + + * SECURITY: + This upstream version fixes some security problems that might at least + allow a DoS attack on the server. + * Fixed an old bug in concurrent accesses to `MERGE' tables (even + one `MERGE' table and `MyISAM' tables), that could've resulted in + a crash or hang of the server. (Bug #2408) + * Fixed bug in privilege checking where, under some conditions, one + was able to grant privileges on the database, he has no privileges + on. (Bug #3933) + * Fixed crash in `MATCH ... AGAINST()' on a phrase search operator + with a missing closing double quote. (Bug #3870) + * Fixed potential memory overrun in `mysql_real_connect()' (which + required a compromised DNS server and certain operating systems). + (Bug #4017) + * New upstream version. + * Fixes bug that made x="foo" in WHERE sometimes the same as x="foo ". + Closes: #211618 + * Updated Japanese Debconf translation (thanks to Hideki Yamane). + Closes: #271097 + + -- Christian Hammers <ch(a)debian.org> Sat, 11 Sep 2004 23:15:44 +0200 + +mysql-dfsg (4.0.20-14) unstable; urgency=low + + * Dave Rolsky spottet that -DBIG_JOINS was not properly enabled. + It allowes joining 64 instead of an 32 tables to join. + + -- Christian Hammers <ch(a)debian.org> Thu, 9 Sep 2004 20:24:02 +0200 + +mysql-dfsg (4.0.20-13) unstable; urgency=medium + + * Fixed a bug in the initscript which caused the check for not properly + closed i.e. corrupt tables that is executed when the server starts + not to run in background as supposed. + Although the check does not repair anything on servers with several + thousand tables the script was reported to take some minutes which + is quite annoying. (Thanks to Jakob Goldbach). Closes: #270800 + + -- Christian Hammers <ch(a)debian.org> Thu, 9 Sep 2004 17:11:05 +0200 + +mysql-dfsg (4.0.20-12) unstable; urgency=medium + + * Filter messages regarding table handles that do not support CHECK TABLE + in the script that checks for corrupted tables on every start which lead + to unnecessary mails (thanks to David Everly). Closes: #269811 + * Added a note to the corrupt-table-check mail which notes that a + false-positive is reported in the case that immediately after starting + the server a client starts using a table (thanks to Uwe Kappe). + Closes: #269985 + * Added "quote-names" as default to the [mysqldump] section in + /etc/mysql/my.cnf as too many users stumble over dump files that + could not be read in again due to the valid use of reserved words + as table names. This has also be done by upstream in 4.1.1 and has + no known drawbacks. Closes: #269865 + * Binary logs can now be rotated as well. Defaults to off, though, for + compatibilty reasons (thanks to Mark Ferlatte). Closes: #94230, #269110 + * The mysql user "debian-sys-maint" now gets all possible rights which + makes binary logging possible and helps other package maintainer who + wants to use it to create package specific databases and users. + * Added example how to change daemon nice level via /etc/mysql/my.cnf + * Updated French debconf translations (thanks to Christian Perrier). + Closes: #265811 + * Renamed options in the default config file that still had old names + (thanks to Yves Kreis). Closes: #266445 + * Fixed spelling in debconf note. + * Added -l and -L to dh_shlibdeps. + + -- Christian Hammers <ch(a)debian.org> Fri, 3 Sep 2004 20:10:46 +0200 + +mysql-dfsg (4.0.20-11) unstable; urgency=high + + * SECURITY + This version fixes a security flaw in mysqlhotcopy which created + temporary files in /tmp which had predictable filenames and such + could be used for a tempfile run attack. + The issue has been recorded as CAN-2004-0457. + + -- Christian Hammers <ch(a)debian.org> Sat, 14 Aug 2004 18:27:19 +0200 + +mysql-dfsg (4.0.20-10) unstable; urgency=low + + * MySQL finally updated their copyright page and installed v1.5 of + the "Free/Libre and Open Source Software License (FLOSS) - Exception" + which will hopefully end the license hell they created by putting the + client libraries under GPL instead of LGPL which conflicts with PHP and + other software that used to link against MySQL. + The license text is not yet in any release MySQL version but visible + on their web site and copied into the debian/copyright file. + Special thanks to Zak Greant <zak(a)mysql.com> and the debian-legal list + for helping to solve this release critical problem. + Closes: #242449 + * Updated Brazil debconf translation (thanks to Andre Luis Lopes). + Closes: #264233 + * Updated Japanese debconf translation (thanks to Hideki Yamane). + Closes: #264620 + * Fixed minor typo in debconf description (thanks to TROJETTE Mohammed + Adnene). Closes: #264840 + * Improved init and preinst script which now detects stalled servers which + do no longer communicate but are present in the process list (thanks to + Henrik Johansson). Closes: #263215 + + -- Christian Hammers <ch(a)debian.org> Mon, 9 Aug 2004 19:44:28 +0200 + +mysql-dfsg (4.0.20-9) unstable; urgency=medium + + * Partly reverted the last patch which gave the mysql-user + "debian-sys-maint" more rights as there are old versions of MySQL which + have fewer privlige columns. Now only those are set (thanks to Alan Tam). + Closes: #263111 + + -- Christian Hammers <ch(a)debian.org> Tue, 3 Aug 2004 13:03:02 +0200 + +mysql-dfsg (4.0.20-8) unstable; urgency=low + + * The mysqlcheck that is started from the initscript will now be + backgrounded because it might else prevent the boot process to continue. + It also now notifies root by mail and syslog if a table is corrupt. + * The "debian-sys-maint" MySQL user now has almost full rights so that other + packages might use this account to create databases and user (thanks to + Andreas Barth). Closes: #262541 + * Added paranoid rules for logcheck. + + -- Christian Hammers <ch(a)debian.org> Sun, 1 Aug 2004 21:00:55 +0200 + +mysql-dfsg (4.0.20-8) unstable; urgency=low + + * Upload stalled. Not released. + + -- Christian Hammers <ch(a)debian.org> Sun, 1 Aug 2004 20:27:55 +0200 + +mysql-dfsg (4.0.20-7) unstable; urgency=medium + + * Solved the upstream bug that error messages of the server are written + in a file that is then rotated away leaving mysqld logging effectively + to /dev/null. It now logs to a /usr/bin/logger process which puts the + messages into the syslog. + Modified files: /etc/init.d/mysql, /usr/bin/mysqld_safe and the + logchecker files. Closes: #254070 + * The initscript does no longer call mysqlcheck directly but via + /etc/mysql/debian-start which is a user customizable config script. + * Splitted the debconf "install and update notes" and only show them + when it is appropriate (thanks to Steve Langasek). Closes: #240515 + * Added NEWS.Debian. + * Added hint to -DBIG_ROWS, which is currently not used, to README.Debian. + * Corrected typo in myisampack manpage (thanks to Marc Lehmann). + Closes: #207090 + * Added Catalan debconf translation (thanks to Aleix Badia i Bosch). + Closes: #236651 + + -- Christian Hammers <ch(a)debian.org> Wed, 28 Jul 2004 01:41:51 +0200 + +mysql-dfsg (4.0.20-6) unstable; urgency=low + + * The build arch detected by configure was "pc-linux-gnu (i686)" + instead of "pc-linux-gnu (i386)". Was no problem AFAIK but + Adam Majer asked me to explicitly change it to i386. Closes: #261382 + * Removed some unused shell scripts from /usr/share/mysql. + * Added lintian overrides. + * Removed rpath by using chrpath. + + -- Christian Hammers <ch(a)debian.org> Mon, 26 Jul 2004 00:17:12 +0200 + +mysql-dfsg (4.0.20-5) unstable; urgency=medium + + * The mysqlcheck in the init script is only called when the server + is really alive. Also, the mysql-user 'debian-sys-maint' now has + global select rights (thanks to Nathan Poznick). Closes: #261130 + * Moved the debconf question whether to remove the databases or not + from mysql-server.config to mysql-server.postrm so that it shows + up on purge time and not months earlier (thanks to Wouter Verhelst). + Closes: #251838 + + -- Christian Hammers <ch(a)debian.org> Fri, 23 Jul 2004 22:41:13 +0200 + +mysql-dfsg (4.0.20-4) unstable; urgency=low + + * Added a "mysqlcheck -A --fast" to the 'start' section of the + init script to help admins detect corrupt tables after a server crash. + Currently it exists with an error message but leaves the server + running. Feedback appreciated! + * Made postinst script more robust by calling db_stop earlier and + so prevent pipe-deadlocks. + * Fixed minor typos in initscript (thanks to "C.Y.M."). Closes: 259518 + * Added the undocumented "-DBIG_JOINS" that MySQL apparently uses in + their MAX binaries. It enables 62 instead of 30 tables in a "join". + (thanks to Dave Rolsky). Closes: #260843 + * Added a "df --portability /var/lib/mysql/." check to the preinst + script as users experienced hard to kill hanging mysqlds in such + a situation (thanks to Vaidas Pilkauskas). Closes: #260306 + + -- Christian Hammers <ch(a)debian.org> Fri, 23 Jul 2004 00:51:32 +0200 + +mysql-dfsg (4.0.20-3) unstable; urgency=low + + * Improved tolerance if the init script has been deleted (thanks to + Leonid Shulov for spotting the problem). + * Minor wording changes to README.Debian generalizing /root/ by $HOME + (thanks to Santiago Vila). Closes: #257725 + * Added Japanese debconf translation (thanks to Hideki Yamane). + Closes: #256485 + * Fixed commend in my.cnf regarding logfile directory (thanks to Jayen + Ashar). Closes: #253434 + * Correted "ease to" by "ease of" in package description (thanks to + Johannes Berg). Closes: #253510 + + -- Christian Hammers <ch(a)debian.org> Fri, 9 Jul 2004 00:57:42 +0200 + +mysql-dfsg (4.0.20-2) unstable; urgency=low + + * Removed RPM .spec file from the included documentation as it is pretty + useless (thanks to Loic Minier). + * Added turkish debconf translation (thanks to Recai Oktas). Closes: #252802 + + -- Christian Hammers <ch(a)debian.org> Sun, 6 Jun 2004 14:48:26 +0200 + +mysql-dfsg (4.0.20-1) unstable; urgency=low + + * New upstream version. + + -- Christian Hammers <ch(a)debian.org> Mon, 31 May 2004 23:36:39 +0200 + +mysql-dfsg (4.0.18-8) unstable; urgency=low + + * Updated french translation (thanks to Christian Perrier). Closes: #246789 + + -- Christian Hammers <ch(a)debian.org> Tue, 4 May 2004 23:26:54 +0200 + +mysql-dfsg (4.0.18-7) unstable; urgency=low + + * Added CVE ids for the recent security fixes. + 4.0.18-4 is CAN-2004-0381 (mysqlbug) and + 4.0.18-6 is CAN-2004-0388 (mysql_multi) + + -- Christian Hammers <ch(a)debian.org> Mon, 19 Apr 2004 18:32:03 +0200 + +mysql-dfsg (4.0.18-6) unstable; urgency=medium + + * SECURITY: + Fixed minor tempfile-run security problem in mysqld_multi. + Unprivileged users could create symlinks to files which were then + unknowingly overwritten by run when this script gets executed. + Upstream informed. Thanks to Martin Schulze for finding this. + + -- Christian Hammers <ch(a)debian.org> Wed, 7 Apr 2004 01:28:22 +0200 + +mysql-dfsg (4.0.18-5) unstable; urgency=low + + * Little improvements in debian scripts for last upload. + * Added check to logrotate script for the case that a mysql + server is running but not be accessible with the username and + password from /etc/mysql/debian.conf (thanks to Jeffrey W. Baker). + Closes: 239421 + + -- Christian Hammers <ch(a)debian.org> Sun, 4 Apr 2004 15:27:40 +0200 + +mysql-dfsg (4.0.18-4) unstable; urgency=medium + + * SECURITY: + Aplied fix for unprobable tempfile-symlink security problem in + mysqlbug reported by Shaun Colley on bugtraq on 2004-03-24. + * Updated french debconf translation (thanks to Christian Perrier). + Closes: #236878 + * Updated portugesian debconf translation (thanks to Nuno Senica). + Closes: #239168 + * Updated german debconf translation (thanks to Alwin Meschede). + Closes: #241749 + * Improved debconf template regarding fix_privileges_tables (thanks + to Matt Zimmermann for suggestions). Closes: #219400 + * Improved README.Debian regarding to password settings (thanks to + Yann Dirson). Closes: #241328 + + -- Christian Hammers <ch(a)debian.org> Sat, 3 Apr 2004 19:52:15 +0200 + +mysql-dfsg (4.0.18-3) unstable; urgency=medium + + * Added Build-Depend to po-debconf to let it build everywhere. + + -- Christian Hammers <ch(a)debian.org> Wed, 31 Mar 2004 23:43:33 +0200 + +mysql-dfsg (4.0.18-2) unstable; urgency=low + + * Added a "2>/dev/null" to a "which" command as there are two + "which" versions in Debian of which one needs it. Closes: #235363 + + -- Christian Hammers <ch(a)debian.org> Tue, 2 Mar 2004 23:31:28 +0100 + +mysql-dfsg (4.0.18-1) unstable; urgency=low + + * New upstream version. + * Should now compile and run on ia64 (thanks to Thorsten Werner and + David Mosberger-Tang). Closes: #226863 #228834 + * Converted init scripts to invoce-rc.d (thanks to Erich Schubert). + Closes: 232118 + * Secondlast upload changed logfile location. Closes: #182655 + * Updated Brasilian translation (thanks to Andre Luis Lopes). Closes: + #219847 + + -- Christian Hammers <ch(a)debian.org> Tue, 17 Feb 2004 23:44:58 +0100 + +mysql-dfsg (4.0.17-2) unstable; urgency=low + + * Improved manpage for mysqldumpslow.1 (thanks to Anthony DeRobertis). + Closes: #231039 + * Improved stopping of crashed daemons in init script (thanks to + Matthias Urlichs). Closes: #230327 + + -- Christian Hammers <ch(a)debian.org> Mon, 9 Feb 2004 21:54:29 +0100 + +mysql-dfsg (4.0.17-1) unstable; urgency=low + + * Made logging into /var/log/mysql/ the default. Closes: #225206 + + * New upstream version. Closes: #225028 + * Turned on a 25MB query cache by default (thanks to Cyril Bouthors). + Closes: #226789 + * Updated russian translation (thanks to Ilgiz Kalmetev). Closes: #219263 + * Upstream fixes the problem that AND was not commutative (thanks for + Iain D Broadfoot for mentioning). Closes: #227927 + * Fixed minor typo in my.cnf comments (thanks to James Renken). + Closes: #221496 + * Better documents regex. Closes: #214952 + * Fixed minor germanism in debconf template (thanks to Marc Haber). + Closes: #224148 + * Added explaining comment to my.cnf regarding quoted passwords + (Thanks to Patrick von der Hagen). Closes: #224906 + * Changed "find -exec" to "find -print0 | xargs -0" in preinst to + speed it up. Thanks to Cyril Bouthors. Closes: #220229 + + -- Christian Hammers <ch(a)debian.org> Sun, 18 Jan 2004 16:16:25 +0100 + +mysql-dfsg (4.0.16-2) unstable; urgency=low + + * Tried to repair undefined weak symbols by adding a little Makefile + patch. Closes: #215973 + + -- Christian Hammers <ch(a)debian.org> Mon, 27 Oct 2003 22:52:10 +0100 + +mysql-dfsg (4.0.16-1) unstable; urgency=low + + * New upstream release. + (Mostly little memory problems and other bugfixes it seems) + * Replaced "." by ":" in chown calls to comply with the env setting + "_POSIX2_VERSION=2000112" (thanks to Robert Luberda). Closes: #217399 + * Adjusted syntax in my.cnf to 4.x standard (thanks to Guillaume Plessis). + Closes: #217273 + * Improved README.Debian password instructions (thanks to Levi Waldron). + Closes: #215046 + * Improved NIS warning debconf-template (thanks to Jeff Breidenbach). + Closes: #215791 + * Explicitly added libssl-dev to the libmysqlclient-dev package as it + is needed for mysql_config and the libmysqlclient package only depends + on libssl which has no unnumbered .so version (thanks to Simon Peter + and Davor Ocelic). Closes: #214436, #216162 + * Added "-lwrap" to "mysql_config --libmysqld-libs" and filed it as + upstream bug #1650 (thanks to Noah Levitt). Closes: #214636 + + -- Christian Hammers <ch(a)debian.org> Sat, 25 Oct 2003 01:09:27 +0200 + +mysql-dfsg (4.0.15a-1) unstable; urgency=low + + * Same package as 4.0.15-2 but I could not convince the Debian + installer to move the packages out of incoming. + + -- Christian Hammers <ch(a)debian.org> Tue, 7 Oct 2003 15:10:26 +0200 + +mysql-dfsg (4.0.15-2) unstable; urgency=low + + * Updated package description (thanks to Adrian Bunk). Closes: #210988 + * Fixed small typos in manpages (thanks to Nicolas Francois). + Closes: #211983 + * More updates to package description (thanks to Matthias Lutz/ddtp). + Closes: #213456 + * Updated standards to 3.6.1. + * Closes "new 4.0.15 available" bug. Closes: #213349 + * Updated README.Debian with notes regarding the MySQL manual section + "2.4 Post-installation Setup and Testing" (thanks to Daniel B.). + Closes: #210841 + + -- Christian Hammers <ch(a)debian.org> Fri, 3 Oct 2003 15:59:39 +0200 + +mysql-dfsg (4.0.15-1) unstable; urgency=high + + * SECURITY: + Users who are able to use the "ALTER TABLE" command on the "mysql" + database may be able to exploit this vulnerability to gain a shell with + the privileges of the mysql server (usually running as the 'mysql' user). + Closes: #210403 + * Fixes small description typos (thanks to Oscar Jarkvik). + * Updated Brazilian Portuguese debconf translation. (thanks to Andre Luis + Lopes). Closes: 208030 + * Replaced depricated '.' by ':' in chown (thanks to Matt Zimmerman). + * Fixed manpage typo (thanks to Marc Lehmann). Closes: #207090 + + -- Christian Hammers <ch(a)debian.org> Fri, 3 Oct 2003 15:59:35 +0200 + +mysql-dfsg (4.0.14-1) unstable; urgency=low + + * New upstream version. + + -- Christian Hammers <ch(a)debian.org> Sun, 24 Aug 2003 16:40:36 +0200 + +mysql-dfsg (4.0.13-3) unstable; urgency=low + + * Now start mysqld as default unless you choose not when configurig + with debconf priority low. So packages depending on the server when + installing can access it. Thanks Matt Zimmermann (Closes: #200277) + * Made mysql-server de-installable if the config and database files were + removed by hand before. Thanks to Ard van Breemen (Closes: #200304) + + -- Christian Hammers <ch(a)debian.org> Tue, 8 Jul 2003 22:30:40 +0200 + +mysql-dfsg (4.0.13-2) unstable; urgency=low + + * Added "nice" option for mysqld_safe to give mysqld a different priority. + Submitted to upstream as MySQL Bug #627. Closes: #192087 + * Fixed possible unbound variable in init script. Closes: #194621 + * Fixed french debconf translation (thx Christian Perrier) Closes: #194739 + * Get rid of automake1.5 (for Eric Dorland). + + -- Christian Hammers <ch(a)debian.org> Wed, 11 Jun 2003 18:58:32 +0200 + +mysql-dfsg (4.0.13-1) unstable; urgency=medium + + * New upstream version. + !!! Fixes a very bad natural join bug which justifies the urgency=medium. + !!! http://bugs.mysql.com/bug.php?id=291 + * Fixed mysql_fix_privileges manpage (Frederic Briere) Closes: #191776 + * preinst: "which" is more chatty normal executable than as builtin. + (Thanks to David B Harris). Closes: #188659 + + -- Christian Hammers <ch(a)debian.org> Tue, 6 May 2003 22:03:45 +0200 + +mysql-dfsg (4.0.12-3) unstable; urgency=medium + + * Reincluded new way of creating my debian-sys-maint user from + an old release from experimental. Now works again with old + and new privilege table format. (Thanks to Vincent Danjean + for spotting the problem) Closes: #188201 + * Reincluded hurd build dependency fix from 3.23 branch. + (Thanks to Robert Millan). Closes: #185929 + * Fixed soname in libmysqlclient-dev. Closes: #188160 + * Remove /var/log/mysql/ when purging the package. Closes: #188064 + * Removed /usr/share/doc/mysql/ from mysql-server. Closes: #188066 + * Let group "adm" be able to read logfiles. Closes: #188067 + * Do not call usermod on every upgrade. Closes: #188248 + (Thanks to Philippe Troin for the last three) + * Fixed mysql-server.preinst so that it works on shells where + which is a builtin, too. (Thanks to Erich Schubert) Closes: #181525 + + -- Christian Hammers <ch(a)debian.org> Fri, 11 Apr 2003 11:32:45 +0200 + +mysql-dfsg (4.0.12-2) unstable; urgency=low + + * + * NEW MAJOR UPSTREAM RELEASE: + * + MySQL 4 has finally been declared as 'stable'. Hurray! Read changelogs. + Thanks to all testers, esp. Jose Luis Tallon, of the versions + that were in the "experimental" section before. + * Modified postinst script to run mysql_fix_privileges on every update. + IMPORTANT: Please report if this breaks anything, it is not supposed to. + * Wrote a SSL-MINI-HOWTO.txt! + * Added zlib1g-dev to libmysqlclient12-dev. Closes: 186656 + * Changed section of libmysqlclient12-dev to libdevel. + * Added even more selfwritten manpages. + * Fixed typos. + + -- Christian Hammers <ch(a)debian.org> Sun, 6 Apr 2003 13:47:32 +0200 + +mysql-dfsg (4.0.10.gamma-1) experimental; urgency=low + + * New upstream version. + * They merged some of my patches from debian/patches. Whoa! + * This release should fix the error-logfile problem where mysqld + keeps the error.log open while logrotate removes it. + + -- Christian Hammers <ch(a)debian.org> Wed, 12 Feb 2003 22:39:48 +0100 + +mysql-dfsg (4.0.9.gamma-1) experimental; urgency=low + + * New upstream version. + * Updated the GNU autoconf files to make building on MIPS work. + See bug #176829. + + -- Christian Hammers <ch(a)debian.org> Wed, 29 Jan 2003 22:07:44 +0100 + +mysql-dfsg (4.0.8.gamma-1) experimental; urgency=low + + * New upstream release. + * Improved logging of init script. Closes: #174790 + * We have now libmysqlclient.so.12 instead of .11. + + -- Christian Hammers <ch(a)debian.org> Thu, 9 Jan 2003 20:14:11 +0100 + +mysql-dfsg (4.0.7.gamma-1) experimental; urgency=high + + * SECURITY: This version fixes an upstream security release that is only + present in the 4.x branch which is currently only in the + experimental distribution and therefore will not get a DSA. + * New upstream release. + + -- Christian Hammers <ch(a)debian.org> Sat, 28 Dec 2002 15:51:39 +0100 + +mysql-dfsg (4.0.6.gamma-2) experimental; urgency=low + + * Added --system to addgroup. Closes: #173866 + + -- Christian Hammers <ch(a)debian.org> Sat, 21 Dec 2002 15:28:26 +0100 + +mysql-dfsg (4.0.6.gamma-1) experimental; urgency=low + + * New upstream version. Now Gamma! + * There are no longer changes to the .orig.tar.gz neccessary to make diff + happy. docs/ has still to be deleted, although, as it is non-free. + * Incorporated patches from unstable. + * Added mysqlmanager and a couple of other new scripts. + * Enabled libmysqld embedded server library. + * Enabled SSL and Virtual-IO support. + (CORBA based MySQL-FS seems to be not existing..) + + -- Christian Hammers <ch(a)debian.org> Fri, 20 Dec 2002 22:30:51 +0100 + +mysql-dfsg (4.0.5a.beta-3) experimental; urgency=low + + * Modified postinst to work with old and new mysql.user table format + and fixed spelling typo in postinst. Thanks to Roger Aich. + * Updated config.{guess,sub} to make the mipsel porters happy. + Thanks to Ryan Murray. Closes: #173553 + + -- Christian Hammers <ch(a)debian.org> Wed, 18 Dec 2002 15:56:34 +0100 + +mysql-dfsg (4.0.5a.beta-2) experimental; urgency=low + + * Upstream removed option "--skip-gemini". So did I. Closes: 173142 + + -- Christian Hammers <ch(a)debian.org> Tue, 17 Dec 2002 10:35:49 +0100 + +mysql-dfsg (4.0.5a.beta-1) experimental; urgency=low + + * First 4.x experimental package due to continuous user requests :-) + Please test and report! + * upstream: safe_mysqld has been renamed to mysqld_safe + * upstream: new library soname version libmysqlclient.so.11 + * Renamed libmysqlclientXX-dev to libmysqlclient-dev as I don't plan to + support more than one development environment and this makes the + dependencies easier. + * FIXME: Skipped parts of the debian/patches/alpha patch as the global.h + is not existing. + * FIXME: How to get rid this? Old ltconfig patch already applied. + "lintian: binary-or-shlib-defines-rpath ./usr/bin/mysql /usr/lib/mysql" + + -- Christian Hammers <ch(a)debian.org> Sun, 1 Dec 2002 18:32:32 +0100 + +mysql-dfsg (3.23.53-4) unstable; urgency=medium + + * Fixed errno.h problem. Closes: #168533, #168535 + + -- Christian Hammers <ch(a)debian.org> Sun, 10 Nov 2002 18:32:08 +0100 + +mysql-dfsg (3.23.53-3) unstable; urgency=medium + + * Changed automake build-dep to unversioned automake1.4. Closes: #166391 + * Fixed description. Closes: #167270 + (Thanks to Soren Boll Overgaard) + + -- Christian Hammers <ch(a)debian.org> Tue, 5 Nov 2002 01:25:01 +0100 + +mysql-dfsg (3.23.53-2) unstable; urgency=low + + * Reverted user creation in init scripts. Closes: #166432 + (Thanks to Birzan George Cristian) + + -- Christian Hammers <ch(a)debian.org> Thu, 31 Oct 2002 15:36:25 +0100 + +mysql-dfsg (3.23.53-1) unstable; urgency=low + + * New upstream release. + + -- Christian Hammers <ch(a)debian.org> Thu, 24 Oct 2002 23:04:16 +0200 + +mysql-dfsg (3.23.52-3) unstable; urgency=low + + * Substituted the first-install 'debian-sys-maint' user creation by + something ANSI SQL compliant. Closes: #163497 + (Thanks to Karl Hammar) + * Tightend dependency to debhelper (>= 4.0.12) to be sure that + debconf-utils gets installed, too, as I use dh_installdebconf. + * Fixed upstream manpage bug in mysqldump.1. Closes: #159779 + (Thanks to Colin Watson) + * Added comment about MIN_WORD_LEN to mysql-server.README.Debian + (Thanks to Philipp Dreimann) + * Added a dependency for zlib1g-dev to libmysqlclient10-dev. + (Thanks to Jordi Mallach) + + -- Christian Hammers <ch(a)debian.org> Sun, 15 Sep 2002 17:14:44 +0200 + +mysql-dfsg (3.23.52-2) unstable; urgency=low + + * Fixed typo in preinst scripts. + * Removed bashism in init script. + * Fixed ambiguous debconf example. Closes: #158884 + + -- Christian Hammers <ch(a)debian.org> Fri, 30 Aug 2002 00:51:29 +0200 + +mysql-dfsg (3.23.52-1) unstable; urgency=low + + * New upstream version. Closes: #157731 + * Clearified the meaning of the debian-sys-maint special user in the + README.Debian file. Closes: #153702 + * Wrote some words regarding the skip-networking in README.Debian. + Closes: #157038 + * Added dependency to passwd. + * Fixes typo and unnecessarily complication in is_mysql_alive(). + * Added check for /etc/mysql/my.cnf in init script. + + -- Christian Hammers <ch(a)debian.org> Tue, 27 Aug 2002 01:53:32 +0200 + +mysql-dfsg (3.23.51-4) unstable; urgency=low + + * Added a compressed "nm mysqld" output to allow people to trace + core dumps with /usr/bin/resolve_stack_dump as suggested in the + INSTALL-SOURCE file. Thanks to atudor(a)labs.agilent.com for the hint. + + -- Christian Hammers <ch(a)debian.org> Wed, 24 Jul 2002 20:44:55 +0200 + +mysql-dfsg (3.23.51-3) unstable; urgency=low + + * Corrected copyright file: the MySQL client library is licenced under + the LGPL-2 not the GPL. From version 4.x it actually will be GPL this + is why parts of http://www.mysql.com/ already say so. Closes: #153591 + * Corrected german translation. + Thanks to Roland Rosenfeld <roland(a)spinnaker.de>. Closes: #151903 + + -- Christian Hammers <ch(a)debian.org> Thu, 11 Jul 2002 20:32:28 +0200 + +mysql-dfsg (3.23.51-2) unstable; urgency=low + + * Improved NIS tolerance in preinst script. + + -- Christian Hammers <ch(a)debian.org> Sun, 7 Jul 2002 04:43:28 +0200 + +mysql-dfsg (3.23.51-1) unstable; urgency=medium + + * New upstream version. + * I applied a patch that fixes a binary imcompatibility in + the shared libary libmysqlclient.so.10 between 3.23.50 and + some versions earlier. Upstream has been contacted and asked + for clarification. Closes: #149952 + * Added support for NIS i.e. it shows a warning and fails if the + needed 'mysql' user does not exists but works if it does. + Closes: #143282, #147869 + * Substituted $0 in init scripts by something really weird so that + "./S20mysql restart" works now, too. (BTW: S20? install file-rc!!!) + Closes: #148658 + * Now postinst works even if /etc/init.d/mysql is removed. Closes: #151021 + * Decided to leave "set +x" in postinst but wrote comment. Closes: #151022 + + -- Christian Hammers <ch(a)debian.org> Sun, 7 Jul 2002 04:43:25 +0200 + +mysql-dfsg (3.23.50-1) unstable; urgency=medium + + * New upstream version. + Fixes a very annoying and important bug that lets all mysql programs + including perl scripts etc. segfault when using the read_default_group() + function. 3.23.50 is currently a pre-release and expected to be released + next week. I plan to propose it for woody as soon as its stability has + been proven. The following bug reports are all regarding this issue. + Closes: #144960, #145322, #136798, #138143, + + -- Christian Hammers <ch(a)debian.org> Sat, 18 May 2002 21:14:01 +0200 + +mysql-dfsg (3.23.49x-1) unstable; urgency=low + + * I had to split the package to seperate the manual as it is not GPL + like the rest of the software and docs but under a license that + e.g. forbids selling printed versions. + . + The upstream authors were contacted a while ago but did not like to + change the situation. + . + The names of the resulting packages have not changed as the manual + already was in a seperate mysql-doc package due to it's size. + The source packages are now splitted from one "mysql" to + "mysql-dfsg" in main and "mysql-nonfree" in non-free. + * No code change! + The "x" at the end of the version number ist just to be able to + upload a new source package. ("a" was already taken by upstream + for their binary upload correction) + + -- Christian Hammers <ch(a)debian.org> Wed, 8 May 2002 02:01:41 +0200 + +mysql (3.23.49-8) unstable; urgency=low + + * Substituted $0 in init script to let e.g. "/etc# ./init.d/mysql restart" + works, too. Closes: #141555 + + -- Christian Hammers <ch(a)debian.org> Sun, 7 Apr 2002 15:00:44 +0200 + +mysql (3.23.49-7) unstable; urgency=low + + * The Makefiles are totally broken for the --enable-local-infile + option. I now patched libmysql/libmysql.c#mysql_init() manually. + Closes: #138347 + + -- Christian Hammers <ch(a)debian.org> Fri, 29 Mar 2002 23:55:15 +0100 + +mysql (3.23.49-6) unstable; urgency=low + + * Moved mysqlcheck from server to client package. Closes: #139799 + * Added manpage for mysqlhotcopy. Regarding: #87097 + * Added 'sharedscripts' directive to the logrotate script. + * Replaced grep by /usr/bin/getent to let the group/user checking work + on NIS/LDAP systems, too. Closes: #115677, #101529 + + -- Christian Hammers <ch(a)debian.org> Fri, 22 Mar 2002 22:40:51 +0100 + +mysql (3.23.49-5) unstable; urgency=low + + * Added skip-innodb to default my.cnf. + * Enabled --enable-local-infile, it seems to be a new option that + defaults to disable a formerly enabled feaure. Closes: #137115 + + -- Christian Hammers <ch(a)debian.org> Sat, 16 Mar 2002 00:29:10 +0100 + +mysql (3.23.49-4) unstable; urgency=medium + + * Recompiled against fixed libz. + + * Enabled --enable-local-infile, it seems to be a new option that + defaults to disable a formerly enabled feaure. Closes: #137115 + * Fixed README.compile_on_potato. Closes: #136529 + * Now a ext3 .jounal file in /var/lib/mysql does not prevent the + installation (happens when creating a jounal on an already mounted + partition). Closes: #137146 + + -- Christian Hammers <ch(a)debian.org> Wed, 13 Mar 2002 13:34:24 +0100 + +mysql (3.23.49-3) unstable; urgency=low + + * Added Russian translation. Closes: #135846 + * Fixed installation of .info documents. Closes: #135030 + + -- Christian Hammers <ch(a)debian.org> Wed, 27 Feb 2002 23:36:35 +0100 + +mysql (3.23.49-2) unstable; urgency=low + + * Updated french translation and split template files. Closes: #134754 + * Fixed a small debian.cnf related bug in mysql-server.postinst. + + -- Christian Hammers <ch(a)debian.org> Tue, 19 Feb 2002 23:13:58 +0100 + +mysql (3.23.49-1) unstable; urgency=low + + * New upstream release. + (Mainly InnoDB related fixes) + * Exported a $HOME variable in the scripts so that /root/.my.cnf + is not read anymore. This will avoid problems when admins put + only passwords but no usernames in this file. Closes: #132048 + * New debian-sys-maint password algorithm (now ~96bit :-)) Closes: #133863 + * Recreating debian-sys-main pwd on every install to help people who + accidently delete user or password files... + * Added /var/log/mysql so that user can put the binary logs in there as + mysql cannot write the .001 etc files itself in /var/log which is + owned by root. + + -- Christian Hammers <ch(a)debian.org> Thu, 14 Feb 2002 22:17:45 +0100 + +mysql (3.23.47-6) unstable; urgency=low + + * Dropped a sentence about the new debian-sys-maint user in the + debconf note and updated the README.Debian. Related: #132048 + * Added more french translation. Closes: #132390 + + -- Christian Hammers <ch(a)debian.org> Wed, 6 Feb 2002 09:41:29 +0100 + +mysql (3.23.47-5) unstable; urgency=low + + * Fixed grammar error in template. Closes: #132238 + * Really fixed typo in logrotate script. Closes: #131711 + + -- Christian Hammers <ch(a)debian.org> Tue, 5 Feb 2002 14:20:08 +0100 + +mysql (3.23.47-4) unstable; urgency=medium + + * Fixes typo in postinst that let init script fail. Closes: #131743 + * Fixed bashism bug that failed on ash. Closes: #131697 + * Fixed typo in logrotate script. Closes: #131711 + + -- Christian Hammers <ch(a)debian.org> Thu, 31 Jan 2002 23:58:46 +0100 + +mysql (3.23.47-3) unstable; urgency=low + + * Added new Debian specific mysql user called 'debian-sys-maint' which + is used for pinging the server status, flushing the logs or shutting + down the server in maintenance scripts. The credentials of this user + are stored in the UID0-only readable file /etc/mysql/debian.cnf. + Closes: #129887, #130326, #99274 + * Fixed unintended server startup at boottime. Closes: #122676, #130105 + * New upstream fixes command line parsing bug: Closes: #128473 + * Fixed manpage headers to let apropos work: Closes: #119122 + * Added "status" options for /etc/init.d/mysql. Closes: #129020 + + -- Christian Hammers <ch(a)debian.org> Sun, 27 Jan 2002 19:46:11 +0100 + +mysql (3.23.47-2) unstable; urgency=low + + * Enhanced init scripts by using mysqladmin instead of kill $pid. + Thanks to Aaron Brick. + + -- Christian Hammers <ch(a)debian.org> Fri, 18 Jan 2002 01:42:23 +0100 + +mysql (3.23.47-1) unstable; urgency=low + + * New upstream release. + * Updated brazilian translation of debconf descriptions. Closes: #123332 + + -- Christian Hammers <ch(a)debian.org> Sun, 6 Jan 2002 21:11:17 +0100 + +mysql (3.23.46-3) unstable; urgency=low + + * Fixed bug in postinst where a script was accidently called with + "bash -c <script> -IN_RPM" prevting the first argument to take effect + and then leading to failures on hosts with unresolvable hostnames. + Closes: #126147 + * Small changes and comments in postinst. + + -- Christian Hammers <ch(a)debian.org> Sat, 22 Dec 2001 14:03:02 +0100 + +mysql (3.23.46-2) unstable; urgency=low + + * Start/stop behaviour now configurable via debconf. Closes: #112174 + + -- Christian Hammers <ch(a)debian.org> Sun, 9 Dec 2001 21:38:54 +0100 + +mysql (3.23.46-1) unstable; urgency=low + + * New upstream release. + Only few fixes, mainly innodb related. + + -- Christian Hammers <ch(a)debian.org> Sun, 2 Dec 2001 03:08:48 +0100 + +mysql (3.23.45-1) unstable; urgency=low + + * New upstream version. + Only few fixes, mainly innodb related. + * Added debconf note regarding the skip-networking option. + + -- Christian Hammers <ch(a)debian.org> Sun, 25 Nov 2001 16:50:37 +0100 + +mysql (3.23.44-2) unstable; urgency=low + + * Finally removed debconf toggled "skip-networking" line add/remove + code for /etc/mysql/my.cnf. I don't like editing a file that's tagged + as configuration file. + I disabled networking by default for security reasons. Better ideas? + + -- Christian Hammers <ch(a)debian.org> Fri, 16 Nov 2001 02:11:02 +0100 + +mysql (3.23.44-1) unstable; urgency=low + + * New upstream release. + - fixes replication bug (core dump) + * Made description better english :) Thanks to D. Welton. + + -- Christian Hammers <ch(a)debian.org> Sun, 11 Nov 2001 15:44:07 +0100 + +mysql (3.23.43-4) unstable; urgency=low + + * Disabled statically linking. + + -- Christian Hammers <ch(a)debian.org> Sat, 10 Nov 2001 03:15:56 +0100 + +mysql (3.23.43-3) unstable; urgency=low + + * Changed compiler settings after one user reported instabilities. + See #116631 for more information. + + -- Christian Hammers <ch(a)debian.org> Tue, 30 Oct 2001 21:39:17 +0100 + +mysql (3.23.43-2) unstable; urgency=low + + * Patched sparc mutexes again. Closes: #113430 + + -- Christian Hammers <ch(a)debian.org> Sun, 7 Oct 2001 15:09:00 +0200 + +mysql (3.23.43-1) unstable; urgency=low + + * New upstream version. + - Fixed some unlikely(sic!) bugs and core dumps. + - Fixed a bug with BDB tables and UNIQUE columns that are NULL. + - [more minor bugs were fixed; see changelog] + * Adjusted build depends on libwrap0 for IA-64. Closes: #114582 + * Added the mysqlcheck binary. Closes: #114490 + * Fixed rules for arm architecture. Closes: #88186 + * Renamed mysql_print_defaults to the original name my_print_defaults. + Isn't as descriptive but else I'd have to patch too much. Closes: #114492 + + -- Christian Hammers <ch(a)debian.org> Fri, 5 Oct 2001 22:24:40 +0200 + +mysql (3.23.42-2) unstable; urgency=low + + * Applied patch for m68k compile. Closes: #112904 + + -- Christian Hammers <ch(a)debian.org> Sun, 23 Sep 2001 21:32:57 +0200 + +mysql (3.23.42-1) unstable; urgency=low + + * New upstream releae. + Fixes critical bug with InnoDB and large BLOBs. + + -- Christian Hammers <ch(a)debian.org> Tue, 18 Sep 2001 22:25:47 +0200 + +mysql (3.23.41-2) unstable; urgency=low + + * Fixed shlibs.local problem. Closes: #111573 + * Replaced emacs by sensible-editor in mysqlbug.sh. Thanks Hans Ginzel. + + -- Christian Hammers <ch(a)debian.org> Sun, 9 Sep 2001 17:16:42 +0200 + +mysql (3.23.41-1) unstable; urgency=low + + * New upstream release + * Fixed build problem on ia64. Closes: #110624 + + -- Christian Hammers <ch(a)debian.org> Tue, 14 Aug 2001 23:20:35 +0200 + +mysql (3.23.40-1) unstable; urgency=low + + * New upstream release + + -- Christian Hammers <ch(a)debian.org> Sun, 5 Aug 2001 19:46:18 +0200 + +mysql (3.23.39-5) unstable; urgency=low + + * Added debconf template for brazil. Closes: #106934, #106752 + * Tightened dependencies on debconf. + * Adjusted mysql.err permissions in logrotate script to 0600. Closes: #105672 + + -- Christian Hammers <ch(a)debian.org> Mon, 30 Jul 2001 00:10:12 +0200 + +mysql (3.23.39-4.1) unstable; urgency=low + + * Maintainer-requested NMU. + * Fixing thread mutexes on Sparc and Alpha + (closes: Bug#101783) + * Added --enable-assembler for sparc. This should + allow mysql on sparc to use assembler versions of + some string functions (read: should speed up a bit). + + -- Christopher C. Chimelis <chris(a)debian.org> Fri, 13 Jul 2001 15:09:30 -0400 + +mysql (3.23.39-4) unstable; urgency=low + + * Porting fixes. + + -- Christian Hammers <ch(a)debian.org> Mon, 9 Jul 2001 17:56:54 +0200 + +mysql (3.23.39-3.1) unstable; urgency=low + + * NMU (for porting) + * Update config.sub and config.guess for hppa, sh & s390. + * Add --with-client-ldflags=-lstdc++ to configure line. Closes: #100884 + + -- Matthew Wilcox <willy(a)debian.org> Sun, 8 Jul 2001 19:26:59 -0600 + +mysql (3.23.39-3) unstable; urgency=low + + * Disabled berkeley-db on sparc again. Mutexes aren't working again :-( + + -- Christian Hammers <ch(a)debian.org> Sat, 7 Jul 2001 18:30:08 +0200 + +mysql (3.23.39-2) unstable; urgency=low + + * Bugfixed the m68k mutex patch. Thanks to Michael Fedrowitz. Closes: #103145 + * Removed config.cache files in bdb/ and innobase/. Closes: #103143 + + -- Christian Hammers <ch(a)debian.org> Wed, 4 Jul 2001 22:06:58 +0200 + +mysql (3.23.39-1) unstable; urgency=low + + * New upstream release. Minor bugfixes only. + + -- Christian Hammers <ch(a)debian.org> Thu, 14 Jun 2001 13:53:03 +0200 + +mysql (3.23.38-4) unstable; urgency=low + + * Added logcheck files. Closes: #99131 + (I can't let the usermod away since I don't know of an easy way to + retrive "passwd" information in a shell script considering that + people use different storage methods like LDAP/NIS instead of passwd.) + + -- Christian Hammers <ch(a)debian.org> Fri, 8 Jun 2001 21:04:25 +0200 + +mysql (3.23.38-3) unstable; urgency=low + + * Explicit pointet to /root/.my.cnf to let /etc/init.d/mysql stop + work in sudo environments with $HOME!=/root work, too. Closes: #98324 + * Removes empty /etc/mysql on purge. Closes: #98164 + + -- Christian Hammers <ch(a)debian.org> Tue, 22 May 2001 10:13:06 +0200 + +mysql (3.23.38-2) unstable; urgency=low + + * Added depends to libdbd-mysql-perl for mysql-server. Closes: #94306 + + -- Christian Hammers <ch(a)debian.org> Sat, 19 May 2001 19:43:26 +0200 + +mysql (3.23.38-1) unstable; urgency=low + + * New upstream release. + * Added Build-Depends to procps. Closes: #96768 + + -- Christian Hammers <ch(a)debian.org> Sun, 13 May 2001 17:30:15 +0200 + +mysql (3.23.37-5) unstable; urgency=low + + * Applied mutex patch for bdb support on m68k. + Thanks to Michael Fedrowitz for the patch. + + -- Christian Hammers <ch(a)debian.org> Mon, 7 May 2001 12:30:40 +0200 + +mysql (3.23.37-4) unstable; urgency=low + + * Enable bdb support for m68k architecture. + + -- Christian Hammers <ch(a)debian.org> Sat, 5 May 2001 16:47:36 +0200 + +mysql (3.23.37-3) unstable; urgency=low + + * Added thread-safe client library. Thanks to Shane Wegner. Closes: #95441 + + -- Christian Hammers <ch(a)debian.org> Sat, 28 Apr 2001 09:45:00 -0400 + +mysql (3.23.37-2) unstable; urgency=low + + * Added sparc to the list of BDB supporting architectures after some + tests on vore.debian.org and mails with Ben Collons. + + -- Christian Hammers <ch(a)debian.org> Fri, 27 Apr 2001 09:30:09 -0400 + +mysql (3.23.37-1) unstable; urgency=low + + * New upstream version. + * Added gemini table support. + * Does anybody know how to enable SSL? + * Fixed ARM compilation problem. Closes: #88186 + + -- Christian Hammers <ch(a)debian.org> Sat, 21 Apr 2001 11:48:46 -0400 + +mysql (3.23.36-2) unstable; urgency=low + + * Added patch by Christopher C. Chimelis <chris(a)debian.org> to make + Berkeley db3 work again on Alpha architecture. Closes: #92787 + + -- Christian Hammers <ch(a)debian.org> Tue, 3 Apr 2001 23:41:46 +0200 + +mysql (3.23.36-1) unstable; urgency=high + + * New upstream version. + * SECURITY FIX: One could place database tables outside the database + directory by using '..' in one of the mysql helper programs where the + table name was not checked correctly. This could lead to root compromise + if the server would be running as root else you could at least do bad + things as user mysql. + * upstream: Fixed bug when thread creation failed. + * upstream: Fixed problem in Innobase with non-latin1 charsets + * upstream: Fixed a core-dump bug when using very complex query with DISTINGT + * upstream: many others so called minor bugs... + * fixes bug in init script. Closes: #90257 + (this report was agains some older problem that has been fixed too in .33) + + -- Christian Hammers <ch(a)debian.org> Fri, 30 Mar 2001 02:55:12 +0200 + +mysql (3.23.35-1) unstable; urgency=medium + + * New upstream relase. + * Fixes problem in ORDER BY clause. People using 3.33.34 should upgrade! + * Includes innobase support. + (Hope this is not such a catastrophe like berkeley db...) + + -- Christian Hammers <ch(a)debian.org> Fri, 16 Mar 2001 23:30:30 +0100 + +mysql (3.23.33-3) unstable; urgency=low + + * Forgot #!/bin/sh at top of mysql-doc.postinst. Closes: #89801 + + -- Christian Hammers <ch(a)vore.debian.org> Thu, 15 Mar 2001 20:38:35 -0500 + +mysql (3.23.33-2) unstable; urgency=low + + * Added some missing scripts and manpages. Closes: #84068 + * Added dependency to perl-5.6. Closes: #81942 + * Added french templates somewhen ago. Closes: #83790 + * Added patch to get db3 working on Alpha. Closes: #86033 + Thanks to Christopher C. Chimelis <chris(a)debian.org>. The patch + itself is included as debian/patch.alpha, too. + + -- Christian Hammers <ch(a)debian.org> Sun, 18 Feb 2001 06:40:40 +0100 + +mysql (3.23.33-1) unstable; urgency=high + + * Fixes two security bugs that allowes crashing the server and maybe + gaining the UID of the process that is linked against libmysqlclient! + + -- Christian Hammers <ch(a)debian.org> Tue, 13 Feb 2001 23:01:18 +0100 + +mysql (3.23.32-1) unstable; urgency=low + + * New upstream releaes. + (just minor fixes) + * Added french and german debconf templates. + + -- Christian Hammers <ch(a)debian.org> Sun, 4 Feb 2001 17:27:07 +0100 + +mysql (3.23.31-1) unstable; urgency=high + + * New upstream release. + * Fixes security bug that was announced at BUGTRAQ mailing list. + (Disappointingly not by mysql.com!) And allows a buffer overflow + and therefore access to the mysql UID and all databases when already + having a valid account. Closes: #82881 + + -- Christian Hammers <ch(a)debian.org> Sat, 20 Jan 2001 11:14:36 +0100 + +mysql (3.23.30-2) unstable; urgency=low + + * Recompiled with new dpkg-dev. + + -- Christian Hammers <ch(a)debian.org> Sun, 14 Jan 2001 22:20:55 +0100 + +mysql (3.23.30-1) unstable; urgency=low + + * New upstream release. + + -- Christian Hammers <ch(a)debian.org> Sun, 7 Jan 2001 22:10:18 +0100 + +mysql (3.23.28-10) testing unstable; urgency=low + + * I must upload to "testing" to get it into woody, right?! + + -- Christian Hammers <ch(a)debian.org> Fri, 29 Dec 2000 14:43:57 +0100 + +mysql (3.23.28-9) unstable; urgency=low + + * Made it a replacement for libmysqlclient9. + + -- Christian Hammers <ch(a)westend.com> Mon, 25 Dec 2000 19:15:04 +0100 + +mysql (3.23.28-8) unstable; urgency=low + + * Applied patch from a user to get the skip-networking option working! + Approved from a mysql employee but please test anyways. + This finally: Closes: #79672, #78634, #79660, #79658 + + -- Christian Hammers <ch(a)debian.org> Sat, 16 Dec 2000 14:01:36 +0100 + +mysql (3.23.28-6) unstable; urgency=medium + + * Fixed error in postinst. Closes: #79392, #79400, #79451, #79550 + * Added .info files again on user request. Closes: #78988, #75737 + + -- Christian Hammers <ch(a)debian.org> Wed, 13 Dec 2000 21:18:24 +0100 + +mysql (3.23.28-5) unstable; urgency=low + + * Fixed a stupid bug in mysql-server.postinst regarding the + configuration of skip-networking. Closes: #78639, 78634 + * Used patched bdb which hopefully enables mutexes on Alpha. Closes: #78197 + * Added dependency to adduser. Closes: #76798 + + -- Christian Hammers <ch(a)debian.org> Sun, 10 Dec 2000 16:55:48 +0100 + +mysql (3.23.28-4) unstable; urgency=low + + [never uploaded] + * Fixed a stupid bug in mysql-server.postinst regarding the + configuration of skip-networking. Closes: #78639, 78634 + * Used patched bdb which hopefully enables mutexes on Alpha. Closes: #78197 + + -- Christian Hammers <ch(a)debian.org> Sun, 3 Dec 2000 17:49:44 +0100 + +mysql (3.23.28-3) unstable; urgency=low + + * This time really fixed m68k build error. Closes: #78235 + + -- Christian Hammers <ch(a)debian.org> Sun, 3 Dec 2000 15:02:55 +0100 + +mysql (3.23.28-2) unstable; urgency=low + + * Adjusted rules file to make it buildable on m86k. Closes: #78235 + + -- Christian Hammers <ch(a)debian.org> Fri, 1 Dec 2000 20:07:26 +0100 + +mysql (3.23.28-1) unstable; urgency=low + + * New upstream vesrion. Now gamma! + * Changed umask of mysql.log making it o-rw + * Disabled listening on network reachable TCP ports by default due to + security considerations. + + -- Christian Hammers <ch(a)debian.org> Thu, 23 Nov 2000 20:12:50 +0100 + +mysql (3.23.27-1) unstable; urgency=low + + * New upstream version. + * Closes: #75711 + + -- Christian Hammers <ch(a)debian.org> Sun, 29 Oct 2000 14:29:51 +0100 + +mysql (3.23.25-4) unstable; urgency=low + + * Recompiled to get rid of the dependency for zlib1 (libc5). + Closes: #74952, #74939 + + -- Christian Hammers <ch(a)debian.org> Tue, 17 Oct 2000 14:34:52 +0200 + +mysql (3.23.25-3.1) unstable; urgency=low + + * Maintainer-approved NMU. + * Includes patch to fix and enable db3 support on Alpha. + * Enable support for thread mutexes in db3 on sparc + (it works after all, according to Ben Collins) + * Removed atomic_ functions for Alpha since they are no + longer supported in the current glibc in woody. + * Cleaned up rules file a bit. + + -- Christopher C. Chimelis <chris(a)debian.org> Sat, 14 Oct 2000 04:22:02 -0400 + +mysql (3.23.25-3) unstable; urgency=low + + * Upstream decided not to include my_config.h,my_dir.h into the installed + header files. As this file contains at least informative material + and more important is checked by several autoconf scripts I + included it by hand again. + * Made building of berkeley db conditional to architecture until + I get response whether it works on sparc/alpha now. + + -- Christian Hammers <ch(a)debian.org> Wed, 11 Oct 2000 23:58:38 +0200 + +mysql (3.23.25-2) unstable; urgency=medium + + * Last build went terrible wrong.. Here's the changelog again: + * New upstream release. + * Shared library version was raised from 9 to 10. + Maintainers of packets using libmysqlclient9 must recompile! + + -- Christian Hammers <ch(a)debian.org> Wed, 11 Oct 2000 01:16:34 +0200 + +mysql (3.23.25-1) unstable; urgency=low + + * New upstream release. + * Shared library version was raised from 9 to 10. + Maintainers of packets using libmysqlclient9 must recompile! + + -- Christian Hammers <ch(a)debian.org> Sat, 7 Oct 2000 18:21:51 +0200 + +mysql (3.23.24-2) unstable; urgency=low + + * Applied upstream patch regarding quoting of mysqldump. + * Updated to db-3.1.17-patched (from www.mysql.com) + + -- Christian Hammers <ch(a)debian.org> Fri, 15 Sep 2000 18:58:14 +0200 + +mysql (3.23.24-1) unstable; urgency=medium + + * New upstream version with some important fixes. + * upstream: Last version corrupted CHAR/VARCHAR/BLOB columns with + chararacters above ASCII 128! Check and repair all these tables. + * upstream: fixed small memory leak + * upstream: fixed problem with BDB tables and reading on unique + (not primary) key. + * Disabled BDB tables on all architectures except i386 due to many + bug reports (see #71206). -> HELP APPRECIATED <- + + -- Christian Hammers <ch(a)debian.org> Tue, 12 Sep 2000 06:18:54 +0200 + +mysql (3.23.23-2) unstable; urgency=low + + * Strange... "nohup nice" gives differnet results and let therefore + crash safe_mysqld when starting up. Apparently it seems to be + kernel dependand. Now fixed by another conditional. This + more or less Closes: #71057 + * This bug was reported (accidently) in the following identical reports: + Closes: #71253, #71254, #71257, #71258, #71259, #71262, #71266, #71267 + Closes: #71268, #71271, #71275, #71277, #71278, #71283, #71291 + + -- Christian Hammers <ch(a)debian.org> Sat, 9 Sep 2000 20:13:50 +0200 + +mysql (3.23.23-1) unstable; urgency=low + + * New upstream version. Feature freeze! + * Fixed source build problem. Closes: #70707 + + -- Christian Hammers <ch(a)debian.org> Thu, 31 Aug 2000 10:03:35 +0200 + +mysql (3.23.22b-1) unstable; urgency=low + + * Reorganised docs. Now we have several small html files instead of + one with almost 2M. Closes: 70431 + * Removed pdf,ps and html from source package shrinked it about 3M + (therefore the .orig.tar.gz is called 3.23.22b!) + * -> Last upload failed due to problems at the FTP site so here the + -> changelog again: + * Fixes memory leak, commit/rollback, reserved word "MASTER" ... + * Added Berkeley DB3 source code to the Debian diff to be able to + compile with bdb transaction support! (Great feature!!!) + * Upstream correction of error message. Closes: #68939 + * Upstream correction of reserved word "source". + + -- Christian Hammers <ch(a)debian.org> Fri, 25 Aug 2000 19:21:24 +0200 + +mysql (3.23.22-1) unstable; urgency=low + + * New upstream version. + * Fixes memory leak, commit/rollback, reserved word "MASTER" ... + * Added Berkeley DB3 source code to the Debian diff to be able to + compile with bdb transaction support! (Great feature!!!) + * Upstream correction of error message. Closes: #68939 + * Upstream correction of reserved word "source". + + -- Christian Hammers <ch(a)debian.org> Sun, 20 Aug 2000 09:05:48 +0200 + +mysql (3.23.21-4) unstable; urgency=low + + * Added libmysqlclient9.shlibs and shlibs.local file. Closes: #68669 + + -- Christian Hammers <ch(a)debian.org> Wed, 9 Aug 2000 14:22:49 +0200 + +mysql (3.23.21-3) unstable; urgency=low + + * Let "/etc/init.d/mysql restart" wait until the pid has been + removed before (but max 6 seconds) before restarting. Closes: 65070 + * Added build dependencies. + + -- Christian Hammers <ch(a)debian.org> Sun, 30 Jul 2000 16:16:48 +0200 + +mysql (3.23.21-2) unstable; urgency=low + + * Typo in safe_mysqld prevents start. + + -- Christian Hammers <ch(a)debian.org> Sat, 29 Jul 2000 13:40:50 +0200 + +mysql (3.23.21-1) unstable; urgency=low + + * New upstream version. + + -- Christian Hammers <ch(a)debian.org> Mon, 10 Jul 2000 22:54:17 +0200 + +mysql (3.23.20-1) unstable; urgency=low + + * MySQL finally got fully GPL'ed! This means that there is only one + souce package and only main/* binary packages from now on. + * Fixed symlink in libmysqlclient9-dev. Closes: 66452 + * Apart from that the usual bug fixes for BETA software. + + -- Christian Hammers <ch(a)debian.org> Mon, 3 Jul 2000 20:05:38 +0200 + +mysql-pd (3.23.16-1) unstable; urgency=low + + * New upstream release. (Actually a brand new upstream branch!) + * Added mysql-common package as the configuration file can be used + by all versions of the mysql client library. + Did some more package reorganisations, too. See README.Debian file! + * libmysqlclient.so raised major version from 6 to 9. + * Minor beautifications in the debian/ directory. + + -- Christian Hammers <ch(a)debian.org> Sat, 27 May 2000 20:30:01 +0200 + +mysql-gpl (3.22.30-2) frozen unstable; urgency=low + + * Fixed path in libmysqlclient.la. Closes: #58875 + + -- Christian Hammers <ch(a)debian.org> Sat, 25 Jan 2000 20:27:29 -0700 + +mysql-gpl (3.22.30-1) frozen unstable; urgency=low + + * A small change in the libmysqlclient6 causes mysqladmin to print an + shared library error when displaying the defaults. Everything else + works fine so this error wasn't detected untill now. Closes: #58033 + * TcX released a new MySQL version that includes another security patch, + this time against mysqlaccess. The author told me that it would be + fine if I just included the new .c in this source since I don't want + go to 3.22.32 in frozen. + * ->Release Manager: Although the version number increased there is + no new coded except for the shared library. The rest is the same + as in mysql-server and mysql-client. + + -- Christian Hammers <ch(a)debian.org> Tue, 15 Feb 2000 23:26:54 +0100 + +mysql-gpl (3.22.29-1) unstable; urgency=low + + * New upstream version. + + -- Christian Hammers <ch(a)debian.org> Thu, 6 Jan 2000 20:37:23 +0100 + +mysql-gpl (3.22.27a-3) unstable; urgency=low + + * Use system readline instead of bundled version. Closes: #50069 + Any objections ? + + -- Christian Hammers <ch(a)debian.org> Sun, 14 Nov 1999 18:09:48 +0100 + +mysql-gpl (3.22.27a-2) unstable; urgency=low + + * Now building mysql-gpl-doc in binary-indep. + + -- Christian Hammers <ch(a)debian.org> Sat, 23 Oct 1999 04:22:36 +0200 + +mysql-gpl (3.22.27a-1) unstable; urgency=low + + * Adjusted version number to allow new orig.tar.gz. + The old seems broken :-( People reported compilation problems. + * Changed mysql-gpl-doc to "Architecture: all". + + -- Christian Hammers <ch(a)debian.org> Sun, 17 Oct 1999 13:01:35 +0200 + +mysql-gpl (3.22.27-1) unstable; urgency=low + + * New upstream release. Fixes charset problem. + + -- Christian Hammers <ch(a)debian.org> Mon, 11 Oct 1999 18:01:40 +0200 + +mysql-gpl (3.22.26a-1) unstable; urgency=low + + * New upstream version. Just some small bug fixes. + * FHS compliance. + + -- Christian Hammers <ch(a)debian.org> Sun, 3 Oct 1999 10:16:14 +0200 + +mysql-gpl (3.22.25-2) unstable; urgency=low + + * Added conflict to all old mysql-dev packages. (fixes: #42966) + + -- Christian Hammers <ch(a)debian.org> Sun, 15 Aug 1999 11:35:46 +0200 + +mysql-gpl (3.22.25-1) unstable; urgency=low + + * New upstream version. (We are waiting for 3.23.x !) + * Fixes some upstream small bugs. + + -- Christian Hammers <ch(a)debian.org> Sun, 18 Jul 1999 22:02:06 +0200 + +mysql-gpl (3.22.23b-4) unstable; urgency=low + + * Rebuild for new perl. + + -- Christian Hammers <ch(a)debian.org> Thu, 8 Jul 1999 01:09:57 +0200 + +mysql-gpl (3.22.23b-3) unstable; urgency=low + + * libmysqlclient had the wrong socket path. + + -- Christian Hammers <ch(a)debian.org> Sun, 03 Jul 1999 23:13:30 +0200 + +mysql-gpl (3.22.23b-2) unstable; urgency=low + + * Missed one replace tag to an very old version of mysql-devel. + + -- Christian Hammers <ch(a)debian.org> Sun, 27 Jun 1999 19:13:30 +0200 + +mysql-gpl (3.22.23b-1) unstable; urgency=low + + * New upstream minor version. + * Cleaned up the dependencies a bit. + + -- Christian Hammers <ch(a)debian.org> Sun, 27 Jun 1999 19:13:30 +0200 + +mysql-gpl (3.22.22-1) unstable; urgency=low + + * New upstream version. (closes Bug#36493,37340) + * New maintainer upload. + * Package reorganisation: We prepare for the GPL'ed server which will + * be released soon and make the structure more clear to the user. + + -- Christian Hammers <ch(a)debian.org> Mon, 3 May 1999 20:43:41 +0200 + +mysql (3.22.21-1) unstable; urgency=low + + * Never released. TcX was too fast :-) + + -- Christian Hammers <ch(a)debian.org> Tue, 20 Apr 1999 17:22:04 +0200 + +mysql-freebits (3.21.33b-3) unstable; urgency=low + + * Recompile with libncurses + + -- Scott Hanson <shanson(a)debian.org> Sat, 31 Oct 1998 15:04:39 +0100 + +mysql-freebits (3.21.33b-2) unstable; urgency=low + + * Recompile with libstdc++2.9 (fixes #27792) + + -- Scott Hanson <shanson(a)debian.org> Mon, 12 Oct 1998 18:47:25 +0200 + +mysql-freebits (3.21.33b-1) unstable; urgency=low + + * New upstream version (probably the last for 3.21) + + -- Scott Hanson <shanson(a)debian.org> Tue, 8 Sep 1998 18:59:37 +0200 + +mysql-freebits (3.21.33-4) unstable; urgency=low + + * Separate out non-free source files, move mysql-base, mysql-dev, and + * mysql-doc to main distribution + * Locale files /usr/share/mysql/ now in server, not base; therefore... + * Add conflict to mysql-server <=3.21.33-3 + + -- Scott Hanson <shanson(a)debian.org> Fri, 31 Jul 1998 19:16:08 +0200 + +mysql (3.21.33-3) unstable; urgency=low + + * Release to unstable with moved socket (fixes #24574) + * Add conflict to old libdbd-mysql-perl package + + -- Scott Hanson <shanson(a)debian.org> Wed, 22 Jul 1998 22:17:43 +0200 + +mysql (3.21.33-2) experimental; urgency=low + + * Move socket from /tmp to /var/run (see #24574) + * Release to experimental, since this breaks everything statically + * linked to libmysqlclient! + + -- Scott Hanson <shanson(a)debian.org> Wed, 15 Jul 1998 19:37:01 +0200 + +mysql (3.21.33-1) unstable; urgency=low + + * New upstream release + + -- Scott Hanson <shanson(a)debian.org> Sun, 12 Jul 1998 08:18:18 +0200 + +mysql (3.21.32a-1) unstable; urgency=low + + * New upstream release + * Lintian bugs: ldconfig, missing manpage, call to perl5 + * Lintian bug shlib-with-non-pic-code _not_ yet fixed + + -- Scott Hanson <shanson(a)debian.org> Sat, 4 Jul 1998 07:57:13 +0200 + +mysql (3.21.31-1) unstable frozen; urgency=low + + * New upstream release for hamm and slink (bug fixes only) + * Fix unsecure use of temp file in mysqlbug (fixes #23606) + * Added brief licensing information to control file + + -- Scott Hanson <shanson(a)debian.org> Tue, 16 Jun 1998 10:52:44 +0200 + +mysql (3.21.30-3) unstable; urgency=low + + * Restore missing shared library dependencies for mysql-server + + -- Scott Hanson <shanson(a)debian.org> Mon, 15 Jun 1998 07:51:58 +0200 + +mysql (3.21.30-2) unstable; urgency=low + + * Simplify debian/rules (fixes #17662) + * Edit manual.texi to add "Debian notes" to documentation + * Add note about passwords on command line (fixes #16471) + * Add note about getting privleges for users (fixes #22891) + * Correct "Possible license changes" heading (fixes #22711) + * Add uninstalled header files to /usr/doc/mysql-dev/examples (fixes #22627) + * Add udf_example.cc to /usr/doc/mysql-dev/examples (fixes #22710) + + -- Scott Hanson <shanson(a)debian.org> Sun, 7 Jun 1998 13:05:37 +0200 + +mysql (3.21.30-1) unstable; urgency=low + + * Stable upstream release + + -- Scott Hanson <shanson(a)debian.org> Tue, 12 May 1998 22:13:25 +0200 + +mysql (3.21.29gamma-1) unstable; urgency=low + + * New upstream release + * Do not create 'mysql' subdirectory for libs and headers (fixes #19020) + * Remove 'CXX=gcc' flag from configure (g++ now standard) + + -- Scott Hanson <shanson(a)debian.org> Sun, 12 Apr 1998 18:38:03 +0200 + +mysql (3.21.28gamma-1) unstable; urgency=low + + * New upstream release + * Unstable-only release; hamm stays at 3.21.25 for now + + -- Scott Hanson <shanson(a)debian.org> Thu, 2 Apr 1998 21:33:51 +0200 + +mysql (3.21.25gamma-3) unstable frozen; urgency=low + + * Have mysql-base suggest perl >= 5.004 for mysqlaccess (fixes #19593) + * Fix shlibs to refer to mysql-base rather than the no-longer-existant mysql + + -- Scott Hanson <shanson(a)debian.org> Thu, 26 Mar 1998 18:22:59 +0100 + +mysql (3.21.25gamma-2) unstable; urgency=low + + * Restore libmysqlclient.so symlink to mysql-dev (fixes #19036) + + -- Scott Hanson <shanson(a)debian.org> Sun, 8 Mar 1998 10:46:43 +0100 + +mysql (3.21.25gamma-1) unstable; urgency=low + + * Check if running as root in init.d script (fixes #18577) + * New upstream release + + -- Scott Hanson <shanson(a)debian.org> Fri, 27 Feb 1998 20:01:30 +0100 + +mysql (3.21.24gamma-1) unstable; urgency=low + + * New upstream release + + -- Scott Hanson <shanson(a)debian.org> Mon, 23 Feb 1998 08:14:17 +0100 + +mysql (3.21.23beta-3) unstable; urgency=low + + * Squashed errors found by lintian + + -- Scott Hanson <shanson(a)debian.org> Tue, 17 Feb 1998 20:19:01 +0100 + +mysql (3.21.23beta-2) unstable; urgency=low + + * Fixed overlaps with old mysql package (fixes #17843) + + -- Scott Hanson <shanson(a)debian.org> Thu, 5 Feb 1998 22:55:00 +0100 + +mysql (3.21.23beta-1) unstable; urgency=low + + * New upstream release + * Fix include lines in mysql.h (fixes #17827) + * Move /usr/include/mysql to mysql-dev + + -- Scott Hanson <shanson(a)debian.org> Wed, 4 Feb 1998 19:59:14 +0100 + +mysql (3.21.22beta-3) unstable; urgency=low + + * Correct descriptions in control file (fixes #17698) + * Clean up output of shutdown script + + -- Scott Hanson <shanson(a)debian.org> Sat, 31 Jan 1998 19:04:29 +0100 + +mysql (3.21.22beta-2) unstable; urgency=low + + * Split out mysql-dev and mysql-bench subpackages + + -- Scott Hanson <shanson(a)debian.org> Wed, 28 Jan 1998 19:52:27 +0100 + +mysql (3.21.22beta-1) unstable; urgency=low + + * New upstream release + + -- Scott Hanson <shanson(a)debian.org> Wed, 28 Jan 1998 18:59:09 +0100 + +mysql (3.21.21a.beta-2) unstable; urgency=low + + * Compile with libpthreads from libc6-dev_2.0.6-3 rather than statically + linking to patched libpthreads (see changes to 3.20.29-2) + + -- Scott Hanson <shanson(a)debian.org> Sun, 25 Jan 1998 13:17:15 +0100 + +mysql (3.21.21a.beta-1) unstable; urgency=low + + * Put initial database, mysql_install_db, safe_mysqld, isamlog and + isamchk in mysql-server + * Correct upstream release number so source packages are correctly built + + -- Scott Hanson <shanson(a)debian.org> Mon, 19 Jan 1998 07:52:48 +0100 + +mysql (3.21.21.beta-1) unstable; urgency=low + + * Use debhelper where possible in rules + * Split binary packages into mysql-base, mysql-client, mysql-doc + * New upstream release + + -- Scott Hanson <shanson(a)debian.org> Thu, 15 Jan 1998 08:12:17 +0100 + +mysql (3.21.19.beta-1) unstable; urgency=low + + * Offer to set root password in mysql_install_db + * Kill `pidof mysqld` on shutdown rather than use mysqladmin + * New upstream version + + -- Scott Hanson <shanson(a)debian.org> Fri, 9 Jan 1998 20:06:35 +0100 + +mysql (3.21.17a.beta-2) unstable; urgency=low + + * Remove perl stuff (it's going back into libdbd-mysql-perl) + * Remove conflict with libdbd-mysql-perl + * Do not compress *html files (fixes #16314) + + -- Scott Hanson <shanson(a)debian.org> Tue, 30 Dec 1997 07:34:20 +0100 + +mysql (3.21.17a.beta-1) unstable; urgency=low + + * Add conflict to libdbd-mysql-perl + * Use --pid-file option to place pid file in /var/run rather than patching + * Add install-info to postinst and postrm + * Add filename to message shown by mysql_install_db (fixes #16621) + * New upstream version + + -- Scott Hanson <shanson(a)debian.org> Sun, 21 Dec 1997 19:41:45 +0100 + +mysql (3.20.32a-5) unstable; urgency=low + + * Move mysqld to /usr/lib/mysql, per policy discussion + * Adjust makefiles so perl libs get installed + + -- Scott Hanson <shanson(a)debian.org> Wed, 3 Dec 1997 22:37:45 +0100 + +mysql (3.20.32a-4) unstable; urgency=low + + * Move mysqld to /usr/sbin to comply with FSSTND + + -- Scott Hanson <shanson(a)debian.org> Mon, 3 Nov 1997 20:12:29 +0100 + +mysql (3.20.32a-3) unstable; urgency=low + + * Comment out tests in mysql_install_db... for real this time! + + -- Scott Hanson <shanson(a)debian.org> Mon, 3 Nov 1997 07:32:53 +0100 + +mysql (3.20.32a-2) unstable; urgency=low + + * Comment out tests in mysql_install_db (fixes #14304) + + -- Scott Hanson <shanson(a)debian.org> Sat, 1 Nov 1997 18:45:25 +0100 + +mysql (3.20.32a-1) unstable; urgency=low + + * New upstream version + + -- Scott Hanson <shanson(a)debian.org> Wed, 29 Oct 1997 07:11:42 +0100 + +mysql (3.20.29-2) unstable; urgency=low + + * New maintainer + * Statically link mysqld to patched glibc-2.0.5 libpthread + (works around #13586; see README.debian.glibc-2.0.5) + * Conflict with libpthread0 (fixes #13448) + * Don't link libg++, avoiding problems with glibc libpthread + + -- Scott Hanson <shanson(a)debian.org> Thu, 16 Oct 1997 19:25:23 +0200 + +mysql (3.20.29-1) unstable; urgency=low + + * New upstream version + * Recompiled with libc6 + * Include mysql-faq_toc.html (fixes #10885) + * Reworked /etc/init.d/mysql script (thanks to Heiko) + * Remove file /usr/lib/libmysqlclient.so.4 when package is removed. + * Use absolute path specification for conffile + * Use /usr/bin/perl instead of /bin/perl (fixes #10654) + * Do not depend on mysql (fixes #12427) + * Installed missing manpage for Mysql perl module + * Don't use debstd anymore + * Pristine source + * Set section to `non-free/devel' + * Upgraded to standards version 2.3.0.0 + + -- Christian Schwarz <schwarz(a)debian.org> Fri, 12 Sep 1997 02:12:58 +0200 + +mysql (3.20.16beta-2) unstable; urgency=low + + * Uses /usr/bin/perl instead of /bin/perl (fixes bug #9731) + * Don't run mysqld with --log option + * Don't install regex manual pages + * Suggest package mysql-manual + * Fixed typo in changelog + * Upgrade to policy 2.1.3.2 + + -- Christian Schwarz <schwarz(a)debian.org> Sun, 11 May 1997 14:19:26 +0200 + +mysql (3.20.16beta-1) unstable; urgency=low + + * Initial Release. + + -- Christian Schwarz <schwarz(a)debian.org> Sat, 12 Apr 1997 13:51:28 +0200 === added file 'storage/xtradb/build/debian/compat' --- a/storage/xtradb/build/debian/compat 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/compat 2010-01-26 18:02:46 +0000 @@ -0,0 +1 @@ +4 === added file 'storage/xtradb/build/debian/control' --- a/storage/xtradb/build/debian/control 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/control 2010-02-22 19:53:53 +0000 @@ -0,0 +1,118 @@ +Source: percona-xtradb-dfsg-5.1 +Section: misc +Priority: optional +Maintainer: Percona SQL Development Team <mysql-dev(a)percona.com> +Uploaders: Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> +Build-Depends: libtool (>= 1.4.2-7), procps | hurd, debhelper (>= 4.1.16), file (>= 3.28-1), libncurses5-dev (>= 5.0-6), perl (>= 5.6.0), libwrap0-dev (>= 7.6-8.3), zlib1g-dev (>= 1:1.1.3-5), libreadline5-dev | libreadline-dev, psmisc, po-debconf, chrpath, automake1.9, doxygen, gs, dpatch, gawk, bison, lsb-release, fakeroot +Standards-Version: 3.8.0 +Homepage: http://www.percona.com/ +Vcs-Browser: http://bazaar.launchpad.net/~percona-dev/percona-xtradb/release-1.0/files +Vcs-Bzr: bzr+ssh://bazaar.launchpad.net/~percona-dev/percona-xtradb/release-1.0/ + +Package: libpercona-xtradb-client16 +Section: libs +Architecture: any +Depends: percona-xtradb-common (>= ${source:Version}), ${shlibs:Depends} +Description: Percona SQL database client library + Percona SQL is a fast, stable and true multi-user, multi-threaded SQL database + server. SQL (Structured Query Language) is the most popular database query + language in the world. The main goals of Percona SQL are speed, robustness and + ease of use. + . + This package includes the client library. + +Package: libpercona-xtradb-client15-dev +Architecture: all +Section: libdevel +Depends: libpercona-xtradb-client-dev (>= ${source:Version}) +Description: Percona SQL database development files - empty transitional package + This is an empty package that depends on libpercona-xtradb-client-dev to ease the + transition for packages with versioned build-deps on libpercona-xtradb-client15-dev. + +Package: libpercona-xtradb-client-dev +Architecture: any +Section: libdevel +Depends: libpercona-xtradb-client16 (>= ${source:Version}), zlib1g-dev, , ${shlibs:Depends} +Conflicts: libmysqlclient14-dev, libmysqlclient12-dev, libmysqlclient10-dev, libmysqlclient15-dev, libmysqlclient16-dev +Replaces: libmysqlclient14-dev, libmysqlclient12-dev, libmysqlclient10-dev, libmysqlclient15-dev, libmysqlclient16-dev +Description: Percona SQL database development files + Percona SQL is a fast, stable and true multi-user, multi-threaded SQL database + server. SQL (Structured Query Language) is the most popular database query + language in the world. The main goals of Percona SQL are speed, robustness and + ease of use. + . + This package includes development libraries and header files. + +Package: percona-xtradb-common +Section: database +Architecture: all +Depends: ${shlibs:Depends}, ${misc:Depends} +Conflicts: mysql-common-4.1, mysql-common-5.0, mysql-common-5.1 +Provides: mysql-common-4.1, percona-xtradb-common +Replaces: mysql-common-4.1, mysql-common-5.0, mysql-common-5.1 +Description: Percona SQL database common files (e.g. /etc/mysql/my.cnf) + Percona SQL is a fast, stable and true multi-user, multi-threaded SQL database + server. SQL (Structured Query Language) is the most popular database query + language in the world. The main goals of Percona SQL are speed, robustness and + ease of use. + . + This package includes files needed by all versions of the client library + (e.g. /etc/mysql/my.cnf). + +Package: percona-xtradb-client-5.1 +Architecture: any +Depends: debianutils (>=1.6), libdbi-perl, percona-xtradb-common (>= ${source:Version}), libpercona-xtradb-client16 (>= ${source:Version}), ${perl:Depends}, ${shlibs:Depends}, ${misc:Depends} +Provides: virtual-mysql-client, mysql-client, mysql-client-4.1, percona-xtradb-client, percona-xtradb-client-5.1 +Conflicts: mysql-client (<< ${source:Version}), mysql-client-5.0, mysql-client-5.1, percona-xtradb-client-5.0 +Replaces: mysql-client (<< ${source:Version}), mysql-client-5.0, mysql-client-5.1, percona-xtradb-client-5.0 +Description: Percona SQL database client binaries + Percona SQL is a fast, stable and true multi-user, multi-threaded SQL database + server. SQL (Structured Query Language) is the most popular database query + language in the world. The main goals of Percona SQL are speed, robustness and + ease of use. + . + This package includes the client binaries and the additional tools + innotop and mysqlreport. + +Package: percona-xtradb-server-5.1 +Architecture: any +Suggests: tinyca +Recommends: mailx, libhtml-template-perl +Pre-Depends: percona-xtradb-common (>= ${source:Version}), adduser (>= 3.40), debconf +Depends: percona-xtradb-client-5.1 (>= ${source:Version}), libdbi-perl, perl (>= 5.6), ${shlibs:Depends}, ${misc:Depends}, psmisc, passwd, lsb-base (>= 3.0-10) +Conflicts: mysql-server (<< ${source:Version}), mysql-server-4.1, percona-xtradb-server-5.0 +Provides: mysql-server, virtual-mysql-server, mysql-server-5.0, percona-xtradb-server-5.1 +Replaces: mysql-server (<< ${source:Version}), mysql-server-5.0, percona-xtradb-server-5.0 +Description: Percona SQL database server binaries + Percona SQL is a fast, stable and true multi-user, multi-threaded SQL database + server. SQL (Structured Query Language) is the most popular database query + language in the world. The main goals of Percona SQL are speed, robustness and + ease of use. + . + This package includes the server binaries. + +Package: percona-xtradb-server +Section: database +Architecture: all +Depends: percona-xtradb-server-5.1 +Description: Percona SQL database server (metapackage depending on the latest version) + This is an empty package that depends on the current "best" version of + percona-xtradb-server (currently percona-xtradb-server-5.1), as determined by the Percona SQL + maintainers. Install this package if in doubt about which Percona SQL + version you need. That will install the version recommended by the + package maintainers. + . + Percona SQL is a fast, stable and true multi-user, multi-threaded SQL database + server. SQL (Structured Query Language) is the most popular database query + language in the world. The main goals of Percona SQL are speed, robustness and + ease of use. + +Package: percona-xtradb-client +Section: database +Architecture: all +Depends: percona-xtradb-client-5.1 +Description: Percona SQL database client (metapackage depending on the latest version) + This is an empty package that depends on the current "best" version of + percona-xtradb-client (currently percona-xtradb-client-5.1), as determined by the Percona SQL + maintainers. Install this package if in doubt about which Percona SQL version + you want, as this is the one we consider to be in the best shape. === added file 'storage/xtradb/build/debian/copyright' --- a/storage/xtradb/build/debian/copyright 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/copyright 2010-01-26 18:02:46 +0000 @@ -0,0 +1,169 @@ + +== MySQL == + +The Debian package of MySQL was first debianzed on 1997-04-12 by Christian +Schwarz <schwarz(a)debian.org> and ist maintained since 1999-04-20 by +Christian Hammers <ch(a)debian.org>. + +It can be downloaded from http://www.mysql.com/ + +Copyright: + +According to the file "COPYING" all parts of this package are licenced +under the terms of the GNU GPL Version 2 of which a copy is available +in /usr/share/common-licenses. + +To allow free software with other licences than the GPL to link against the +shared library, special terms for "derived works" are defined in the file +"EXCEPTIONS-CLIENT" which is quoted below. + +More information can be found on http://www.mysql.com/company/legal/licensing/ + +The manual had to be removed as it is not free in the sense of the +Debian Free Software Guidelines (DFSG). + +> Appendix I MySQL FLOSS License Exception +> **************************************** +> +> Version 0.3, 10 February 2005 +> +> The MySQL AB Exception for Free/Libre and Open Source Software-only +> Applications Using MySQL Client Libraries (the "FLOSS Exception"). +> +> Exception Intent +> ================ +> +> We want specified Free/Libre and Open Source Software ("FLOSS") +> applications to be able to use specified GPL-licensed MySQL client +> libraries (the "Program") despite the fact that not all FLOSS licenses +> are compatible with version 2 of the GNU General Public License (the +> "GPL"). +> +> Legal Terms and Conditions +> ========================== +> +> As a special exception to the terms and conditions of version 2.0 of the +> GPL: +> +> 1. You are free to distribute a Derivative Work that is formed +> entirely from the Program and one or more works (each, a "FLOSS +> Work") licensed under one or more of the licenses listed below in +> section 1, as long as: +> +> a. You obey the GPL in all respects for the Program and the +> Derivative Work, except for identifiable sections of the +> Derivative Work which are not derived from the Program, and +> which can reasonably be considered independent and separate +> works in themselves, +> +> b. all identifiable sections of the Derivative Work which are not +> derived from the Program, and which can reasonably be +> considered independent and separate works in themselves, +> +> i +> are distributed subject to one of the FLOSS licenses +> listed below, and +> +> ii +> the object code or executable form of those sections are +> accompanied by the complete corresponding +> machine-readable source code for those sections on the +> same medium and under the same FLOSS license as the +> corresponding object code or executable forms of those +> sections, and +> +> c. any works which are aggregated with the Program or with a +> Derivative Work on a volume of a storage or distribution +> medium in accordance with the GPL, can reasonably be +> considered independent and separate works in themselves which +> are not derivatives of either the Program, a Derivative Work +> or a FLOSS Work. +> +> If the above conditions are not met, then the Program may only be +> copied, modified, distributed or used under the terms and +> conditions of the GPL or another valid licensing option from MySQL +> AB. +> +> 2. FLOSS License List +> +> *License name* *Version(s)/Copyright Date* +> Academic Free License 2.0 +> Apache Software License 1.0/1.1/2.0 +> Apple Public Source License 2.0 +> Artistic license From Perl 5.8.0 +> BSD license "July 22 1999" +> Common Public License 1.0 +> GNU Library or "Lesser" General Public 2.0/2.1 +> License (LGPL) +> Jabber Open Source License 1.0 +> MIT license - +> Mozilla Public License (MPL) 1.0/1.1 +> Open Software License 2.0 +> OpenSSL license (with original SSLeay "2003" ("1998") +> license) +> PHP License 3.0 +> Python license (CNRI Python License) - +> Python Software Foundation License 2.1.1 +> Sleepycat License "1999" +> W3C License "2001" +> X11 License "2001" +> Zlib/libpng License - +> Zope Public License 2.0 +> +> Due to the many variants of some of the above licenses, we require +> that any version follow the 2003 version of the Free Software +> Foundation's Free Software Definition +> (`http://www.gnu.org/philosophy/free-sw.html') or version 1.9 of +> the Open Source Definition by the Open Source Initiative +> (`http://www.opensource.org/docs/definition.php'). +> +> 3. Definitions +> +> a. Terms used, but not defined, herein shall have the meaning +> provided in the GPL. +> +> b. Derivative Work means a derivative work under copyright law. +> +> 4. Applicability This FLOSS Exception applies to all Programs that +> contain a notice placed by MySQL AB saying that the Program may be +> distributed under the terms of this FLOSS Exception. If you +> create or distribute a work which is a Derivative Work of both the +> Program and any other work licensed under the GPL, then this FLOSS +> Exception is not available for that work; thus, you must remove +> the FLOSS Exception notice from that work and comply with the GPL +> in all respects, including by retaining all GPL notices. You may +> choose to redistribute a copy of the Program exclusively under the +> terms of the GPL by removing the FLOSS Exception notice from that +> copy of the Program, provided that the copy has never been +> modified by you or any third party. + + +== innotop == + +Author: Baron Schwartz <baron(a)xaprb.com> +URL: http://innotop.sourceforge.net + +License: +> This software is dual licensed, either GPL version 2 or Artistic License. +> +> This package is free software; you can redistribute it and/or modify +> it under the terms of the GNU General Public License as published by +> the Free Software Foundation; either version 2 of the License, or +> (at your option) any later version. +> +> This package is distributed in the hope that it will be useful, +> but WITHOUT ANY WARRANTY; without even the implied warranty of +> MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +> GNU General Public License for more details. +> +> You should have received a copy of the GNU General Public License +> along with this package; if not, write to the Free Software +> Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +On Debian systems, the complete text of the GNU General Public License and the +Artistic License can be found in `/usr/share/common-licenses/'. + +The upstream author explained here: http://bugs.gentoo.org/show_bug.cgi?id=14760 +that these licenses also apply to the following files: +- innotop.html +- InnoDBParser.pm === added file 'storage/xtradb/build/debian/libpercona-xtradb-client-dev.README.Maintainer' --- a/storage/xtradb/build/debian/libpercona-xtradb-client-dev.README.Maintainer 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client-dev.README.Maintainer 2010-02-18 23:53:03 +0000 @@ -0,0 +1,4 @@ +The examples directory includes files that might be needed by some +developers: +- header files not installed by default +- the example file udf_example.c === added file 'storage/xtradb/build/debian/libpercona-xtradb-client-dev.dirs' --- a/storage/xtradb/build/debian/libpercona-xtradb-client-dev.dirs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client-dev.dirs 2010-02-18 23:53:03 +0000 @@ -0,0 +1,2 @@ +usr/include/ +usr/lib/ === added file 'storage/xtradb/build/debian/libpercona-xtradb-client-dev.docs' --- a/storage/xtradb/build/debian/libpercona-xtradb-client-dev.docs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client-dev.docs 2010-02-18 23:53:03 +0000 @@ -0,0 +1 @@ +EXCEPTIONS-CLIENT === added file 'storage/xtradb/build/debian/libpercona-xtradb-client-dev.examples' --- a/storage/xtradb/build/debian/libpercona-xtradb-client-dev.examples 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client-dev.examples 2010-02-18 23:53:03 +0000 @@ -0,0 +1 @@ +sql/udf_example.c === added file 'storage/xtradb/build/debian/libpercona-xtradb-client-dev.files' --- a/storage/xtradb/build/debian/libpercona-xtradb-client-dev.files 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client-dev.files 2010-02-18 23:53:03 +0000 @@ -0,0 +1,7 @@ +usr/bin/mysql_config +usr/include/mysql/*.h +usr/lib/libmysqlclient.a +usr/lib/libmysqlclient.la +usr/lib/mysql/*.a +usr/lib/mysql/*.la +usr/share/man/man1/mysql_config.1 === added file 'storage/xtradb/build/debian/libpercona-xtradb-client-dev.links' --- a/storage/xtradb/build/debian/libpercona-xtradb-client-dev.links 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client-dev.links 2010-02-18 23:53:03 +0000 @@ -0,0 +1,2 @@ +usr/lib/libmysqlclient.so.16 usr/lib/libmysqlclient.so +usr/lib/libmysqlclient_r.so.16 usr/lib/libmysqlclient_r.so === added file 'storage/xtradb/build/debian/libpercona-xtradb-client16.dirs' --- a/storage/xtradb/build/debian/libpercona-xtradb-client16.dirs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client16.dirs 2010-02-18 23:53:03 +0000 @@ -0,0 +1 @@ +usr/lib/ === added file 'storage/xtradb/build/debian/libpercona-xtradb-client16.docs' --- a/storage/xtradb/build/debian/libpercona-xtradb-client16.docs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client16.docs 2010-02-18 23:53:03 +0000 @@ -0,0 +1 @@ +EXCEPTIONS-CLIENT === added file 'storage/xtradb/build/debian/libpercona-xtradb-client16.files' --- a/storage/xtradb/build/debian/libpercona-xtradb-client16.files 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client16.files 2010-02-18 23:53:03 +0000 @@ -0,0 +1 @@ +usr/lib/libmysqlclient*.so.* === added file 'storage/xtradb/build/debian/libpercona-xtradb-client16.postinst' --- a/storage/xtradb/build/debian/libpercona-xtradb-client16.postinst 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/libpercona-xtradb-client16.postinst 2010-02-18 23:53:03 +0000 @@ -0,0 +1,12 @@ +#!/bin/bash -e + +# dh_installdeb will replace this with shell code automatically +# generated by other debhelper scripts. + +#DEBHELPER# + +exit 0 + +# vim: ts=4 + + === added directory 'storage/xtradb/build/debian/patches' === added file 'storage/xtradb/build/debian/patches/00list' --- a/storage/xtradb/build/debian/patches/00list 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/00list 2010-01-26 18:02:46 +0000 @@ -0,0 +1,6 @@ +33_scripts__mysql_create_system_tables__no_test.dpatch +38_scripts__mysqld_safe.sh__signals.dpatch +41_scripts__mysql_install_db.sh__no_test.dpatch +44_scripts__mysql_config__libs.dpatch +50_mysql-test__db_test.dpatch +60_percona_support.dpatch === added file 'storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Images_Makefile.in.dpatch' --- a/storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Images_Makefile.in.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Images_Makefile.in.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,776 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run +## 01_MAKEFILES__Docs_Makefile.in.dpatch by <ch(a)debian.org> +## +## All lines beginning with `## DP:' are a description of the patch. +## DP: Creates Docs/Makefile.in + +@DPATCH@ + +--- old/Docs/Images/Makefile.in 2005-03-01 02:08:01.877429040 +0100 ++++ new/Docs/Images/Makefile.in 2005-02-28 21:21:24.000000000 +0100 +@@ -0,0 +1,765 @@ ++# Makefile.in generated by automake 1.7.9 from Makefile.am. ++# @configure_input@ ++ ++# Copyright 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003 ++# Free Software Foundation, Inc. ++# This Makefile.in is free software; the Free Software Foundation ++# gives unlimited permission to copy and/or distribute it, ++# with or without modifications, as long as this notice is preserved. ++ ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY, to the extent permitted by law; without ++# even the implied warranty of MERCHANTABILITY or FITNESS FOR A ++# PARTICULAR PURPOSE. ++ ++@SET_MAKE@ ++ ++# Copyright (C) 2000 MySQL AB & MySQL Finland AB & TCX DataKonsult AB ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program; if not, write to the Free Software ++# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ++ ++# Process this file with automake to create Makefile.in ++ ++srcdir = @srcdir@ ++top_srcdir = @top_srcdir@ ++VPATH = @srcdir@ ++pkgdatadir = $(datadir)/@PACKAGE@ ++pkglibdir = $(libdir)/@PACKAGE@ ++pkgincludedir = $(includedir)/@PACKAGE@ ++top_builddir = . ++ ++am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd ++INSTALL = @INSTALL@ ++install_sh_DATA = $(install_sh) -c -m 644 ++install_sh_PROGRAM = $(install_sh) -c ++install_sh_SCRIPT = $(install_sh) -c ++INSTALL_HEADER = $(INSTALL_DATA) ++transform = $(program_transform_name) ++NORMAL_INSTALL = : ++PRE_INSTALL = : ++POST_INSTALL = : ++NORMAL_UNINSTALL = : ++PRE_UNINSTALL = : ++POST_UNINSTALL = : ++build_triplet = @build@ ++host_triplet = @host@ ++target_triplet = @target@ ++ACLOCAL = @ACLOCAL@ ++ALLOCA = @ALLOCA@ ++AMDEP_FALSE = @AMDEP_FALSE@ ++AMDEP_TRUE = @AMDEP_TRUE@ ++AMTAR = @AMTAR@ ++AR = @AR@ ++AS = @AS@ ++ASSEMBLER_FALSE = @ASSEMBLER_FALSE@ ++ASSEMBLER_TRUE = @ASSEMBLER_TRUE@ ++ASSEMBLER_sparc32_FALSE = @ASSEMBLER_sparc32_FALSE@ ++ASSEMBLER_sparc32_TRUE = @ASSEMBLER_sparc32_TRUE@ ++ASSEMBLER_sparc64_FALSE = @ASSEMBLER_sparc64_FALSE@ ++ASSEMBLER_sparc64_TRUE = @ASSEMBLER_sparc64_TRUE@ ++ASSEMBLER_x86_FALSE = @ASSEMBLER_x86_FALSE@ ++ASSEMBLER_x86_TRUE = @ASSEMBLER_x86_TRUE@ ++AUTOCONF = @AUTOCONF@ ++AUTOHEADER = @AUTOHEADER@ ++AUTOMAKE = @AUTOMAKE@ ++AVAILABLE_LANGUAGES = @AVAILABLE_LANGUAGES@ ++AVAILABLE_LANGUAGES_ERRORS = @AVAILABLE_LANGUAGES_ERRORS@ ++AWK = @AWK@ ++CC = @CC@ ++CCAS = @CCAS@ ++CCASFLAGS = @CCASFLAGS@ ++CCDEPMODE = @CCDEPMODE@ ++CC_VERSION = @CC_VERSION@ ++CFLAGS = @CFLAGS@ ++CHARSETS_NEED_SOURCE = @CHARSETS_NEED_SOURCE@ ++CHARSET_OBJS = @CHARSET_OBJS@ ++CHARSET_SRCS = @CHARSET_SRCS@ ++CHECK_PID = @CHECK_PID@ ++CHMOD = @CHMOD@ ++CLIENT_EXTRA_LDFLAGS = @CLIENT_EXTRA_LDFLAGS@ ++CLIENT_LIBS = @CLIENT_LIBS@ ++CMP = @CMP@ ++COMPILATION_COMMENT = @COMPILATION_COMMENT@ ++COMPILE_PSTACK_FALSE = @COMPILE_PSTACK_FALSE@ ++COMPILE_PSTACK_TRUE = @COMPILE_PSTACK_TRUE@ ++CONF_COMMAND = @CONF_COMMAND@ ++CP = @CP@ ++CPP = @CPP@ ++CPPFLAGS = @CPPFLAGS@ ++CXX = @CXX@ ++CXXCPP = @CXXCPP@ ++CXXDEPMODE = @CXXDEPMODE@ ++CXXFLAGS = @CXXFLAGS@ ++CXXLDFLAGS = @CXXLDFLAGS@ ++CXX_VERSION = @CXX_VERSION@ ++CYGPATH_W = @CYGPATH_W@ ++DEFS = @DEFS@ ++DEPDIR = @DEPDIR@ ++DOT_FRM_VERSION = @DOT_FRM_VERSION@ ++DVIS = @DVIS@ ++ECHO = @ECHO@ ++ECHO_C = @ECHO_C@ ++ECHO_N = @ECHO_N@ ++ECHO_T = @ECHO_T@ ++EGREP = @EGREP@ ++EXEEXT = @EXEEXT@ ++F77 = @F77@ ++FFLAGS = @FFLAGS@ ++FIND_PROC = @FIND_PROC@ ++GETCONF = @GETCONF@ ++GXX = @GXX@ ++HAVE_NETWARE_FALSE = @HAVE_NETWARE_FALSE@ ++HAVE_NETWARE_TRUE = @HAVE_NETWARE_TRUE@ ++HOSTNAME = @HOSTNAME@ ++INSTALL_DATA = @INSTALL_DATA@ ++INSTALL_PROGRAM = @INSTALL_PROGRAM@ ++INSTALL_SCRIPT = @INSTALL_SCRIPT@ ++INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ ++IS_LINUX = @IS_LINUX@ ++KILL = @KILL@ ++LD = @LD@ ++LDFLAGS = @LDFLAGS@ ++LIBDL = @LIBDL@ ++LIBOBJS = @LIBOBJS@ ++LIBS = @LIBS@ ++LIBTOOL = @LIBTOOL@ ++LIB_EXTRA_CCFLAGS = @LIB_EXTRA_CCFLAGS@ ++LM_CFLAGS = @LM_CFLAGS@ ++LN = @LN@ ++LN_CP_F = @LN_CP_F@ ++LN_S = @LN_S@ ++LOCAL_FALSE = @LOCAL_FALSE@ ++LOCAL_TRUE = @LOCAL_TRUE@ ++LTLIBOBJS = @LTLIBOBJS@ ++MACHINE_TYPE = @MACHINE_TYPE@ ++MAINT = @MAINT@ ++MAINTAINER_MODE_FALSE = @MAINTAINER_MODE_FALSE@ ++MAINTAINER_MODE_TRUE = @MAINTAINER_MODE_TRUE@ ++MAKEINFO = @MAKEINFO@ ++MAKE_BINARY_DISTRIBUTION_OPTIONS = @MAKE_BINARY_DISTRIBUTION_OPTIONS@ ++MAKE_SHELL = @MAKE_SHELL@ ++MT_INCLUDES = @MT_INCLUDES@ ++MT_LD_ADD = @MT_LD_ADD@ ++MV = @MV@ ++MYSQLD_DEFAULT_SWITCHES = @MYSQLD_DEFAULT_SWITCHES@ ++MYSQLD_EXTRA_LDFLAGS = @MYSQLD_EXTRA_LDFLAGS@ ++MYSQLD_USER = @MYSQLD_USER@ ++MYSQL_BASE_VERSION = @MYSQL_BASE_VERSION@ ++MYSQL_NO_DASH_VERSION = @MYSQL_NO_DASH_VERSION@ ++MYSQL_SERVER_SUFFIX = @MYSQL_SERVER_SUFFIX@ ++MYSQL_TCP_PORT = @MYSQL_TCP_PORT@ ++MYSQL_TCP_PORT_DEFAULT = @MYSQL_TCP_PORT_DEFAULT@ ++MYSQL_UNIX_ADDR = @MYSQL_UNIX_ADDR@ ++MYSQL_VERSION_ID = @MYSQL_VERSION_ID@ ++NOINST_LDFLAGS = @NOINST_LDFLAGS@ ++OBJEXT = @OBJEXT@ ++PACKAGE = @PACKAGE@ ++PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ ++PACKAGE_NAME = @PACKAGE_NAME@ ++PACKAGE_STRING = @PACKAGE_STRING@ ++PACKAGE_TARNAME = @PACKAGE_TARNAME@ ++PACKAGE_VERSION = @PACKAGE_VERSION@ ++PATH_SEPARATOR = @PATH_SEPARATOR@ ++PDFMANUAL = @PDFMANUAL@ ++PERL = @PERL@ ++PERL5 = @PERL5@ ++PROTOCOL_VERSION = @PROTOCOL_VERSION@ ++PS = @PS@ ++RANLIB = @RANLIB@ ++RM = @RM@ ++SAVE_ASFLAGS = @SAVE_ASFLAGS@ ++SAVE_CFLAGS = @SAVE_CFLAGS@ ++SAVE_CXXFLAGS = @SAVE_CXXFLAGS@ ++SAVE_CXXLDFLAGS = @SAVE_CXXLDFLAGS@ ++SAVE_LDFLAGS = @SAVE_LDFLAGS@ ++SED = @SED@ ++SET_MAKE = @SET_MAKE@ ++SHARED_LIB_VERSION = @SHARED_LIB_VERSION@ ++SHELL = @SHELL@ ++STRIP = @STRIP@ ++SYSTEM_TYPE = @SYSTEM_TYPE@ ++TAR = @TAR@ ++TERMCAP_LIB = @TERMCAP_LIB@ ++THREAD_LOBJECTS = @THREAD_LOBJECTS@ ++THREAD_LPROGRAMS = @THREAD_LPROGRAMS@ ++VERSION = @VERSION@ ++WRAPLIBS = @WRAPLIBS@ ++YACC = @YACC@ ++ac_ct_AR = @ac_ct_AR@ ++ac_ct_CC = @ac_ct_CC@ ++ac_ct_CXX = @ac_ct_CXX@ ++ac_ct_F77 = @ac_ct_F77@ ++ac_ct_GETCONF = @ac_ct_GETCONF@ ++ac_ct_RANLIB = @ac_ct_RANLIB@ ++ac_ct_STRIP = @ac_ct_STRIP@ ++am__fastdepCC_FALSE = @am__fastdepCC_FALSE@ ++am__fastdepCC_TRUE = @am__fastdepCC_TRUE@ ++am__fastdepCXX_FALSE = @am__fastdepCXX_FALSE@ ++am__fastdepCXX_TRUE = @am__fastdepCXX_TRUE@ ++am__include = @am__include@ ++am__leading_dot = @am__leading_dot@ ++am__quote = @am__quote@ ++bdb_includes = @bdb_includes@ ++bdb_libs = @bdb_libs@ ++bdb_libs_with_path = @bdb_libs_with_path@ ++bench_dirs = @bench_dirs@ ++bindir = @bindir@ ++build = @build@ ++build_alias = @build_alias@ ++build_cpu = @build_cpu@ ++build_os = @build_os@ ++build_vendor = @build_vendor@ ++datadir = @datadir@ ++default_charset = @default_charset@ ++docs_dirs = @docs_dirs@ ++exec_prefix = @exec_prefix@ ++host = @host@ ++host_alias = @host_alias@ ++host_cpu = @host_cpu@ ++host_os = @host_os@ ++host_vendor = @host_vendor@ ++includedir = @includedir@ ++infodir = @infodir@ ++innodb_includes = @innodb_includes@ ++innodb_libs = @innodb_libs@ ++innodb_system_libs = @innodb_system_libs@ ++install_sh = @install_sh@ ++isam_libs = @isam_libs@ ++libdir = @libdir@ ++libexecdir = @libexecdir@ ++libmysqld_dirs = @libmysqld_dirs@ ++linked_client_targets = @linked_client_targets@ ++linked_netware_sources = @linked_netware_sources@ ++localstatedir = @localstatedir@ ++man_dirs = @man_dirs@ ++mandir = @mandir@ ++netware_dir = @netware_dir@ ++oldincludedir = @oldincludedir@ ++openssl_includes = @openssl_includes@ ++openssl_libs = @openssl_libs@ ++orbit_idl = @orbit_idl@ ++orbit_includes = @orbit_includes@ ++orbit_libs = @orbit_libs@ ++prefix = @prefix@ ++program_transform_name = @program_transform_name@ ++pstack_dirs = @pstack_dirs@ ++pstack_libs = @pstack_libs@ ++readline_dir = @readline_dir@ ++readline_link = @readline_link@ ++sbindir = @sbindir@ ++server_scripts = @server_scripts@ ++sharedstatedir = @sharedstatedir@ ++sql_client_dirs = @sql_client_dirs@ ++sql_server_dirs = @sql_server_dirs@ ++sysconfdir = @sysconfdir@ ++target = @target@ ++target_alias = @target_alias@ ++target_cpu = @target_cpu@ ++target_os = @target_os@ ++target_vendor = @target_vendor@ ++thread_dirs = @thread_dirs@ ++tools_dirs = @tools_dirs@ ++uname_prog = @uname_prog@ ++vio_dir = @vio_dir@ ++vio_libs = @vio_libs@ ++ ++AUTOMAKE_OPTIONS = foreign ++ ++# These are built from source in the Docs directory ++EXTRA_DIST = INSTALL-SOURCE README COPYING EXCEPTIONS-CLIENT ++SUBDIRS = . include @docs_dirs@ @readline_dir@ \ ++ @thread_dirs@ pstack @sql_client_dirs@ \ ++ @sql_server_dirs@ scripts @man_dirs@ tests \ ++ BUILD netware os2 @libmysqld_dirs@ \ ++ @bench_dirs@ support-files @tools_dirs@ ++ ++ ++# Relink after clean ++linked_sources = linked_client_sources linked_server_sources \ ++ linked_libmysql_sources linked_libmysql_r_sources \ ++ linked_libmysqld_sources linked_libmysqldex_sources \ ++ linked_include_sources @linked_netware_sources@ ++ ++ ++CLEANFILES = $(linked_sources) ++subdir = . ++ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 ++mkinstalldirs = $(SHELL) $(top_srcdir)/mkinstalldirs ++CONFIG_HEADER = config.h ++CONFIG_CLEAN_FILES = bdb/Makefile ++DIST_SOURCES = ++ ++RECURSIVE_TARGETS = info-recursive dvi-recursive pdf-recursive \ ++ ps-recursive install-info-recursive uninstall-info-recursive \ ++ all-recursive install-data-recursive install-exec-recursive \ ++ installdirs-recursive install-recursive uninstall-recursive \ ++ check-recursive installcheck-recursive ++DIST_COMMON = README $(srcdir)/Makefile.in $(srcdir)/configure COPYING \ ++ ChangeLog Makefile.am acconfig.h acinclude.m4 aclocal.m4 \ ++ config.guess config.h.in config.sub configure configure.in \ ++ depcomp install-sh ltconfig ltmain.sh missing mkinstalldirs ++DIST_SUBDIRS = $(SUBDIRS) ++all: config.h ++ $(MAKE) $(AM_MAKEFLAGS) all-recursive ++ ++.SUFFIXES: ++ ++am__CONFIG_DISTCLEAN_FILES = config.status config.cache config.log \ ++ configure.lineno ++$(srcdir)/Makefile.in: @MAINTAINER_MODE_TRUE@ Makefile.am $(top_srcdir)/configure.in $(ACLOCAL_M4) ++ cd $(top_srcdir) && \ ++ $(AUTOMAKE) --foreign Makefile ++Makefile: @MAINTAINER_MODE_TRUE@ $(srcdir)/Makefile.in $(top_builddir)/config.status ++ cd $(top_builddir) && $(SHELL) ./config.status $@ $(am__depfiles_maybe) ++ ++$(top_builddir)/config.status: $(srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) ++ $(SHELL) ./config.status --recheck ++$(srcdir)/configure: @MAINTAINER_MODE_TRUE@ $(srcdir)/configure.in $(ACLOCAL_M4) $(CONFIGURE_DEPENDENCIES) ++ cd $(srcdir) && $(AUTOCONF) ++ ++$(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ configure.in acinclude.m4 ++ cd $(srcdir) && $(ACLOCAL) $(ACLOCAL_AMFLAGS) ++ ++stamp-h1: $(srcdir)/config.h.in $(top_builddir)/config.status ++ @rm -f stamp-h1 ++ cd $(top_builddir) && $(SHELL) ./config.status config.h ++ ++$(srcdir)/config.h.in: @MAINTAINER_MODE_TRUE@ $(top_srcdir)/configure.in $(ACLOCAL_M4) $(top_srcdir)/acconfig.h ++ cd $(top_srcdir) && $(AUTOHEADER) ++ touch $(srcdir)/config.h.in ++ ++distclean-hdr: ++ -rm -f config.h stamp-h1 ++bdb/Makefile: $(top_builddir)/config.status $(top_srcdir)/bdb/Makefile.in ++ cd $(top_builddir) && $(SHELL) ./config.status $@ ++ ++mostlyclean-libtool: ++ -rm -f *.lo ++ ++clean-libtool: ++ -rm -rf .libs _libs ++ ++distclean-libtool: ++ -rm -f libtool ++uninstall-info-am: ++ ++# This directory's subdirectories are mostly independent; you can cd ++# into them and run `make' without going through this Makefile. ++# To change the values of `make' variables: instead of editing Makefiles, ++# (1) if the variable is set in `config.status', edit `config.status' ++# (which will cause the Makefiles to be regenerated when you run `make'); ++# (2) otherwise, pass the desired values on the `make' command line. ++$(RECURSIVE_TARGETS): ++ @set fnord $$MAKEFLAGS; amf=$$2; \ ++ dot_seen=no; \ ++ target=`echo $@ | sed s/-recursive//`; \ ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ echo "Making $$target in $$subdir"; \ ++ if test "$$subdir" = "."; then \ ++ dot_seen=yes; \ ++ local_target="$$target-am"; \ ++ else \ ++ local_target="$$target"; \ ++ fi; \ ++ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$local_target) \ ++ || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \ ++ done; \ ++ if test "$$dot_seen" = "no"; then \ ++ $(MAKE) $(AM_MAKEFLAGS) "$$target-am" || exit 1; \ ++ fi; test -z "$$fail" ++ ++mostlyclean-recursive clean-recursive distclean-recursive \ ++maintainer-clean-recursive: ++ @set fnord $$MAKEFLAGS; amf=$$2; \ ++ dot_seen=no; \ ++ case "$@" in \ ++ distclean-* | maintainer-clean-*) list='$(DIST_SUBDIRS)' ;; \ ++ *) list='$(SUBDIRS)' ;; \ ++ esac; \ ++ rev=''; for subdir in $$list; do \ ++ if test "$$subdir" = "."; then :; else \ ++ rev="$$subdir $$rev"; \ ++ fi; \ ++ done; \ ++ rev="$$rev ."; \ ++ target=`echo $@ | sed s/-recursive//`; \ ++ for subdir in $$rev; do \ ++ echo "Making $$target in $$subdir"; \ ++ if test "$$subdir" = "."; then \ ++ local_target="$$target-am"; \ ++ else \ ++ local_target="$$target"; \ ++ fi; \ ++ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$local_target) \ ++ || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \ ++ done && test -z "$$fail" ++tags-recursive: ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ test "$$subdir" = . || (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) tags); \ ++ done ++ctags-recursive: ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ test "$$subdir" = . || (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) ctags); \ ++ done ++ ++ETAGS = etags ++ETAGSFLAGS = ++ ++CTAGS = ctags ++CTAGSFLAGS = ++ ++ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) ++ list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ ++ unique=`for i in $$list; do \ ++ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ ++ done | \ ++ $(AWK) ' { files[$$0] = 1; } \ ++ END { for (i in files) print i; }'`; \ ++ mkid -fID $$unique ++ ++TAGS: tags-recursive $(HEADERS) $(SOURCES) config.h.in $(TAGS_DEPENDENCIES) \ ++ $(TAGS_FILES) $(LISP) ++ tags=; \ ++ here=`pwd`; \ ++ if (etags --etags-include --version) >/dev/null 2>&1; then \ ++ include_option=--etags-include; \ ++ else \ ++ include_option=--include; \ ++ fi; \ ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ if test "$$subdir" = .; then :; else \ ++ test -f $$subdir/TAGS && \ ++ tags="$$tags $$include_option=$$here/$$subdir/TAGS"; \ ++ fi; \ ++ done; \ ++ list='$(SOURCES) $(HEADERS) config.h.in $(LISP) $(TAGS_FILES)'; \ ++ unique=`for i in $$list; do \ ++ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ ++ done | \ ++ $(AWK) ' { files[$$0] = 1; } \ ++ END { for (i in files) print i; }'`; \ ++ test -z "$(ETAGS_ARGS)$$tags$$unique" \ ++ || $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ ++ $$tags $$unique ++ ++ctags: CTAGS ++CTAGS: ctags-recursive $(HEADERS) $(SOURCES) config.h.in $(TAGS_DEPENDENCIES) \ ++ $(TAGS_FILES) $(LISP) ++ tags=; \ ++ here=`pwd`; \ ++ list='$(SOURCES) $(HEADERS) config.h.in $(LISP) $(TAGS_FILES)'; \ ++ unique=`for i in $$list; do \ ++ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ ++ done | \ ++ $(AWK) ' { files[$$0] = 1; } \ ++ END { for (i in files) print i; }'`; \ ++ test -z "$(CTAGS_ARGS)$$tags$$unique" \ ++ || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ ++ $$tags $$unique ++ ++GTAGS: ++ here=`$(am__cd) $(top_builddir) && pwd` \ ++ && cd $(top_srcdir) \ ++ && gtags -i $(GTAGS_ARGS) $$here ++ ++distclean-tags: ++ -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags ++DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) ++ ++top_distdir = . ++distdir = $(PACKAGE)-$(VERSION) ++ ++am__remove_distdir = \ ++ { test ! -d $(distdir) \ ++ || { find $(distdir) -type d ! -perm -200 -exec chmod u+w {} ';' \ ++ && rm -fr $(distdir); }; } ++ ++GZIP_ENV = --best ++distuninstallcheck_listfiles = find . -type f -print ++distcleancheck_listfiles = find . -type f -print ++ ++distdir: $(DISTFILES) ++ $(am__remove_distdir) ++ mkdir $(distdir) ++ $(mkinstalldirs) $(distdir)/bdb $(distdir)/include ++ @srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; \ ++ topsrcdirstrip=`echo "$(top_srcdir)" | sed 's|.|.|g'`; \ ++ list='$(DISTFILES)'; for file in $$list; do \ ++ case $$file in \ ++ $(srcdir)/*) file=`echo "$$file" | sed "s|^$$srcdirstrip/||"`;; \ ++ $(top_srcdir)/*) file=`echo "$$file" | sed "s|^$$topsrcdirstrip/|$(top_builddir)/|"`;; \ ++ esac; \ ++ if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ ++ dir=`echo "$$file" | sed -e 's,/[^/]*$$,,'`; \ ++ if test "$$dir" != "$$file" && test "$$dir" != "."; then \ ++ dir="/$$dir"; \ ++ $(mkinstalldirs) "$(distdir)$$dir"; \ ++ else \ ++ dir=''; \ ++ fi; \ ++ if test -d $$d/$$file; then \ ++ if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ ++ cp -pR $(srcdir)/$$file $(distdir)$$dir || exit 1; \ ++ fi; \ ++ cp -pR $$d/$$file $(distdir)$$dir || exit 1; \ ++ else \ ++ test -f $(distdir)/$$file \ ++ || cp -p $$d/$$file $(distdir)/$$file \ ++ || exit 1; \ ++ fi; \ ++ done ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ if test "$$subdir" = .; then :; else \ ++ test -d $(distdir)/$$subdir \ ++ || mkdir $(distdir)/$$subdir \ ++ || exit 1; \ ++ (cd $$subdir && \ ++ $(MAKE) $(AM_MAKEFLAGS) \ ++ top_distdir="$(top_distdir)" \ ++ distdir=../$(distdir)/$$subdir \ ++ distdir) \ ++ || exit 1; \ ++ fi; \ ++ done ++ $(MAKE) $(AM_MAKEFLAGS) \ ++ top_distdir="$(top_distdir)" distdir="$(distdir)" \ ++ dist-hook ++ -find $(distdir) -type d ! -perm -777 -exec chmod a+rwx {} \; -o \ ++ ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \ ++ ! -type d ! -perm -400 -exec chmod a+r {} \; -o \ ++ ! -type d ! -perm -444 -exec $(SHELL) $(install_sh) -c -m a+r {} {} \; \ ++ || chmod -R a+r $(distdir) ++dist-gzip: distdir ++ $(AMTAR) chof - $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).tar.gz ++ $(am__remove_distdir) ++ ++dist dist-all: distdir ++ $(AMTAR) chof - $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).tar.gz ++ $(am__remove_distdir) ++ ++# This target untars the dist file and tries a VPATH configuration. Then ++# it guarantees that the distribution is self-contained by making another ++# tarfile. ++distcheck: dist ++ $(am__remove_distdir) ++ GZIP=$(GZIP_ENV) gunzip -c $(distdir).tar.gz | $(AMTAR) xf - ++ chmod -R a-w $(distdir); chmod a+w $(distdir) ++ mkdir $(distdir)/_build ++ mkdir $(distdir)/_inst ++ chmod a-w $(distdir) ++ dc_install_base=`$(am__cd) $(distdir)/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \ ++ && dc_destdir="$${TMPDIR-/tmp}/am-dc-$$$$/" \ ++ && cd $(distdir)/_build \ ++ && ../configure --srcdir=.. --prefix="$$dc_install_base" \ ++ $(DISTCHECK_CONFIGURE_FLAGS) \ ++ && $(MAKE) $(AM_MAKEFLAGS) \ ++ && $(MAKE) $(AM_MAKEFLAGS) dvi \ ++ && $(MAKE) $(AM_MAKEFLAGS) check \ ++ && $(MAKE) $(AM_MAKEFLAGS) install \ ++ && $(MAKE) $(AM_MAKEFLAGS) installcheck \ ++ && $(MAKE) $(AM_MAKEFLAGS) uninstall \ ++ && $(MAKE) $(AM_MAKEFLAGS) distuninstallcheck_dir="$$dc_install_base" \ ++ distuninstallcheck \ ++ && chmod -R a-w "$$dc_install_base" \ ++ && ({ \ ++ (cd ../.. && $(mkinstalldirs) "$$dc_destdir") \ ++ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" install \ ++ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" uninstall \ ++ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" \ ++ distuninstallcheck_dir="$$dc_destdir" distuninstallcheck; \ ++ } || { rm -rf "$$dc_destdir"; exit 1; }) \ ++ && rm -rf "$$dc_destdir" \ ++ && $(MAKE) $(AM_MAKEFLAGS) dist-gzip \ ++ && rm -f $(distdir).tar.gz \ ++ && $(MAKE) $(AM_MAKEFLAGS) distcleancheck ++ $(am__remove_distdir) ++ @echo "$(distdir).tar.gz is ready for distribution" | \ ++ sed 'h;s/./=/g;p;x;p;x' ++distuninstallcheck: ++ @cd $(distuninstallcheck_dir) \ ++ && test `$(distuninstallcheck_listfiles) | wc -l` -le 1 \ ++ || { echo "ERROR: files left after uninstall:" ; \ ++ if test -n "$(DESTDIR)"; then \ ++ echo " (check DESTDIR support)"; \ ++ fi ; \ ++ $(distuninstallcheck_listfiles) ; \ ++ exit 1; } >&2 ++distcleancheck: distclean ++ @if test '$(srcdir)' = . ; then \ ++ echo "ERROR: distcleancheck can only run from a VPATH build" ; \ ++ exit 1 ; \ ++ fi ++ @test `$(distcleancheck_listfiles) | wc -l` -eq 0 \ ++ || { echo "ERROR: files left in build directory after distclean:" ; \ ++ $(distcleancheck_listfiles) ; \ ++ exit 1; } >&2 ++check-am: all-am ++check: check-recursive ++all-am: Makefile config.h ++installdirs: installdirs-recursive ++installdirs-am: ++ ++install: install-recursive ++install-exec: install-exec-recursive ++install-data: install-data-recursive ++uninstall: uninstall-recursive ++ ++install-am: all-am ++ @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am ++ ++installcheck: installcheck-recursive ++install-strip: ++ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ ++ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ ++ `test -z '$(STRIP)' || \ ++ echo "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'"` install ++mostlyclean-generic: ++ ++clean-generic: ++ -test -z "$(CLEANFILES)" || rm -f $(CLEANFILES) ++ ++distclean-generic: ++ -rm -f $(CONFIG_CLEAN_FILES) ++ ++maintainer-clean-generic: ++ @echo "This command is intended for maintainers to use" ++ @echo "it deletes files that may require special tools to rebuild." ++clean: clean-recursive ++ ++clean-am: clean-generic clean-libtool mostlyclean-am ++ ++distclean: distclean-recursive ++ -rm -f $(am__CONFIG_DISTCLEAN_FILES) ++ -rm -f Makefile ++distclean-am: clean-am distclean-generic distclean-hdr distclean-libtool \ ++ distclean-tags ++ ++dvi: dvi-recursive ++ ++dvi-am: ++ ++info: info-recursive ++ ++info-am: ++ ++install-data-am: ++ ++install-exec-am: ++ ++install-info: install-info-recursive ++ ++install-man: ++ ++installcheck-am: ++ ++maintainer-clean: maintainer-clean-recursive ++ -rm -f $(am__CONFIG_DISTCLEAN_FILES) ++ -rm -rf $(top_srcdir)/autom4te.cache ++ -rm -f Makefile ++maintainer-clean-am: distclean-am maintainer-clean-generic ++ ++mostlyclean: mostlyclean-recursive ++ ++mostlyclean-am: mostlyclean-generic mostlyclean-libtool ++ ++pdf: pdf-recursive ++ ++pdf-am: ++ ++ps: ps-recursive ++ ++ps-am: ++ ++uninstall-am: uninstall-info-am ++ ++uninstall-info: uninstall-info-recursive ++ ++.PHONY: $(RECURSIVE_TARGETS) CTAGS GTAGS all all-am check check-am clean \ ++ clean-generic clean-libtool clean-recursive ctags \ ++ ctags-recursive dist dist-all dist-gzip distcheck distclean \ ++ distclean-generic distclean-hdr distclean-libtool \ ++ distclean-recursive distclean-tags distcleancheck distdir \ ++ distuninstallcheck dvi dvi-am dvi-recursive info info-am \ ++ info-recursive install install-am install-data install-data-am \ ++ install-data-recursive install-exec install-exec-am \ ++ install-exec-recursive install-info install-info-am \ ++ install-info-recursive install-man install-recursive \ ++ install-strip installcheck installcheck-am installdirs \ ++ installdirs-am installdirs-recursive maintainer-clean \ ++ maintainer-clean-generic maintainer-clean-recursive mostlyclean \ ++ mostlyclean-generic mostlyclean-libtool mostlyclean-recursive \ ++ pdf pdf-am pdf-recursive ps ps-am ps-recursive tags \ ++ tags-recursive uninstall uninstall-am uninstall-info-am \ ++ uninstall-info-recursive uninstall-recursive ++ ++ ++# This is just so that the linking is done early. ++config.h: $(linked_sources) ++ ++linked_include_sources: ++ cd include; $(MAKE) link_sources ++ echo timestamp > linked_include_sources ++ ++linked_client_sources: @linked_client_targets@ ++ cd client; $(MAKE) link_sources ++ echo timestamp > linked_client_sources ++ ++linked_libmysql_sources: ++ cd libmysql; $(MAKE) link_sources ++ echo timestamp > linked_libmysql_sources ++ ++linked_libmysql_r_sources: linked_libmysql_sources ++ cd libmysql_r; $(MAKE) link_sources ++ echo timestamp > linked_libmysql_r_sources ++ ++linked_libmysqld_sources: ++ cd libmysqld; $(MAKE) link_sources ++ echo timestamp > linked_libmysqld_sources ++ ++linked_libmysqldex_sources: ++ cd libmysqld/examples; $(MAKE) link_sources ++ echo timestamp > linked_libmysqldex_sources ++ ++linked_netware_sources: ++ cd @netware_dir@; $(MAKE) link_sources ++ echo timestamp > linked_netware_sources ++ ++#avoid recursive make calls in sql directory ++linked_server_sources: ++ cd sql; rm -f mini_client_errors.c;@LN_CP_F@ ../libmysql/errmsg.c mini_client_errors.c ++ echo timestamp > linked_server_sources ++ ++# Create permission databases ++init-db: all ++ $(top_builddir)/scripts/mysql_install_db ++ ++bin-dist: all ++ $(top_builddir)/scripts/make_binary_distribution @MAKE_BINARY_DISTRIBUTION_OPTIONS@ ++ ++# Remove BK's "SCCS" subdirectories from source distribution ++dist-hook: ++ rm -rf `find $(distdir) -type d -name SCCS` ++ ++tags: ++ support-files/build-tags ++.PHONY: init-db bin-dist ++ ++# Test installation ++ ++test: ++ cd mysql-test ; ./mysql-test-run ++# Tell versions [3.59,3.63) of GNU make to not export all variables. ++# Otherwise a system limit (for SysV at least) may be exceeded. ++.NOEXPORT: === added file 'storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Makefile.in.dpatch' --- a/storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Makefile.in.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/01_MAKEFILES__Docs_Makefile.in.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,776 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run +## 01_MAKEFILES__Docs_Makefile.in.dpatch by <ch(a)debian.org> +## +## All lines beginning with `## DP:' are a description of the patch. +## DP: Creates Docs/Makefile.in + +@DPATCH@ + +--- old/Docs/Makefile.in 2005-03-01 02:08:01.877429040 +0100 ++++ new/Docs/Makefile.in 2005-02-28 21:21:24.000000000 +0100 +@@ -0,0 +1,765 @@ ++# Makefile.in generated by automake 1.7.9 from Makefile.am. ++# @configure_input@ ++ ++# Copyright 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003 ++# Free Software Foundation, Inc. ++# This Makefile.in is free software; the Free Software Foundation ++# gives unlimited permission to copy and/or distribute it, ++# with or without modifications, as long as this notice is preserved. ++ ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY, to the extent permitted by law; without ++# even the implied warranty of MERCHANTABILITY or FITNESS FOR A ++# PARTICULAR PURPOSE. ++ ++@SET_MAKE@ ++ ++# Copyright (C) 2000 MySQL AB & MySQL Finland AB & TCX DataKonsult AB ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program; if not, write to the Free Software ++# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ++ ++# Process this file with automake to create Makefile.in ++ ++srcdir = @srcdir@ ++top_srcdir = @top_srcdir@ ++VPATH = @srcdir@ ++pkgdatadir = $(datadir)/@PACKAGE@ ++pkglibdir = $(libdir)/@PACKAGE@ ++pkgincludedir = $(includedir)/@PACKAGE@ ++top_builddir = . ++ ++am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd ++INSTALL = @INSTALL@ ++install_sh_DATA = $(install_sh) -c -m 644 ++install_sh_PROGRAM = $(install_sh) -c ++install_sh_SCRIPT = $(install_sh) -c ++INSTALL_HEADER = $(INSTALL_DATA) ++transform = $(program_transform_name) ++NORMAL_INSTALL = : ++PRE_INSTALL = : ++POST_INSTALL = : ++NORMAL_UNINSTALL = : ++PRE_UNINSTALL = : ++POST_UNINSTALL = : ++build_triplet = @build@ ++host_triplet = @host@ ++target_triplet = @target@ ++ACLOCAL = @ACLOCAL@ ++ALLOCA = @ALLOCA@ ++AMDEP_FALSE = @AMDEP_FALSE@ ++AMDEP_TRUE = @AMDEP_TRUE@ ++AMTAR = @AMTAR@ ++AR = @AR@ ++AS = @AS@ ++ASSEMBLER_FALSE = @ASSEMBLER_FALSE@ ++ASSEMBLER_TRUE = @ASSEMBLER_TRUE@ ++ASSEMBLER_sparc32_FALSE = @ASSEMBLER_sparc32_FALSE@ ++ASSEMBLER_sparc32_TRUE = @ASSEMBLER_sparc32_TRUE@ ++ASSEMBLER_sparc64_FALSE = @ASSEMBLER_sparc64_FALSE@ ++ASSEMBLER_sparc64_TRUE = @ASSEMBLER_sparc64_TRUE@ ++ASSEMBLER_x86_FALSE = @ASSEMBLER_x86_FALSE@ ++ASSEMBLER_x86_TRUE = @ASSEMBLER_x86_TRUE@ ++AUTOCONF = @AUTOCONF@ ++AUTOHEADER = @AUTOHEADER@ ++AUTOMAKE = @AUTOMAKE@ ++AVAILABLE_LANGUAGES = @AVAILABLE_LANGUAGES@ ++AVAILABLE_LANGUAGES_ERRORS = @AVAILABLE_LANGUAGES_ERRORS@ ++AWK = @AWK@ ++CC = @CC@ ++CCAS = @CCAS@ ++CCASFLAGS = @CCASFLAGS@ ++CCDEPMODE = @CCDEPMODE@ ++CC_VERSION = @CC_VERSION@ ++CFLAGS = @CFLAGS@ ++CHARSETS_NEED_SOURCE = @CHARSETS_NEED_SOURCE@ ++CHARSET_OBJS = @CHARSET_OBJS@ ++CHARSET_SRCS = @CHARSET_SRCS@ ++CHECK_PID = @CHECK_PID@ ++CHMOD = @CHMOD@ ++CLIENT_EXTRA_LDFLAGS = @CLIENT_EXTRA_LDFLAGS@ ++CLIENT_LIBS = @CLIENT_LIBS@ ++CMP = @CMP@ ++COMPILATION_COMMENT = @COMPILATION_COMMENT@ ++COMPILE_PSTACK_FALSE = @COMPILE_PSTACK_FALSE@ ++COMPILE_PSTACK_TRUE = @COMPILE_PSTACK_TRUE@ ++CONF_COMMAND = @CONF_COMMAND@ ++CP = @CP@ ++CPP = @CPP@ ++CPPFLAGS = @CPPFLAGS@ ++CXX = @CXX@ ++CXXCPP = @CXXCPP@ ++CXXDEPMODE = @CXXDEPMODE@ ++CXXFLAGS = @CXXFLAGS@ ++CXXLDFLAGS = @CXXLDFLAGS@ ++CXX_VERSION = @CXX_VERSION@ ++CYGPATH_W = @CYGPATH_W@ ++DEFS = @DEFS@ ++DEPDIR = @DEPDIR@ ++DOT_FRM_VERSION = @DOT_FRM_VERSION@ ++DVIS = @DVIS@ ++ECHO = @ECHO@ ++ECHO_C = @ECHO_C@ ++ECHO_N = @ECHO_N@ ++ECHO_T = @ECHO_T@ ++EGREP = @EGREP@ ++EXEEXT = @EXEEXT@ ++F77 = @F77@ ++FFLAGS = @FFLAGS@ ++FIND_PROC = @FIND_PROC@ ++GETCONF = @GETCONF@ ++GXX = @GXX@ ++HAVE_NETWARE_FALSE = @HAVE_NETWARE_FALSE@ ++HAVE_NETWARE_TRUE = @HAVE_NETWARE_TRUE@ ++HOSTNAME = @HOSTNAME@ ++INSTALL_DATA = @INSTALL_DATA@ ++INSTALL_PROGRAM = @INSTALL_PROGRAM@ ++INSTALL_SCRIPT = @INSTALL_SCRIPT@ ++INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ ++IS_LINUX = @IS_LINUX@ ++KILL = @KILL@ ++LD = @LD@ ++LDFLAGS = @LDFLAGS@ ++LIBDL = @LIBDL@ ++LIBOBJS = @LIBOBJS@ ++LIBS = @LIBS@ ++LIBTOOL = @LIBTOOL@ ++LIB_EXTRA_CCFLAGS = @LIB_EXTRA_CCFLAGS@ ++LM_CFLAGS = @LM_CFLAGS@ ++LN = @LN@ ++LN_CP_F = @LN_CP_F@ ++LN_S = @LN_S@ ++LOCAL_FALSE = @LOCAL_FALSE@ ++LOCAL_TRUE = @LOCAL_TRUE@ ++LTLIBOBJS = @LTLIBOBJS@ ++MACHINE_TYPE = @MACHINE_TYPE@ ++MAINT = @MAINT@ ++MAINTAINER_MODE_FALSE = @MAINTAINER_MODE_FALSE@ ++MAINTAINER_MODE_TRUE = @MAINTAINER_MODE_TRUE@ ++MAKEINFO = @MAKEINFO@ ++MAKE_BINARY_DISTRIBUTION_OPTIONS = @MAKE_BINARY_DISTRIBUTION_OPTIONS@ ++MAKE_SHELL = @MAKE_SHELL@ ++MT_INCLUDES = @MT_INCLUDES@ ++MT_LD_ADD = @MT_LD_ADD@ ++MV = @MV@ ++MYSQLD_DEFAULT_SWITCHES = @MYSQLD_DEFAULT_SWITCHES@ ++MYSQLD_EXTRA_LDFLAGS = @MYSQLD_EXTRA_LDFLAGS@ ++MYSQLD_USER = @MYSQLD_USER@ ++MYSQL_BASE_VERSION = @MYSQL_BASE_VERSION@ ++MYSQL_NO_DASH_VERSION = @MYSQL_NO_DASH_VERSION@ ++MYSQL_SERVER_SUFFIX = @MYSQL_SERVER_SUFFIX@ ++MYSQL_TCP_PORT = @MYSQL_TCP_PORT@ ++MYSQL_TCP_PORT_DEFAULT = @MYSQL_TCP_PORT_DEFAULT@ ++MYSQL_UNIX_ADDR = @MYSQL_UNIX_ADDR@ ++MYSQL_VERSION_ID = @MYSQL_VERSION_ID@ ++NOINST_LDFLAGS = @NOINST_LDFLAGS@ ++OBJEXT = @OBJEXT@ ++PACKAGE = @PACKAGE@ ++PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ ++PACKAGE_NAME = @PACKAGE_NAME@ ++PACKAGE_STRING = @PACKAGE_STRING@ ++PACKAGE_TARNAME = @PACKAGE_TARNAME@ ++PACKAGE_VERSION = @PACKAGE_VERSION@ ++PATH_SEPARATOR = @PATH_SEPARATOR@ ++PDFMANUAL = @PDFMANUAL@ ++PERL = @PERL@ ++PERL5 = @PERL5@ ++PROTOCOL_VERSION = @PROTOCOL_VERSION@ ++PS = @PS@ ++RANLIB = @RANLIB@ ++RM = @RM@ ++SAVE_ASFLAGS = @SAVE_ASFLAGS@ ++SAVE_CFLAGS = @SAVE_CFLAGS@ ++SAVE_CXXFLAGS = @SAVE_CXXFLAGS@ ++SAVE_CXXLDFLAGS = @SAVE_CXXLDFLAGS@ ++SAVE_LDFLAGS = @SAVE_LDFLAGS@ ++SED = @SED@ ++SET_MAKE = @SET_MAKE@ ++SHARED_LIB_VERSION = @SHARED_LIB_VERSION@ ++SHELL = @SHELL@ ++STRIP = @STRIP@ ++SYSTEM_TYPE = @SYSTEM_TYPE@ ++TAR = @TAR@ ++TERMCAP_LIB = @TERMCAP_LIB@ ++THREAD_LOBJECTS = @THREAD_LOBJECTS@ ++THREAD_LPROGRAMS = @THREAD_LPROGRAMS@ ++VERSION = @VERSION@ ++WRAPLIBS = @WRAPLIBS@ ++YACC = @YACC@ ++ac_ct_AR = @ac_ct_AR@ ++ac_ct_CC = @ac_ct_CC@ ++ac_ct_CXX = @ac_ct_CXX@ ++ac_ct_F77 = @ac_ct_F77@ ++ac_ct_GETCONF = @ac_ct_GETCONF@ ++ac_ct_RANLIB = @ac_ct_RANLIB@ ++ac_ct_STRIP = @ac_ct_STRIP@ ++am__fastdepCC_FALSE = @am__fastdepCC_FALSE@ ++am__fastdepCC_TRUE = @am__fastdepCC_TRUE@ ++am__fastdepCXX_FALSE = @am__fastdepCXX_FALSE@ ++am__fastdepCXX_TRUE = @am__fastdepCXX_TRUE@ ++am__include = @am__include@ ++am__leading_dot = @am__leading_dot@ ++am__quote = @am__quote@ ++bdb_includes = @bdb_includes@ ++bdb_libs = @bdb_libs@ ++bdb_libs_with_path = @bdb_libs_with_path@ ++bench_dirs = @bench_dirs@ ++bindir = @bindir@ ++build = @build@ ++build_alias = @build_alias@ ++build_cpu = @build_cpu@ ++build_os = @build_os@ ++build_vendor = @build_vendor@ ++datadir = @datadir@ ++default_charset = @default_charset@ ++docs_dirs = @docs_dirs@ ++exec_prefix = @exec_prefix@ ++host = @host@ ++host_alias = @host_alias@ ++host_cpu = @host_cpu@ ++host_os = @host_os@ ++host_vendor = @host_vendor@ ++includedir = @includedir@ ++infodir = @infodir@ ++innodb_includes = @innodb_includes@ ++innodb_libs = @innodb_libs@ ++innodb_system_libs = @innodb_system_libs@ ++install_sh = @install_sh@ ++isam_libs = @isam_libs@ ++libdir = @libdir@ ++libexecdir = @libexecdir@ ++libmysqld_dirs = @libmysqld_dirs@ ++linked_client_targets = @linked_client_targets@ ++linked_netware_sources = @linked_netware_sources@ ++localstatedir = @localstatedir@ ++man_dirs = @man_dirs@ ++mandir = @mandir@ ++netware_dir = @netware_dir@ ++oldincludedir = @oldincludedir@ ++openssl_includes = @openssl_includes@ ++openssl_libs = @openssl_libs@ ++orbit_idl = @orbit_idl@ ++orbit_includes = @orbit_includes@ ++orbit_libs = @orbit_libs@ ++prefix = @prefix@ ++program_transform_name = @program_transform_name@ ++pstack_dirs = @pstack_dirs@ ++pstack_libs = @pstack_libs@ ++readline_dir = @readline_dir@ ++readline_link = @readline_link@ ++sbindir = @sbindir@ ++server_scripts = @server_scripts@ ++sharedstatedir = @sharedstatedir@ ++sql_client_dirs = @sql_client_dirs@ ++sql_server_dirs = @sql_server_dirs@ ++sysconfdir = @sysconfdir@ ++target = @target@ ++target_alias = @target_alias@ ++target_cpu = @target_cpu@ ++target_os = @target_os@ ++target_vendor = @target_vendor@ ++thread_dirs = @thread_dirs@ ++tools_dirs = @tools_dirs@ ++uname_prog = @uname_prog@ ++vio_dir = @vio_dir@ ++vio_libs = @vio_libs@ ++ ++AUTOMAKE_OPTIONS = foreign ++ ++# These are built from source in the Docs directory ++EXTRA_DIST = INSTALL-SOURCE README COPYING EXCEPTIONS-CLIENT ++SUBDIRS = . include @docs_dirs@ @readline_dir@ \ ++ @thread_dirs@ pstack @sql_client_dirs@ \ ++ @sql_server_dirs@ scripts @man_dirs@ tests \ ++ BUILD netware os2 @libmysqld_dirs@ \ ++ @bench_dirs@ support-files @tools_dirs@ ++ ++ ++# Relink after clean ++linked_sources = linked_client_sources linked_server_sources \ ++ linked_libmysql_sources linked_libmysql_r_sources \ ++ linked_libmysqld_sources linked_libmysqldex_sources \ ++ linked_include_sources @linked_netware_sources@ ++ ++ ++CLEANFILES = $(linked_sources) ++subdir = . ++ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 ++mkinstalldirs = $(SHELL) $(top_srcdir)/mkinstalldirs ++CONFIG_HEADER = config.h ++CONFIG_CLEAN_FILES = bdb/Makefile ++DIST_SOURCES = ++ ++RECURSIVE_TARGETS = info-recursive dvi-recursive pdf-recursive \ ++ ps-recursive install-info-recursive uninstall-info-recursive \ ++ all-recursive install-data-recursive install-exec-recursive \ ++ installdirs-recursive install-recursive uninstall-recursive \ ++ check-recursive installcheck-recursive ++DIST_COMMON = README $(srcdir)/Makefile.in $(srcdir)/configure COPYING \ ++ ChangeLog Makefile.am acconfig.h acinclude.m4 aclocal.m4 \ ++ config.guess config.h.in config.sub configure configure.in \ ++ depcomp install-sh ltconfig ltmain.sh missing mkinstalldirs ++DIST_SUBDIRS = $(SUBDIRS) ++all: config.h ++ $(MAKE) $(AM_MAKEFLAGS) all-recursive ++ ++.SUFFIXES: ++ ++am__CONFIG_DISTCLEAN_FILES = config.status config.cache config.log \ ++ configure.lineno ++$(srcdir)/Makefile.in: @MAINTAINER_MODE_TRUE@ Makefile.am $(top_srcdir)/configure.in $(ACLOCAL_M4) ++ cd $(top_srcdir) && \ ++ $(AUTOMAKE) --foreign Makefile ++Makefile: @MAINTAINER_MODE_TRUE@ $(srcdir)/Makefile.in $(top_builddir)/config.status ++ cd $(top_builddir) && $(SHELL) ./config.status $@ $(am__depfiles_maybe) ++ ++$(top_builddir)/config.status: $(srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) ++ $(SHELL) ./config.status --recheck ++$(srcdir)/configure: @MAINTAINER_MODE_TRUE@ $(srcdir)/configure.in $(ACLOCAL_M4) $(CONFIGURE_DEPENDENCIES) ++ cd $(srcdir) && $(AUTOCONF) ++ ++$(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ configure.in acinclude.m4 ++ cd $(srcdir) && $(ACLOCAL) $(ACLOCAL_AMFLAGS) ++ ++stamp-h1: $(srcdir)/config.h.in $(top_builddir)/config.status ++ @rm -f stamp-h1 ++ cd $(top_builddir) && $(SHELL) ./config.status config.h ++ ++$(srcdir)/config.h.in: @MAINTAINER_MODE_TRUE@ $(top_srcdir)/configure.in $(ACLOCAL_M4) $(top_srcdir)/acconfig.h ++ cd $(top_srcdir) && $(AUTOHEADER) ++ touch $(srcdir)/config.h.in ++ ++distclean-hdr: ++ -rm -f config.h stamp-h1 ++bdb/Makefile: $(top_builddir)/config.status $(top_srcdir)/bdb/Makefile.in ++ cd $(top_builddir) && $(SHELL) ./config.status $@ ++ ++mostlyclean-libtool: ++ -rm -f *.lo ++ ++clean-libtool: ++ -rm -rf .libs _libs ++ ++distclean-libtool: ++ -rm -f libtool ++uninstall-info-am: ++ ++# This directory's subdirectories are mostly independent; you can cd ++# into them and run `make' without going through this Makefile. ++# To change the values of `make' variables: instead of editing Makefiles, ++# (1) if the variable is set in `config.status', edit `config.status' ++# (which will cause the Makefiles to be regenerated when you run `make'); ++# (2) otherwise, pass the desired values on the `make' command line. ++$(RECURSIVE_TARGETS): ++ @set fnord $$MAKEFLAGS; amf=$$2; \ ++ dot_seen=no; \ ++ target=`echo $@ | sed s/-recursive//`; \ ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ echo "Making $$target in $$subdir"; \ ++ if test "$$subdir" = "."; then \ ++ dot_seen=yes; \ ++ local_target="$$target-am"; \ ++ else \ ++ local_target="$$target"; \ ++ fi; \ ++ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$local_target) \ ++ || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \ ++ done; \ ++ if test "$$dot_seen" = "no"; then \ ++ $(MAKE) $(AM_MAKEFLAGS) "$$target-am" || exit 1; \ ++ fi; test -z "$$fail" ++ ++mostlyclean-recursive clean-recursive distclean-recursive \ ++maintainer-clean-recursive: ++ @set fnord $$MAKEFLAGS; amf=$$2; \ ++ dot_seen=no; \ ++ case "$@" in \ ++ distclean-* | maintainer-clean-*) list='$(DIST_SUBDIRS)' ;; \ ++ *) list='$(SUBDIRS)' ;; \ ++ esac; \ ++ rev=''; for subdir in $$list; do \ ++ if test "$$subdir" = "."; then :; else \ ++ rev="$$subdir $$rev"; \ ++ fi; \ ++ done; \ ++ rev="$$rev ."; \ ++ target=`echo $@ | sed s/-recursive//`; \ ++ for subdir in $$rev; do \ ++ echo "Making $$target in $$subdir"; \ ++ if test "$$subdir" = "."; then \ ++ local_target="$$target-am"; \ ++ else \ ++ local_target="$$target"; \ ++ fi; \ ++ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$local_target) \ ++ || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \ ++ done && test -z "$$fail" ++tags-recursive: ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ test "$$subdir" = . || (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) tags); \ ++ done ++ctags-recursive: ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ test "$$subdir" = . || (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) ctags); \ ++ done ++ ++ETAGS = etags ++ETAGSFLAGS = ++ ++CTAGS = ctags ++CTAGSFLAGS = ++ ++ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) ++ list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ ++ unique=`for i in $$list; do \ ++ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ ++ done | \ ++ $(AWK) ' { files[$$0] = 1; } \ ++ END { for (i in files) print i; }'`; \ ++ mkid -fID $$unique ++ ++TAGS: tags-recursive $(HEADERS) $(SOURCES) config.h.in $(TAGS_DEPENDENCIES) \ ++ $(TAGS_FILES) $(LISP) ++ tags=; \ ++ here=`pwd`; \ ++ if (etags --etags-include --version) >/dev/null 2>&1; then \ ++ include_option=--etags-include; \ ++ else \ ++ include_option=--include; \ ++ fi; \ ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ if test "$$subdir" = .; then :; else \ ++ test -f $$subdir/TAGS && \ ++ tags="$$tags $$include_option=$$here/$$subdir/TAGS"; \ ++ fi; \ ++ done; \ ++ list='$(SOURCES) $(HEADERS) config.h.in $(LISP) $(TAGS_FILES)'; \ ++ unique=`for i in $$list; do \ ++ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ ++ done | \ ++ $(AWK) ' { files[$$0] = 1; } \ ++ END { for (i in files) print i; }'`; \ ++ test -z "$(ETAGS_ARGS)$$tags$$unique" \ ++ || $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ ++ $$tags $$unique ++ ++ctags: CTAGS ++CTAGS: ctags-recursive $(HEADERS) $(SOURCES) config.h.in $(TAGS_DEPENDENCIES) \ ++ $(TAGS_FILES) $(LISP) ++ tags=; \ ++ here=`pwd`; \ ++ list='$(SOURCES) $(HEADERS) config.h.in $(LISP) $(TAGS_FILES)'; \ ++ unique=`for i in $$list; do \ ++ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ ++ done | \ ++ $(AWK) ' { files[$$0] = 1; } \ ++ END { for (i in files) print i; }'`; \ ++ test -z "$(CTAGS_ARGS)$$tags$$unique" \ ++ || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ ++ $$tags $$unique ++ ++GTAGS: ++ here=`$(am__cd) $(top_builddir) && pwd` \ ++ && cd $(top_srcdir) \ ++ && gtags -i $(GTAGS_ARGS) $$here ++ ++distclean-tags: ++ -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags ++DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) ++ ++top_distdir = . ++distdir = $(PACKAGE)-$(VERSION) ++ ++am__remove_distdir = \ ++ { test ! -d $(distdir) \ ++ || { find $(distdir) -type d ! -perm -200 -exec chmod u+w {} ';' \ ++ && rm -fr $(distdir); }; } ++ ++GZIP_ENV = --best ++distuninstallcheck_listfiles = find . -type f -print ++distcleancheck_listfiles = find . -type f -print ++ ++distdir: $(DISTFILES) ++ $(am__remove_distdir) ++ mkdir $(distdir) ++ $(mkinstalldirs) $(distdir)/bdb $(distdir)/include ++ @srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; \ ++ topsrcdirstrip=`echo "$(top_srcdir)" | sed 's|.|.|g'`; \ ++ list='$(DISTFILES)'; for file in $$list; do \ ++ case $$file in \ ++ $(srcdir)/*) file=`echo "$$file" | sed "s|^$$srcdirstrip/||"`;; \ ++ $(top_srcdir)/*) file=`echo "$$file" | sed "s|^$$topsrcdirstrip/|$(top_builddir)/|"`;; \ ++ esac; \ ++ if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ ++ dir=`echo "$$file" | sed -e 's,/[^/]*$$,,'`; \ ++ if test "$$dir" != "$$file" && test "$$dir" != "."; then \ ++ dir="/$$dir"; \ ++ $(mkinstalldirs) "$(distdir)$$dir"; \ ++ else \ ++ dir=''; \ ++ fi; \ ++ if test -d $$d/$$file; then \ ++ if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ ++ cp -pR $(srcdir)/$$file $(distdir)$$dir || exit 1; \ ++ fi; \ ++ cp -pR $$d/$$file $(distdir)$$dir || exit 1; \ ++ else \ ++ test -f $(distdir)/$$file \ ++ || cp -p $$d/$$file $(distdir)/$$file \ ++ || exit 1; \ ++ fi; \ ++ done ++ list='$(SUBDIRS)'; for subdir in $$list; do \ ++ if test "$$subdir" = .; then :; else \ ++ test -d $(distdir)/$$subdir \ ++ || mkdir $(distdir)/$$subdir \ ++ || exit 1; \ ++ (cd $$subdir && \ ++ $(MAKE) $(AM_MAKEFLAGS) \ ++ top_distdir="$(top_distdir)" \ ++ distdir=../$(distdir)/$$subdir \ ++ distdir) \ ++ || exit 1; \ ++ fi; \ ++ done ++ $(MAKE) $(AM_MAKEFLAGS) \ ++ top_distdir="$(top_distdir)" distdir="$(distdir)" \ ++ dist-hook ++ -find $(distdir) -type d ! -perm -777 -exec chmod a+rwx {} \; -o \ ++ ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \ ++ ! -type d ! -perm -400 -exec chmod a+r {} \; -o \ ++ ! -type d ! -perm -444 -exec $(SHELL) $(install_sh) -c -m a+r {} {} \; \ ++ || chmod -R a+r $(distdir) ++dist-gzip: distdir ++ $(AMTAR) chof - $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).tar.gz ++ $(am__remove_distdir) ++ ++dist dist-all: distdir ++ $(AMTAR) chof - $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).tar.gz ++ $(am__remove_distdir) ++ ++# This target untars the dist file and tries a VPATH configuration. Then ++# it guarantees that the distribution is self-contained by making another ++# tarfile. ++distcheck: dist ++ $(am__remove_distdir) ++ GZIP=$(GZIP_ENV) gunzip -c $(distdir).tar.gz | $(AMTAR) xf - ++ chmod -R a-w $(distdir); chmod a+w $(distdir) ++ mkdir $(distdir)/_build ++ mkdir $(distdir)/_inst ++ chmod a-w $(distdir) ++ dc_install_base=`$(am__cd) $(distdir)/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \ ++ && dc_destdir="$${TMPDIR-/tmp}/am-dc-$$$$/" \ ++ && cd $(distdir)/_build \ ++ && ../configure --srcdir=.. --prefix="$$dc_install_base" \ ++ $(DISTCHECK_CONFIGURE_FLAGS) \ ++ && $(MAKE) $(AM_MAKEFLAGS) \ ++ && $(MAKE) $(AM_MAKEFLAGS) dvi \ ++ && $(MAKE) $(AM_MAKEFLAGS) check \ ++ && $(MAKE) $(AM_MAKEFLAGS) install \ ++ && $(MAKE) $(AM_MAKEFLAGS) installcheck \ ++ && $(MAKE) $(AM_MAKEFLAGS) uninstall \ ++ && $(MAKE) $(AM_MAKEFLAGS) distuninstallcheck_dir="$$dc_install_base" \ ++ distuninstallcheck \ ++ && chmod -R a-w "$$dc_install_base" \ ++ && ({ \ ++ (cd ../.. && $(mkinstalldirs) "$$dc_destdir") \ ++ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" install \ ++ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" uninstall \ ++ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" \ ++ distuninstallcheck_dir="$$dc_destdir" distuninstallcheck; \ ++ } || { rm -rf "$$dc_destdir"; exit 1; }) \ ++ && rm -rf "$$dc_destdir" \ ++ && $(MAKE) $(AM_MAKEFLAGS) dist-gzip \ ++ && rm -f $(distdir).tar.gz \ ++ && $(MAKE) $(AM_MAKEFLAGS) distcleancheck ++ $(am__remove_distdir) ++ @echo "$(distdir).tar.gz is ready for distribution" | \ ++ sed 'h;s/./=/g;p;x;p;x' ++distuninstallcheck: ++ @cd $(distuninstallcheck_dir) \ ++ && test `$(distuninstallcheck_listfiles) | wc -l` -le 1 \ ++ || { echo "ERROR: files left after uninstall:" ; \ ++ if test -n "$(DESTDIR)"; then \ ++ echo " (check DESTDIR support)"; \ ++ fi ; \ ++ $(distuninstallcheck_listfiles) ; \ ++ exit 1; } >&2 ++distcleancheck: distclean ++ @if test '$(srcdir)' = . ; then \ ++ echo "ERROR: distcleancheck can only run from a VPATH build" ; \ ++ exit 1 ; \ ++ fi ++ @test `$(distcleancheck_listfiles) | wc -l` -eq 0 \ ++ || { echo "ERROR: files left in build directory after distclean:" ; \ ++ $(distcleancheck_listfiles) ; \ ++ exit 1; } >&2 ++check-am: all-am ++check: check-recursive ++all-am: Makefile config.h ++installdirs: installdirs-recursive ++installdirs-am: ++ ++install: install-recursive ++install-exec: install-exec-recursive ++install-data: install-data-recursive ++uninstall: uninstall-recursive ++ ++install-am: all-am ++ @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am ++ ++installcheck: installcheck-recursive ++install-strip: ++ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ ++ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ ++ `test -z '$(STRIP)' || \ ++ echo "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'"` install ++mostlyclean-generic: ++ ++clean-generic: ++ -test -z "$(CLEANFILES)" || rm -f $(CLEANFILES) ++ ++distclean-generic: ++ -rm -f $(CONFIG_CLEAN_FILES) ++ ++maintainer-clean-generic: ++ @echo "This command is intended for maintainers to use" ++ @echo "it deletes files that may require special tools to rebuild." ++clean: clean-recursive ++ ++clean-am: clean-generic clean-libtool mostlyclean-am ++ ++distclean: distclean-recursive ++ -rm -f $(am__CONFIG_DISTCLEAN_FILES) ++ -rm -f Makefile ++distclean-am: clean-am distclean-generic distclean-hdr distclean-libtool \ ++ distclean-tags ++ ++dvi: dvi-recursive ++ ++dvi-am: ++ ++info: info-recursive ++ ++info-am: ++ ++install-data-am: ++ ++install-exec-am: ++ ++install-info: install-info-recursive ++ ++install-man: ++ ++installcheck-am: ++ ++maintainer-clean: maintainer-clean-recursive ++ -rm -f $(am__CONFIG_DISTCLEAN_FILES) ++ -rm -rf $(top_srcdir)/autom4te.cache ++ -rm -f Makefile ++maintainer-clean-am: distclean-am maintainer-clean-generic ++ ++mostlyclean: mostlyclean-recursive ++ ++mostlyclean-am: mostlyclean-generic mostlyclean-libtool ++ ++pdf: pdf-recursive ++ ++pdf-am: ++ ++ps: ps-recursive ++ ++ps-am: ++ ++uninstall-am: uninstall-info-am ++ ++uninstall-info: uninstall-info-recursive ++ ++.PHONY: $(RECURSIVE_TARGETS) CTAGS GTAGS all all-am check check-am clean \ ++ clean-generic clean-libtool clean-recursive ctags \ ++ ctags-recursive dist dist-all dist-gzip distcheck distclean \ ++ distclean-generic distclean-hdr distclean-libtool \ ++ distclean-recursive distclean-tags distcleancheck distdir \ ++ distuninstallcheck dvi dvi-am dvi-recursive info info-am \ ++ info-recursive install install-am install-data install-data-am \ ++ install-data-recursive install-exec install-exec-am \ ++ install-exec-recursive install-info install-info-am \ ++ install-info-recursive install-man install-recursive \ ++ install-strip installcheck installcheck-am installdirs \ ++ installdirs-am installdirs-recursive maintainer-clean \ ++ maintainer-clean-generic maintainer-clean-recursive mostlyclean \ ++ mostlyclean-generic mostlyclean-libtool mostlyclean-recursive \ ++ pdf pdf-am pdf-recursive ps ps-am ps-recursive tags \ ++ tags-recursive uninstall uninstall-am uninstall-info-am \ ++ uninstall-info-recursive uninstall-recursive ++ ++ ++# This is just so that the linking is done early. ++config.h: $(linked_sources) ++ ++linked_include_sources: ++ cd include; $(MAKE) link_sources ++ echo timestamp > linked_include_sources ++ ++linked_client_sources: @linked_client_targets@ ++ cd client; $(MAKE) link_sources ++ echo timestamp > linked_client_sources ++ ++linked_libmysql_sources: ++ cd libmysql; $(MAKE) link_sources ++ echo timestamp > linked_libmysql_sources ++ ++linked_libmysql_r_sources: linked_libmysql_sources ++ cd libmysql_r; $(MAKE) link_sources ++ echo timestamp > linked_libmysql_r_sources ++ ++linked_libmysqld_sources: ++ cd libmysqld; $(MAKE) link_sources ++ echo timestamp > linked_libmysqld_sources ++ ++linked_libmysqldex_sources: ++ cd libmysqld/examples; $(MAKE) link_sources ++ echo timestamp > linked_libmysqldex_sources ++ ++linked_netware_sources: ++ cd @netware_dir@; $(MAKE) link_sources ++ echo timestamp > linked_netware_sources ++ ++#avoid recursive make calls in sql directory ++linked_server_sources: ++ cd sql; rm -f mini_client_errors.c;@LN_CP_F@ ../libmysql/errmsg.c mini_client_errors.c ++ echo timestamp > linked_server_sources ++ ++# Create permission databases ++init-db: all ++ $(top_builddir)/scripts/mysql_install_db ++ ++bin-dist: all ++ $(top_builddir)/scripts/make_binary_distribution @MAKE_BINARY_DISTRIBUTION_OPTIONS@ ++ ++# Remove BK's "SCCS" subdirectories from source distribution ++dist-hook: ++ rm -rf `find $(distdir) -type d -name SCCS` ++ ++tags: ++ support-files/build-tags ++.PHONY: init-db bin-dist ++ ++# Test installation ++ ++test: ++ cd mysql-test ; ./mysql-test-run ++# Tell versions [3.59,3.63) of GNU make to not export all variables. ++# Otherwise a system limit (for SysV at least) may be exceeded. ++.NOEXPORT: === added file 'storage/xtradb/build/debian/patches/33_scripts__mysql_create_system_tables__no_test.dpatch' --- a/storage/xtradb/build/debian/patches/33_scripts__mysql_create_system_tables__no_test.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/33_scripts__mysql_create_system_tables__no_test.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,29 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run +## 33_scripts__mysql_create_system_tables__no_test.dpatch by <ch(a)debian.org> +## +## All lines beginning with `## DP:' are a description of the patch. +## DP: scripts__mysql_create_system_tables__no_test +## DP: A user with no password prevents a normal user from login under certain +## DP: circumstances as it is checked first. See #301741. +## DP: http://bugs.mysql.com/bug.php?id=6901 + +@DPATCH@ +--- old/scripts/mysql_system_tables_data.sql 2008-12-04 22:59:44.000000000 +0100 ++++ new/scripts/mysql_system_tables_data.sql 2008-12-04 23:00:07.000000000 +0100 +@@ -11,8 +11,6 @@ + -- Fill "db" table with default grants for anyone to + -- access database 'test' and 'test_%' if "db" table didn't exist + CREATE TEMPORARY TABLE tmp_db LIKE db; +-INSERT INTO tmp_db VALUES ('%','test','','Y','Y','Y','Y','Y','Y','N','Y','Y','Y','Y','Y','Y','Y','Y','N','N','Y','Y'); +-INSERT INTO tmp_db VALUES ('%','test\_%','','Y','Y','Y','Y','Y','Y','N','Y','Y','Y','Y','Y','Y','Y','Y','N','N','Y','Y'); + INSERT INTO db SELECT * FROM tmp_db WHERE @had_db_table=0; + DROP TABLE tmp_db; + +@@ -24,7 +22,5 @@ + INSERT INTO tmp_user VALUES ('localhost','root','','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','','','','',0,0,0,0); + REPLACE INTO tmp_user SELECT @current_hostname,'root','','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','','','','',0,0,0,0 FROM dual WHERE LOWER( @current_hostname) != 'localhost'; + REPLACE INTO tmp_user VALUES ('127.0.0.1','root','','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','','','','',0,0,0,0); +-INSERT INTO tmp_user (host,user) VALUES ('localhost',''); +-INSERT INTO tmp_user (host,user) SELECT @current_hostname,'' FROM dual WHERE LOWER(@current_hostname ) != 'localhost'; + INSERT INTO user SELECT * FROM tmp_user WHERE @had_user_table=0; + DROP TABLE tmp_user; === added file 'storage/xtradb/build/debian/patches/38_scripts__mysqld_safe.sh__signals.dpatch' --- a/storage/xtradb/build/debian/patches/38_scripts__mysqld_safe.sh__signals.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/38_scripts__mysqld_safe.sh__signals.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,43 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run +## 38_scripts__mysqld_safe.sh__signals.dpatch by <ch(a)debian.org> +## +## All lines beginning with `## DP:' are a description of the patch. +## DP: Executes /etc/init.d/mysql on signals +## DP: Reported as http://bugs.mysql.com/bug.php?id=31361 + +@DPATCH@ + +--- old/scripts/mysqld_safe.sh 2006-07-29 13:12:34.000000000 +0200 ++++ old/scripts/mysqld_safe.sh 2006-07-29 13:14:08.000000000 +0200 +@@ -16,8 +16,6 @@ + # This command can be used as pipe to syslog. With "-s" it also logs to stderr. + ERR_LOGGER="logger -p daemon.err -t mysqld_safe -i" + +-trap '' 1 2 3 15 # we shouldn't let anyone kill us +- + umask 007 + + defaults= +@@ -122,7 +122,7 @@ + # sed buffers output (only GNU sed supports a -u (unbuffered) option) + # which means that messages may not get sent to syslog until the + # mysqld process quits. +- cmd="$cmd 2>&1 | logger -t '$syslog_tag_mysqld' -p daemon.error" ++ cmd="$cmd 2>&1 | logger -t '$syslog_tag_mysqld' -p daemon.error & wait" + ;; + *) + echo "Internal program error (non-fatal):" \ +@@ -352,6 +350,13 @@ + fi + + # ++# From now on, we catch signals to do a proper shutdown of mysqld ++# when signalled to do so. ++# ++trap '/usr/bin/mysqladmin --defaults-extra-file=/etc/mysql/debian.cnf refresh' 1 # HUP ++trap '/usr/bin/mysqladmin --defaults-extra-file=/etc/mysql/debian.cnf shutdown' 2 3 15 # INT QUIT and TERM ++ ++# + # Uncomment the following lines if you want all tables to be automatically + # checked and repaired during startup. You should add sensible key_buffer + # and sort_buffer values to my.cnf to improve check performance or require === added file 'storage/xtradb/build/debian/patches/41_scripts__mysql_install_db.sh__no_test.dpatch' --- a/storage/xtradb/build/debian/patches/41_scripts__mysql_install_db.sh__no_test.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/41_scripts__mysql_install_db.sh__no_test.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,20 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run +## 41_scripts__mysql_install_db.sh__no_test.dpatch by <ch(a)debian.org> +## +## All lines beginning with `## DP:' are a description of the patch. +## DP: scripts__mysql_install_db.sh__no_test +## DP: http://bugs.mysql.com/bug.php?id=6901 + +@DPATCH@ + +--- mysql-dfsg-5.1-5.1.23rc.orig/scripts/mysql_install_db.sh 2008-01-29 22:41:20.000000000 +0100 ++++ mysql-dfsg-5.1-5.1.23rc/scripts/mysql_install_db.sh 2008-02-28 10:08:11.000000000 +0100 +@@ -306,7 +306,7 @@ + fi + + # Create database directories +-for dir in $ldata $ldata/mysql $ldata/test ++for dir in $ldata $ldata/mysql + do + if test ! -d $dir + then === added file 'storage/xtradb/build/debian/patches/44_scripts__mysql_config__libs.dpatch' --- a/storage/xtradb/build/debian/patches/44_scripts__mysql_config__libs.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/44_scripts__mysql_config__libs.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,24 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run +## 99-unnamed.dpatch by <ch(a)debian.org> +## +## All lines beginning with `## DP:' are a description of the patch. +## DP: Removes unnecessary library dependencies. See #390692 + +@DPATCH@ +diff -Nur mysql-dfsg-5.1-5.1.31.orig/scripts/mysql_config.sh mysql-dfsg-5.1-5.1.31/scripts/mysql_config.sh +--- mysql-dfsg-5.1-5.1.31.orig/scripts/mysql_config.sh 2009-01-19 17:30:55.000000000 +0100 ++++ mysql-dfsg-5.1-5.1.31/scripts/mysql_config.sh 2009-02-08 17:17:48.000000000 +0100 +@@ -104,10 +104,10 @@ + + # Create options + # We intentionally add a space to the beginning and end of lib strings, simplifies replace later +-libs=" $ldflags -L$pkglibdir -lmysqlclient @ZLIB_DEPS@ @NON_THREADED_LIBS@" ++libs=" $ldflags -L$pkglibdir -lmysqlclient" + libs="$libs @openssl_libs@ @STATIC_NSS_FLAGS@ " +-libs_r=" $ldflags -L$pkglibdir -lmysqlclient_r @ZLIB_DEPS@ @LIBS@ @openssl_libs@ " +-embedded_libs=" $ldflags -L$pkglibdir -lmysqld @LIBDL@ @ZLIB_DEPS@ @LIBS@ @WRAPLIBS@ @innodb_system_libs@ @openssl_libs@ " ++libs_r=" $ldflags -L$pkglibdir -lmysqlclient_r @openssl_libs@ " ++embedded_libs=" $ldflags -L$pkglibdir -lmysqld @LIBDL@ @WRAPLIBS@ @innodb_system_libs@ @openssl_libs@ " + + if [ -r "$pkglibdir/libmygcc.a" ]; then + # When linking against the static library with a different version of GCC === added file 'storage/xtradb/build/debian/patches/50_mysql-test__db_test.dpatch' --- a/storage/xtradb/build/debian/patches/50_mysql-test__db_test.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/50_mysql-test__db_test.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,23 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run +## 50_mysql-test__db_test.dpatch by Christian Hammers <ch(a)debian.org> +## +## All lines beginning with `## DP:' are a description of the patch. +## DP: Patch 33_scripts__mysql_create_system_tables__no_test removes the +## DP: rights for anybody to connect to the test database but the test +## DP: suite depends on them. + +@DPATCH@ + +--- old/mysql-test/mysql-test-run.pl 2009-06-16 14:24:09.000000000 +0200 ++++ new/mysql-test/mysql-test-run.pl 2009-07-04 00:03:34.000000000 +0200 +@@ -2717,6 +2717,10 @@ + mtr_appendfile_to_file("$sql_dir/mysql_system_tables_data.sql", + $bootstrap_sql_file); + ++ mtr_tofile($bootstrap_sql_file, "-- Debian removed the default privileges on the 'test' database\n"); ++ mtr_tofile($bootstrap_sql_file, "INSERT INTO mysql.db VALUES ('%','test','','Y','Y','Y','Y','Y','Y','N','Y','Y','Y','Y','Y','Y','Y','Y','N','N','Y','Y');\n"); ++ ++ + # Add test data for timezone - this is just a subset, on a real + # system these tables will be populated either by mysql_tzinfo_to_sql + # or by downloading the timezone table package from our website === added file 'storage/xtradb/build/debian/patches/60_percona_support.dpatch' --- a/storage/xtradb/build/debian/patches/60_percona_support.dpatch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/patches/60_percona_support.dpatch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,15 @@ +#! /bin/sh /usr/share/dpatch/dpatch-run + +@DPATCH@ + +--- a/scripts/mysql_install_db.sh 2009-08-08 09:20:07.000000000 +0000 ++++ b/scripts/mysql_install_db.sh 2009-08-08 09:29:23.000000000 +0000 +@@ -471,7 +471,7 @@ + echo "Please report any problems with the $scriptdir/mysqlbug script!" + echo + echo "The latest information about MySQL is available at http://www.mysql.com/" +- echo "Support MySQL by buying support/licenses from http://shop.mysql.com/" ++ echo "For commercial support please contact Percona at http://www.percona.com/contacts.html" + echo + fi + === added file 'storage/xtradb/build/debian/percona-xtradb-client-5.1.README.Debian' --- a/storage/xtradb/build/debian/percona-xtradb-client-5.1.README.Debian 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-client-5.1.README.Debian 2010-02-18 23:53:03 +0000 @@ -0,0 +1,4 @@ +FAQ: + +Q: My <tab> completition is gone, why? +A: You have "no-auto-rehash" in the "[mysql]" section of /etc/mysql/my.cnf! === added file 'storage/xtradb/build/debian/percona-xtradb-client-5.1.dirs' --- a/storage/xtradb/build/debian/percona-xtradb-client-5.1.dirs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-client-5.1.dirs 2010-02-18 23:53:03 +0000 @@ -0,0 +1,3 @@ +usr/bin/ +usr/share/man/man1/ +usr/share/perl5/ === added file 'storage/xtradb/build/debian/percona-xtradb-client-5.1.docs' --- a/storage/xtradb/build/debian/percona-xtradb-client-5.1.docs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-client-5.1.docs 2010-02-18 23:53:03 +0000 @@ -0,0 +1,3 @@ +debian/additions/innotop/changelog.innotop +EXCEPTIONS-CLIENT +README === added file 'storage/xtradb/build/debian/percona-xtradb-client-5.1.files' --- a/storage/xtradb/build/debian/percona-xtradb-client-5.1.files 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-client-5.1.files 2010-02-18 23:53:03 +0000 @@ -0,0 +1,39 @@ +usr/bin/innotop +usr/bin/myisam_ftdump +usr/bin/mysql +usr/bin/mysqlaccess +usr/bin/mysqladmin +usr/bin/mysqlbug +usr/bin/mysqlcheck +usr/bin/mysql_client_test +usr/bin/mysqldump +usr/bin/mysqldumpslow +usr/bin/mysql_find_rows +usr/bin/mysql_fix_extensions +usr/bin/mysqlimport +usr/bin/mysqlreport +usr/bin/mysqlshow +usr/bin/mysql_waitpid +usr/sbin/mysqlmanager +usr/share/lintian/overrides/percona-xtradb-client-5.1 +usr/share/man/man1/innotop.1 +usr/share/man/man1/myisam_ftdump.1 +usr/share/man/man1/mysql.1 +usr/share/man/man1/mysqlaccess.1 +usr/share/man/man1/mysqladmin.1 +usr/share/man/man1/mysqlbug.1 +usr/share/man/man1/mysqlcheck.1 +usr/share/man/man1/mysqldump.1 +usr/share/man/man1/mysqldumpslow.1 +usr/share/man/man1/mysql_find_rows.1 +usr/share/man/man1/mysql_fix_extensions.1 +usr/share/man/man1/mysqlimport.1 +usr/share/man/man1/mysqlmanager.1 +usr/share/man/man1/mysqlmanagerc.1 +usr/share/man/man1/mysqlmanager-pwgen.1 +usr/share/man/man1/mysqlreport.1 +usr/share/man/man1/mysqlshow.1 +usr/share/man/man1/mysql_tableinfo.1 +usr/share/man/man1/mysql_waitpid.1 +usr/share/man/man1/mysql_client_test.1 +usr/share/perl5/InnoDBParser.pm === added file 'storage/xtradb/build/debian/percona-xtradb-client-5.1.links' --- a/storage/xtradb/build/debian/percona-xtradb-client-5.1.links 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-client-5.1.links 2010-02-18 23:53:03 +0000 @@ -0,0 +1,3 @@ +usr/bin/mysqlcheck usr/bin/mysqlrepair +usr/bin/mysqlcheck usr/bin/mysqlanalyze +usr/bin/mysqlcheck usr/bin/mysqloptimize === added file 'storage/xtradb/build/debian/percona-xtradb-client-5.1.lintian-overrides' --- a/storage/xtradb/build/debian/percona-xtradb-client-5.1.lintian-overrides 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-client-5.1.lintian-overrides 2010-02-18 23:53:03 +0000 @@ -0,0 +1,3 @@ +percona-xtradb-client-5.1: package-has-a-duplicate-relation +percona-xtradb-client-5.1: wrong-name-for-upstream-changelog usr/share/doc/percona-xtradb-client-5.1/changelog.innotop.gz +percona-xtradb-client-5.1: pkg-not-in-package-test innotop === added file 'storage/xtradb/build/debian/percona-xtradb-client-5.1.menu' --- a/storage/xtradb/build/debian/percona-xtradb-client-5.1.menu 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-client-5.1.menu 2010-02-18 23:53:03 +0000 @@ -0,0 +1,3 @@ +# According to /usr/share/menu/ policy 1.4, not /usr/share/doc/debian-policy/ +?package(innotop):needs="text" section="Applications/Data Management"\ + title="innotop" command="/usr/bin/innotop" === added file 'storage/xtradb/build/debian/percona-xtradb-common.dirs' --- a/storage/xtradb/build/debian/percona-xtradb-common.dirs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-common.dirs 2010-02-18 23:53:03 +0000 @@ -0,0 +1 @@ +etc/mysql/conf.d/ === added file 'storage/xtradb/build/debian/percona-xtradb-common.files' --- a/storage/xtradb/build/debian/percona-xtradb-common.files 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-common.files 2010-02-18 23:53:03 +0000 @@ -0,0 +1,2 @@ +etc/mysql/my.cnf +usr/share/percona-xtradb-common/internal-use-only === added file 'storage/xtradb/build/debian/percona-xtradb-common.lintian-overrides' --- a/storage/xtradb/build/debian/percona-xtradb-common.lintian-overrides 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-common.lintian-overrides 2010-02-18 23:53:03 +0000 @@ -0,0 +1,2 @@ +script-not-executable ./usr/share/percona-xtradb-common/internal-use-only/_etc_init.d_mysql +script-not-executable ./usr/share/percona-xtradb-common/internal-use-only/_etc_mysql_debian-start === added file 'storage/xtradb/build/debian/percona-xtradb-common.postrm' --- a/storage/xtradb/build/debian/percona-xtradb-common.postrm 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-common.postrm 2010-02-18 23:53:03 +0000 @@ -0,0 +1,7 @@ +#!/bin/bash + +if [ "$1" = "purge" ]; then + rmdir /etc/mysql 2>/dev/null || true +fi + +#DEBHELPER# === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.NEWS' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.NEWS 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.NEWS 2010-02-18 23:53:03 +0000 @@ -0,0 +1,34 @@ +mysql-dfsg-5.1 (5.1.38-1) unstable; urgency=low + + * Please read http://dev.mysql.com/doc/refman/5.1/en/upgrading-from-5-0.html + * Make sure to do a REPAIR TABLE on all tables that use UTF-8 and have a + FULLTEXT index. + + -- Christian Hammers <ch(a)debian.org> Sat, 4 Jul 2009 02:31:21 +0200 + +mysql-dfsg-5.0 (5.1.14beta-2) unstable; urgency=low + + * The BerkeleyDB Storage Engine is no longer supported. If the options + have-bdb or skip-bdb are found, MySQL will not start. If you have BDB + tables, you should change them to use another storage engine before + upgrading to 5.1. + + -- Monty Taylor <mordred(a)inaugust.com> Thu, 18 Jan 2007 12:28:21 -0800 + +mysql-dfsg-5.0 (5.0.45-2) unstable; urgency=low + + * Binary logging is now disabled by default. If you really need it (e.g. on + a replication master), remove the comment from the log_bin line in my.cnf. + + -- Norbert Tretkowski <nobse(a)debian.org> Sat, 10 Nov 2007 16:26:35 +0100 + +mysql-dfsg-5.0 (5.0.18-9) unstable; urgency=low + + * Rotation of the binary logs is now configured in /etc/mysql/my.cnf with + "expire-logs-days" which defaults to 20 days. The old file + /etc/mysql/debian-log-rotate.conf should be removed together with + /etc/cron.daily/mysql-server after this value has been adjusted. Note that + the old variable defined the number of files whereas the new one defines + a time span in days. + + -- Christian Hammers <ch(a)debian.org> Tue, 24 Jan 2006 22:18:21 +0100 === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.README.Debian' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.README.Debian 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.README.Debian 2010-02-18 23:53:03 +0000 @@ -0,0 +1,109 @@ +* MYSQL WON'T START OR STOP?: +============================= +You may never ever delete the special mysql user "debian-sys-maint". This +user together with the credentials in /etc/mysql/debian.cnf are used by the +init scripts to stop the server as they would require knowledge of the mysql +root users password else. +So in most of the times you can fix the situation by making sure that the +debian.cnf file contains the right password, e.g. by setting a new one +(remember to do a "flush privileges" then). + +* WHAT TO DO AFTER UPGRADES: +============================ +The privilege tables are automatically updated so all there is left is read +the changelogs on dev.mysql.com to see if any changes affect custom apps. + +* WHAT TO DO AFTER INSTALLATION: +================================ +The MySQL manual describes certain steps to do at this stage in a separate +chapter. They are not necessary as the Debian packages does them +automatically. + +The only thing that is left over for the admin is + - setting the passwords + - creating new users and databases + - read the rest of this text + +* DOWNGRADING TO 4.0 or 4.1: +============================ +Unsupported. Period. +But if you do and get problems or make interesting experiences, mail me, it +might help others. +Ok, if you really want, I would recommend to "mysqldump --opt" all tables, +then purge 4.1, delete /var/lib/mysql, install 4.0 and insert the dumps. Be +carefully, though, with the "mysql" table, you might not simply overwrite that +one as the password for the mysql "debian-sys-maint" user is stored in +/etc/mysql/debian.cnf and needed by /etc/init.d/ to start mysql and check if +it's alive. + +* SOME APPLICATION CAN NO LONGER CONNECT: +========================================= +This application is probably linked against libmysqlclient12 or below and +somebody has created a mysql user with new-style passwords. +The old_passwords=1 option in /etc/mysql/my.cnf might help. If not the +application that inserted the user has to be changed or the application that +tries to connect updated to libmysqlclient14 or -15. + +* NETWORKING: +============= +For security reasons, the Debian package has enabled networking only on the +loop-back device using "bind-address" in /etc/mysql/my.cnf. Check with +"netstat -tlnp" where it is listening. If your connection is aborted +immediately see if "mysqld: all" or similar is in /etc/hosts.allow and read +hosts_access(5). + +* WHERE IS THE DOCUMENTATION?: +============================== +Unfortunately due to licensing restrictions, debian currently not able +to provide the mysql-doc package in any format. For the most up to date +documentation, please go to http://dev.mysql.com/doc. + +* PASSWORDS: +============ +It is strongly recommended to set a password for the mysql root user (which + /usr/bin/mysql -u root -D mysql -e "update user set password=password('new-password') where user='root'" + /usr/bin/mysql -u root -e "flush privileges" +If you already had a password set add "-p" before "-u" to the lines above. + + +If you are tired to type the password in every time or want to automate your +scripts you can store it in the file $HOME/.my.cnf. It should be chmod 0600 +(-rw------- username username .my.cnf) to ensure that nobody else can read +it. Every other configuration parameter can be stored there, too. You will +find an example below and more information in the MySQL manual in +/usr/share/doc/mysql-doc or www.mysql.com. + +ATTENTION: It is necessary, that a .my.cnf from root always contains a "user" +line wherever there is a "password" line, else, the Debian maintenance +scripts, that use /etc/mysql/debian.cnf, will use the username +"debian-sys-maint" but the password that is in root's .my.cnf. Also note, +that every change you make in the /root/.my.cnf will affect the mysql cron +script, too. + + # an example of $HOME/.my.cnf + [client] + user = your-mysql-username + password = enter-your-good-new-password-here + +* BIG_ROWS FOR EVEN MORE ROWS IN A TABLE: +========================================= +If you ever run out of rows in a table there is the possibility of building +the package with "-DBIG_ROWS" which, according to a MySQL employee on +packagers(a)lists.mysql.com should lead to a 64bit row index (I guess > 2^32 +rows) but also to an approx. 5% performance loss. + +* BerkeleyDB Storage Engine +=========================== +Support for BerkeleyDB has been removed in 5.1, and consequently both the +have-bdb and skip-bdb configuration options will cause the server to fail. +Removing the options from /etc/mysql/my.cnf will fix this problem. + +* FURTHER NOTES ON REPLICATION +=============================== +If the MySQL server is acting as a replication slave, you should not +set --tmpdir to point to a directory on a memory-based filesystem or to +a directory that is cleared when the server host restarts. A replication +slave needs some of its temporary files to survive a machine restart so +that it can replicate temporary tables or LOAD DATA INFILE operations. If +files in the temporary file directory are lost when the server restarts, +replication fails. === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.config' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.config 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.config 2010-02-18 23:53:03 +0000 @@ -0,0 +1,46 @@ +#!/bin/bash -e + +. /usr/share/debconf/confmodule + +if [ -n "$DEBIAN_SCRIPT_DEBUG" ]; then set -v -x; DEBIAN_SCRIPT_TRACE=1; fi +${DEBIAN_SCRIPT_TRACE:+ echo "#42#DEBUG# RUNNING $0 $*" 1>&2 } + +CNF=/etc/mysql/my.cnf + +# Beware that there are two ypwhich one of them needs the 2>/dev/null! +if test -n "`which ypwhich 2>/dev/null`" && ypwhich >/dev/null 2>&1; then + db_input high percona-xtradb-server-5.1/nis_warning || true + db_go +fi + +# only ask this question on fresh installs, during "reconfiguration" and when +# not upgrading from an existing 5.0 installation. +# there is also an additional check for empty root passwords in the +# postinst script when the tools are available for us to use. +if [ "$1" = "configure" ] && ([ -z "$2" ] && [ ! -e "/var/lib/mysql/debian-5.0.flag" ] ) || [ "$1" = "reconfigure" ]; then + while :; do + RET="" + db_input high percona-xtradb-server/root_password || true + db_go + db_get percona-xtradb-server/root_password + # if password isn't empty we ask for password verification + if [ -z "$RET" ]; then + db_fset percona-xtradb-server/root_password seen false + db_fset percona-xtradb-server/root_password_again seen false + break + fi + ROOT_PW="$RET" + db_input high percona-xtradb-server/root_password_again || true + db_go + db_get percona-xtradb-server/root_password_again + if [ "$RET" == "$ROOT_PW" ]; then + ROOT_PW='' + break + fi + db_fset percona-xtradb-server/password_mismatch seen false + db_input critical percona-xtradb-server/password_mismatch + db_set percona-xtradb-server/root_password "" + db_set percona-xtradb-server/root_password_again "" + db_go + done +fi === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.dirs' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.dirs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.dirs 2010-02-18 23:53:03 +0000 @@ -0,0 +1,9 @@ +etc/init.d +etc/logrotate.d +etc/mysql/conf.d +usr/bin +usr/sbin +usr/share/man/man8 +usr/share/mysql +var/run/mysqld +var/lib/mysql-upgrade === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.docs' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.docs 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.docs 2010-02-18 23:53:03 +0000 @@ -0,0 +1 @@ +EXCEPTIONS-CLIENT === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.files' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.files 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.files 2010-02-18 23:53:03 +0000 @@ -0,0 +1,53 @@ +usr/lib/mysql/*so* +etc/mysql/debian-start +etc/mysql/conf.d/mysqld_safe_syslog.cnf +usr/bin/msql2mysql +usr/bin/my_print_defaults +usr/bin/myisamchk +usr/bin/myisamlog +usr/bin/myisampack +usr/bin/mysql_convert_table_format +usr/bin/mysql_fix_privilege_tables +usr/bin/mysql_install_db +usr/bin/mysql_secure_installation +usr/bin/mysql_setpermission +usr/bin/mysql_tzinfo_to_sql +usr/bin/mysql_upgrade +usr/bin/mysql_zap +usr/bin/mysqlbinlog +usr/bin/mysqld_multi +usr/bin/mysqld_safe +usr/bin/mysqlhotcopy +usr/bin/mysqltest +usr/bin/perror +usr/bin/replace +usr/bin/resolve_stack_dump +usr/bin/resolveip +usr/sbin/mysqld +usr/share/doc/percona-xtradb-server-5.1/ +usr/share/lintian/overrides/percona-xtradb-server-5.1 +usr/share/man/man1/msql2mysql.1 +usr/share/man/man1/myisamchk.1 +usr/share/man/man1/myisamlog.1 +usr/share/man/man1/myisampack.1 +usr/share/man/man1/my_print_defaults.1 +usr/share/man/man1/mysqlbinlog.1 +usr/share/man/man1/mysql_convert_table_format.1 +usr/share/man/man1/mysqld_multi.1 +usr/share/man/man1/mysqld_safe.1 +usr/share/man/man1/mysql_fix_privilege_tables.1 +usr/share/man/man1/mysqlhotcopy.1 +usr/share/man/man1/mysql_install_db.1 +usr/share/man/man1/mysql_secure_installation.1 +usr/share/man/man1/mysql_setpermission.1 +usr/share/man/man1/mysql_upgrade.1 +usr/share/man/man1/mysqltest.1 +usr/share/man/man1/mysql_zap.1 +usr/share/man/man1/perror.1 +usr/share/man/man1/replace.1 +usr/share/man/man1/resolveip.1 +usr/share/man/man1/resolve_stack_dump.1 +usr/share/man/man1/innochecksum.1 +usr/share/man/man1/mysql_tzinfo_to_sql.1 +usr/share/man/man8/mysqld.8 +usr/share/mysql/ === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.links' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.links 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.links 2010-02-18 23:53:03 +0000 @@ -0,0 +1,2 @@ +usr/share/mysql/mysql-test/mysql-test-run.pl usr/share/mysql/mysql-test/mysql-test-run +usr/share/mysql/mysql-test/mysql-test-run.pl usr/share/mysql/mysql-test/mtr === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.lintian-overrides' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.lintian-overrides 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.lintian-overrides 2010-02-18 23:53:03 +0000 @@ -0,0 +1,4 @@ +percona-xtradb-server-5.1: possible-bashism-in-maintainer-script postinst:81 'p{("a".."z","A".."Z",0..9)[int(rand(62))]}' +percona-xtradb-server-5.1: possible-bashism-in-maintainer-script preinst:33 '${cmd/ */}' +percona-xtradb-server-5.1: statically-linked-binary ./usr/bin/mysql_tzinfo_to_sql +percona-xtradb-server-5.1: statically-linked-binary ./usr/sbin/mysqld === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.paranoid' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.paranoid 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.paranoid 2010-02-18 23:53:03 +0000 @@ -0,0 +1,9 @@ +/etc/init.d/mysql\[[0-9]+\]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists\!$ +/etc/init.d/mysql\[[0-9]+\]: '/usr/bin/mysqladmin --defaults-(extra-)?file=/etc/mysql/debian.cnf ping' resulted in$ +/etc/mysql/debian-start\[[0-9]+\]: Checking for crashed MySQL tables\.$ +mysqld\[[0-9]+\]: $ +mysqld\[[0-9]+\]: Version: .* socket: '/var/run/mysqld/mysqld.sock' port: 3306$ +mysqld\[[0-9]+\]: Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line$ +mysqld_safe\[[0-9]+\]: started$ +usermod\[[0-9]+\]: change user `mysql' GID from `([0-9]+)' to `\1'$ +usermod\[[0-9]+\]: change user `mysql' shell from `/bin/false' to `/bin/false'$ === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.server' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.server 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.server 2010-02-18 23:53:03 +0000 @@ -0,0 +1,32 @@ +/etc/init.d/mysql\[[0-9]+\]: [0-9]+ processes alive and '/usr/bin/mysqladmin --defaults-(extra-)?file=/etc/mysql/debian.cnf ping' resulted in$ +/etc/init.d/mysql\[[0-9]+\]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists\!$ +/etc/init.d/mysql\[[0-9]+\]: '/usr/bin/mysqladmin --defaults-(extra-)?file=/etc/mysql/debian.cnf ping' resulted in$ +/etc/mysql/debian-start\[[0-9]+\]: Checking for crashed MySQL tables\.$ +mysqld\[[0-9]+\]: ?$ +mysqld\[[0-9]+\]: .*InnoDB: Shutdown completed +mysqld\[[0-9]+\]: .*InnoDB: Started; +mysqld\[[0-9]+\]: .*InnoDB: Starting shutdown\.\.\.$ +mysqld\[[0-9]+\]: .*\[Note\] /usr/sbin/mysqld: Normal shutdown$ +mysqld\[[0-9]+\]: .*\[Note\] /usr/sbin/mysqld: ready for connections\.$ +mysqld\[[0-9]+\]: .*\[Note\] /usr/sbin/mysqld: Shutdown complete$ +mysqld\[[0-9]+\]: /usr/sbin/mysqld: ready for connections\.$ +mysqld\[[0-9]+\]: .*/usr/sbin/mysqld: Shutdown Complete$ +mysqld\[[0-9]+\]: Version: .* socket +mysqld\[[0-9]+\]: Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line$ +mysqld_safe\[[0-9]+\]: ?$ +mysqld_safe\[[0-9]+\]: able to use the new GRANT command!$ +mysqld_safe\[[0-9]+\]: ended$ +mysqld_safe\[[0-9]+\]: http://www.mysql.com$ +mysqld_safe\[[0-9]+\]: NOTE: If you are upgrading from a MySQL <= 3.22.10 you should run$ +mysqld_safe\[[0-9]+\]: PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !$ +mysqld_safe\[[0-9]+\]: Please report any problems with the /usr/bin/mysqlbug script!$ +mysqld_safe\[[0-9]+\]: See the manual for more instructions.$ +mysqld_safe\[[0-9]+\]: started$ +mysqld_safe\[[0-9]+\]: Support MySQL by buying support/licenses at https://order.mysql.com$ +mysqld_safe\[[0-9]+\]: The latest information about MySQL is available on the web at$ +mysqld_safe\[[0-9]+\]: the /usr/bin/mysql_fix_privilege_tables. Otherwise you will not be$ +mysqld_safe\[[0-9]+\]: To do so, start the server, then issue the following commands:$ +mysqld_safe\[[0-9]+\]: /usr/bin/mysqladmin -u root -h app109 password 'new-password'$ +mysqld_safe\[[0-9]+\]: /usr/bin/mysqladmin -u root password 'new-password'$ +usermod\[[0-9]+\]: change user `mysql' GID from `([0-9]+)' to `\1'$ +usermod\[[0-9]+\]: change user `mysql' shell from `/bin/false' to `/bin/false'$ === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.workstation' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.workstation 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.logcheck.ignore.workstation 2010-02-18 23:53:03 +0000 @@ -0,0 +1,32 @@ +/etc/init.d/mysql\[[0-9]+\]: [0-9]+ processes alive and '/usr/bin/mysqladmin --defaults-(extra-)?file=/etc/mysql/debian.cnf ping' resulted in$ +/etc/init.d/mysql\[[0-9]+\]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists\!$ +/etc/init.d/mysql\[[0-9]+\]: '/usr/bin/mysqladmin --defaults-(extra-)?file=/etc/mysql/debian.cnf ping' resulted in$ +/etc/mysql/debian-start\[[0-9]+\]: Checking for crashed MySQL tables\.$ +mysqld\[[0-9]+\]: ?$ +mysqld\[[0-9]+\]: .*InnoDB: Shutdown completed +mysqld\[[0-9]+\]: .*InnoDB: Started; +mysqld\[[0-9]+\]: .*InnoDB: Starting shutdown\.\.\.$ +mysqld\[[0-9]+\]: .*\[Note\] /usr/sbin/mysqld: Normal shutdown$ +mysqld\[[0-9]+\]: .*\[Note\] /usr/sbin/mysqld: ready for connections\.$ +mysqld\[[0-9]+\]: .*\[Note\] /usr/sbin/mysqld: Shutdown complete$ +mysqld\[[0-9]+\]: /usr/sbin/mysqld: ready for connections\.$ +mysqld\[[0-9]+\]: .*/usr/sbin/mysqld: Shutdown Complete$ +mysqld\[[0-9]+\]: Version: .* socket +mysqld\[[0-9]+\]: Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line$ +mysqld_safe\[[0-9]+\]: ?$ +mysqld_safe\[[0-9]+\]: able to use the new GRANT command!$ +mysqld_safe\[[0-9]+\]: ended$ +mysqld_safe\[[0-9]+\]: http://www.mysql.com$ +mysqld_safe\[[0-9]+\]: NOTE: If you are upgrading from a MySQL <= 3.22.10 you should run$ +mysqld_safe\[[0-9]+\]: PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !$ +mysqld_safe\[[0-9]+\]: Please report any problems with the /usr/bin/mysqlbug script!$ +mysqld_safe\[[0-9]+\]: See the manual for more instructions.$ +mysqld_safe\[[0-9]+\]: started$ +mysqld_safe\[[0-9]+\]: Support MySQL by buying support/licenses at https://order.mysql.com$ +mysqld_safe\[[0-9]+\]: The latest information about MySQL is available on the web at$ +mysqld_safe\[[0-9]+\]: the /usr/bin/mysql_fix_privilege_tables. Otherwise you will not be$ +mysqld_safe\[[0-9]+\]: To do so, start the server, then issue the following commands:$ +mysqld_safe\[[0-9]+\]: /usr/bin/mysqladmin -u root -h app109 password 'new-password'$ +mysqld_safe\[[0-9]+\]: /usr/bin/mysqladmin -u root password 'new-password'$ +usermod\[[0-9]+\]: change user `mysql' GID from `([0-9]+)' to `\1'$ +usermod\[[0-9]+\]: change user `mysql' shell from `/bin/false' to `/bin/false'$ === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.mysql.init' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.mysql.init 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.mysql.init 2010-02-18 23:53:03 +0000 @@ -0,0 +1,182 @@ +#!/bin/bash +# +### BEGIN INIT INFO +# Provides: mysql +# Required-Start: $remote_fs $syslog +# Required-Stop: $remote_fs $syslog +# Should-Start: $network $named $time +# Should-Stop: $network $named $time +# Default-Start: 2 3 4 5 +# Default-Stop: 0 1 6 +# Short-Description: Start and stop the mysql database server daemon +# Description: Controls the main MySQL database server daemon "mysqld" +# and its wrapper script "mysqld_safe". +### END INIT INFO +# +set -e +set -u +${DEBIAN_SCRIPT_DEBUG:+ set -v -x} + +test -x /usr/sbin/mysqld || exit 0 + +. /lib/lsb/init-functions + +SELF=$(cd $(dirname $0); pwd -P)/$(basename $0) +CONF=/etc/mysql/my.cnf +MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf" + +# priority can be overriden and "-s" adds output to stderr +ERR_LOGGER="logger -p daemon.err -t /etc/init.d/mysql -i" + +# Safeguard (relative paths, core dumps..) +cd / +umask 077 + +# mysqladmin likes to read /root/.my.cnf. This is usually not what I want +# as many admins e.g. only store a password without a username there and +# so break my scripts. +export HOME=/etc/mysql/ + +## Fetch a particular option from mysql's invocation. +# +# Usage: void mysqld_get_param option +mysqld_get_param() { + /usr/sbin/mysqld --print-defaults \ + | tr " " "\n" \ + | grep -- "--$1" \ + | tail -n 1 \ + | cut -d= -f2 +} + +## Do some sanity checks before even trying to start mysqld. +sanity_checks() { + # check for config file + if [ ! -r /etc/mysql/my.cnf ]; then + log_warning_msg "$0: WARNING: /etc/mysql/my.cnf cannot be read. See README.Debian.gz" + echo "WARNING: /etc/mysql/my.cnf cannot be read. See README.Debian.gz" | $ERR_LOGGER + fi + + # check for diskspace shortage + datadir=`mysqld_get_param datadir` + if LC_ALL=C BLOCKSIZE= df --portability $datadir/. | tail -n 1 | awk '{ exit ($4>4096) }'; then + log_failure_msg "$0: ERROR: The partition with $datadir is too full!" + echo "ERROR: The partition with $datadir is too full!" | $ERR_LOGGER + exit 1 + fi +} + +## Checks if there is a server running and if so if it is accessible. +# +# check_alive insists on a pingable server +# check_dead also fails if there is a lost mysqld in the process list +# +# Usage: boolean mysqld_status [check_alive|check_dead] [warn|nowarn] +mysqld_status () { + ping_output=`$MYADMIN ping 2>&1`; ping_alive=$(( ! $? )) + + ps_alive=0 + pidfile=`mysqld_get_param pid-file` + if [ -f "$pidfile" ] && ps `cat $pidfile` >/dev/null 2>&1; then ps_alive=1; fi + + if [ "$1" = "check_alive" -a $ping_alive = 1 ] || + [ "$1" = "check_dead" -a $ping_alive = 0 -a $ps_alive = 0 ]; then + return 0 # EXIT_SUCCESS + else + if [ "$2" = "warn" ]; then + echo -e "$ps_alive processes alive and '$MYADMIN ping' resulted in\n$ping_output\n" | $ERR_LOGGER -p daemon.debug + fi + return 1 # EXIT_FAILURE + fi +} + +# +# main() +# + +case "${1:-''}" in + 'start') + sanity_checks; + # Start daemon + log_daemon_msg "Starting MySQL database server" "mysqld" + if mysqld_status check_alive nowarn; then + log_progress_msg "already running" + log_end_msg 0 + else + /usr/bin/mysqld_safe > /dev/null 2>&1 & + # 6s was reported in #352070 to be too few when using ndbcluster + for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14; do + sleep 1 + if mysqld_status check_alive nowarn ; then break; fi + log_progress_msg "." + done + if mysqld_status check_alive warn; then + log_end_msg 0 + # Now start mysqlcheck or whatever the admin wants. + output=$(/etc/mysql/debian-start) + [ -n "$output" ] && log_action_msg "$output" + else + log_end_msg 1 + log_failure_msg "Please take a look at the syslog" + fi + fi + ;; + + 'stop') + # * As a passwordless mysqladmin (e.g. via ~/.my.cnf) must be possible + # at least for cron, we can rely on it here, too. (although we have + # to specify it explicit as e.g. sudo environments points to the normal + # users home and not /root) + log_daemon_msg "Stopping MySQL database server" "mysqld" + if ! mysqld_status check_dead nowarn; then + set +e + shutdown_out=`$MYADMIN shutdown 2>&1`; r=$? + set -e + if [ "$r" -ne 0 ]; then + log_end_msg 1 + [ "$VERBOSE" != "no" ] && log_failure_msg "Error: $shutdown_out" + log_daemon_msg "Killing MySQL database server by signal" "mysqld" + killall -15 mysqld + server_down= + for i in 1 2 3 4 5 6 7 8 9 10; do + sleep 1 + if mysqld_status check_dead nowarn; then server_down=1; break; fi + done + if test -z "$server_down"; then killall -9 mysqld; fi + fi + fi + + if ! mysqld_status check_dead warn; then + log_end_msg 1 + log_failure_msg "Please stop MySQL manually and read /usr/share/doc/percona-xtradb-server-5.1/README.Debian.gz!" + exit -1 + else + log_end_msg 0 + fi + ;; + + 'restart') + set +e; $SELF stop; set -e + $SELF start + ;; + + 'reload'|'force-reload') + log_daemon_msg "Reloading MySQL database server" "mysqld" + $MYADMIN reload + log_end_msg 0 + ;; + + 'status') + if mysqld_status check_alive nowarn; then + log_action_msg "$($MYADMIN version)" + else + log_action_msg "MySQL is stopped." + exit 3 + fi + ;; + + *) + echo "Usage: $SELF start|stop|restart|reload|force-reload|status" + exit 1 + ;; +esac + === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.percona-xtradb-server.logrotate' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.percona-xtradb-server.logrotate 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.percona-xtradb-server.logrotate 2010-02-18 23:53:03 +0000 @@ -0,0 +1,27 @@ +# - I put everything in one block and added sharedscripts, so that mysql gets +# flush-logs'd only once. +# Else the binary logs would automatically increase by n times every day. +# - The error log is obsolete, messages go to syslog now. +/var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log { + daily + rotate 7 + missingok + create 640 mysql adm + compress + sharedscripts + postrotate + test -x /usr/bin/mysqladmin || exit 0 + + # If this fails, check debian.conf! + MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf" + if [ -z "`$MYADMIN ping 2>/dev/null`" ]; then + # Really no mysqld or rather a missing debian-sys-maint user? + # If this occurs and is not a error please report a bug. + if ps cax | grep -q mysqld; then + exit 1 + fi + else + $MYADMIN flush-logs + fi + endscript +} === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.postinst' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.postinst 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.postinst 2010-02-18 23:53:03 +0000 @@ -0,0 +1,277 @@ +#!/bin/bash -e + +. /usr/share/debconf/confmodule + +if [ -n "$DEBIAN_SCRIPT_DEBUG" ]; then set -v -x; DEBIAN_SCRIPT_TRACE=1; fi +${DEBIAN_SCRIPT_TRACE:+ echo "#42#DEBUG# RUNNING $0 $*" 1>&2 } + +export PATH=$PATH:/sbin:/usr/sbin:/bin:/usr/bin + +# This command can be used as pipe to syslog. With "-s" it also logs to stderr. +ERR_LOGGER="logger -p daemon.err -t mysqld_safe -i" + +invoke() { + if [ -x /usr/sbin/invoke-rc.d ]; then + invoke-rc.d mysql $1 + else + /etc/init.d/mysql $1 + fi +} + +MYSQL_BOOTSTRAP="/usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables" + +test_mysql_access() { + mysql --no-defaults -u root -h localhost </dev/null >/dev/null 2>&1 +} + +# call with $1 = "online" to connect to the server, otherwise it bootstraps +set_mysql_rootpw() { + # forget we ever saw the password. don't use reset to keep the seen status + db_set percona-xtradb-server/root_password "" + + tfile=`mktemp` + if [ ! -f "$tfile" ]; then + return 1 + fi + + # this avoids us having to call "test" or "[" on $rootpw + cat << EOF > $tfile +USE mysql; +UPDATE user SET password=PASSWORD("$rootpw") WHERE user='root'; +FLUSH PRIVILEGES; +EOF + if grep -q 'PASSWORD("")' $tfile; then + retval=0 + elif [ "$1" = "online" ]; then + mysql --no-defaults -u root -h localhost <$tfile >/dev/null + retval=$? + else + $MYSQL_BOOTSTRAP <$tfile + retval=$? + fi + rm -f $tfile + return $retval +} + +# This is necessary because mysql_install_db removes the pid file in /var/run +# and because changed configuration options should take effect immediately. +# In case the server wasn't running at all it should be ok if the stop +# script fails. I can't tell at this point because of the cleaned /var/run. +set +e; invoke stop; set -e + +case "$1" in + configure) + mysql_datadir=/usr/share/mysql + mysql_statedir=/var/lib/mysql + mysql_rundir=/var/run/mysqld + mysql_logdir=/var/log + mysql_cfgdir=/etc/mysql + mysql_newlogdir=/var/log/mysql + mysql_upgradedir=/var/lib/mysql-upgrade + + # first things first, if the following symlink exists, it is a preserved + # copy the old data dir from a mysql upgrade that would have otherwise + # been replaced by an empty mysql dir. this should restore it. + for dir in DATADIR LOGDIR; do + if [ "$dir" = "DATADIR" ]; then targetdir=$mysql_statedir; else targetdir=$mysql_newlogdir; fi + savelink="$mysql_upgradedir/$dir.link" + if [ -L "$savelink" ]; then + # If the targetdir was a symlink before we upgraded it is supposed + # to be either still be present or not existing anymore now. + if [ -L "$targetdir" ]; then + rm "$savelink" + elif [ ! -d "$targetdir" ]; then + mv "$savelink" "$targetdir" + else + # this should never even happen, but just in case... + mysql_tmp=`mktemp -d -t mysql-symlink-restore-XXXXXX` + echo "this is very strange! see $mysql_tmp/README..." >&2 + mv "$targetdir" "$mysql_tmp" + cat << EOF > "$mysql_tmp/README" + +if you're reading this, it's most likely because you had replaced /var/lib/mysql +with a symlink, then upgraded to a new version of mysql, and then dpkg +removed your symlink (see #182747 and others). the mysql packages noticed +that this happened, and as a workaround have restored it. however, because +/var/lib/mysql seems to have been re-created in the meantime, and because +we don't want to rm -rf something we don't know as much about, we're going +to leave this unexpected directory here. if your database looks normal, +and this is not a symlink to your database, you should be able to blow +this all away. + +EOF + fi + fi + rmdir $mysql_upgradedir 2>/dev/null || true + done + + # Ensure the existence and right permissions for the database and + # log files. + if [ ! -d "$mysql_statedir" -a ! -L "$mysql_statedir" ]; then mkdir "$mysql_statedir"; fi + if [ ! -d "$mysql_statedir/mysql" -a ! -L "$mysql_statedir/mysql" ]; then mkdir "$mysql_statedir/mysql"; fi + if [ ! -d "$mysql_newlogdir" -a ! -L "$mysql_newlogdir" ]; then mkdir "$mysql_newlogdir"; fi + # When creating an ext3 jounal on an already mounted filesystem like e.g. + # /var/lib/mysql, you get a .journal file that is not modifyable by chown. + # The mysql_datadir must not be writable by the mysql user under any + # circumstances as it contains scripts that are executed by root. + set +e + chown -R 0:0 $mysql_datadir + chown -R mysql $mysql_statedir + chown -R mysql $mysql_rundir + chown -R mysql:adm $mysql_newlogdir; chmod 2750 $mysql_newlogdir; + for i in log err; do + touch $mysql_logdir/mysql.$i + chown mysql:adm $mysql_logdir/mysql.$i + chmod 0640 $mysql_logdir/mysql.$i + done + set -e + + # This is important to avoid dataloss when there is a removed + # percona-xtradb-server version from Woody lying around which used the same + # data directory and then somewhen gets purged by the admin. + db_set percona-xtradb-server/postrm_remove_database false || true + + # To avoid downgrades. + touch $mysql_statedir/debian-5.1.flag + + # initiate databases. Output is not allowed by debconf :-( + # Debian: beware of the bashisms... + # Debian: can safely run on upgrades with existing databases + set +e + /bin/bash /usr/bin/mysql_install_db --rpm 2>&1 | $ERR_LOGGER + if [ "$?" != "0" ]; then + echo "ATTENTION: An error has occured. More info is in the syslog!" + fi + set -e + + ## On every reconfiguration the maintenance user is recreated. + # + # - It is easier to regenerate the password every time but as people + # use fancy rsync scripts and file alteration monitors, the existing + # password is used and existing files not touched. + # - The mysqld statement is like that in mysql_install_db because the + # server is not already running. This has some implications: + # - The amount of newlines and semicolons in the query is important! + # - GRANT is not possible with --skip-grant-tables and "INSERT + # (user,host..) VALUES" is not --ansi compliant + # - The echo is just for readability. ash's buildin has no "-e" so use /bin/echo. + # - The Super_priv, Show_db_priv, Create_tmp_table_priv and Lock_tables_priv + # may not be present as old Woody 3.23 databases did not have it and the + # admin might not already have run mysql_upgrade which adds them. + # As the binlog cron scripts to need at least the Super_priv, I do first + # the old query which always succeeds and then the new which may or may not. + + # recreate the credentials file if not present or without mysql_upgrade stanza + dc=$mysql_cfgdir/debian.cnf; + if [ -e "$dc" -a -n "`fgrep mysql_upgrade $dc 2>/dev/null`" ]; then + pass="`sed -n 's/^[ ]*password *= *// p' $dc | head -n 1`" + else + pass=`perl -e 'print map{("a".."z","A".."Z",0..9)[int(rand(62))]}(1..16)'`; + if [ ! -d "$mysql_cfgdir" ]; then install -o 0 -g 0 -m 0755 -d $mysql_cfgdir; fi + cat /dev/null > $dc + echo "# Automatically generated for Debian scripts. DO NOT TOUCH!" >>$dc + echo "[client]" >>$dc + echo "host = localhost" >>$dc + echo "user = debian-sys-maint" >>$dc + echo "password = $pass" >>$dc + echo "socket = $mysql_rundir/mysqld.sock" >>$dc + echo "[mysql_upgrade]" >>$dc + echo "host = localhost" >>$dc + echo "user = debian-sys-maint" >>$dc + echo "password = $pass" >>$dc + echo "socket = $mysql_rundir/mysqld.sock" >>$dc + echo "basedir = /usr" >>$dc + fi + # If this dir chmod go+w then the admin did it. But this file should not. + chown 0:0 $dc + chmod 0600 $dc + + # update privilege tables + password_column_fix_query=`/bin/echo -e \ + "USE mysql\n" \ + "ALTER TABLE user CHANGE Password Password char(41) character set latin1 collate latin1_bin DEFAULT '' NOT NULL"`; + replace_query=`/bin/echo -e \ + "USE mysql\n" \ + "REPLACE INTO user SET " \ + " host='localhost', user='debian-sys-maint', password=password('$pass'), " \ + " Select_priv='Y', Insert_priv='Y', Update_priv='Y', Delete_priv='Y', " \ + " Create_priv='Y', Drop_priv='Y', Reload_priv='Y', Shutdown_priv='Y', " \ + " Process_priv='Y', File_priv='Y', Grant_priv='Y', References_priv='Y', " \ + " Index_priv='Y', Alter_priv='Y', Super_priv='Y', Show_db_priv='Y', "\ + " Create_tmp_table_priv='Y', Lock_tables_priv='Y', Execute_priv='Y', "\ + " Repl_slave_priv='Y', Repl_client_priv='Y', Create_view_priv='Y', "\ + " Show_view_priv='Y', Create_routine_priv='Y', Alter_routine_priv='Y', "\ + " Create_user_priv='Y', Event_priv='Y', Trigger_priv='Y' "`; + fix_privs=`/bin/echo -e \ + "USE mysql;\n" \ + "ALTER TABLE user ADD column Create_view_priv enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N'; " \ + "ALTER TABLE user ADD column Show_view_priv enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N'; " \ + "ALTER TABLE user ADD column Create_routine_priv enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N'; " \ + "ALTER TABLE user ADD column Alter_routine_priv enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N'; " \ + "ALTER TABLE user ADD column Create_user_priv enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N'; " \ + "ALTER TABLE user ADD column Event_priv enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N'; " \ + "ALTER TABLE user ADD column Trigger_priv enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N'; " ` + # Engines supported by etch should be installed per default. The query sequence is supposed + # to be aborted if the CREATE TABLE fails due to an already existent table in which case the + # admin might already have chosen to remove one or more plugins. Newlines are necessary. + install_plugins=`/bin/echo -e \ + "USE mysql;\n" \ + "CREATE TABLE plugin (name char(64) COLLATE utf8_bin NOT NULL DEFAULT '', " \ + " dl char(128) COLLATE utf8_bin NOT NULL DEFAULT '', " \ + " PRIMARY KEY (name)) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='MySQL plugins';\n" \ + "INSERT INTO plugin VALUES ('innodb', 'ha_innodb.so');\n" \ + "INSERT INTO plugin VALUES ('federated', 'ha_federated.so');\n" \ + "INSERT INTO plugin VALUES ('blackhole', 'ha_blackhole.so');\n" \ + "INSERT INTO plugin VALUES ('archive', 'ha_archive.so');" ` + + # Upgrade password column format before the root password gets set. + echo "$password_column_fix_query" | $MYSQL_BOOTSTRAP 2>&1 | $ERR_LOGGER + + db_get percona-xtradb-server/root_password && rootpw="$RET" + if ! set_mysql_rootpw; then + password_error="yes" + fi + + echo "$fix_privs" | $MYSQL_BOOTSTRAP 2>&1 | $ERR_LOGGER + echo "$replace_query" | $MYSQL_BOOTSTRAP 2>&1 | $ERR_LOGGER + set +e + echo "$install_plugins" | $MYSQL_BOOTSTRAP 2>&1 | $ERR_LOGGER + set -e + ;; + + abort-upgrade|abort-remove|abort-configure) + ;; + + *) + echo "postinst called with unknown argument '$1'" 1>&2 + exit 1 + ;; +esac + +# here we check to see if we can connect as root without a password +# this should catch upgrades from previous versions where the root +# password wasn't set. if there is a password, or if the connection +# fails for any other reason, nothing happens. +if [ "$1" = "configure" ]; then + if test_mysql_access; then + db_input medium percona-xtradb-server/root_password || true + db_go + db_get percona-xtradb-server/root_password && rootpw="$RET" + + if ! set_mysql_rootpw "online"; then + password_error="yes" + fi + fi + + if [ "$password_error" = "yes" ]; then + db_input high percona-xtradb-server/error_setting_password || true + db_go + fi + +fi + +db_stop # in case invoke failes + +#DEBHELPER# + +exit 0 === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.postrm' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.postrm 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.postrm 2010-02-18 23:53:03 +0000 @@ -0,0 +1,83 @@ +#!/bin/bash -e + +# It is possible that Debconf has already been removed, too. +if [ -f /usr/share/debconf/confmodule ]; then + . /usr/share/debconf/confmodule +fi + +if [ -n "$DEBIAN_SCRIPT_DEBUG" ]; then set -v -x; DEBIAN_SCRIPT_TRACE=1; fi +${DEBIAN_SCRIPT_TRACE:+ echo "#42#DEBUG# RUNNING $0 $*" 1>&2 } + +MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf" + +# Try to stop the server in a sane way. If it does not success let the admin +# do it himself. No database directories should be removed while the server +# is running! +stop_server() { + set +e + if [ -x /usr/sbin/invoke-rc.d ]; then + invoke-rc.d mysql stop + else + /etc/init.d/mysql stop + fi + errno=$? + set -e + + if [ "$?" != 0 ]; then + echo "Trying to stop the MySQL server resulted in exitcode $?." 1>&2 + echo "Stop it yourself and try again!" 1>&2 + exit 1 + fi +} + +case "$1" in + purge|remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear) + if [ -n "`$MYADMIN ping 2>/dev/null`" ]; then + stop_server + sleep 2 + fi + ;; + *) + echo "postrm called with unknown argument '$1'" 1>&2 + exit 1 + ;; +esac + +# +# - Do NOT purge logs or data if another percona-xtradb-server* package is installed (#307473) +# - Remove the mysql user only after all his owned files are purged. +# +if [ "$1" = "purge" -a ! $ -x /usr/sbin/mysqld -o -L /usr/sbin/mysqld $ ]; then + # we remove the mysql user only after all his owned files are purged + rm -f /var/log/mysql.{log,err}{,.0,.[1234567].gz} + rm -rf /var/log/mysql + + db_input high percona-xtradb-server-5.1/postrm_remove_databases || true + db_go || true + db_get percona-xtradb-server-5.1/postrm_remove_databases || true + if [ "$RET" = "true" ]; then + # never remove the debian.cnf when the databases are still existing + # else we ran into big trouble on the next install! + rm -f /etc/mysql/debian.cnf + rm -rf /var/lib/mysql + rm -rf /var/run/mysqld + userdel mysql || true + fi + + # (normally) Automatically added by dh_installinit + if [ "$1" = "purge" ] ; then + update-rc.d mysql remove >/dev/null || exit 0 + fi + # (normally) End automatically added section +fi + +# (normally) Automatically added by dh_installdebconf +if [ "$1" = purge ] && [ -e /usr/share/debconf/confmodule ]; then + . /usr/share/debconf/confmodule + db_purge +fi +# (normally) End automatically added section + +# no DEBHELPER here, "update-rc.d remove" fails if percona-xtradb-server-5.1 is installed + +exit 0 === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.preinst' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.preinst 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.preinst 2010-02-18 23:53:03 +0000 @@ -0,0 +1,186 @@ +#!/bin/bash -e +# +# summary of how this script can be called: +# * <new-preinst> install +# * <new-preinst> install <old-version> +# * <new-preinst> upgrade <old-version> +# * <old-preinst> abort-upgrade <new-version> +# + +. /usr/share/debconf/confmodule + +if [ -n "$DEBIAN_SCRIPT_DEBUG" ]; then set -v -x; DEBIAN_SCRIPT_TRACE=1; fi +${DEBIAN_SCRIPT_TRACE:+ echo "#42#DEBUG# RUNNING $0 $*" 1>&2 } + +export PATH=$PATH:/sbin:/usr/sbin:/bin:/usr/bin +MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf" +DATADIR=/var/lib/mysql +LOGDIR=/var/log/mysql +UPGRADEDIR=/var/lib/mysql-upgrade + +# Try to stop the server in a sane way. If it does not success let the admin +# do it himself. No database directories should be removed while the server +# is running! Another mysqld in e.g. a different chroot is fine for us. +stop_server() { + if [ ! -x /etc/init.d/mysql ]; then return; fi + + set +e + if [ -x /usr/sbin/invoke-rc.d ]; then + cmd="invoke-rc.d mysql stop" + else + cmd="/etc/init.d/mysql stop" + fi + $cmd + errno=$? + set -e + + # 0=ok, 100=no init script (fresh install) + if [ "$errno" != 0 -a "$errno" != 100 ]; then + echo "${cmd/ */} returned $errno" 1>&2 + echo "There is a MySQL server running, but we failed in our attempts to stop it." 1>&2 + echo "Stop it yourself and try again!" 1>&2 + db_stop + exit 1 + fi +} + +################################ main() ########################## + +this_version=5.1 + +# Check kernel version +if dpkg --compare-versions `uname -r` lt 2.6; then + /bin/echo -e "\nPROBLEM: MySQL-5.x is currently incompatible with kernel 2.4. Aborting."; + /bin/echo -e "See http://bugs.debian.org/416841 for more information.\n" + exit 1 +fi + +# Abort if an NDB cluster is in use. +if egrep -q -r '^[^#]*ndb.connectstring' /etc/mysql/; then + db_fset percona-xtradb-server/no_upgrade_when_using_ndb seen false || true + db_input high percona-xtradb-server/no_upgrade_when_using_ndb || true + db_go + db_stop + exit 1 +fi + +# Safe the user from stupidities. +show_downgrade_warning=0 +for i in `ls $DATADIR/debian-*.flag 2>/dev/null`; do + found_version=`echo $i | sed 's/.*debian-$[0-9\.]\+$.flag/\1/'` + if dpkg --compare-versions "$this_version" '<<' "$found_version"; then + show_downgrade_warning=1 + break; + fi +done +if [ "$show_downgrade_warning" = 1 ]; then + db_fset percona-xtradb-server-$this_version/really_downgrade seen false || true + db_input medium percona-xtradb-server-$this_version/really_downgrade || true + db_go + db_get percona-xtradb-server-$this_version/really_downgrade || true + if [ "$RET" = "true" ]; then + rm -f $DATADIR/debian-*.flag + touch $DATADIR/debian-$this_version.flag + else + echo "Aborting downgrade from (at least) $found_version to $this_version." 1>&2 + echo "If are sure you want to downgrade to $this_version, remove the file" 1>&2 + echo "$DATADIR/debian-*.flag and try installing again." 1>&2 + db_stop + exit 1 + fi +fi + +# to be sure +stop_server + +# If we use NIS then errors should be tolerated. It's up to the +# user to ensure that the mysql user is correctly setup. +# Beware that there are two ypwhich one of them needs the 2>/dev/null! +if test -n "`which ypwhich 2>/dev/null`" && ypwhich >/dev/null 2>&1; then + set +e +fi + +# +# Now we have to ensure the following state: +# /etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false +# /etc/group: mysql:x:101: +# +# Sadly there could any state be present on the system so we have to +# modify everything carefully i.e. not doing a chown before creating +# the user etc... +# + +# creating mysql group if he isn't already there +if ! getent group mysql >/dev/null; then + # Adding system group: mysql. + addgroup --system mysql >/dev/null +fi + +# creating mysql user if he isn't already there +if ! getent passwd mysql >/dev/null; then + # Adding system user: mysql. + adduser \ + --system \ + --disabled-login \ + --ingroup mysql \ + --home $DATADIR \ + --gecos "MySQL Server" \ + --shell /bin/false \ + mysql >/dev/null +fi + +# end of NIS tolerance zone +set -e + +# if there's a symlink, let's store where it's pointing, because otherwise +# it's going to be lost in some situations +for dir in DATADIR LOGDIR; do + checkdir=`eval echo "$"$dir` + if [ -L "$checkdir" ]; then + mkdir -p "$UPGRADEDIR" + cp -d "$checkdir" "$UPGRADEDIR/$dir.link" + fi +done + +# creating mysql home directory +if [ ! -d $DATADIR -a ! -L $DATADIR ]; then + mkdir $DATADIR +fi + +# checking disc space +if LC_ALL=C BLOCKSIZE= df --portability $DATADIR/. | tail -n 1 | awk '{ exit ($4>1000) }'; then + echo "ERROR: There's not enough space in $DATADIR/" 1>&2 + db_stop + exit 1 +fi + +# Since the home directory was created before putting the user into +# the mysql group and moreover we cannot guarantee that the +# permissions were correctly *before* calling this script, we fix them now. +# In case we use NIS and no mysql user is present then this script should +# better fail now than later.. +# The "set +e" is necessary as e.g. a ".journal" of a ext3 partition is +# not chgrp'able (#318435). +set +e +chown mysql:mysql $DATADIR +find $DATADIR -follow -not -group mysql -print0 2>/dev/null \ + | xargs -0 --no-run-if-empty chgrp mysql +set -e + +# Some files below /etc/ were possibly in the percona-xtradb-server-5.0/etch package +# before. They get overwritten by current ones to avoid unnecessary dpkg questions. +while read md5 file; do + if [ "`md5sum $file 2>/dev/null`" = "$md5 $file" ]; then + cp /usr/share/percona-xtradb-common/internal-use-only/`echo $file | sed 's�/�_�g'` $file + fi +done <<EOT +6691f2fdc5c6d27ff0260eb79813e1bc /etc/init.d/mysql +b53b9552d44661361d39157c3c7c51d3 /etc/logrotate.d/percona-xtradb-server +57f3e58f72582ca55100dc1ba0f1a8ae /etc/mysql/debian-start +EOT + +db_stop + +#DEBHELPER# + +exit 0 === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.prerm' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.prerm 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.prerm 2010-02-18 23:53:03 +0000 @@ -0,0 +1,8 @@ +#!/bin/bash -e + +. /usr/share/debconf/confmodule + +if [ -n "$DEBIAN_SCRIPT_DEBUG" ]; then set -v -x; DEBIAN_SCRIPT_TRACE=1; fi +${DEBIAN_SCRIPT_TRACE:+ echo "#42#DEBUG# RUNNING $0 $*" 1>&2 } + +#DEBHELPER# === added file 'storage/xtradb/build/debian/percona-xtradb-server-5.1.templates' --- a/storage/xtradb/build/debian/percona-xtradb-server-5.1.templates 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/percona-xtradb-server-5.1.templates 2010-02-18 23:53:03 +0000 @@ -0,0 +1,90 @@ +# These templates have been reviewed by the debian-l10n-english +# team +# +# If modifications/additions/rewording are needed, please ask +# for an advice to debian-l10n-english(a)lists.debian.org +# +# Even minor modifications require translation updates and such +# changes should be coordinated with translators and reviewers. + +Template: percona-xtradb-server-5.1/really_downgrade +Type: boolean +Default: false +_Description: Really proceed with downgrade? + A file named /var/lib/mysql/debian-*.flag exists on this system. + . + Such file is an indication that a percona-xtradb-server package with a higher + version has been installed earlier. + . + There is no guarantee that the version you're currently installing + will be able to use the current databases. + +Template: percona-xtradb-server-5.1/nis_warning +Type: note +#flag:translate!:3,5 +_Description: Important note for NIS/YP users + To use MySQL, the following entries for users and groups should be added + to the system: + . + /etc/passwd : mysql:x:100:101:Percona SQL Server:/var/lib/mysql:/bin/false + /etc/group : mysql:x:101: + . + You should also check the permissions and the owner of the + /var/lib/mysql directory: + . + /var/lib/mysql: drwxr-xr-x mysql mysql + +Template: percona-xtradb-server-5.1/postrm_remove_databases +Type: boolean +Default: false +_Description: Remove all Percona SQL databases? + The /var/lib/mysql directory which contains the Percona SQL databases is about + to be removed. + . + If you're removing the Percona SQL package in order to later install a more + recent version or if a different percona-xtradb-server package is already + using it, the data should be kept. + +Template: percona-xtradb-server-5.1/start_on_boot +Type: boolean +Default: true +_Description: Start the Percona SQL server on boot? + The Percona SQL server can be launched automatically at boot time or manually + with the '/etc/init.d/mysql start' command. + +Template: percona-xtradb-server/root_password +Type: password +_Description: New password for the Percona SQL "root" user: + While not mandatory, it is highly recommended that you set a password + for the Percona SQL administrative "root" user. + . + If that field is left blank, the password will not be changed. + +Template: percona-xtradb-server/root_password_again +Type: password +_Description: Repeat password for the Percona SQL "root" user: + +Template: percona-xtradb-server/error_setting_password +Type: error +_Description: Unable to set password for the Percona SQL "root" user + An error occurred while setting the password for the Percona SQL + administrative user. This may have happened because the account + already has a password, or because of a communication problem with + the Percona SQL server. + . + You should check the account's password after the package installation. + . + Please read the /usr/share/doc/percona-xtradb-server-5.1/README.Debian file + for more information. + +Template: percona-xtradb-server/password_mismatch +Type: error +_Description: Password input error + The two passwords you entered were not the same. Please try again. + +Template: percona-xtradb-server/no_upgrade_when_using_ndb +Type: error +_Description: NDB Cluster seems to be in use + Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new + mysql-cluster package and remove all lines starting with "ndb" from + all config files below /etc/mysql/. === added directory 'storage/xtradb/build/debian/po' === added file 'storage/xtradb/build/debian/po/POTFILES.in' --- a/storage/xtradb/build/debian/po/POTFILES.in 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/POTFILES.in 2010-02-18 23:51:40 +0000 @@ -0,0 +1 @@ +[type: gettext/rfc822deb] percona-xtradb-server-5.1.templates === added file 'storage/xtradb/build/debian/po/ar.po' --- a/storage/xtradb/build/debian/po/ar.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/ar.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,267 @@ +# translation of templates.po to Arabic +# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER +# This file is distributed under the same license as the PACKAGE package. +# +# Ossama M. Khayat <okhayat(a)yahoo.com>, 2007. +msgid "" +msgstr "" +"Project-Id-Version: templates\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-05-01 13:04+0300\n" +"Last-Translator: Ossama M. Khayat <okhayat(a)yahoo.com>\n" +"Language-Team: Arabic <support(a)arabeyes.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"X-Generator: KBabel 1.11.4\n" +"Plural-Forms: nplurals=6; plural=n==1 ? 0 : n==0 ? 1 : n==2 ? 2: n%100>=3 && " +"n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5\n" +": n%100>=3 && n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5\n" +": n%100>=3 && n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5\n" +": n%100>=3 && n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5\n" +": n%100>=3 && n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5\n" +": n%100>=3 && n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5\n" +": n%100>=3 && n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "هل فعلاً تريد التثبيط؟" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "هناك ملف مسمى /var/lib/mysql/debian-*.flag موجود على هذا النظام." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"هذا الملف دلالة على أن نسخة أحدث من حزمة mysql-server تم تثبيتها مسبقاً." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"ليست هناك أية ضمانة أن النسخة التي تقوم بتثبيتها ستكون قادرة على استخدام " +"قواعد البيانات الحالية." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "ملاحظة هامة لمستخدمي NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"كي تستخدم MySQL، يجب إضافة المُدخلات التالية الخاصة بالمستخدمين والمجموعات " +"إلى النظام:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "عليك أيضاً أن تقوم بالتأكد من صلاحيات مالك الملف /var/lib/mysql: " + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "إزالة جميع قواعد بيانات MySQL؟" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "الدليل /var/lib/mysql الذي يحتوي قواعد بيانات MySQL ستتم إزالته." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"إن كنت تقوم بإزالة حزمة MySQL كي تقوم لاحقاً بتثبيت نسخة أحدث أو إن كانت حزمة " +"mysql-server مختلفة تستخدمها، فيجب إبقاء البيانات." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "تشغيل خادم MySQL عند الإقلاع؟" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"يمكن تشغيل خادم MySQL آلياً وقت الإقلاع أو يدوياً باستخدام الأمر '/etc/init.d/" +"mysql start'." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "كلمة المرور الجديدة لمستخد \"root\" الخاص بـMySQL:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"مع أنه ليس إجبارياً، ولكن من المستحسن أن تقوم بتعيين كلمة مرور خاصة بمستخدم " +"MySQL الإداري \"root\"." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "إن ترك الحقل فارغاً، فلن يتم تغيير كلمة المرور." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "كلمة المرور الجديدة لمستخد \"root\" الخاص بـMySQL:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "تعذر تعيين كلمة مرور للمستخدم \"root\" الخاص بـMySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"حدث خطأ أثناء تعيين كلمة المرور لمستخدم MySQL الإداري. قد يكون هذا حدث بسبب " +"أن حساب المستخدم له كلمة مرور معيّنة مسبقاً، أو بسبب مشكلة في الاتصال مع خادم " +"MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "You should check the account's password after tha package installation." +msgid "You should check the account's password after the package installation." +msgstr "يجب عليك التحقق من كلمة مرور الحساب عقب تثبيت الحزمة." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"الرجاء قراءة الملف /usr/share/doc/mysql-server-5.1/README.Debian للمزيد من " +"المعلومات." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "هل تريد دعم اتصالات MySQL من الأجهزة التي تعمل على ديبيان \"sarge\" أو " +#~ "أقدم؟" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "في إصدارات عملاء MySQL القديمة من ديبيان، لم تكن كلمات المرور تحفظ بشكل " +#~ "آمن. ولقد حل هذه المشكلة بعدها، غير أن العملاء (مثل PHP) المتصلين من " +#~ "أجهزة تعمل على ديبيان Sarge 3.1 لن يكونوا قادرين على الاتصال باستخدام " +#~ "الحسابات الحديثة أو الحسابات التي تم تغيير كلمة مرورها." === added file 'storage/xtradb/build/debian/po/ca.po' --- a/storage/xtradb/build/debian/po/ca.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/ca.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,342 @@ +# mysql-dfsg (debconf) translation to Catalan. +# Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004 Free Software Foundation, Inc. +# Aleix Badia i Bosch <abadia(a)ica.es> 2004 +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-4.1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2004-01-31 19:20GMT\n" +"Last-Translator: Aleix Badia i Bosch <abadia(a)ica.es>\n" +"Language-Team: Debian L10n Catalan <debian-l10n-catalan(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=ISO-8859-1\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "Nota important pels usuaris de NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Voleu que el MySQL s'inici� l'arrencada ?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"El MySQL es pot executar a l'arrencada o nom�si executeu manualment '/etc/" +"init.d/mysql start'. Seleccioneu 's�si voleu que s'inicialitzi " +"autom�cament." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "New password for the Percona SQL \"root\" user:" +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#, fuzzy +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Per utilitzar la base de dades de MySQL heu d'afegir un usuari i grup " +#~ "equivalent al seg� assegurar-vos que el directori /var/lib/mysql " +#~ "tingui els permisos correctes." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#, fuzzy +#~ msgid "Please also read http://www.mysql.com/doc/en/Upgrade.html" +#~ msgstr "" +#~ "Feu una ullada al document: http://www.mysql.com/doc/en/Upgrade.html" + +#, fuzzy +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "El MySQL nom�s'instal�la en cas de tenir un nom d'ordinador central que " +#~ "no sigui num�c i que es pugui resoldre a trav�del fitxer /etc/hosts. " +#~ "Ex. si l'ordre \"hostname\" retorna \"myhostname\", llavors hi ha d'haver " +#~ "una l�a com la seg�"10.0.0.1 myhostname\"." + +#, fuzzy +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "Es crea un nou usuari de mysql \"debian-sys-maint\". S'utilitza per les " +#~ "seq�s d'inicialitzaci�aturada del cron, no el suprimiu." + +#, fuzzy +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "Recordeu posar una contrasenya al superusuari del MySQL. Si utilitzeu un " +#~ "fitxer /root/.my.cnf, escriviu sempre all�es l�es \"user\" i " +#~ "\"password\".; mai nom�la contrasenya. Per a m�informaci�u una " +#~ "ullada a /usr/share/doc/mysql-server/README.Debian." + +#, fuzzy +#~ msgid "" +#~ "Should I remove all databases below /var/lib/mysql as you are purging the " +#~ "mysql-server package?" +#~ msgstr "" +#~ "Voleu suprimir totes les bases de dades en purgar el paquet mysql-server ?" + +#~ msgid "" +#~ "Networking is disabled by default for security reasons. You can enable it " +#~ "by commenting out the skip-networking option in /etc/mysql/my.cnf." +#~ msgstr "" +#~ "La xarxa est�nhabilitada per defecte per a raons de seguretat. La podeu " +#~ "habilitar descomentant l'opci� skip-networking del fitxer /etc/mysql/" +#~ "my.cnf." + +#~ msgid "security and update notice" +#~ msgstr "Av�de seguretat i actualitzaci� +#~ msgid "Please run mysql_fix_privilege_tables !" +#~ msgstr "Executeu mysql_fix_privilege_tables" + +#~ msgid "" +#~ "I will ensure secure permissions of /var/lib/mysql by replacing GIDs " +#~ "other than root and mysql with mysql." +#~ msgstr "" +#~ "S'asseguren els permisos de seguretat de /var/lib/mysql canviant a mysql " +#~ "tots els GIDs diferents a root i mysql." + +#~ msgid "" +#~ "Instructions how to enable SSL support are in /usr/share/doc/mysql-server/" +#~ msgstr "" +#~ "Per habilitar el suport de SSL podeu seguir les instruccions de /usr/" +#~ "share/doc/mysql-server/" + +#~ msgid "mysql_fix_privileges_tables will be executed" +#~ msgstr "s'executa mysql_fix_privileges_tables" + +#~ msgid "" +#~ "The latest MySQL versions have an enhanced, more fine grained, privilege " +#~ "system. To make use of it, some new fields must be added to the tables " +#~ "in the \"mysql\" database. This is done by the " +#~ "mysql_fix_privilege_tables script during this upgrade regardless of if " +#~ "the server is currently running or not!" +#~ msgstr "" +#~ "Les �es versions de MySQL tenen un sistema de privilegis m�" +#~ "elaborat. Per utilitzar-lo cal afegir nous camps a les taules de la base " +#~ "de dades \"mysql\". Aquesta tasca la realitza la seq� " +#~ "mysql_fix_privilege_tables durant l'actualitzaci�dependentment de si " +#~ "el servidor s'est�xecutant o no!" + +#~ msgid "" +#~ "This script is not supposed to give any user more rights that he had " +#~ "before, if you encounter such a case, please contact me." +#~ msgstr "" +#~ "Aquesta seq� no assigna privilegis d'usuari diferents als que ja " +#~ "tenia, en cas que us trob�iu en aquesta situaci�oseu-vos en contacte " +#~ "amb mi." + +#~ msgid "" +#~ "Should I remove everything below /var/lib/mysql when you purge the mysql-" +#~ "server package with the \"dpkg --purge mysql-server\" command (i.e. " +#~ "remove everything including the configuration) somewhen? (default is not)" +#~ msgstr "" +#~ "Voleu suprimir tots els continguts de /var/lib/mysql quan es purgui el " +#~ "paquet mysql-server amb l'ordre \"dpkg --purge mysql-server\". (ex. " +#~ "suprimir-ho tot incl�a configuraci� (per defecte no)" + +#~ msgid "Make MySQL reachable via network?" +#~ msgstr "Voleu fer accessible el MySQL via xarxa ?" + +#~ msgid "" +#~ "Should MySQL listen on a network reachable TCP port? This is not " +#~ "necessary for use on a single computer and could be a security problem." +#~ msgstr "" +#~ "Voleu que el MySQL escolti a un port TCP accessible des de la xarxa ? " +#~ "Aquesta opci� �imprescindible en ordinadors a�ats i podria " +#~ "provocar un problema de seguretat." + +#~ msgid "Enable chroot mode?" +#~ msgstr "Permetre el mode chroot ?" + +#~ msgid "" +#~ "MySQL is able to jail itself into the /var/lib/mysql_jail directory so " +#~ "that users cannot modify any files outside this directory. This improves " +#~ "resistence against crackers, too, as they are not able to modify system " +#~ "files." +#~ msgstr "" +#~ "El MySQL es pot executar en una entorn tancat al directori /var/lib/" +#~ "mysql_jail perqu�ls usuaris no puguin modificar cap fitxer fora del " +#~ "directori.Aquesta opci�mb�ugmenta la seguretat envers els crackers, " +#~ "jaque no poden modificar els fitxers del sistema." === added file 'storage/xtradb/build/debian/po/cs.po' --- a/storage/xtradb/build/debian/po/cs.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/cs.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,361 @@ +# +# Translators, if you are not familiar with the PO format, gettext +# documentation is worth reading, especially sections dedicated to +# this format, e.g. by running: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry' +# +# Some information specific to po-debconf are available at +# /usr/share/doc/po-debconf/README-trans +# or http://www.debian.org/intl/l10n/po-debconf/README-trans +# +# Developers do not need to manually edit POT or PO files. +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-05-01 13:01+0200\n" +"Last-Translator: Miroslav Kure <kurem(a)debian.cz>\n" +"Language-Team: Czech <debian-l10n-czech(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "Opravdu pokračovat v degradaci?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "V systému existuje soubor /var/lib/mysql/debian-*.flag." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "To znamená, že již byl nainstalován balík mysql-server s vyšší verzí." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"Neexistuje žádná záruka, že momentálně instalovaná verze bude umět pracovat " +"se stávajícími databázemi." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Důležitá poznámka pro uživatele NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Abyste mohli MySQL používat, musíte v systému založit následující uživatele " +"a skupiny:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Také byste měli zkontrolovat vlastníka a oprávnění adresáře /var/lib/mysql:" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "Odstranit všechny MySQL databáze?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"Adresář /var/lib/mysql, ve kterém se nachází MySQL databáze, bude odstraněn." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Jestliže odstraňujete balík MySQL za účelem instalace novější verze MySQL, " +"nebo pokud tato data souběžně využívá jiný balík mysql-server, měli byste " +"data ponechat." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Spustit MySQL server při startu systému?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL se může spouštět automaticky při startu systému, nebo ručně příkazem '/" +"etc/init.d/mysql start'." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nové heslo MySQL uživatele \"root\":" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Přestože to není nezbytné, je silně doporučeno nastavit heslo u " +"správcovského MySQL účtu \"root\"." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "Ponecháte-li pole prázdné, heslo se nezmění." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nové heslo MySQL uživatele \"root\":" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Nelze nastavit heslo MySQL uživatele \"root\"" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Během nastavování hesla pro správcovského uživatele MySQL se vyskytla chyba. " +"To se mohlo stát třeba proto, protože uživatel již měl heslo nastaveno, nebo " +"protože nastal problém v komunikaci s MySQL serverem." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "You should check the account's password after tha package installation." +msgid "You should check the account's password after the package installation." +msgstr "Po instalaci balíku byste měli heslo ověřit." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Více informací naleznete v /usr/share/doc/mysql-server-5.1/README.Debian." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "Cannot upgrade if ISAM tables are present!" +#~ msgstr "Aktualizace nelze provést pokud jsou přítomny tabulky ISAM!" + +#~ msgid "" +#~ "Recent versions of MySQL can no longer use the old ISAM table format and " +#~ "it is necessary to convert your tables to e.g. MyISAM before upgrading by " +#~ "using \"mysql_convert_table_format\" or \"ALTER TABLE x ENGINE=MyISAM\". " +#~ "The installation of mysql-server-5.1 will now abort. In case your old " +#~ "mysql-server-4.1 gets removed nevertheless just reinstall it to convert " +#~ "those tables." +#~ msgstr "" +#~ "Poslední verze MySQL již nemohou používat starý formát tabulek ISAM a " +#~ "před aktualizací je nutné převést tyto tabulky např. do formátu MyISAM " +#~ "pomocí \"mysql_convert_table_format\" nebo \"ALTER TABLE x ENGINE=MyISAM" +#~ "\". Instalace mysql-server-5.1 se nyní přeruší. V případě, že se mezitím " +#~ "odinstaloval původní mysql-server-4.1, jednoduše jej znovu nainstalujte a " +#~ "tabulky převeďte." + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Podporovat MySQL připojení z počítačů používajících Debian Sarge nebo " +#~ "starší?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Způsob, jakým se dříve ukládala hesla, nebyl příliš bezpečný. To se nyní " +#~ "zlepšilo, ale nevýhodou je, že se klienti z Debianu 3.1 Sarge (např. PHP) " +#~ "nebudou moci připojit na nové účty, nebo na účty, u nichž se heslo " +#~ "změnilo." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Abyste mohli mysql používat, musíte do následujících souborů přidat " +#~ "ekvivalentního uživatele a skupinu a zajistit, že /var/lib/mysql má " +#~ "správná práva (uid/gid se mohou lišit)." + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Odstranit databáze používané všemi verzemi MySQL?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "Nezadáte-li heslo, žádné změny se s účtem neprovedou." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Po skončení instalace byste měli ověřit, že je účet chráněn heslem (více " +#~ "informací naleznete v souboru README.Debian)." + +#~ msgid "Update Hints" +#~ msgstr "Poznámky k aktualizaci" + +#~ msgid "" +#~ "You have to run \"mysql_upgrade\" after the upgrade, else tables can be " +#~ "corrupted! This script also enhances the privilege tables but is not " +#~ "supposed to give any user more rights that he had before," +#~ msgstr "" +#~ "Po aktualizaci ještě musíte spustit \"mysql_upgrade\", protože jinak by " +#~ "se tabulky mohly narušit! Tento skript také rozšiřuje tabulky privilegií, " +#~ "ovšem neměl by uživatelům přidat více práv, než měli dosud." + +#~ msgid "Please also read http://www.mysql.com/doc/en/Upgrade.html" +#~ msgstr "Také si přečtěte http://www.mysql.com/doc/en/Upgrade.html" + +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "MySQL se nainstaluje pouze v případě, že používáte nenumerické jméno " +#~ "počítače, které se dá přeložit přes soubor /etc/hosts. Např. když příkaz " +#~ "\"hostname\" vrátí \"diamond\", tak v /etc/hosts musí existovat obdobný " +#~ "řádek jako \"10.0.0.1 diamond\"." + +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "Bude vytvořen nový mysql uživatel \"debian-sys-maint\". Tento mysql účet " +#~ "se používá ve startovacích, ukončovacích a cronových skriptech. Nemažte " +#~ "jej." + +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "Nezapomeňte nastavit heslo pro účet administrátora MySQL! Používáte-li /" +#~ "root/.my.cnf, vždy zde zadejte jak řádek \"user\", tak řádek \"password" +#~ "\". Nikdy zde nezadávejte jenom heslo!" + +#~ msgid "" +#~ "Should I remove the complete /var/lib/mysql directory tree which is used " +#~ "by all MySQL versions, not necessarily only the one you are about to " +#~ "purge?" +#~ msgstr "" +#~ "Mám odstranit kompletní adresářový strom /var/lib/mysql, který se používá " +#~ "pro všechny verze MySQL, tedy ne nutně pouze pro verzi, kterou se " +#~ "chystáte vyčistit?" === added file 'storage/xtradb/build/debian/po/da.po' --- a/storage/xtradb/build/debian/po/da.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/da.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,397 @@ +# +# Translators, if you are not familiar with the PO format, gettext +# documentation is worth reading, especially sections dedicated to +# this format, e.g. by running: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry' +# Some information specific to po-debconf are available at +# /usr/share/doc/po-debconf/README-trans +# or http://www.debian.org/intl/l10n/po-debconf/README-trans# +# Developers do not need to manually edit POT or PO files. +# +# Claus Hindsgaul <claus_h(a)image.dk>, 2005, 2006. +# Claus Hindsgaul <claus.hindsgaul(a)gmail.com>, 2006, 2007. +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-4.1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-05-30 22:41+0200\n" +"Last-Translator: Claus Hindsgaul <claus.hindsgaul(a)gmail.com>\n" +"Language-Team: Danish\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=ISO-8859-1\n" +"Content-Transfer-Encoding: 8bit\n" +"X-Generator: KBabel 1.11.4\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "�sker du virkelig at forts�e nedgraderingen?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" +"Der er en fil med navnet /var/lib/mysql/debian-*.flag p�ette system." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"S�n en fil tyder p�t der tidligere har v�t installeret en h� " +"version af mysql-server-pakken." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"Det kan ikke garanteres at den version, du er ved at installere, kan benytte " +"data fra de eksisterende databaser." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Vigtig oplysning til NIS/YP-brugere" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Nedenst�de linjer for brugere og grupper skal tilf�dette system for at " +"benytte MySQL:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Du b�s�jekke filrettighederne og ejerskabet af mappen /var/lib/mysql:" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "Fjern alle MySQL-databaser?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"Mappen /var/lib/mysql, der indeholder MySQL-databaserne, er ved at blive " +"fjernet." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Hvis du fjerner MySQL-pakken for senere at installere en nyere version, " +"eller hvis en anden mysql-server-pakke allerede benytter den, b�taene " +"bevares." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Start MySQL-serveren under systemopstart?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL-serveren kan enten startes op automatisk under systemopstarten, eller " +"manuelt med kommandoen '/etc/init.d/mysql start'." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Ny adgangskode for MySQL's \"root\"-bruger:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Selvom det ikke kr�s, anbefales det kraftigt, at du s�er en adgangskode " +"for MySQL's administrationsbruger \"root\"." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "Hvis du lader dette felt st�omt, vil adgangskoden ikke blive �ret." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Ny adgangskode for MySQL's \"root\"-bruger:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Kunne ikke s�e adgangskoden for MySQL's \"root\"-bruger" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Der opstod en fejl, da adgangskoden for MySQL's administrationsbruger blev " +"fors�ndret. Dette kan v� sket, fordi brugeren allerede har en " +"adgangskode, eller fordi der var problemer med at kommunikere med MySQL-" +"serveren." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "Du b�ekke kontoens adgangskode efter pakkeinstallationen." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Se filen /usr/share/doc/mysql-server-5.1/README.Debian for yderligere " +"oplysninger." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "Cannot upgrade if ISAM tables are present!" +#~ msgstr "Kan ikke opgradere hvis der er ISAM-tabeller!" + +#~ msgid "" +#~ "Recent versions of MySQL can no longer use the old ISAM table format and " +#~ "it is necessary to convert your tables to e.g. MyISAM before upgrading by " +#~ "using \"mysql_convert_table_format\" or \"ALTER TABLE x ENGINE=MyISAM\". " +#~ "The installation of mysql-server-5.1 will now abort. In case your old " +#~ "mysql-server-4.1 gets removed nevertheless just reinstall it to convert " +#~ "those tables." +#~ msgstr "" +#~ "Nyere versioner af MySQL kan ikke l�ere benytte det gamle ISAM-" +#~ "tabelformat, og det er derfor n�digt at konvertere dine tabeller til " +#~ "f.eks. MyISAM forud for opgraderingen med \"mysql_convert_table_format\" " +#~ "eller \"ALTER TABLE x ENGINE=MyISAM\". Installationen af mysql-server-5.1 " +#~ "afbrydes nu. Skulle din gamle mysql-server-4.1 alligevel bliver " +#~ "afinstalleret, s�eninstall�den blot og konverter tabellerne." + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Underst�SQL-forbindelser fra maskiner, der k�Debian \"Sarge\" " +#~ "eller �re?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Gamle udgaver af MySQL-klienter p�ebian gemte ikke adgangskoderne " +#~ "sikkert. Dette er blevet forbedret siden da, men klienter (f.eks. PHP) " +#~ "fra maskiner, der k�Debian 3.1 Sarge vil ikke kunne forbinde til " +#~ "nyere konti eller konti, hvis adgangskode er blevet �ret." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "For at kunne bruge mysql skal du installere en bruger og en gruppe, der " +#~ "svarer til nedenst�de, og sikre dig at /var/lib/mysql har de rigtige " +#~ "adgangsrettigheder (uid/gid kan afvige)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Fjern de databaser, der benyttes af samtlige MySQL-versioner?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "Hvis du ikke angiver en adgangskode, vil kontoen ikke blive �ret." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "N�installationen afsluttes, b� tjekke at kontoen er ordentligt " +#~ "beskyttet med en adgangskode (se README.Debian for yderligere " +#~ "oplysninger)." + +#~ msgid "Update Hints" +#~ msgstr "Opdateringstips" + +#~ msgid "" +#~ "You have to run \"mysql_upgrade\" after the upgrade, else tables can be " +#~ "corrupted! This script also enhances the privilege tables but is not " +#~ "supposed to give any user more rights that he had before," +#~ msgstr "" +#~ "Du skal k�"mysql_upgrade\" efter opgraderingen, da tabellerne eller " +#~ "kan blive �gt! Dette script forbedrer ogs�ettighedstabellerne, men " +#~ "burde ikke give nogen bruger flere rettigheder, end han havde tidligere," + +#~ msgid "Please also read http://www.mysql.com/doc/en/Upgrade.html" +#~ msgstr "L�ogs�ttp://www.mysql.com/doc/en/Upgrade.html" + +#~ msgid "Install Hints" +#~ msgstr "Installationstips" + +#~ msgid "" +#~ "On upgrades from MySQL 3.23, as shipped with Debian Woody, symlinks in " +#~ "place of /var/lib/mysql or /var/log/mysql gets accidently removed and " +#~ "have manually be restored." +#~ msgstr "" +#~ "Ved opgraderinger fra MySQL 3.23, der fulgte med Debian Woody, kan de " +#~ "symbolske /var/lib/mysql or /var/log/mysql blive fjernet ved et uheld, og " +#~ "m�enskabes manuelt." + +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "MySQL vil kun blive installeret, hvis du har et ikke-numerisk v�snavn, " +#~ "som kan sl�op i filen /ets/hosts. Hvis f.eks. kommandoen \"hostname\" " +#~ "svarer med \"mitvaertsnavn\", skal du have en linje a'la \"10.0.0.1 " +#~ "mitvaertsnavn\" i /etc/hosts." + +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "Det vil blive oprettet en ny mysql-bruger, \"debian-sys-maint\". Denne " +#~ "mysql-konto bruges i start/stop-cron-scripterne. Slet den ikke." + +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "Husk at s�e en ADGANGSKODE for MySQLs root-bruger! Hvis du bruger en /" +#~ "etc/.my.cnf, s�kriv altid \"user\"- og \"password\"-linjer ind her, " +#~ "ikke kun adgangskoden!" + +#~ msgid "" +#~ "Should I remove the complete /var/lib/mysql directory tree which is used " +#~ "by all MySQL versions, not necessarily only the one you are about to " +#~ "purge?" +#~ msgstr "" +#~ "Skal jeg fjerne hele mappetr� /var/lib/mysql, som benyttes af alle " +#~ "MySQL-versioner, ikke kun den version, du er ved at slette?" + +#~ msgid "" +#~ "Rarely, e.g. on new major versions, the privilege system is improved. To " +#~ "make use of it mysql_fix_privilege_tables must be executed manually. The " +#~ "script is not supposed to give any user more rights that he had before," +#~ msgstr "" +#~ "En sj�en gang imellem, f.eks. ved nye hovedversioner, sker det at " +#~ "rettighedssystemet forbedres. For at g�rug af dette, skal " +#~ "mysql_fix_privilege_tables k�manuelt. Scriptet vil ikke give nogen " +#~ "bruger flere rettigheder, end vedkommende havde tidligere," === added file 'storage/xtradb/build/debian/po/de.po' --- a/storage/xtradb/build/debian/po/de.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/de.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,277 @@ +# translation of mysql-dfsg-5.1_5.0.41-2_de.po to german +# +# Translators, if you are not familiar with the PO format, gettext +# documentation is worth reading, especially sections dedicated to +# this format, e.g. by running: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry' +# Some information specific to po-debconf are available at +# /usr/share/doc/po-debconf/README-trans +# or http://www.debian.org/intl/l10n/po-debconf/README-trans# +# Developers do not need to manually edit POT or PO files. +# +# Alwin Meschede <ameschede(a)gmx.de>, 2006, 2007. +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1_5.0.41-2_de\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-05-29 16:05+0200\n" +"Last-Translator: Alwin Meschede <ameschede(a)gmx.de>\n" +"Language-Team: german <debian-l10n-german(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"X-Generator: KBabel 1.11.4\n" +"Plural-Forms: nplurals=2; plural=(n != 1);\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "Möchten Sie wirklich eine ältere Version einspielen?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" +"Auf diesem System existiert eine Datei mit dem Namen /var/lib/mysql/debian-*." +"flag" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Diese Datei ist ein Hinweis darauf, dass früher ein MySQL-Server-Paket mit " +"einer höheren Version installiert war." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"Es kann nicht garantiert werden, dass die gegenwärtig zu installierende " +"Version dessen Daten benutzen kann." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Wichtige Anmerkung für NIS/YP-Benutzer!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Um MySQL benutzen zu können, sollten die folgenden Benutzer und Gruppen dem " +"System hinzugefügt werden:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Sie sollten außerdem Besitzer und Zugriffsrechte des Verzeichnisses /var/lib/" +"mysql überprüfen:" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "Alle MySQL-Datenbanken entfernen?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"Das Verzeichnis /var/lib/mysql mit den MySQL-Datenbanken soll entfernt " +"werden." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Falls geplant ist, nur eine höhere Version von MySQL zu installieren oder " +"ein anderes mysql-server-Paket dieses bereits benutzt, sollten die Daten " +"behalten werden." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Soll MySQL automatisch beim Booten starten?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"Der MySQL-Dienst kann entweder automatisch beim Systemstart oder manuell " +"durch Eingabe des Befehls »/etc/init.d/mysql start« gestartet werden." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Neues Passwort für den MySQL »root«-Benutzer:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Obwohl es nicht zwingend erforderlich ist, wird nachdrücklich empfohlen für " +"den administrativen MySQL »root«-Benutzer ein Passwort zu setzen." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "Wenn dieses Feld freigelassen wird, wird das Passwort nicht geändert." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Neues Passwort für den MySQL »root«-Benutzer:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Konnte für den MySQL-»root«-Benutzer kein Passwort setzen" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Beim setzen des Passworts für den administrativen MySQL-Benutzer ist ein " +"Fehler aufgetreten. Dies könnte daran liegen, dass der Benutzer bereits ein " +"Passwort hat oder dass es ein Problem mit der Kommunikation mit dem MySQL-" +"Server gibt." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" +"Sie sollten das Passwort des administrativen Benutzers nach der " +"Paketinstallation prüfen." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Für weitere Informationen lesen Sie /usr/share/doc/mysql-server-5.1/README." +"Debian" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Sollen MySQL-Verbindungen von Rechnern mit Debian »Sarge« oder älter " +#~ "unterstützt werden?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Alte Versionen der MySQL-Clients für Debian speicherten Passwörter nicht " +#~ "sehr sicher. Dies wurde verbessert, allerdings werden Clients (z. B. PHP) " +#~ "von Hosts mit Debian 3.1 Sarge sich nicht mehr mit MySQL-Konten verbinden " +#~ "können, die neu angelegt werden oder deren Passwort geändert wird. Siehe " +#~ "auch /usr/share/doc/mysql-server-5.1/README.Debian." === added file 'storage/xtradb/build/debian/po/es.po' --- a/storage/xtradb/build/debian/po/es.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/es.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,405 @@ +# mysql-dfsg-5 translation to spanish +# Copyright (C) 2005-2007 Software in the Public Interest, SPI Inc. +# This file is distributed under the same license as the XXXX package. +# +# Changes: +# - Initial translation +# Jesus Aneiros, 2006 +# - Updated +# Javier Fernandez-Sanguino, 2006-2007 +# - Revision +# Nacho Barrientos Arias +# Fernando Cerezal +# David Martínez Moreno +# Ricardo Mones +# Carlos Galisteo +# Javier Fernandez-Sanguino +# +# +# Traductores, si no conoce el formato PO, merece la pena leer la +# documentación de gettext, especialmente las secciones dedicadas a este +# formato, por ejemplo ejecutando: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry' +# +# Equipo de traducción al español, por favor lean antes de traducir +# los siguientes documentos: +# +# - El proyecto de traducción de Debian al español +# http://www.debian.org/intl/spanish/ +# especialmente las notas y normas de traducción en +# http://www.debian.org/intl/spanish/notas +# +# - La guía de traducción de po's de debconf: +# /usr/share/doc/po-debconf/README-trans +# o http://www.debian.org/intl/l10n/po-debconf/README-trans +# +# Si tiene dudas o consultas sobre esta traducción consulte con el último +# traductor (campo Last-Translator) y ponga en copia a la lista de +# traducción de Debian al español (<debian-l10n-spanish(a)lists.debian.org>) +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1_5.0.24-3\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-05-28 22:21+0200\n" +"Last-Translator: Javier Fernández-Sanguino <jfs(a)debian.org>\n" +"Language-Team: Debian l10 Spanish <debian-l10n-spanish(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "¿Desea realmente continuar con la desactualización?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" +"Existe un archivo con el nombre /var/lib/mysql/debian-*.flag en este sistema." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Este fichero indica que se instaló previamente una versión superior del " +"paquete mysql-server." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"No se puede garantizar que la versión que está instalando pueda usar la base " +"de datos actual." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Nota importante para los usuarios de NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Deben añadirse las siguientes entradas para usuarios y grupos en el sistema " +"para poder utilizar MySQL:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"También debería comprobar los permisos y el propietario del directorio /var/" +"lib/mysql:" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "¿Desea eliminar todas las bases de datos MySQL?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"El directorio /var/lib/mysql contiene bases de datos MySQL que van a " +"eliminarse." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Debería mantener los datos si tiene planificado instalar una versión de " +"MySQL más reciente o si hay un paquete «mysql-server» distinto que los está " +"utilizando." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "¿Debería ejecutarse el servidor MySQL al iniciarse el sistema?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"El servidor MySQL puede iniciarse en el momento de arranque del sistema o " +"manualmente si escribe la orden «/etc/init.d/mysql start»." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nueva contraseña para el usuario «root» de MySQL:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Se recomienda que configure una contraseña para el usuario " +"«root» (administrador) de MySQL, aunque no es obligatorio." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "No se modificará la contraseña si deja el espacio en blanco." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nueva contraseña para el usuario «root» de MySQL:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "No se pudo fijar la contraseña para el usuario «root» de MySQL" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Se produjo un error mientras intentaba fijar la contraseña para el usuario " +"administrador de MySQL. Esto puede haber sucedido porque la cuenta ya tenía " +"una contraseña o porque se produjo un error de comunicación con el servidor " +"MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" +"Debería comprobar la contraseña de la cuenta después de la instalación del " +"paquete." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Consulte /usr/share/doc/mysql-server-5.1/README.Debian para más información." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "Cannot upgrade if ISAM tables are present!" +#~ msgstr "¡No se puede actualizar si ya hay tablas ISAM!" + +#~ msgid "" +#~ "Recent versions of MySQL can no longer use the old ISAM table format and " +#~ "it is necessary to convert your tables to e.g. MyISAM before upgrading by " +#~ "using \"mysql_convert_table_format\" or \"ALTER TABLE x ENGINE=MyISAM\". " +#~ "The installation of mysql-server-5.1 will now abort. In case your old " +#~ "mysql-server-4.1 gets removed nevertheless just reinstall it to convert " +#~ "those tables." +#~ msgstr "" +#~ "Las versiones recientes de MySQL ya no soportan el antiguo formato de " +#~ "tabla ISAM. Antes de realizar la actualización es necesario convertir sus " +#~ "tablas a por ejemplo, MyISAM, usando «mysql_convert_table_format» o «ALTER " +#~ "TABLE x ENGINE=MyISAM». Se va a interrumpir ahora la instalación de mysql-" +#~ "server-5.1. Si aún así su mysql-server-4.1 se elimina aún así, puede " +#~ "reinstalarlo para convertir ese tipo de tablas." + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "¿Soportar las conexiones MySQL establecidadas desde sistemas que ejecutan " +#~ "Debian Sarge o versiones anteriores?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "No era muy segura la forma en la que se almacenaban las contraseñas en " +#~ "versiones anteriores del cliente de MySQL en Debian. Este problema se ha " +#~ "mejorado posteriormente con el inconveniente, sin embargo, de que " +#~ "clientes (por ejemplo, PHP) en sistemas que ejecutan Debian 3.1 «Sarge» no " +#~ "podrán conectarse a cuentas que son nuevas o a las que se le haya " +#~ "cambiado la contraseña." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Para utilizar mysql debe instalar un usuario y grupo equivalente al " +#~ "siguiente y asegurarse de que /var/lib/mysql tiene los permisos correctos " +#~ "(los valores del «uid» y del «gid» pueden ser diferentes)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "" +#~ "¿Eliminar las bases de datos utilizadas por todas las versiones de MySQL?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "No se hará ningún cambio en la cuenta si no introduce una contraseña." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Debería confirmar que la contraseña está correctamente protegida con una " +#~ "contraseña cuando termine la instalación (consulte el fichero README." +#~ "Debian si desea más información)." + +#~ msgid "Install Hints" +#~ msgstr "Sugerencias para la instalación" + +#~ msgid "" +#~ "On upgrades from MySQL 3.23, as shipped with Debian Woody, symlinks in " +#~ "place of /var/lib/mysql or /var/log/mysql gets accidently removed and " +#~ "have manually be restored." +#~ msgstr "" +#~ "Al actualizar a la versión de MySQL 3.23, la vrsión proporcionada en " +#~ "Debian Woody, se eliminan de manera accidental, los enlaces simbólicos a «/" +#~ "var/lib/mysql» o «/var/log/mysql» y tienen que restaurarse manualmente." + +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "Sólo se instalará MySQL si tiene un nombre de equipo que no sea una " +#~ "dirección IP y pueda resolverse a través del archivo /etc/hosts. Por " +#~ "ejemplo, si la orden «hostname» devuelve «MiNombreEquipo» entonces deberá " +#~ "existir una línea «10.0.0.1 MiNombreEquipo» en dicho archivo." + +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "Se creará un nuevo usuario «debian-sys-maint». Esta cuenta de mysql se " +#~ "utilizará en los scripts de inicio y parada y en los scripts «cron». No la " +#~ "elimine." + +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "¡Por favor, recuerde crear una CONTRASEÑA para el usuario «root» de MySQL! " +#~ "¡Si utiliza /root/.my.cnf debe escribir las líneas «user» y «password» en " +#~ "dicho archivo, no incluya sólo la contraseña!" + +#~ msgid "" +#~ "Should I remove the complete /var/lib/mysql directory tree which is used " +#~ "by all MySQL versions, not necessarily only the one you are about to " +#~ "purge?" +#~ msgstr "" +#~ "¿Debería eliminar el árbol de directorio /var/lib/mysql completo? Tenga " +#~ "en cuenta que lo utilizan todas las versiones de MySQL y no sólo la que " +#~ "está a punto de purgar." === added file 'storage/xtradb/build/debian/po/eu.po' --- a/storage/xtradb/build/debian/po/eu.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/eu.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,295 @@ +# SOME DESCRIPTIVE TITLE. +# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER +# This file is distributed under the same license as the PACKAGE package. +# Piarres BEobide <pi(a)beobide.net>, 2006. +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1_5.0.26-3-debconf_eu\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-02-19 09:33+0100\n" +"Last-Translator: Piarres Beobide <pi(a)beobide.net>\n" +"Language-Team: Euskara <Librezale(a)librezale.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Plural-Forms: nplurals=2; plural=(n != 1);\n" +"X-Generator: Pootle 0.10.1\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "Do you really want to downgrade?" +msgid "Really proceed with downgrade?" +msgstr "Benetan bertsio zaharragora itzuli nahi duzu?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "" +#| "WARNING: The file /var/lib/mysql/debian-*.flag exists. This indicates " +#| "that a mysql-server package with a higher version has been installed " +#| "before. It can not be guaranteed that this version can use its data." +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Oharra: /var/lib/mysql/debian-*.flag dago.. Honek aurretik bertsio " +"berriagoko mysql-zerbitzari bat instalatu dela adierazten du. Ezin da " +"ziurtatu bertsio honek datu horiek erabili ahal izango dituenik." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "NIS/YP erabiltzaileentzat ohar garrantzitsua!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The script is about to remove the data directory /var/lib/mysql. If it is " +#| "planned to just install a higher MySQL version or if a different mysql-" +#| "server package is already using it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Script-a /var/lib/mysql data direktorioa ezabatzera doa. MySQL bertsio " +"berriago bat instalatu behar bada edo beste mysql-server pakete bat berau " +"erabiltzen ari bada, datuak mantendu egingo dira." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Sistema abiaraztean MySQL abiarazi behar al da?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL can start automatically on boot time or only if you manually " +#| "type '/etc/init.d/mysql start'." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL abiaraztean automatikoki abiarazi daiteke edo eskuz /etc/init.d/mysql " +"start' eginaz." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "MySQL \"root\" erabiltzailearen pasahitz berria:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "It is highly recommended that you set a password for the MySQL " +#| "administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Oso gomendagarria da MySQL administratzaile \"root\" erabiltzaileari " +"pasahitz bat ezartzea." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "MySQL \"root\" erabiltzailearen pasahitz berria:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Ezinda MySQL \"root\" erabiltzailearen pasahitza ezarri" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "It seems an error occurred while setting the password for the MySQL " +#| "administrative user. This may have happened because the user already has " +#| "a password, or because there was a problem communicating with the MySQL " +#| "server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Dirudienez errore bat gertatu da MySQL administratzaile kontuaren pasahitza " +"ezartzean. Hau erabiltzaileak dagoeneko pasahitz bat duelako edo MySQL " +"zerbitzariarekiko konexioan erroreak daudelako gertatu daiteke." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Debian \"sarge\" edo zaharragoak erabiltzen duten ostalarietatik MySQL " +#~ "konexioak onartu?" + +#, fuzzy +#~| msgid "" +#~| "The way passwords were stored was not very secure. This has been " +#~| "improved with the drawback that clients (e.g. PHP) from hosts running " +#~| "Debian 3.1 Sarge will not be able to connect to account which are new or " +#~| "whose password have been changed. See /usr/share/doc/mysql-server-5.1/" +#~| "README.Debian." +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Pasahitzak biltegiratzeko modua ez da oso ziurra. Hau hobetua izan da " +#~ "baina Debian 3.1 Sarge erabiltzaileak ezingo dira kontu berri edo " +#~ "pasahitza aldatu duten kontuetara konektatu. Begiratu /usr/share/doc/" +#~ "mysql-server-5.1/README.Debian argibide gehiagorako." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Mysql erabili ahal izateko beharrezko erabiltzaile eta taldea sortu eta /" +#~ "var/lib/mysql-ek beharrezko baimenak dituela ziurtatu behar duzu (uid/gid-" +#~ "a ezberdina izan daiteke)" + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "MySQL bertsio guztiek erabilitako databaseak ezabatu?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "Ez baduzu pasahitzik ezartzen ez da aldaketarik egingo kontuan." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Instalazio amaitzean, kontua pasahitzez babesturik dagoela ziurtatu " +#~ "beharko zenuke (README.Debian irakurri xehetasun gehiagotarako)" === added file 'storage/xtradb/build/debian/po/fr.po' --- a/storage/xtradb/build/debian/po/fr.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/fr.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,274 @@ +# translation of fr.po to French +# Translators, if you are not familiar with the PO format, gettext +# documentation is worth reading, especially sections dedicated to +# this format, e.g. by running: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry' +# +# Some information specific to po-debconf are available at +# /usr/share/doc/po-debconf/README-trans +# or http://www.debian.org/intl/l10n/po-debconf/README-trans +# +# Developers do not need to manually edit POT or PO files. +# +# Christian Perrier <bubulle(a)debian.org>, 2004, 2006, 2007. +msgid "" +msgstr "" +"Project-Id-Version: fr\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-04-19 22:43+0200\n" +"Last-Translator: Christian Perrier <bubulle(a)debian.org>\n" +"Language-Team: French <debian-l10n-french(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"debian.org>\n" +"X-Generator: KBabel 1.11.4\n" +"Plural-Forms: Plural-Forms: nplurals=2; plural=n>1;\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "Faut-il vraiment revenir à la version précédente ?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "Un fichier /var/lib/mysql/debian-*.flag est présent sur ce système." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Cela indique qu'une version plus récente du paquet mysql-server a été " +"précédemment installée." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "Il n'est pas garanti que cette version puisse en utiliser les données." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Note importante pour les utilisateurs NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Pour pouvoir utiliser MySQL, les utilisateurs et les groupes suivants " +"doivent être ajoutés au système :" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Vous devez également vérifier le propriétaire et les permissions du " +"répertoire /var/lib/mysql :" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "Faut-il supprimer toutes les bases de données MySQL ?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"Le répertoire /var/lib/mysql qui contient les bases de données de MySQL va " +"être supprimé." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Si vous prévoyez d'installer une version plus récente de MySQL ou si un " +"autre paquet mysql-server les utilise déjà, vous devriez les conserver." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Faut-il lancer MySQL au démarrage ?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL peut être lancé soit au démarrage, soit en entrant la commande « /etc/" +"init.d/mysql start »." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nouveau mot de passe du superutilisateur de MySQL :" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Il est très fortement recommandé d'établir un mot de passe pour le compte " +"d'administration de MySQL (« root »)." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "Si ce champ est laissé vide, le mot de passe ne sera pas changé." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nouveau mot de passe du superutilisateur de MySQL :" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "" +"Impossible de changer le mot de passe de l'utilisateur « root » de MySQL" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Une erreur s'est produite lors du changement de mot de passe du compte " +"d'administration. Un mot de passe existait peut-être déjà ou il n'a pas été " +"possible de communiquer avec le serveur MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" +"Vous devriez vérifier le mot de passe de ce compte après l'installation du " +"paquet." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Veuillez consulter le fichier /usr/share/doc/mysql-server-5.1/README.Debian " +"pour plus d'informations." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Gérer les connexions d'hôtes qui utilisent les versions Debian « sarge » " +#~ "ou antérieures ?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "La méthode de stockage des mots de passe n'était pas très sûre dans les " +#~ "version précédentes de ce paquet. Cette méthode a été améliorée mais les " +#~ "modifications empêchent la connexion avec de nouveaux comptes ou des " +#~ "comptes dont le mot de passe a été modifié, pour les clients (p. ex. PHP) " +#~ "depuis des hôtes qui utilisent Debian 3.1 « sarge »." === added file 'storage/xtradb/build/debian/po/gl.po' --- a/storage/xtradb/build/debian/po/gl.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/gl.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,264 @@ +# Galician translation of mysql-dfsg-5.1's debconf templates +# This file is distributed under the same license as the mysql-dfsg-5.1 package. +# Jacobo Tarrio <jtarrio(a)debian.org>, 2007. +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-04-20 09:44+0200\n" +"Last-Translator: Jacobo Tarrio <jtarrio(a)debian.org>\n" +"Language-Team: Galician <proxecto(a)trasno.net>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "¿Quere pasar a unha versión anterior?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "Neste sistema hai un ficheiro chamado /var/lib/mysql/debian-*.flag." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Este ficheiro indica que antes se instalou un paquete mysql-server cunha " +"versión superior." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"Non se pode garantir que a versión que está a instalar poida empregar as " +"bases de datos actuais." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Nota importante para os usuarios de NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Para empregar MySQL deberían engadirse ao sistema as seguintes entradas de " +"usuarios e grupos:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Tamén debería comprobar os permisos e o propietario do directorio /var/lib/" +"mysql:" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "¿Eliminar tódalas bases de datos de MySQL?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"Hase eliminar o directorio /var/lib/mysql, que contén as bases de datos de " +"MySQL." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Se está a eliminar o paquete MySQL para instalar despois unha versión máis " +"recente ou se xa hai un paquete mysql-server diferente a empregalo, debería " +"conservar os datos." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "¿Iniciar o servidor MySQL co ordenador?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"Pódese iniciar automaticamente o servidor MySQL ao iniciar o ordenador, ou " +"manualmente coa orde \"/etc/init.d/mysql start\"." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Novo contrasinal para o usuario \"root\" de MySQL:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Aínda que non é obrigatorio, recoméndase encarecidamente que estableza un " +"contrasinal para o usuario administrativo \"root\" de MySQL." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "Se deixa o campo en branco, non se ha cambiar o contrasinal." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Novo contrasinal para o usuario \"root\" de MySQL:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Non se puido establecer o contrasinal do usuario \"root\" de MySQL" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Houbo un erro ao establecer o contrasinal do usuario administrativo de " +"MySQL. Puido ocorrer porque o usuario xa teña un contrasinal ou debido a un " +"problema de comunicacións co servidor MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "You should check the account's password after tha package installation." +msgid "You should check the account's password after the package installation." +msgstr "Debería comprobar o contrasinal da conta trala instalación do paquete." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Consulte o ficheiro /usr/share/doc/mysql-server-5.1/README.Debian para máis " +"información." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "¿Soportar as conexións a MySQL de máquinas que empreguen Debian \"sarge\" " +#~ "ou anterior?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Nas versións antigas dos clientes MySQL de Debian, os contrasinais non se " +#~ "armacenaban de xeito seguro. Isto mellorouse desde aquela; nembargantes, " +#~ "os clientes (tales coma PHP) das máquinas que executen Debian 3.1 Sarge " +#~ "non se han poder conectar a contas recentes ou a contas nas que se " +#~ "cambiara o contrasinal." === added file 'storage/xtradb/build/debian/po/it.po' --- a/storage/xtradb/build/debian/po/it.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/it.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,266 @@ +# Italian (it) translation of debconf templates for mysql-dfsg-5.1 +# Copyright (C) 2006 Software in the Public Interest +# This file is distributed under the same license as the mysql-dfsg-5.1 package. +# Luca Monducci <luca.mo(a)tiscali.it>, 2006, 2007. +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1 5.0.38 italian debconf templates\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-04-22 15:43+0200\n" +"Last-Translator: Luca Monducci <luca.mo(a)tiscali.it>\n" +"Language-Team: Italian <debian-l10n-italian(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "Procedere realmente con l'abbassamento di versione?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" +"Su questo sistema esiste un file con nome /var/lib/mysql/debian-*.flag." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Quel file indica che in precedenza è stata installata una versione superiore " +"del pacchetto mysql-server." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"Non è garantito che la versione che si sta installando sia in grado di usare " +"i database presenti." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Nota importante per gli utenti NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Per usare MySQL i seguenti utenti e gruppi devono essere aggiunti al sistema:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Inoltre si devono verificare i permessi e il proprietario della directory /" +"var/lib/mysql:" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "Eliminare tutti i database MySQL?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"La directory /var/lib/mysql contenente i database di MySQL sta per essere " +"eliminata." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Se si rimuove il pacchetto MySQL per poi installare una versione più recente " +"oppure se sono già in uso da un altro pacchetto mysql-server, i dati non " +"devono essere eliminati." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Lanciare il server MySQL all'avvio?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"Il server MySQL può essere lanciato automaticamente all'avvio del sistema " +"oppure manualmente con il comando «/etc/init.d/mysql start»." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nuova password per l'utente «root» di MySQL:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Sebbene non sia obbligatoria, si raccomanda d'impostare una password per " +"l'utente d'amministrazione «root» di MySQL." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "Se questo campo è lasciato vuoto, la password non viene cambiata." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nuova password per l'utente «root» di MySQL:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Impossibile impostare la password per l'utente «root» di MySQL" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Si è verificato un errore durante l'impostazione della password per l'utente " +"d'amministrazione di MySQL. Questo può essere accaduto perché l'utente ha " +"già una password oppure a causa di un problema di connessione con il server " +"MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "You should check the account's password after tha package installation." +msgid "You should check the account's password after the package installation." +msgstr "" +"Al termine dell'installazione si deve verificare la password dell'account." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Per maggiori informazioni si consulti il file /usr/share/doc/mysql-server-" +"5.1/README.Debian." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Supporto a connessioni MySQL da macchine con Debian «sarge» o antecedente" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Nelle precedenti versioni dei client MySQL su Debian le password non " +#~ "erano memorizzate in modo sicuro. Questo è stato migliorato ma i client " +#~ "(per esempio PHP) presenti su una macchina con Debian 3.1 Sarge non sono " +#~ "più in grado di connettersi a un nuovo account né ad account le cui " +#~ "password siano state cambiate." === added file 'storage/xtradb/build/debian/po/ja.po' --- a/storage/xtradb/build/debian/po/ja.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/ja.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,394 @@ +# +# Translators, if you are not familiar with the PO format, gettext +# documentation is worth reading, especially sections dedicated to +# this format, e.g. by running: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry' +# +# Some information specific to po-debconf are available at +# /usr/share/doc/po-debconf/README-trans +# or http://www.debian.org/intl/l10n/po-debconf/README-trans +# +# Developers do not need to manually edit POT or PO files. +# +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1 5.0.32-6\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-02-18 22:25+0900\n" +"Last-Translator: Hideki Yamane (Debian-JP) <henrich(a)debian.or.jp>\n" +"Language-Team: Japanese <debian-japanese(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "Do you really want to downgrade?" +msgid "Really proceed with downgrade?" +msgstr "本当にダウングレードしますか?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "" +#| "WARNING: The file /var/lib/mysql/debian-*.flag exists. This indicates " +#| "that a mysql-server package with a higher version has been installed " +#| "before. It can not be guaranteed that this version can use its data." +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"警告: /var/lib/mysql/debian-*.flag ファイルが存在しています。これは、以前によ" +"り新しいバージョンの mysql-server パッケージがインストールされていたことを示" +"します。データをこのバージョンで使えるかどうか、保証できません。" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "NIS/YP ユーザへ重要な注意!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The script is about to remove the data directory /var/lib/mysql. If it is " +#| "planned to just install a higher MySQL version or if a different mysql-" +#| "server package is already using it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"このスクリプトはデータのディレクトリ /var/lib/mysql を削除するためのもので" +"す。単に新しいバージョンの MySQL をインストールしようとしている、あるいは別" +"の mysql-server パッケージを既に使っている場合、データは保持する必要がありま" +"す。" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "MySQL をシステム起動時に開始しますか?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL can start automatically on boot time or only if you manually " +#| "type '/etc/init.d/mysql start'." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL の起動方法について、システム起動時に自動的に開始するか、あるいは '/etc/" +"init.d/mysql start' と手で入力した時のみ起動するかを選べます。" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "MySQL の \"root\" ユーザに対する新しいパスワード:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "It is highly recommended that you set a password for the MySQL " +#| "administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"MySQL を管理する \"root\" ユーザのパスワードを設定することを強くお勧めしま" +"す。" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "MySQL の \"root\" ユーザに対する新しいパスワード:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "MySQL の \"root\" ユーザのパスワードを設定できません" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "It seems an error occurred while setting the password for the MySQL " +#| "administrative user. This may have happened because the user already has " +#| "a password, or because there was a problem communicating with the MySQL " +#| "server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"MySQL の管理者ユーザに対してパスワードを設定しようとした際、エラーが発生した" +"ようです。これは既に管理者ユーザにパスワードが設定されていたか、MySQL サーバ" +"との接続に問題があったためだと思われます。" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "See /usr/share/doc/mysql-server-5.1/README.Debian for more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"詳細は /usr/share/doc/mysql-server-5.1/README.Debian を参照してください。" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "Cannot upgrade if ISAM tables are present!" +#~ msgstr "ISAM テーブルがある場合はアップグレードできません!" + +#~ msgid "" +#~ "Recent versions of MySQL can no longer use the old ISAM table format and " +#~ "it is necessary to convert your tables to e.g. MyISAM before upgrading by " +#~ "using \"mysql_convert_table_format\" or \"ALTER TABLE x ENGINE=MyISAM\". " +#~ "The installation of mysql-server-5.1 will now abort. In case your old " +#~ "mysql-server-4.1 gets removed nevertheless just reinstall it to convert " +#~ "those tables." +#~ msgstr "" +#~ "MySQL の最近のバージョンでは以前の ISAM テーブル形式は利用できなくなってい" +#~ "ます。そのため、例えば \"mysql_convert_table_format\" あるいは \"ALTER " +#~ "TABLE x ENGINE=MyISAM\" としてアップグレード前に MyISAM にコンバートするこ" +#~ "となどが必要です。mysql-server-5.1 のインストールを中断します。以前の " +#~ "mysql-server-4.1 が削除されてしまった場合であっても、テーブルをコンバート" +#~ "するために再インストールをしてください。" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Debian \"Sarge\" あるいはそれよりも古いバージョンが稼働しているホストから" +#~ "の MySQL 接続をサポートしますか?" + +#, fuzzy +#~| msgid "" +#~| "The way passwords were stored was not very secure. This has been " +#~| "improved with the drawback that clients (e.g. PHP) from hosts running " +#~| "Debian 3.1 Sarge will not be able to connect to account which are new or " +#~| "whose password have been changed. See /usr/share/doc/mysql-server-5.1/" +#~| "README.Debian." +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "パスワードの保存方法は、あまり安全な方法で行われていませんでした。これは改" +#~ "善されましたが、Debian 3.1 Sarge が稼働しているホストからクライアント " +#~ "(PHP など) が新しいアカウントやパスワードが変更されたアカウントには接続で" +#~ "きなくなるという欠点もでています。詳細については /usr/share/doc/mysql-" +#~ "server-5.1/README.Debian を参照してください。" + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "mysql を利用するには以下のユーザとグループを作成し、/var/lib/mysql が正し" +#~ "い権限になっているかどうかを確認する必要があります (おそらく uid/gid が違" +#~ "います)。" + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "全バージョンの MySQL で利用されているデータベースを削除しますか?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "パスワードを入力しない場合、アカウントに対して変更は加えられません。" + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "インストールが終了した際、アカウントがパスワードできちんと保護されているか" +#~ "どうかを確認してください (詳細については README.Debian を参照してくださ" +#~ "い)。" + +#~ msgid "Install Hints" +#~ msgstr "インストールのヒント" + +#~ msgid "" +#~ "On upgrades from MySQL 3.23, as shipped with Debian Woody, symlinks in " +#~ "place of /var/lib/mysql or /var/log/mysql gets accidently removed and " +#~ "have manually be restored." +#~ msgstr "" +#~ "Debian Woody でリリースされた MySQL 3.23 からのアップグレードでは、/var/" +#~ "lib/mysql あるいは /var/log/mysql の代わりにシンボリックリンクは偶然にも削" +#~ "除されてしまっているので、手動でのリストアが必要になります。" + +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "MySQL は /etc/hosts ファイル経由で解決できる「数字のみで構成されてない」ホ" +#~ "スト名の場合のみ、インストールされます。つまり、\"hostname\" コマンドが " +#~ "\"myhostname\" を返すなら、\"10.0.0.1 myhostname\" という行が /etc/hosts " +#~ "ファイルにあるはずです。" + +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "新規に mysql ユーザとして \"debian-sys-maint\" が作成されます。この mysql " +#~ "アカウントは start/stop 時と cron スクリプトで利用されます。消さないでくだ" +#~ "さい。" + +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "MySQL の root ユーザに対して「パスワードの設定」を忘れないでください! /" +#~ "root/.my.cnf を使っている場合、このファイル中の \"user\" 行と \"password" +#~ "\" 行を記述してください。決してパスワードだけではいけません!" + +#~ msgid "" +#~ "Should I remove the complete /var/lib/mysql directory tree which is used " +#~ "by all MySQL versions, not necessarily only the one you are about to " +#~ "purge?" +#~ msgstr "" +#~ "これから purge しようとしているバージョンだけでは無く、全てのバージョンの " +#~ "MySQL が /var/lib/mysql ディレクトリを使用しています。このディレクトリを完" +#~ "全に削除しますか?" + +#~ msgid "Update Hints" +#~ msgstr "更新のヒント" + +#~ msgid "" +#~ "Rarely, e.g. on new major versions, the privilege system is improved. To " +#~ "make use of it mysql_fix_privilege_tables must be executed manually. The " +#~ "script is not supposed to give any user more rights that he had before," +#~ msgstr "" +#~ "まれに、つまりは新しいメジャーバージョンにおいて、特権システムが改善されま" +#~ "す。これを行うには、mysql_fix_privilege_table を手動で実行する必要がありま" +#~ "す。スクリプトは、どのようなユーザに対しても、以前に保持していた以上の権限" +#~ "を与えるようにはなっていません。" + +#~ msgid "Please also read http://www.mysql.com/doc/en/Upgrade.html" +#~ msgstr "http://www.mysql.com/doc/ja/Upgrade.html も参照してください" === added file 'storage/xtradb/build/debian/po/nb.po' --- a/storage/xtradb/build/debian/po/nb.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/nb.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,297 @@ +# translation of mysql_nb.po to Norwegian Bokmål +# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER +# This file is distributed under the same license as the PACKAGE package. +# +# Bjørn Steensrud <bjornst(a)powertech.no>, 2007. +msgid "" +msgstr "" +"Project-Id-Version: mysql_nb\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-02-18 12:13+0100\n" +"Last-Translator: Bjørn Steensrud <bjornst(a)powertech.no>\n" +"Language-Team: Norwegian Bokmål <i18n-nb(a)lister.ping.uio.no>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"X-Generator: KBabel 1.11.2\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "Do you really want to downgrade?" +msgid "Really proceed with downgrade?" +msgstr "Er du sikker på at du vil nedgradere?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "" +#| "WARNING: The file /var/lib/mysql/debian-*.flag exists. This indicates " +#| "that a mysql-server package with a higher version has been installed " +#| "before. It can not be guaranteed that this version can use its data." +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"ADVARSEL: Fila /var/lib/mysql/debian-*.flag finnes. Dette viser at en mysql-" +"server-pakke med et høyere versjonsnummer har vært installert før. Det kan " +"ikke garanteres at denne versjonen kan bruke data fra den høyere versjonen." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "Viktig merknad for NIS/YP-brukere!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The script is about to remove the data directory /var/lib/mysql. If it is " +#| "planned to just install a higher MySQL version or if a different mysql-" +#| "server package is already using it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Dette skriptet skal til å fjerne data-mappa /var/lib/mysql. Denne mappa bør " +"beholdes hvis det bare skal installeres en høyere MySQL-versjon, eller hvis " +"en annen mysql-server-pakke allerede bruker den." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Skal MySQL startes ved maskinoppstart?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL can start automatically on boot time or only if you manually " +#| "type '/etc/init.d/mysql start'." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL kan startes automatisk når maskinen starter, eller bare hvis du " +"skriver «/etc/init.d/mysql start»." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nytt passord for MySQLs «root»-bruker:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "It is highly recommended that you set a password for the MySQL " +#| "administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Det anbefales sterkt at du oppgir et passord for den administrative «root»-" +"brukeren i MySQl." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nytt passord for MySQLs «root»-bruker:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Klarer ikke angi passord for MySQLs «root»-bruker" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "It seems an error occurred while setting the password for the MySQL " +#| "administrative user. This may have happened because the user already has " +#| "a password, or because there was a problem communicating with the MySQL " +#| "server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Det ser ut til at det oppsto en feil mens det ble satt et passord for MySQLs " +"administrative bruker. Dette kan være fordi brukeren allerede har et " +"passord, eller fordi det var et kommunikasjonsproblem med MySQL-tjeneren." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Skal MySQL-tilkoblinger støttes fra vertsmaskiner som kjører Debian " +#~ "«sarge» eller eldre?" + +#, fuzzy +#~| msgid "" +#~| "The way passwords were stored was not very secure. This has been " +#~| "improved with the drawback that clients (e.g. PHP) from hosts running " +#~| "Debian 3.1 Sarge will not be able to connect to account which are new or " +#~| "whose password have been changed. See /usr/share/doc/mysql-server-5.1/" +#~| "README.Debian." +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Passord ble tidligere lagret på en lite sikker måte. Dette er nå " +#~ "forbedret, med den ulempen at klienter (f.eks. PHP) fra verter som kjører " +#~ "Debian 3.1 Sarge ikke vil kunne koble til en konto som er ny eller har " +#~ "fått endret passordet. Se /usr/share/doc/mysql-server-5.1/README.Debian." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "For å bruke MySQL må du installere en bruker og gruppe tilsvarende den " +#~ "nedenfor og se til at /var/lib/mysql har riktige rettigheter (uid/gid kan " +#~ "være forskjellig)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Skal databasene brukt av alle MySQL-versjoner fjernes?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "Hvis du ikke oppgir et passord blir det ikke gjort noen endringer med " +#~ "kontoen." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Når installasjonen er ferdig bør det sjekkes at kontoen er ordentlig " +#~ "beskyttet med et passord (mer informasjon finnes i README.Debian)." === added file 'storage/xtradb/build/debian/po/nl.po' --- a/storage/xtradb/build/debian/po/nl.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/nl.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,302 @@ +# Dutch mysql-dfsg-5.1 po-debconf translation, +# Copyright (C) 2006 THE PACKAGE'S COPYRIGHT HOLDER +# This file is distributed under the same license as the mysql-dfsg-5.1 package. +# Vincent Zweije <zweije(a)xs4all.nl>, 2006. +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1 5.0.30-1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2006-02-19 10:20+0100\n" +"Last-Translator: Thijs Kinkhorst <thijs(a)debian.org>\n" +"Language-Team: Debian-Dutch <debian-l10n-dutch(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=utf-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "Do you really want to downgrade?" +msgid "Really proceed with downgrade?" +msgstr "Wilt u echt een oude versie herstellen?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "" +#| "WARNING: The file /var/lib/mysql/debian-*.flag exists. This indicates " +#| "that a mysql-server package with a higher version has been installed " +#| "before. It can not be guaranteed that this version can use its data." +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Waarschuwing: waarschijnlijk is een hogere versie van het mysql-server " +"pakket geïnstalleerd geweest (het bestand /var/lib/mysql/debian-*.flag " +"bestaat). Er is geen garantie dat de gegevensbestanden, bewerkt met die " +"hogere versie, kunnen worden gebruikt met de versie van mysql die u nu " +"installeert." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "Belangrijke opmerking voor gebruikers van NIS/YP!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The script is about to remove the data directory /var/lib/mysql. If it is " +#| "planned to just install a higher MySQL version or if a different mysql-" +#| "server package is already using it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Het script staat op het punt de datamap /var/lib/mysql te verwijderen. Als " +"het plan alleen maar is om een hogere MySQL versie te installeren, of als " +"een ander mysql-serverpakket de datamap al gebruikt, dan zou de data moeten " +"worden behouden." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Moet MySQL starten als de computer start?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL can start automatically on boot time or only if you manually " +#| "type '/etc/init.d/mysql start'." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL kan automatisch starten bij het starten van de computer, of slechts " +"wanneer u '/etc/init.d/mysql start' handmatig uitvoert." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nieuw wachtwoord voor de MySQL \"root\"-gebruiker:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "It is highly recommended that you set a password for the MySQL " +#| "administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Het wordt sterk aangeraden een wachtwoord in te stellen voor de " +"administratieve MySQL \"root\"-gebruiker." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nieuw wachtwoord voor de MySQL \"root\"-gebruiker:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Kan het wachtwoord voor de MySQL \"root\"-gebruiker niet instellen" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "It seems an error occurred while setting the password for the MySQL " +#| "administrative user. This may have happened because the user already has " +#| "a password, or because there was a problem communicating with the MySQL " +#| "server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Er lijkt een fout te zijn opgetreden bij het instellen van het wachtwoord " +"van de MySQL administratieve gebruiker. Dat kan komen doordat de gebruiker " +"al een wachtwoord heeft, of omdat er een probleem was bij het communiceren " +"met de MySQL server." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Moet u MySQL-verbindingen accepteren van computers die Debian \"sarge\" " +#~ "of ouder draaien?" + +#, fuzzy +#~| msgid "" +#~| "The way passwords were stored was not very secure. This has been " +#~| "improved with the drawback that clients (e.g. PHP) from hosts running " +#~| "Debian 3.1 Sarge will not be able to connect to account which are new or " +#~| "whose password have been changed. See /usr/share/doc/mysql-server-5.1/" +#~| "README.Debian." +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "De wijze waarop wachtwoorden werden bewaard was niet erg veilig. Dit is " +#~ "verbeterd, maar helaas zullen programma's van computers die Debian 3.1 " +#~ "Sarge draaien, geen verbinding meer kunnen maken met accounts die nieuw " +#~ "zijn, of waarvan het wachtwoord is gewijzigd. Zie /usr/share/doc/mysql-" +#~ "server-5.1/README.Debian." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Om mysql te gebruiken dient u een gebruiker en groep aan te maken, " +#~ "gelijkwaardig aan onderstaand voorbeeld, en u dient ervoor te zorgen dat /" +#~ "var/lib/mysql de bijbehorende toegangsrechten heeft (uid en gid mogen " +#~ "anders zijn)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Databases verwijderen die door alle MySQL versies worden gebruikt?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "Indien u geen wachtwoord opgeeft zal het account niet worden gewijzigd." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Wanneer de installatie klaar is, dient u te verifiëren dat het account " +#~ "netjes beschermd is met een wachtwoord (zie README.Debian voor meer " +#~ "informatie)." === added file 'storage/xtradb/build/debian/po/pt.po' --- a/storage/xtradb/build/debian/po/pt.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/pt.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,322 @@ +# Portuguese translation for mysql-dfsg-5.1's debconf messages +# Copyright (C) 2006 Miguel Figueiredo <elmig(a)debianpt.org> +# This file is distributed under the same license as the mysql-dfsg-5.1 package. +# Miguel Figueiredo <elmig(a)debianpt.org> +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-05-05 21:01+0100\n" +"Last-Translator: Miguel Figueiredo <elmig(a)debianpt.org>\n" +"Language-Team: Portuguese <traduz(a)debianpt.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "Deseja mesmo fazer downgrade?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "Existe um ficheiro chamado /var/lib/mysql/debian-*.flag neste sistema." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"Tal ficheiro significa que anteriormente foi instalado um pacote mysql-" +"server com um número de versão superior." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"Não existe nenhuma garantia que a versão que está actualmente a instalar " +"seja capaz de utilizar as bases de dados actuais." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Nota importante para utilizadores de NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Para utilizar o MySQL, têm de ser acrescentadas as seguintes entradas para " +"os utilizadores e grupos:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Deve também verificar as permissões e o dono do directório /var/lib/mysql :" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "Remover todas as bases de dados MySQL?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"O directório /var/lib/mysql que contém as bases de dados MySQL está prestes " +"a ser removido." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Se está a remover o pacote MySQL de modo a posteriormente instalar uma " +"versão mais recente ou se um pacote mysq-server já está os está a utilizar, " +"os dados devem ser mantidos." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Iniciar o servidor MySQL no arranque?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"O MySQL pode ser automaticamente lançado no arranque ou manualmente através " +"do comando '/etc/init.d/mysql start'." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nova palavra-passe para o utilizador \"root\" do MySQL:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Embora não seja mandatório, É fortemente recomendado que defina uma palavra-" +"passe para o utilizador administrativo \"root\" do MySQL." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" +"Se esse campo for deixado em branco, a palavra-passe não irá ser alterada." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nova palavra-passe para o utilizador \"root\" do MySQL:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "" +"Não foi possível definir a palavra-passe para o utilizador \"root\" do MySQL" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Ocorreu um erro enquanto era definida a palavra-passe para o utilizador " +"administrativo do MySQL. Isto pode ter acontecido porque a cona já tem uma " +"palavra-passe, ou porque ocorreu um problema ao comunicação com o servidor " +"MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "You should check the account's password after tha package installation." +msgid "You should check the account's password after the package installation." +msgstr "" +"Você deve verificar a palavra-passe da conta após a instalação do pacote." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Para mais informação por favor leia o ficheiro /usr/share/doc/mysql-server-" +"5.1/README.Debian." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "Cannot upgrade if ISAM tables are present!" +#~ msgstr "Não é possível actualizar se estiverem presentes tabelas ISAM!" + +#~ msgid "" +#~ "Recent versions of MySQL can no longer use the old ISAM table format and " +#~ "it is necessary to convert your tables to e.g. MyISAM before upgrading by " +#~ "using \"mysql_convert_table_format\" or \"ALTER TABLE x ENGINE=MyISAM\". " +#~ "The installation of mysql-server-5.1 will now abort. In case your old " +#~ "mysql-server-4.1 gets removed nevertheless just reinstall it to convert " +#~ "those tables." +#~ msgstr "" +#~ "As versões recentes de MySQL já não podem utilizar o antigo formato de " +#~ "tabelas ISAM e é por isso necessário converter as suas tabelas pra e.g. " +#~ "MyISAM antes da actualização, utilizando \"mysql_convert_table_format\" " +#~ "ou \"ALTER TABLE x ENGINE=MyISAM\". A instalação de mysql-server-5.1 irá " +#~ "agora ser cancelada. Se o seu antigo mysql-server-4.1 for removido apenas " +#~ "reinstale para converter essas tabelas." + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Suportar ligações MySQL de máquinas que corram Debian \"sarge\" ou mais " +#~ "antigos?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Nas versões antigas de clientes de MySQL em Debian, as palavras-passe não " +#~ "eram guardadas de forma segura. Isto foi melhorado desde aí, no entanto " +#~ "os clientes (como o PHP) de máquinas que corram Debian 3.1 Sarge não irão " +#~ "conseguir ligar-se a contas novas ou cuja palavra-passe foi alterada." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Para utilizar mysql e instalar um utilizador e grupo equivalentes para o " +#~ "seguinte e assegurar-se que /var/lib/mysql têm as permissões correctas (o " +#~ "uid/gid podem ser diferentes)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Remover as bases de dados utilizadas por todas as versões de MySQL?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "Se não disponibilizar uma password não serão feitas alterações nesta " +#~ "conta." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Quando terminar a instalação, deve verificar se a conta está devidamente " +#~ "protegida com uma password (para mais informações veja README.Debian)." === added file 'storage/xtradb/build/debian/po/pt_BR.po' --- a/storage/xtradb/build/debian/po/pt_BR.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/pt_BR.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,458 @@ +# Brazilian Portuguese (pt_BR) debconf template translation for +# Debian's mysql-dfsg source package. +# Debian-BR Project <debian-l10n-portuguese(a)lists.debian.org> +# André Luís Lopes, <andrelop(a)debian.org> , 2004 +# André Luís Lopes, <andrelop(a)debian.org> , 2006 +# André Luís Lopes, <andrelop(a)debian.org> , 2007 +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-04-21 15:59-0300\n" +"Last-Translator: André Luís Lopes <andrelop(a)debian.org>\n" +"Language-Team: Debian-BR Project <debian-l10n-portuguese(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"pt_BR utf-8\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "Realmente proceder com o rebaixamento de versão?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "Um arquivo de nome /var/lib/mysql/debian-*.flag existe no sistema." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"A presença de um arquivo como este é uma indicação de que um pacote mysql-" +"server com um número de versão mais alto já foi instalado anteriormente." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" +"Não há garantias de que a versão que você está instalando no momento " +"conseguirá utilizar as bases de dados existentes." + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "Aviso importante para usuários NIS/YP" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" +"Para utilizar o MySQL, as seguintes entradas para usuários e grupos devem " +"ser adicionadas ao sistema:" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" +"Você deverá também checar as permissões e o dono do diretório /var/lib/mysql:" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "Remove all MySQL databases?" +msgid "Remove all Percona SQL databases?" +msgstr "Remover todas as bases de dados do MySQL?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The /var/lib/mysql directory which contains the MySQL databases is about " +#| "to be removed." +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" +"O diretório /var/lib/mysql, o qual contém as bases de dados do MySQL, está " +"prestes a ser removido." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "If you're removing the MySQL package in order to later install a more " +#| "recent version or if a different mysql-server package is already using " +#| "it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Caso você esteja removendo o pacote MySQL para posteriormente instalar uma " +"versão mais recente ou, caso uma versão diferente do pacote mysql-server " +"esteja sendo utilizada, os dados deverão ser mantidos." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Start the MySQL server on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Iniciar o servidor MySQL junto a inicialização da máquina?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL server can be launched automatically at boot time or manually " +#| "with the '/etc/init.d/mysql start' command." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"O servidor MySQL pode ser iniciado automaticamente junto a inicialização da " +"máquina ou manualmente com o comando '/etc/init.d/mysql start'." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nova senha para o usuário \"root\" do MySQL:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "While not mandatory, it is highly recommended that you set a password for " +#| "the MySQL administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Apesar de não ser mandatório, é altamente recomendado que você defina uma " +"senha para o usuário administrativo \"root\" do MySQL." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "Caso este campo seja deixado em branco, a senha não sera mudada." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for the MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nova senha para o usuário \"root\" do MySQL:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for the MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Impossível definir senha para o usuário \"root\" do MySQL" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "An error occurred while setting the password for the MySQL administrative " +#| "user. This may have happened because the account already has a password, " +#| "or because of a communication problem with the MySQL server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Um erro ocorreu durante a definição da senha para o usuário administrativo " +"do MySQL. Isso pode ter acontecido devido a esse usuário já possuir uma " +"senha definida ou devido a ocorrência de um problema de comunicação com o " +"servidor MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "You should check the account's password after tha package installation." +msgid "You should check the account's password after the package installation." +msgstr "Você deverá checar a senha dessa conta após a instalação deste pacote." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for " +#| "more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" +"Por favor, leia o arquivo /usr/share/doc/mysql-server-5.1/README.Debian para " +"maiores informações." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Suportar conexões MySQL originadas de hosts executando o Debian \"sarge\" " +#~ "ou mais antigos ?" + +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Em versões antigas dos clientes MySQL no Debian, as senhas não eram " +#~ "armazenadas de forma segura. Isto foi corrigido desde então, porém, " +#~ "clientes (como o PHP) em hosts executando o Debian 3.1 Sarge não serão " +#~ "capazes de conectar em contas recentes ou contas as quais as senhas " +#~ "tenham sido modificadas." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Para utilizar o MySQL, você deve instalar um usuário e um grupo " +#~ "equivalentes ao usuário e grupo a seguir para se certificar de que o " +#~ "diretório /var/lib/mysql possua as permissões correctas (o uid/gid podem " +#~ "ser diferentes)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Remover as bases de dados utilizadas por todas as versões do MySQL?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "Caso você não forneça uma senha, nenhuma mudança será feita na conta." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Quando a instalação finalizar, você deverá verificar se a conta está " +#~ "apropriadamente protegida com uma senha (consulte o arquivo README.Debian " +#~ "para maiores informações)." + +#~ msgid "internal" +#~ msgstr "interno" + +#~ msgid "Only internally used." +#~ msgstr "Somente utilizado internamente." + +#, fuzzy +#~ msgid "Update Hints" +#~ msgstr "Dicas de atualização" + +#, fuzzy +#~ msgid "" +#~ "Rarely, e.g. on new major versions, the privilege system is improved. To " +#~ "make use of it mysql_fix_privilege_tables must be executed manually. The " +#~ "script is not supposed to give any user more rights that he had before," +#~ msgstr "" +#~ "Raramente, por exemplo, em novas versões maiores, o sistema de " +#~ "privilégios é melhorado. Para fazer uso disso, o script " +#~ "mysql_fix_privilege_tables deve ser executado manualmente. O script não " +#~ "atribuirá a nenhum usuário mais direitos do que os mesmos já possuíam " +#~ "anteriormente." + +#~ msgid "Please also read http://www.mysql.com/doc/en/Upgrade.html" +#~ msgstr "Por favor, leia http://www.mysql.com/doc/en/Upgrade.html" + +#, fuzzy +#~ msgid "Install Hints" +#~ msgstr "Dicas de instalação" + +#, fuzzy +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "O MySQL será instalado somente caso você possua um nome de host NÃO " +#~ "NUMÉRICO que possa ser resolvido através do arquivo /etc/hosts, ou seja, " +#~ "caso o comando \"hostname\" retorne \"myhostname\", uma linha como " +#~ "\"10.0.0.1 myhostname\" deverá existir no arquivo /etc/hosts." + +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "Um novo usuário MySQL de nome \"debian-sys-maint\" será criado. Essa " +#~ "conta MySQL é utilizada pelos scripts de inicialização/parada e pelos " +#~ "scripts cron. Não remova esse usuário." + +#, fuzzy +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "Por favor, lembre-se de definir uma SENHA para o usuário root do MySQL ! " +#~ "Caso você utilize um arquivo /root/.my.cnf, sempre inclua as linhas \"user" +#~ "\" e \"password\" nesse arquivo, nunca somente a senha ! Consulte o " +#~ "arquivo /usr/share/doc/mysql-server/README.Debian para mais informações." + +#~ msgid "" +#~ "Should I remove all databases below /var/lib/mysql as you are purging the " +#~ "mysql-server package?" +#~ msgstr "" +#~ "Todas as base de dados sob o diretório /var/lib/mysql devem ser removidas " +#~ "quando você remover o pacote pacote mysql-server ?" + +#~ msgid "" +#~ "Networking is disabled by default for security reasons. You can enable it " +#~ "by commenting out the skip-networking option in /etc/mysql/my.cnf." +#~ msgstr "" +#~ "O suporte ao funcionamento em rede está desativado por padrão por " +#~ "questões de segurança. Você poderá ativá-lo comentando a opção 'skip-" +#~ "networking' no arquivo /etc/mysql/my.cnf." + +#~ msgid "security and update notice" +#~ msgstr "aviso de segurança e actualização" + +#~ msgid "" +#~ "Should I remove everything below /var/lib/mysql when you purge the mysql-" +#~ "server package with the \"dpkg --purge mysql-server\" command (i.e. " +#~ "remove everything including the configuration) somewhen? (default is not)" +#~ msgstr "" +#~ "Devo remover tudo abaixo de /var/lib/mysql quando fizer o purge do pacote " +#~ "mysql-server com o comando \"dpkg --purge mysql-server\" (ou seja, " +#~ "remover tudo incluíndo a configuração)? (o padrão é não remover)" + +#~ msgid "Make MySQL reachable via network?" +#~ msgstr "Fazer com que o MySQL seja acessível via rede?" + +#~ msgid "" +#~ "Should MySQL listen on a network reachable TCP port? This is not " +#~ "necessary for use on a single computer and could be a security problem." +#~ msgstr "" +#~ "O MySQL deve aguardar ligações numa porta TCP acessível via rede? Isto " +#~ "não é necessário para uso num único computador e pode ser um problema de " +#~ "segurança." + +#~ msgid "Enable chroot mode?" +#~ msgstr "Activar o modo chroot?" + +#~ msgid "" +#~ "MySQL is able to jail itself into the /var/lib/mysql_jail directory so " +#~ "that users cannot modify any files outside this directory. This improves " +#~ "resistence against crackers, too, as they are not able to modify system " +#~ "files." +#~ msgstr "" +#~ "O MySQL é capaz de se prender no diretório /var/lib/mysql_jail, assim os " +#~ "utilizadores não poderão modificar ficheiros fora deste directório. Isto " +#~ "aumenta também a resistência contra crackers, pois eles não poderão " +#~ "modificar arquivos de sistema." + +#~ msgid "Please run mysql_fix_privilege_tables !" +#~ msgstr "Por favor execute mysql_fix_privilege_tables !" + +#~ msgid "" +#~ "I will ensure secure permissions of /var/lib/mysql by replacing GIDs " +#~ "other than root and mysql with mysql." +#~ msgstr "" +#~ "Permissões seguras para o diretório /var/lib/mysql serão asseguradas " +#~ "substituíndo GIDs diferentes de root e mysql por mysql." + +#~ msgid "" +#~ "Instructions how to enable SSL support are in /usr/share/doc/mysql-server/" +#~ msgstr "" +#~ "Instruções sobre como activar o suporte de SSL estão disponíveis no " +#~ "directório /usr/share/doc/mysql-server/." + +#, fuzzy +#~ msgid "mysql_fix_privileges_tables should be executed" +#~ msgstr "mysql_fix_privileges_tables será executado" + +#, fuzzy +#~ msgid "" +#~ "The latest MySQL versions have an enhanced, more fine grained, privilege " +#~ "system. To make use of it, some new fields must be added to the tables " +#~ "in the \"mysql\" database. This will not happen automatically." +#~ msgstr "" +#~ "As últimas versões do MySQL possuem um sistema de privilégios melhorado e " +#~ "mais refinado. Para utilizá-lo, alguns novos campos devem ser adicionados " +#~ "as tabelas na base de dados \"mysql\". Isto é feito pelo script " +#~ "mysql_fix_privileges_tables durante esta actualização independente do " +#~ "servidor estar a correr ou não !" + +#~ msgid "" +#~ "This script is not supposed to give any user more rights that he had " +#~ "before, if you encounter such a case, please contact me." +#~ msgstr "" +#~ "Este script não deverá fornecer mais direitos a um utilizador além dos " +#~ "quais ele já possua anteriormente. SE encontrar um caso desses, por favor " +#~ "entre em contacto com o mantainer deste pacote Debian." === added file 'storage/xtradb/build/debian/po/ro.po' --- a/storage/xtradb/build/debian/po/ro.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/ro.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,319 @@ +# Romanian translation of mysql-dfsg. +# Copyright (C) 2006 THE mysql-dfsg'S COPYRIGHT HOLDER +# This file is distributed under the same license as the mysql-dfsg package. +# +# Stan Ioan-Eugen <stan.ieugen(a)gmail.com>, 2006. +msgid "" +msgstr "" +"Project-Id-Version: po-debconf://mysql-dfsg\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2006-12-20 21:27+0200\n" +"Last-Translator: stan ioan-eugen <stan.ieugen(a)gmail.com>\n" +"Language-Team: romanian <debian-l10n-romanian(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"X-Generator: KBabel 1.11.4\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "Do you really want to downgrade?" +msgid "Really proceed with downgrade?" +msgstr "Sunteţi sigur că doriţi să instalaţi o versiune mai veche?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "" +#| "WARNING: The file /var/lib/mysql/debian-*.flag exists. This indicates " +#| "that a mysql-server package with a higher version has been installed " +#| "before. It can not be guaranteed that this version can use its data." +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"AVERTISMENT: Fişierul /var/lib/mysql/debian-*.flag există. Acest lucru " +"indică faptul că anterior a fost instalată o versiune nouă a pachetului " +"mysql-server. Nu se poate garanta că versiunea instalată acum poate folosi " +"datele versiunii instalate anterior." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "Notă importantă pentru utilizatorii NIS/YP!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The script is about to remove the data directory /var/lib/mysql. If it is " +#| "planned to just install a higher MySQL version or if a different mysql-" +#| "server package is already using it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Scriptul urmează să şteargă directorul de date /var/lib/mysql. Dacă plănuiţi " +"doar să instalaţi o versiune nouă MySQL sau datele sunt folosite de către un " +"alt pachet mysql-server, atunci ar trebui păstraţi datele." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Doriţi ca MySQL să pornească la initializarea sistemului?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL can start automatically on boot time or only if you manually " +#| "type '/etc/init.d/mysql start'." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL poate porni automat la iniţializarea sistemului sau doar dacă rulaţi " +"comanda „/etc/init.d/mysql start”." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Noua parolă pentru utilizatorul „root” al MySQL:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "It is highly recommended that you set a password for the MySQL " +#| "administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Este recomandat să stabiliţi o parolă pentru utilizatorul administrativ " +"„root” al MySQL." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Noua parolă pentru utilizatorul „root” al MySQL:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Nu s-a putut stabili parola pentru utilizatorul „root” al MySQL" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "It seems an error occurred while setting the password for the MySQL " +#| "administrative user. This may have happened because the user already has " +#| "a password, or because there was a problem communicating with the MySQL " +#| "server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Se pare că a intervenit o eroare în stabilirea parolei pentru utilizatorul " +"administrativ al MySQL. Acest lucru se poate întâmpla dacă utilizatorul are " +"deja o parolă, sau a existat o problemă în comunicarea cu serverul MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "Cannot upgrade if ISAM tables are present!" +#~ msgstr "Nu se poate face actualizarea dacă sunt prezente tabele ISAM!" + +#~ msgid "" +#~ "Recent versions of MySQL can no longer use the old ISAM table format and " +#~ "it is necessary to convert your tables to e.g. MyISAM before upgrading by " +#~ "using \"mysql_convert_table_format\" or \"ALTER TABLE x ENGINE=MyISAM\". " +#~ "The installation of mysql-server-5.1 will now abort. In case your old " +#~ "mysql-server-4.1 gets removed nevertheless just reinstall it to convert " +#~ "those tables." +#~ msgstr "" +#~ "Versiunile recente MySQL nu mai pot folosi vechiul format de tabele ISAM " +#~ "şieste necesar să convertiţi tabelele dumneavoastră de ex. la formatul " +#~ "MyISAM înainte de a face actualizarea folosind comanda " +#~ "„mysql_convert_table_format” sau „ALTER TABLE x ENGINE=MyISAM”. " +#~ "Instalarea mysql-server-5.1 va eşua. În caz că ştergeţiversiunea " +#~ "anterioară mysql-server-4.1 va trebui reinstalată pentru a converti " +#~ "tabelele." + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Suportaţi conexiuni MySQL de la staţii ce rulează sistemul Debian „sarge” " +#~ "sau mai vechi?" + +#, fuzzy +#~| msgid "" +#~| "The way passwords were stored was not very secure. This has been " +#~| "improved with the drawback that clients (e.g. PHP) from hosts running " +#~| "Debian 3.1 Sarge will not be able to connect to account which are new or " +#~| "whose password have been changed. See /usr/share/doc/mysql-server-5.1/" +#~| "README.Debian." +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Modul în care erau păstrate parolele nu era foarte sigur. Acest lucru a " +#~ "fost îmbunătăţitcu dezajantajul că clienţii (de ex. PHP) de pe staţii ce " +#~ "rulează sistemul Debian 3.1 Sargenu se vor putea conecta la conturi noi " +#~ "sau ale căror parole au fost schimbate. Citiţi /usr/share/doc/mysql-" +#~ "server-5.1/README.Debian." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Pentru a folosi mysql trebuie să adăugaţi un utilizator şi grup " +#~ "echivalent şi să vă asiguraţi că /var/lib/mysql are permisiunile " +#~ "stabilite corect (uid/gid pot aveavalori diferite)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd:\tmysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group:\tmysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql:\tdrwxr-xr-x\tmysql\tmysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Doriţi să ştergeţi bazele de date folosite de toate versiune MySQL?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "Dacă nu introduceţi nici o parolă, nici o schimbare nu va fi luată în " +#~ "considerare." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "După finalizarea instalării, ar trebui să verificaţi dacă contul este " +#~ "protejat cu o parolă (citiţi fişierul README.Debian pentru informaţii " +#~ "suplimentare)." === added file 'storage/xtradb/build/debian/po/ru.po' --- a/storage/xtradb/build/debian/po/ru.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/ru.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,305 @@ +# translation of mysql-dfsg-5.1_5.0.32-6_ru.po to Russian +# Russian messages: +# Translators, if you are not familiar with the PO format, gettext +# documentation is worth reading, especially sections dedicated to +# this format, e.g. by running: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry'# +# Some information specific to po-debconf are available at +# /usr/share/doc/po-debconf/README-trans +# or http://www.debian.org/intl/l10n/po-debconf/README-trans# +# Developers do not need to manually edit POT or PO files. +# Ilgiz Kalmetev <translator(a)ilgiz.pp.ru>, 2003. +# Yuriy Talakan' <yt(a)amur.elektra.ru>, 2005, 2006. +# Yuriy Talakan' <yt(a)drsk.ru>, 2007. +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1_5.0.32-6_ru\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-02-19 11:28+0900\n" +"Last-Translator: Yuriy Talakan' <yt(a)drsk.ru>\n" +"Language-Team: Russian <debian-l10n-russian(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"X-Generator: KBabel 1.9.1\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "Do you really want to downgrade?" +msgid "Really proceed with downgrade?" +msgstr "Вы действительно желаете понизить версию?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "" +#| "WARNING: The file /var/lib/mysql/debian-*.flag exists. This indicates " +#| "that a mysql-server package with a higher version has been installed " +#| "before. It can not be guaranteed that this version can use its data." +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"ВНИМАНИЕ: Найден файл /var/lib/mysql/debian-*.flag. Это означает, что ранее " +"был установлен пакет mysql-server более высокой версии. Невозможно " +"гарантировать, что текущая версия сможет использовать его данные." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "Важное замечание для пользователей NIS/YP!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The script is about to remove the data directory /var/lib/mysql. If it is " +#| "planned to just install a higher MySQL version or if a different mysql-" +#| "server package is already using it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Сценарий собирается удалить директорию данных /var/lib/mysql. Если " +"планируется установить новую версию MySQL или есть другие пакеты mysql-" +"server, использующие эту директорию, то данные надо сохранить." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Запускать MySQL при загрузке системы?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL can start automatically on boot time or only if you manually " +#| "type '/etc/init.d/mysql start'." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL может запускаться при загрузке системы, либо только если вы вручную " +"введете команду '/etc/init.d/mysql start'. " + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Новый пароль для MySQL пользователя \"root\":" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "It is highly recommended that you set a password for the MySQL " +#| "administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Крайне рекомендуется установить пароль для административного MySQL " +"пользователя \"root\"." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Новый пароль для MySQL пользователя \"root\":" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Невозможно задать пароль MySQL пользователю \"root\"" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "It seems an error occurred while setting the password for the MySQL " +#| "administrative user. This may have happened because the user already has " +#| "a password, or because there was a problem communicating with the MySQL " +#| "server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"В процессе задания пароля административного MySQL пользователя произошла " +"ошибка. Это могло произойти если у пользователя уже был задан пароль, или " +"из-за проблем соединения с сервером MySQL." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Нужна поддержка подключений к MySQL с машин, работающих под Debian \"sarge" +#~ "\" или старше?" + +#, fuzzy +#~| msgid "" +#~| "The way passwords were stored was not very secure. This has been " +#~| "improved with the drawback that clients (e.g. PHP) from hosts running " +#~| "Debian 3.1 Sarge will not be able to connect to account which are new or " +#~| "whose password have been changed. See /usr/share/doc/mysql-server-5.1/" +#~| "README.Debian." +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "Метод хранения пароля был не очень безопасен. Это было сделано из-за " +#~ "того, клиенты (например, PHP) с машин, работающих под Debian 3.1 Sarge не " +#~ "смогут подключиться к учетной записи если она новая, или пароль был " +#~ "изменен. См. /usr/share/doc/mysql-server-5.1/README.Debian." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Чтобы использовать mysql, Вы должны установить эквивалентные пользователя " +#~ "и группу, как указано ниже и убедиться, что /var/lib/mysql имеет " +#~ "правильные права (uid/gid могут отличаться)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Удалить базы данных, используемые всеми версиями MySQL?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "Если вы не зададите пароль, то учетная запись не будет изменена." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "Когда установка завершится, вы должны убедиться, что учетная запись " +#~ "защищена паролем (подробную информацию см. в README.Debian)." === added file 'storage/xtradb/build/debian/po/sv.po' --- a/storage/xtradb/build/debian/po/sv.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/sv.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,400 @@ +# Translators, if you are not familiar with the PO format, gettext +# documentation is worth reading, especially sections dedicated to +# this format, e.g. by running: +# info -n '(gettext)PO Files' +# info -n '(gettext)Header Entry' +# Some information specific to po-debconf are available at +# /usr/share/doc/po-debconf/README-trans +# or http://www.debian.org/intl/l10n/po-debconf/README-trans +# Developers do not need to manually edit POT or PO files. +# , fuzzy +# +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-5.1 5.0.21-3\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2007-02-18 14:48+0100\n" +"Last-Translator: Andreas Henriksson <andreas(a)fatal.se>\n" +"Language-Team: Swedish <tp-sv(a)listor.tp-sv.se>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=iso-8859-1\n" +"Content-Transfer-Encoding: 8bit\n" +"X-Poedit-Language: Swedish\n" +"X-Poedit-Country: SWEDEN\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "Do you really want to downgrade?" +msgid "Really proceed with downgrade?" +msgstr "Vill du verkligen nedgradera?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +#, fuzzy +#| msgid "" +#| "WARNING: The file /var/lib/mysql/debian-*.flag exists. This indicates " +#| "that a mysql-server package with a higher version has been installed " +#| "before. It can not be guaranteed that this version can use its data." +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" +"VARNING: Filen /var/lib/mysql/debian-*.flag existerar. Detta betyder att " +"paketet mysql-server med h� versionsnummer har installerats tidigare. Det " +"kan inte garanteras att denna version kan anv�a dess data." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "Viktig notering f�IS/YP-anv�are!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +#, fuzzy +#| msgid "" +#| "The script is about to remove the data directory /var/lib/mysql. If it is " +#| "planned to just install a higher MySQL version or if a different mysql-" +#| "server package is already using it, the data should be kept." +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" +"Scriptet kommer strax ta bort data-katalogen /var/lib/mysql. Om det " +"planerade var att bara installera en h� MySQL-version eller om ett annan " +"mysql-server paket redan anv�e det, skall datan sparas." + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "Ska MySQL startas n�systemet startar upp?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "" +#| "The MySQL can start automatically on boot time or only if you manually " +#| "type '/etc/init.d/mysql start'." +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL kan startas n�systemet startas upp eller endast om du manuellt " +"skriver '/etc/init.d/mysql start'." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "New password for the Percona SQL \"root\" user:" +msgstr "Nytt l�ord f�ySQLs \"root\"-anv�are:" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +#, fuzzy +#| msgid "" +#| "It is highly recommended that you set a password for the MySQL " +#| "administrative \"root\" user." +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" +"Det �starkt rekommenderat att du s�er ett l�ord f�ySQLs " +"administrativa \"root\"-anv�are." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +#, fuzzy +#| msgid "New password for MySQL \"root\" user:" +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "Nytt l�ord f�ySQLs \"root\"-anv�are:" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "Unable to set password for MySQL \"root\" user" +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "Lyckades inte s�a l�ord f�ySQLs \"root\"-anv�are" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "It seems an error occurred while setting the password for the MySQL " +#| "administrative user. This may have happened because the user already has " +#| "a password, or because there was a problem communicating with the MySQL " +#| "server." +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" +"Det verkar som ett fel uppstod n�det skulle s�as ett l�ord f�ySQLs " +"administrativa anv�are. Detta kan ha skett f�tt anv�aren redan har " +"ett l�ord satt, eller p�rund av problem med att kommunicera med MySQL-" +"servern." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +#, fuzzy +#| msgid "" +#| "See /usr/share/doc/mysql-server-5.1/README.Debian for more information." +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "Se /usr/share/doc/mysql-server-5.1/README.Debian f�er information." + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "Cannot upgrade if ISAM tables are present!" +#~ msgstr "Kan inte uppgradera om ISAM-tabeller finns!" + +#~ msgid "" +#~ "Recent versions of MySQL can no longer use the old ISAM table format and " +#~ "it is necessary to convert your tables to e.g. MyISAM before upgrading by " +#~ "using \"mysql_convert_table_format\" or \"ALTER TABLE x ENGINE=MyISAM\". " +#~ "The installation of mysql-server-5.1 will now abort. In case your old " +#~ "mysql-server-4.1 gets removed nevertheless just reinstall it to convert " +#~ "those tables." +#~ msgstr "" +#~ "Senaste versionerna av MySQL kan inte l�re anv�a gamla ISAM-" +#~ "tabellformatet och det �n�ndigt att konvertera dina tabeller till " +#~ "exempelvis MyISAM f�uppgradering med \"mysql_convert_table_format\" " +#~ "eller \"ALTER TABLE x ENGINE=MyISAM\". Installationen av mysql-server-5.1 " +#~ "kommer nu att avbrytas. Om ditt gamla mysql-server-4.1-paket tas bort �" +#~ "det bara att installera om det f�tt konvertera de tabellerna." + +#~ msgid "" +#~ "Support MySQL connections from hosts running Debian \"sarge\" or older?" +#~ msgstr "" +#~ "Beh� du MySQL-anslutningar fr�system som k�ebian \"Sarge\" eller " +#~ "�re?" + +#, fuzzy +#~| msgid "" +#~| "The way passwords were stored was not very secure. This has been " +#~| "improved with the drawback that clients (e.g. PHP) from hosts running " +#~| "Debian 3.1 Sarge will not be able to connect to account which are new or " +#~| "whose password have been changed. See /usr/share/doc/mysql-server-5.1/" +#~| "README.Debian." +#~ msgid "" +#~ "In old versions of MySQL clients on Debian, passwords were not stored " +#~ "securely. This has been improved since then, however clients (such as " +#~ "PHP) from hosts running Debian 3.1 Sarge will not be able to connect to " +#~ "recent accounts or accounts whose password have been changed." +#~ msgstr "" +#~ "S�et som l�orden lagrades p�ar inte s�kilt s�rt. Detta har " +#~ "f�ttrats p�ekostnad av att klienter (t.ex. PHP) fr�system som k� +#~ "Debian 3.1 Sarge inte kan ansluta till konton som �nya eller vars " +#~ "l�ord har �rats. Se /usr/share/doc/mysql-server-5.1/README.Debian." + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "F�tt anv�a MySQL m�e du installera en motsvarande anv�are och " +#~ "grupp till f�nde och se till att /var/lib/mysql har korrekta " +#~ "r�igheter satta (uid/gid kan vara olika)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#~ msgid "Remove the databases used by all MySQL versions?" +#~ msgstr "Ta bort databaserna som anv�s av alla MySQL-versioner?" + +#~ msgid "" +#~ "If you do not provide a password no changes will be made to the account." +#~ msgstr "" +#~ "Om du inte anger ett l�ord kommer inga �ringar att g� f�ontot." + +#~ msgid "" +#~ "When installation finishes, you should verify that the account is " +#~ "properly protected with a password (see README.Debian for more " +#~ "information)." +#~ msgstr "" +#~ "N�installationen �klar, b�u kontrollera s�tt kontot �riktigt " +#~ "skyddat av ett l�ord (l�README.Debian f�er information)." + +#~ msgid "Update Hints" +#~ msgstr "Uppdateringstips" + +#~ msgid "" +#~ "You have to run \"mysql_upgrade\" after the upgrade, else tables can be " +#~ "corrupted! This script also enhances the privilege tables but is not " +#~ "supposed to give any user more rights that he had before," +#~ msgstr "" +#~ "Du m�e k�\"mysql_upgrade\" efter uppgraderingen, annars kan " +#~ "tabellerna vara skadade! Detta skript ut� �n privilegietabellerna " +#~ "men �inte t�te att ge n�n anv�are mer befogenhet �vad han hade " +#~ "tidigare," + +#~ msgid "Please also read http://www.mysql.com/doc/en/Upgrade.html" +#~ msgstr "L��n http://www.mysql.com/doc/en/Upgrade.html" + +#~ msgid "Install Hints" +#~ msgstr "Installationstips" + +#~ msgid "" +#~ "On upgrades from MySQL 3.23, as shipped with Debian Woody, symlinks in " +#~ "place of /var/lib/mysql or /var/log/mysql gets accidently removed and " +#~ "have manually be restored." +#~ msgstr "" +#~ "Vid uppgraderingar fr�MySQL 3.23 som skickades med Debian Woody har " +#~ "symboliska l�ar i /var/lib/mysql eller /var/log/mysql av misstag tagits " +#~ "bort och m�e manuellt �rskapas." + +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "MySQL kan endast installeras om du har ett icke-numeriskt v�namn som " +#~ "kan sl�upp via filen /etc/hosts. Exempelvis om kommandot \"hostname\" " +#~ "returnerar \"mittnamn\" s��et finnas en rad som liknar \"10.0.0.1 " +#~ "mittnamn\"." + +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "En ny MySQL-anv�are kallad \"debian-sys-maint\" kommer att skapas. " +#~ "Detta MySQL-konto anv�s f�tart/stopp och cron-skript. Ta inte bort " +#~ "det." + +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "T� p�tt s�a ett L�ENORD f�ySQL:s root-anv�are! Om du " +#~ "anv�er /root/.my.cnf, skriv d�lltid en \"user\"-rad och en \"password" +#~ "\"-rad i den, aldrig med endast l�ordet!" + +#~ msgid "" +#~ "Should I remove the complete /var/lib/mysql directory tree which is used " +#~ "by all MySQL versions, not necessarily only the one you are about to " +#~ "purge?" +#~ msgstr "" +#~ "Ska jag ta bort hela katalogtr�t i /var/lib/mysql som anv�s av alla " +#~ "MySQL-versioner och inte bara f�en som du nu kommer att rensa ut?" + +#~ msgid "" +#~ "Rarely, e.g. on new major versions, the privilege system is improved. To " +#~ "make use of it mysql_fix_privilege_tables must be executed manually. The " +#~ "script is not supposed to give any user more rights that he had before," +#~ msgstr "" +#~ "S�an, exempelvis i nya st� versioner, har beh�hetssystemet " +#~ "f�ttrats. F�tt anv�a det m�e skriptet " +#~ "mysql_fix_privilege_tables k� manuellt. Skriptet �inte t�t att ge " +#~ "n�n anv�are h� beh�het �han hade tidigare." === added file 'storage/xtradb/build/debian/po/templates.pot' --- a/storage/xtradb/build/debian/po/templates.pot 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/templates.pot 2010-02-18 23:51:40 +0000 @@ -0,0 +1,187 @@ +# SOME DESCRIPTIVE TITLE. +# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER +# This file is distributed under the same license as the PACKAGE package. +# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR. +# +#, fuzzy +msgid "" +msgstr "" +"Project-Id-Version: PACKAGE VERSION\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" +"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" +"Language-Team: LANGUAGE <LL(a)li.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=CHARSET\n" +"Content-Transfer-Encoding: 8bit\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "Important note for NIS/YP users" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +msgid "Start the Percona SQL server on boot?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "New password for the Percona SQL \"root\" user:" +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" === added file 'storage/xtradb/build/debian/po/tr.po' --- a/storage/xtradb/build/debian/po/tr.po 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/po/tr.po 2010-02-18 23:51:40 +0000 @@ -0,0 +1,342 @@ +# Turkish translation of mysql-server. +# This file is distributed under the same license as the mysql-server package. +# Gürkan Aslan <gurkan(a)iaslan.com>, 2004 +# +msgid "" +msgstr "" +"Project-Id-Version: mysql-dfsg-4.1\n" +"Report-Msgid-Bugs-To: percona-xtradb-dfsg-5.1(a)packages.debian.org\n" +"POT-Creation-Date: 2010-02-15 17:10-0500\n" +"PO-Revision-Date: 2004-06-05 08:53+0300\n" +"Last-Translator: Gürkan Aslan <gurkan(a)iaslan.com>\n" +"Language-Team: Turkish <debian-l10n-turkish(a)lists.debian.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Plural-Forms: nplurals=1; plural=0;\n" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "Really proceed with downgrade?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "A file named /var/lib/mysql/debian-*.flag exists on this system." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"Such file is an indication that a mysql-server package with a higher version " +"has been installed earlier." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:2001 +msgid "" +"There is no guarantee that the version you're currently installing will be " +"able to use the current databases." +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +#, fuzzy +#| msgid "Important note for NIS/YP users!" +msgid "Important note for NIS/YP users" +msgstr "NIS/YP kullanıcıları için önemli not!" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"To use MySQL, the following entries for users and groups should be added to " +"the system:" +msgstr "" + +#. Type: note +#. Description +#: ../percona-xtradb-server-5.1.templates:3001 +msgid "" +"You should also check the permissions and the owner of the /var/lib/mysql " +"directory:" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "Remove all Percona SQL databases?" +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"The /var/lib/mysql directory which contains the Percona SQL databases is " +"about to be removed." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:4001 +msgid "" +"If you're removing the Percona SQL package in order to later install a more " +"recent version or if a different mysql-server package is already using it, " +"the data should be kept." +msgstr "" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +#| msgid "Should MySQL start on boot?" +msgid "Start the Percona SQL server on boot?" +msgstr "MySQL açılış sırasında başlatılsın mı?" + +#. Type: boolean +#. Description +#: ../percona-xtradb-server-5.1.templates:5001 +#, fuzzy +msgid "" +"The Percona SQL server can be launched automatically at boot time or " +"manually with the '/etc/init.d/mysql start' command." +msgstr "" +"MySQL açılış sırasında veya '/etc/init.d/mysql start' komutunu vermeniz " +"halinde elle başlatılabilir. Eğer açılışta otomatik olarak başlatılmasını " +"istiyorsanız burada 'evet'i seçin." + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "New password for the Percona SQL \"root\" user:" +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "" +"While not mandatory, it is highly recommended that you set a password for " +"the Percona SQL administrative \"root\" user." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:6001 +msgid "If that field is left blank, the password will not be changed." +msgstr "" + +#. Type: password +#. Description +#: ../percona-xtradb-server-5.1.templates:7001 +msgid "Repeat password for the Percona SQL \"root\" user:" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "Unable to set password for the Percona SQL \"root\" user" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"An error occurred while setting the password for the Percona SQL " +"administrative user. This may have happened because the account already has " +"a password, or because of a communication problem with the Percona SQL " +"server." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "You should check the account's password after the package installation." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:8001 +msgid "" +"Please read the /usr/share/doc/mysql-server-5.1/README.Debian file for more " +"information." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "Password input error" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:9001 +msgid "The two passwords you entered were not the same. Please try again." +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "NDB Cluster seems to be in use" +msgstr "" + +#. Type: error +#. Description +#: ../percona-xtradb-server-5.1.templates:10001 +msgid "" +"Percona-SQL-5.1 has orphaned NDB Cluster support. Please migrate to the new " +"mysql-cluster package and remove all lines starting with \"ndb\" from all " +"config files below /etc/mysql/." +msgstr "" + +#~ msgid "" +#~ "To use mysql you must install an equivalent user and group to the " +#~ "following and ensure yourself that /var/lib/mysql has the right " +#~ "permissions (the uid/gid may be different)." +#~ msgstr "" +#~ "Mysql'i kullanmak için aşağıdakiyle eşdeğer bir kullanıcı ve grup " +#~ "tanımlamalı, ve /var/lib/mysql izinlerinin uygun şekilde ayarlandığından " +#~ "emin olmalısınız (uid/gid farklı olabilir)." + +#~ msgid "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" +#~ msgstr "" +#~ "/etc/passwd: mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false" + +#~ msgid "/etc/group: mysql:x:101:" +#~ msgstr "/etc/group: mysql:x:101:" + +#~ msgid "/var/lib/mysql: drwxr-xr-x mysql mysql" +#~ msgstr "/var/lib/mysql: drwxr-xr-x mysql mysql" + +#, fuzzy +#~ msgid "Please also read http://www.mysql.com/doc/en/Upgrade.html" +#~ msgstr "Lütfen http://www.mysql.com/doc/en/Upgrade.html belgesini okuyun" + +#, fuzzy +#~ msgid "" +#~ "MySQL will only install if you have a non-numeric hostname that is " +#~ "resolvable via the /etc/hosts file. E.g. if the \"hostname\" command " +#~ "returns \"myhostname\" then there must be a line like \"10.0.0.1 " +#~ "myhostname\"." +#~ msgstr "" +#~ "MySQL sadece /etc/hosts dosyası yoluyla çözülebilir NUMERİK OLMAYAN bir " +#~ "makine adına sahipseniz kurulacaktır. Örneğin, eğer \"hostname\" komutu " +#~ "\"makinem\" ismini döndürüyorsa, bu dosya içinde \"10.0.0.1 makinem\" " +#~ "gibi bir satır olmalıdır." + +#, fuzzy +#~ msgid "" +#~ "A new mysql user \"debian-sys-maint\" will be created. This mysql account " +#~ "is used in the start/stop and cron scripts. Don't delete." +#~ msgstr "" +#~ "Yeni mysql kullanıcısı \"debian-sys-maint\" yaratılacak. Bu hesap, " +#~ "başlangıç betiklerinde ve cron içinde kullanılıyor. Bu hesabı silmeyin." + +#, fuzzy +#~ msgid "" +#~ "Please remember to set a PASSWORD for the MySQL root user! If you use a /" +#~ "root/.my.cnf, always write the \"user\" and the \"password\" lines in " +#~ "there, never only the password!" +#~ msgstr "" +#~ "Lütfen MySQL root kullanıcısı için bir PAROLA girmeyi unutmayın! Eğer /" +#~ "root/.my.cnf kullanıyorsanız, \"user\" ve \"password\" satırlarını her " +#~ "zaman buraya ekleyin, sadece parolayı değil! Daha fazla bilgi için /usr/" +#~ "share/doc/mysql-server/README.Debian dosyasını okuyun." + +#, fuzzy +#~ msgid "" +#~ "Should I remove all databases below /var/lib/mysql as you are purging the " +#~ "mysql-server package?" +#~ msgstr "" +#~ "mysql-server paketi kaldırıldıktan sonra bütün veritabanları silinsin mi?" + +#~ msgid "" +#~ "Networking is disabled by default for security reasons. You can enable it " +#~ "by commenting out the skip-networking option in /etc/mysql/my.cnf." +#~ msgstr "" +#~ "Ağ, öntanımlı olarak güvenlik gerekçeleriyle devre dışı bırakıldı. Bu " +#~ "özelliği /etc/mysql/my.cnf dosyası içindeki \"skip-networking\" " +#~ "seçeneğini kaldırarak etkinleştirebilirsiniz." + +#~ msgid "security and update notice" +#~ msgstr "güvenlik ve güncelleme duyurusu" + +#~ msgid "" +#~ "Should I remove everything below /var/lib/mysql when you purge the mysql-" +#~ "server package with the \"dpkg --purge mysql-server\" command (i.e. " +#~ "remove everything including the configuration) somewhen? (default is not)" +#~ msgstr "" +#~ "mysql-server paketini temizlemek için \"dpkg --purge mysql-server\" " +#~ "komutunu kullandığınızda (yani yapılandırma dahil herşeyi silmek) /var/" +#~ "lib/mysql altındaki herşeyi sileyim mi? (öntanımlı cevap hayır'dır)." + +#~ msgid "Please run mysql_fix_privilege_tables !" +#~ msgstr "Lütfen mysql_fix_privilege_tables komutunu çalıştırın!" + +#~ msgid "" +#~ "I will ensure secure permissions of /var/lib/mysql by replacing GIDs " +#~ "other than root and mysql with mysql." +#~ msgstr "" +#~ "/var/lib/mysql'in izinlerinin güvenli olmasını sağlamak amacıyla, buna " +#~ "ait GID'leri root ve mysql'den farklı olacak şekilde değiştireceğim." + +#~ msgid "" +#~ "Instructions how to enable SSL support are in /usr/share/doc/mysql-server/" +#~ msgstr "" +#~ "SSL desteğini nasıl etkinleştirebileceğinize ilişkin talimatlar /usr/" +#~ "share/doc/mysql-server/ içinde." + +#~ msgid "mysql_fix_privileges_tables will be executed" +#~ msgstr "mysql_fix_privileges_tables çalıştırılacak" + +#~ msgid "" +#~ "The latest MySQL versions have an enhanced, more fine grained, privilege " +#~ "system. To make use of it, some new fields must be added to the tables " +#~ "in the \"mysql\" database. This is done by the " +#~ "mysql_fix_privilege_tables script during this upgrade regardless of if " +#~ "the server is currently running or not!" +#~ msgstr "" +#~ "En son MySQL sürümleri zenginleştirilmiş, daha ayrıntılandırılmış bir " +#~ "ayrıcalık (privilege) sistemine sahiptir. Yeni sistemi kullanmak için, " +#~ "\"mysql\" veritabanındaki tablolara bazı yeni alanlar eklenmelidir. Bu " +#~ "işlem, sunucunun çalışıp çalışmamasına bağlı olmaksızın " +#~ "mysql_fix_privilege_tables betiği tarafından bu yükseltme sırasında " +#~ "yapılır." + +#~ msgid "" +#~ "This script is not supposed to give any user more rights that he had " +#~ "before, if you encounter such a case, please contact me." +#~ msgstr "" +#~ "Bu betiğin hiç bir kullanıcıya öncekinden daha fazla hak kazandırmadığı " +#~ "varsayılıyor. Eğer bunun aksinde bir durumla karşılaşırsanız, lütfen " +#~ "benimle bağlantıya geçin." + +#~ msgid "Make MySQL reachable via network?" +#~ msgstr "MySQL network üzerinden ulaşılabilir olsun mu?" + +#~ msgid "" +#~ "Should MySQL listen on a network reachable TCP port? This is not " +#~ "necessary for use on a single computer and could be a security problem." +#~ msgstr "" +#~ "MySQL ağ üzerinde ulaşılabilen bir TCP portunu dinlesin mi? Tek olan bir " +#~ "bilgisayar için bu ayar gerekli değildir ve bir güvenlik sorunu " +#~ "oluşturabilir." + +#~ msgid "Enable chroot mode?" +#~ msgstr "chroot kipi etkinleştirilsin mi?" + +#~ msgid "" +#~ "MySQL is able to jail itself into the /var/lib/mysql_jail directory so " +#~ "that users cannot modify any files outside this directory. This improves " +#~ "resistence against crackers, too, as they are not able to modify system " +#~ "files." +#~ msgstr "" +#~ "MySQL kendini /var/lib/mysql_jail dizinine hapsederek kullanıcıların bu " +#~ "dizin dışındaki hiç bir dosyayı değiştirmemesini sağlayabilir. Bu " +#~ "düzenleme, sistem dosyalarını değiştirmelerini engelleyeceğinden, " +#~ "cracker'lara karşı dayanıklılığı arttırır." === added file 'storage/xtradb/build/debian/rules' --- a/storage/xtradb/build/debian/rules 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/rules 2010-02-18 23:51:40 +0000 @@ -0,0 +1,322 @@ +#!/usr/bin/make -f + +export DH_VERBOSE=1 + +PACKAGE=percona-xtradb-dfsg-5.1 + +include /usr/share/dpatch/dpatch.make + +TMP=$(CURDIR)/debian/tmp/ + +ARCH = $(shell dpkg-architecture -qDEB_BUILD_ARCH) +ARCH_OS = $(shell dpkg-architecture -qDEB_BUILD_ARCH_OS) +DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) +DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) +DEBVERSION = $(shell dpkg-parsechangelog | awk '/^Version: / { print $$2 }' | sed 's/^.*-//' ) + +DEB_SOURCE_PACKAGE ?= $(strip $(shell egrep '^Source: ' debian/control | cut -f 2 -d ':')) +DEB_VERSION ?= $(shell dpkg-parsechangelog | egrep '^Version:' | cut -f 2 -d ' ') +DEB_NOEPOCH_VERSION ?= $(shell echo $(DEB_VERSION) | cut -d: -f2-) +DEB_UPSTREAM_VERSION ?= $(shell echo $(DEB_NOEPOCH_VERSION) | sed 's/-[^-]*$$//') +DEB_UPSTREAM_VERSION_MAJOR_MINOR := $(shell echo $(DEB_UPSTREAM_VERSION) | sed -r -n 's/^([0-9]+\.[0-9]+).*/\1/p') + +DISTRIBUTION = $(shell echo "Percona SQL Server (GPL), XtraDB $(BB_PERCONA_VERSION), Revision $(BB_PERCONA_REVISION)") + +MAKE_J = -j$(shell if [ -f /proc/cpuinfo ] ; then grep -c processor.* /proc/cpuinfo ; else echo 1 ; fi) +ifeq (${MAKE_J}, -j0) + MAKE_J = -j1 +endif + +MAKE_TEST_TARGET=test-force +ifneq ($(findstring $(DEB_BUILD_OPTIONS),fulltest),) +# make test-bt is the testsuite run by the MySQL build team +# before a release, but it is long + MAKE_TEST_TARGET=test-bt +endif + +USE_ASSEMBLER=--enable-assembler + +#ifneq ($(findstring $(ARCH), alpha amd64 arm armel ia64 i386 hppa mipsel powerpc s390 sparc),) +# TESTSUITE_FAIL_CMD=true +#else + TESTSUITE_FAIL_CMD=exit 1 +#endif + +# This causes seg11 crashes if LDAP is used for groups in /etc/nsswitch.conf +# so it is disabled by default although, according to MySQL, it brings >10% +# performance gain if enabled. See #299382. +ifeq ($(STATIC_MYSQLD), 1) + USE_STATIC_MYSQLD=--with-mysqld-ldflags=-all-static +endif + +configure: patch configure-stamp +configure-stamp: + @echo "RULES.configure-stamp" + dh_testdir + +ifneq ($(ARCH_OS),hurd) + if [ ! -d /proc/self ]; then echo "/proc IS NEEDED" 1>&2; exit 1; fi +endif + + sh -c 'PATH=$${MYSQL_BUILD_PATH:-"/bin:/usr/bin"} \ + CC=$${MYSQL_BUILD_CC:-gcc} \ + CFLAGS=$${MYSQL_BUILD_CFLAGS:-"-O3 -DBIG_JOINS=1 ${FORCE_FPIC_CFLAGS}"} \ + CXX=$${MYSQL_BUILD_CXX:-g++} \ + CXXFLAGS=$${MYSQL_BUILD_CXXFLAGS:-"-O3 -DBIG_JOINS=1 -felide-constructors -fno-exceptions -fno-rtti ${FORCE_FPIC_CFLAGS}"} \ + ./configure \ + --build=${DEB_BUILD_GNU_TYPE} \ + --host=${DEB_HOST_GNU_TYPE} \ + \ + --prefix=/usr \ + --exec-prefix=/usr \ + --libexecdir=/usr/sbin \ + --datadir=/usr/share \ + --localstatedir=/var/lib/mysql \ + --includedir=/usr/include \ + --infodir=/usr/share/info \ + --mandir=/usr/share/man \ + \ + --with-server-suffix="-$(DEBVERSION)" \ + --with-comment="($(DISTRIBUTION))" \ + --with-system-type="debian-linux-gnu" \ + \ + --enable-shared \ + --enable-static \ + --enable-thread-safe-client \ + $(USE_ASSEMBLER) \ + --enable-local-infile \ + $(FORCE_FPIC) \ + --with-fast-mutexes \ + --with-big-tables \ + --with-unix-socket-path=/var/run/mysqld/mysqld.sock \ + --with-mysqld-user=mysql \ + --with-libwrap \ + $(USE_STATIC_MYSQLD) \ + --with-ssl \ + --without-docs \ + --with-extra-charsets=all \ + --with-plugins=max-no-ndb \ + \ + --without-embedded-server \ + --with-embedded-privilege-control' + + # --sysconfdir=/etc/mysql -- Appends /etc/mysql after ~/ in the my.cnf search patch! + # + # --with-debug + + touch configure-stamp + + +build: build-stamp +build-stamp: configure + dh_testdir + + $(MAKE) $(MAKE_J) + +ifeq ($(findstring $(DEB_BUILD_OPTIONS),nocheck),) + if [ ! -f testsuite-stamp ] ; then \ + $(MAKE) $(MAKE_TEST_TARGET) || $(TESTSUITE_FAIL_CMD) ; \ + fi +endif + + touch testsuite-stamp + + touch build-stamp + + +clean: clean-patched unpatch + rm -rf debian/patched +clean-patched: + @echo "RULES.clean-patched" + dh_testdir + dh_testroot + rm -f configure-stamp + rm -f build-stamp + rm -f testsuite-stamp + + [ ! -f Makefile ] || $(MAKE) clean + [ ! -d mysql-test/var ] || rm -rf mysql-test/var + + # We like to see how long this is neccessary + @echo "CRUFT BEGIN" + @find -type l -print0 | xargs --no-run-if-empty -0 rm -v + @find -name .deps -type d -print0 | xargs --no-run-if-empty -0 rm -rfv + @rm -vrf ndb/docs/.doxy* ndb/docs/*html ndb/docs/*pdf innobase/autom4te.cache + @for i in \ + readline/Makefile \ + sql-bench/Makefile \ + scripts/make_win_binary_distribution \ + scripts/mysqlbug \ + sql/gen_lex_hash \ + sql/lex_hash.h \ + strings/ctype_autoconf.c \ + config.log \ + config.cache \ + ; \ + do \ + rm -vf $$i; \ + done + @echo "CRUFT END" + + debconf-updatepo + dh_clean -v + + +install: +install: build + @echo "RULES.install" + dh_testdir + dh_testroot + dh_clean -k + dh_installdirs + + # some self written manpages which hopefully + # gets overwritten sooner or later with upstreams + mkdir -p $(TMP)/usr/share/man/man1/ + mkdir -p $(TMP)/usr/share/man/man8/ + cp debian/additions/*.1 $(TMP)/usr/share/man/man1/ + mkdir -p $(TMP)/etc/mysql/conf.d/ + cp debian/additions/mysqld_safe_syslog.cnf $(TMP)/etc/mysql/conf.d/ + ln -s mysqlmanager.1 $(TMP)/usr/share/man/man1/mysqlmanager-pwgen.1 + ln -s mysqlmanager.1 $(TMP)/usr/share/man/man1/mysqlmanagerc.1 + + # make install (trailing slash needed for innobase) + $(MAKE) install DESTDIR=$(TMP)/ + + # After installing, remove rpath to make lintian happy. + set +e; \ + find ./debian/tmp/ -type f -print0 \ + | xargs -0 --no-run-if-empty chrpath -k 2>/dev/null \ + | fgrep RPATH= \ + | cut -d: -f 1 \ + | xargs --no-run-if-empty chrpath -d; \ + set -e + + # libmysqlclient: move shared libraries (but not the rest like libheap.a & co) + mv $(TMP)/usr/lib/mysql/libmysqlclient* $(TMP)/usr/lib + perl -pi -e 's#/usr/lib/mysql#/usr/lib#' $(TMP)/usr/lib/libmysqlclient.la + perl -pi -e 's#/usr/lib/mysql#/usr/lib#' $(TMP)/usr/lib/libmysqlclient_r.la + # Check if our beloved versioned symbols are really there + if [ "`objdump -T $(TMP)/usr/lib/libmysqlclient.so.16.0.0 | grep -c libmysqlclient_16`" -lt 500 ]; then \ + echo "ERROR: versioned symbols are absent"; \ + exit 1; \ + fi + + # libmysqlclient-dev: forgotten header file since 3.23.25? + cp include/my_config.h $(TMP)/usr/include/mysql/ + cp include/my_dir.h $(TMP)/usr/include/mysql/ + + # percona-xtradb-common: We now provide our own config file. + install -d $(TMP)/etc/mysql + install -m 0644 debian/additions/my.cnf $(TMP)/etc/mysql/my.cnf + + # percona-xtradb-client + install -m 0755 debian/additions/mysqlreport $(TMP)/usr/bin/ + install -m 0755 debian/additions/innotop/innotop $(TMP)/usr/bin/ + install -m 0644 debian/additions/innotop/innotop.1 $(TMP)/usr/share/man/man1/ + install -m 0644 -D debian/additions/innotop/InnoDBParser.pm $(TMP)/usr/share/perl5/InnoDBParser.pm + + # percona-xtradb-server + install -m 0755 scripts/mysqld_safe $(TMP)/usr/bin/mysqld_safe + mkdir -p $(TMP)/usr/share/doc/percona-xtradb-server-5.1/examples + mv $(TMP)/usr/share/mysql/*cnf $(TMP)/usr/share/doc/percona-xtradb-server-5.1/examples/ + rm -vf $(TMP)/usr/share/mysql/mi_test_all* \ + $(TMP)/usr/share/mysql/mysql-log-rotate \ + $(TMP)/usr/share/mysql/mysql.server \ + $(TMP)/usr/share/mysql/binary-configure + nm -n sql/mysqld |gzip -9 > $(TMP)/usr/share/doc/percona-xtradb-server-5.1/mysqld.sym.gz + install -m 0755 debian/additions/echo_stderr $(TMP)/usr/share/mysql/ + install -m 0755 debian/additions/debian-start $(TMP)/etc/mysql/ + install -m 0755 debian/additions/debian-start.inc.sh $(TMP)/usr/share/mysql/ + # lintian overrides + mkdir -p $(TMP)/usr/share/lintian/overrides/ + cp debian/percona-xtradb-common.lintian-overrides $(TMP)/usr/share/lintian/overrides/percona-xtradb-common + cp debian/percona-xtradb-server-5.1.lintian-overrides $(TMP)/usr/share/lintian/overrides/percona-xtradb-server-5.1 + cp debian/percona-xtradb-client-5.1.lintian-overrides $(TMP)/usr/share/lintian/overrides/percona-xtradb-client-5.1 + + # For 5.0 -> 5.1 transition + d=$(TMP)/usr/share/percona-xtradb-common/internal-use-only/; \ + mkdir -p $$d; \ + cp debian/percona-xtradb-server-5.1.mysql.init $$d/_etc_init.d_mysql; \ + cp debian/percona-xtradb-server-5.1.logrotate $$d/_etc_logrotate.d_percona-xtradb-server; \ + cp debian/additions/debian-start $$d/_etc_mysql_debian-start; + + dh_movefiles + +# Build architecture-independent files here. +binary-indep: build install + @echo "RULES.binary-indep" + dh_testdir -i + dh_testroot -i + dh_installdebconf -i + dh_installdocs -i + dh_installexamples -i + dh_installmenu -i + dh_installlogrotate -i + dh_installinit -i + dh_installcron -i + dh_installman -i + dh_installinfo -i + dh_installlogcheck -i + dh_installchangelogs -i + dh_link -i + dh_compress -i + dh_fixperms -i + dh_installdeb -i + dh_perl -i + dh_gencontrol -i + dh_md5sums -i + dh_builddeb -i + +# Build architecture-dependent files here. +binary-arch: build install + @echo "RULES.binary-arch" + dh_testdir + dh_testroot + + dh_installdebconf -a + dh_installdocs -a + dh_installexamples -a + dh_installmenu -a + dh_installlogrotate -a --name percona-xtradb-server + # Start mysql in runlevel 19 before 20 where apache, proftpd etc gets + # started which might depend on a running database server. + dh_installinit -a --name=mysql -- defaults 19 21 + dh_installcron -a --name percona-xtradb-server + dh_installman -a + dh_installinfo -a + dh_installlogcheck -a + dh_installchangelogs -a + dh_strip -a + dh_link -a # .so muss nach .so.1.2.3 installier werden! + dh_compress -a + dh_fixperms -a + dh_makeshlibs -a + dh_makeshlibs -plibmysqlclient16 -V'libmysqlclient16 (>= 5.1.21-1)' + dh_installdeb -a + dh_perl -a + dh_shlibdeps -a -l debian/libmysqlclient16/usr/lib -L libmysqlclient16 + dh_gencontrol -a + dh_md5sums -a + dh_builddeb -a + +source diff: + @echo >&2 'source and diff are obsolete - use dpkg-source -b'; false + +binary: binary-indep binary-arch + +get-orig-source: + @wget -nv -T10 -t3 \ + -O /tmp/mysql-$(DEB_UPSTREAM_VERSION).tar.gz \ + http://ftp.gwdg.de/pub/misc/mysql/Downloads/MySQL-$(DEB_UPSTREAM_VERSION_MA… + @tar xfz /tmp/mysql-$(DEB_UPSTREAM_VERSION).tar.gz -C /tmp + @rm -rf /tmp/mysql-$(DEB_UPSTREAM_VERSION)/Docs + @rm -rf /tmp/mysql-$(DEB_UPSTREAM_VERSION)/debian + @mv /tmp/mysql-$(DEB_UPSTREAM_VERSION) /tmp/$(DEB_SOURCE_PACKAGE)-$(DEB_UPSTREAM_VERSION).orig + @cd /tmp ; tar czf $(DEB_SOURCE_PACKAGE)_$(DEB_UPSTREAM_VERSION).orig.tar.gz $(DEB_SOURCE_PACKAGE)-$(DEB_UPSTREAM_VERSION).orig + @rm -f /tmp/mysql-$(DEB_UPSTREAM_VERSION).tar.gz + @rm -rf /tmp/$(DEB_SOURCE_PACKAGE)-$(DEB_UPSTREAM_VERSION).orig + +.PHONY: clean clean-patched configure build binary binary-indep binary-arch install patch unpatch + +# vim: ts=8 === added file 'storage/xtradb/build/debian/source.lintian-overrides' --- a/storage/xtradb/build/debian/source.lintian-overrides 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/source.lintian-overrides 2010-01-26 18:02:46 +0000 @@ -0,0 +1,2 @@ +maintainer-script-lacks-debhelper-token debian/mysql-server-5.1.postinst +maintainer-script-lacks-debhelper-token debian/mysql-server-5.1.postrm === added file 'storage/xtradb/build/debian/watch' --- a/storage/xtradb/build/debian/watch 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/debian/watch 2010-01-26 18:02:46 +0000 @@ -0,0 +1,3 @@ +version=3 +opts="uversionmangle=s/-(rc|beta)/$1/" \ + ftp://sunsite.informatik.rwth-aachen.de/pub/mirror/www.mysql.com/Downloads/… debian === added file 'storage/xtradb/build/percona-sql.spec' --- a/storage/xtradb/build/percona-sql.spec 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/build/percona-sql.spec 2010-02-18 18:47:04 +0000 @@ -0,0 +1,1635 @@ +############################################################################# +# +# This is the spec file for the distribution specific RPM files +# +############################################################################## + +############################################################################## +# Some common macro definitions +############################################################################## + +# Required arguments +# mysqlversion - e.g. 5.1.37 +# pluginversion - Version of InnoDB plugin taken as the basis, e.g. 1.0.3 +# redhatversion - 5 or 4 +# xtradbversion - The XtraDB release, eg. 6 +# gotrevision - bzr revision of the sources the package is built of + +%define mysql_vendor Percona, Inc +%{!?redhatversion:%define redhatversion 5} +%define distribution rhel%{redhatversion} +%define release %{xtradbversion}.%{distribution} + +%define mysqld_user mysql +%define mysqld_group mysql +%define mysqldatadir /var/lib/mysql +%define see_base For a description of MySQL see the base MySQL RPM or http://www.mysql.com + +# ------------------------------------------------------------------------------ +# Meta information, don't remove! +# ------------------------------------------------------------------------------ +# norootforbuild + +# ------------------------------------------------------------------------------ +# On SuSE 9 no separate "debuginfo" package is built. To enable basic +# debugging on that platform, we don't strip binaries on SuSE 9. We +# disable the strip of binaries by redefining the RPM macro +# "__os_install_post" leaving out the script calls that normally does +# this. We do this in all cases, as on platforms where "debuginfo" is +# created, a script "find-debuginfo.sh" will be called that will do +# the strip anyway, part of separating the executable and debug +# information into separate files put into separate packages. +# +# Some references (shows more advanced conditional usage): +# http://www.redhat.com/archives/rpm-list/2001-November/msg00257.html +# http://www.redhat.com/archives/rpm-list/2003-February/msg00275.html +# http://www.redhat.com/archives/rhl-devel-list/2004-January/msg01546.html +# http://lists.opensuse.org/archive/opensuse-commit/2006-May/1171.html +# ------------------------------------------------------------------------------ +%define __os_install_post /usr/lib/rpm/brp-compress + +# ------------------------------------------------------------------------------ +# We don't package all files installed into the build root by intention - +# See BUG#998 for details. +# ------------------------------------------------------------------------------ +%define _unpackaged_files_terminate_build 0 + +# ------------------------------------------------------------------------------ +# RPM build tools now automatically detects Perl module dependencies. This +# detection gives problems as it is broken in some versions, and it also +# give unwanted dependencies from mandatory scripts in our package. +# Might not be possible to disable in all RPM tool versions, but here we +# try. We keep the "AutoReqProv: no" for the "test" sub package, as disabling +# here might fail, and that package has the most problems. +# See http://fedoraproject.org/wiki/Packaging/Perl#Filtering_Requires:_and_Provid… +# http://www.wideopen.com/archives/rpm-list/2002-October/msg00343.html +# ------------------------------------------------------------------------------ +%undefine __perl_provides +%undefine __perl_requires + +############################################################################## +# Command line handling +############################################################################## + +# ---------------------------------------------------------------------- +# use "rpmbuild --with yassl" or "rpm --define '_with_yassl 1'" (for RPM 3.x) +# to build with yaSSL support (off by default) +# ---------------------------------------------------------------------- +%{?_with_yassl:%define YASSL_BUILD 1} +%{!?_with_yassl:%define YASSL_BUILD 0} + +# ---------------------------------------------------------------------- +# use "rpmbuild --without libgcc" or "rpm --define '_without_libgcc 1'" (for RPM 3.x) +# to include libgcc (as libmygcc) (on by default) +# ---------------------------------------------------------------------- +%{!?_with_libgcc: %{!?_without_libgcc: %define WITH_LIBGCC 1}} +%{?_with_libgcc:%define WITH_LIBGCC 1} +%{?_without_libgcc:%define WITH_LIBGCC 0} + + +# On SuSE 9 no separate "debuginfo" package is built. To enable basic +# debugging on that platform, we don't strip binaries on SuSE 9. We +# disable the strip of binaries by redefining the RPM macro +# "__os_install_post" leaving out the script calls that normally does +# this. We do this in all cases, as on platforms where "debuginfo" is +# created, a script "find-debuginfo.sh" will be called that will do +# the strip anyway, part of separating the executable and debug +# information into separate files put into separate packages. +# +# Some references (shows more advanced conditional usage): +# http://www.redhat.com/archives/rpm-list/2001-November/msg00257.html +# http://www.redhat.com/archives/rpm-list/2003-February/msg00275.html +# http://www.redhat.com/archives/rhl-devel-list/2004-January/msg01546.html +# http://lists.opensuse.org/archive/opensuse-commit/2006-May/1171.html + +%define __os_install_post /usr/lib/rpm/brp-compress + +%define server_suffix -51 +%define package_suffix -51 +%define ndbug_comment Percona SQL Server (GPL), XtraDB %{xtradbversion}, Revision %{gotrevision} +%define debug_comment Percona SQL Server - Debug (GPL), XtraDB %{xtradbversion}, Revision %{gotrevision} +%define commercial 0 +%define YASSL_BUILD 1 +%define EMBEDDED_BUILD 0 +%define PARTITION_BUILD 1 +%define CLUSTER_BUILD 0 +%define COMMUNITY_BUILD 1 +%define INNODB_BUILD 1 +%define PERCONA_PLUGIN_BUILD 1 +%define MARIA_BUILD 0 +%define NORMAL_TEST_MODE test-bt +%define DEBUG_TEST_MODE test-bt-debug + +%define BUILD_DEBUG 0 + + +%if %{COMMUNITY_BUILD} +%define cluster_package_prefix -cluster +%else +%define cluster_package_prefix - +%endif + +%define lic_type GNU GPL v2 +%define lic_files COPYING README +%define src_dir mysql-%{mysqlversion} + +Source1: percona-xtradb-%{pluginversion}-%{xtradbversion}.tar.gz +Patch0: percona-support.patch + +Patch01: show_patches.patch +Patch02: slow_extended.patch +Patch03: profiling_slow.patch +Patch04: microsec_process.patch +Patch05: userstat.patch +Patch06: optimizer_fix.patch +Patch07: mysql-test_for_xtradb.diff + + +%define perconaxtradbplugin percona-xtradb-%{pluginversion}-%{xtradbversion}.tar.gz + +############################################################################## +# Main spec file section +############################################################################## + +Name: Percona-XtraDB%{package_suffix} +Summary: Percona-XtraDB: a very fast and reliable SQL database server +Group: Applications/Databases +Version: %{mysqlversion} +Release: %{release} +Distribution: Red Hat Enterprise Linux %{redhatversion} +License: GPL version 2 http://www.gnu.org/licenses/gpl-2.0.html +Source: %{src_dir}.tar.gz +URL: http://www.percona.com/ +Packager: %{mysql_vendor} MySQL Development Team <mysql-dev(a)percona.com> +Vendor: %{mysql_vendor} +Provides: msqlormysql MySQL-server mysql Percona-XtraDB-server +BuildRequires: gperf perl readline-devel gcc-c++ ncurses-devel zlib-devel libtool automake autoconf time ccache + +# Think about what you use here since the first step is to +# run a rm -rf +BuildRoot: %{_tmppath}/%{name}-%{version}-build + +# From the manual +%description +The Percona-XtraDB software delivers a very fast, multi-threaded, multi-user, +and robust SQL (Structured Query Language) database server. Percona-XtraDB Server +is intended for mission-critical, heavy-load production systems as well +as for embedding into mass-deployed software. + +Percona Inc. provides commercial support of Percona-XtraDB Server. +For more information visist our web site http://www.percona.com/ + +############################################################################## +# Sub package definition +############################################################################## + +%package -n Percona-XtraDB-server%{package_suffix} +Summary: %{ndbug_comment} for Red Hat Enterprise Linux %{redhatversion} +Group: Applications/Databases +Requires: chkconfig coreutils shadow-utils grep procps +Provides: msqlormysql mysql-server mysql MySQL MySQL-server Percona-XtraDB-server +Obsoletes: MySQL mysql mysql-server MySQL-server MySQL-server-community MySQL-server-percona + +%description -n Percona-XtraDB-server%{package_suffix} +The Percona-XtraDB software delivers a very fast, multi-threaded, multi-user, +and robust SQL (Structured Query Language) database server. Percona-XtraDB Server +is intended for mission-critical, heavy-load production systems as well +as for embedding into mass-deployed software. + +Percona Inc. provides commercial support of Percona-XtraDB Server. +For more information visist our web site http://www.percona.com/ + +This package includes the Percona-XtraDB server binary +%if %{INNODB_BUILD} +(configured including XtraDB) +%endif +as well as related utilities to run and administer a Percona-XtraDB server. + +If you want to access and work with the database, you have to install +package "Percona-XtraDB-client%{package_suffix}" as well! + +# ------------------------------------------------------------------------------ + +%package -n Percona-XtraDB-client%{package_suffix} +Summary: Percona-XtraDB - Client +Group: Applications/Databases +Obsoletes: mysql-client MySQL-client MySQL-client-community MySQL-client-percona +Provides: mysql-client MySQL-client Percona-XtraDB-client + +%description -n Percona-XtraDB-client%{package_suffix} +This package contains the standard Percona-XtraDB clients and administration tools. + +%{see_base} + + +# ------------------------------------------------------------------------------ + +%package -n Percona-XtraDB-test%{package_suffix} +Requires: mysql-client perl +Summary: Percona-XtraDB - Test suite +Group: Applications/Databases +Provides: mysql-test MySQL-test Percona-XtraDB-test +Obsoletes: mysql-test MySQL-test MySQL-test-community MySQL-test-percona +AutoReqProv: no + +%description -n Percona-XtraDB-test%{package_suffix} +This package contains the Percona-XtraDB regression test suite. + +%{see_base} + +# ------------------------------------------------------------------------------ + +%package -n Percona-XtraDB-devel%{package_suffix} +Summary: Percona-XtraDB - Development header files and libraries +Group: Applications/Databases +Provides: mysql-devel MySQL-devel Percona-XtraDB-devel +Obsoletes: mysql-devel MySQL-devel MySQL-devel-community MySQL-devel-percona + +%description -n Percona-XtraDB-devel%{package_suffix} +This package contains the development header files and libraries +necessary to develop Percona-XtraDB client applications. + +%{see_base} + +# ------------------------------------------------------------------------------ + +%package -n Percona-XtraDB-shared%{package_suffix} +Summary: Percona-XtraDB - Shared libraries +Group: Applications/Databases +Provides: mysql-shared MySQL-shared Percona-XtraDB-shared +# Obsoletes below to correct old missing Provides:/Obsoletes +Obsoletes: mysql-shared MySQL-shared-standard MySQL-shared-pro +Obsoletes: MySQL-shared-pro-cert MySQL-shared-pro-gpl +Obsoletes: MySQL-shared-pro-gpl-cert MySQL-shared MySQL-shared-community MySQL-shared-percona + +%description -n Percona-XtraDB-shared%{package_suffix} +This package contains the shared libraries (*.so*) which certain +languages and applications need to dynamically load and use MySQL. + +# ------------------------------------------------------------------------------ + +%if %{PERCONA_PLUGIN_BUILD} + +%package -n Percona-XtraDB-%{pluginversion}-%{xtradbversion} +Requires: Percona-XtraDB-devel +Summary: Percona XtraDB Storage engine for MySQL +Group: Applications/Databases +Provides: percona-xtradb-plugin Percona-XtraDB-plugin +Obsoletes: percona-xtradb-plugin Percona-XtraDB-plugin + +%description -n Percona-XtraDB-%{pluginversion}-%{xtradbversion} +This package contains the Percona-XtraDB storage engine for MySQL server. + +An enhanced version of the InnoDB storage engine, including all +of InnoDB's robust, reliable ACID-compliant design and advanced +MVCC architecture, and builds on that solid foundation with more +features, more tunability, more metrics, and more scalability. +In particular, it is designed to scale better on many cores, +to use memory more efficiently, and to be more convenient and useful. +The new features are especially designed to reduce the need for +awkward workarounds to many of InnoDB's limitations. We choose +features and fixes based on customer requests and on our best +judgment as a high-performance consulting company. + +%endif + +############################################################################## +# +############################################################################## + +%prep + +%setup -n %{src_dir} + +%patch01 -p1 +%patch02 -p1 +%patch03 -p1 +%patch04 -p1 +%patch05 -p1 +%patch06 -p1 +%patch07 -p1 + +if [ "%{redhatversion}" = "5" ] ; then +tar xfz $RPM_SOURCE_DIR/%{perconaxtradbplugin} -C storage/innobase --strip-components=1 +else +tar xfz $RPM_SOURCE_DIR/%{perconaxtradbplugin} -C storage/innobase --strip-path=1 +fi +%patch0 -p1 + +cd storage/innobase && bash -x ./setup.sh + +############################################################################## +# The actual build +############################################################################## + +%build + +BuildMySQL() { +# Get flags from environment. RPM_OPT_FLAGS seems not to be set anywhere. +CFLAGS=${CFLAGS:-$RPM_OPT_FLAGS} +CXXFLAGS=${CXXFLAGS:-$RPM_OPT_FLAGS} +# Evaluate current setting of $DEBUG +if [ $DEBUG -gt 0 ] ; then + OPT_COMMENT='--with-comment="%{debug_comment}"' + OPT_DEBUG='--with-debug' + CFLAGS=`echo " $CFLAGS " | \ + sed -e 's/ -O[0-9]* / /' -e 's/ -unroll2 / /' -e 's/ -ip / /' \ + -e 's/^ //' -e 's/ $//'` + CXXFLAGS=`echo " $CXXFLAGS " | \ + sed -e 's/ -O[0-9]* / /' -e 's/ -unroll2 / /' -e 's/ -ip / /' \ + -e 's/^ //' -e 's/ $//'` +else + OPT_COMMENT='--with-comment="%{ndbug_comment}"' + OPT_DEBUG='' +fi + +echo "BUILD ==================" +echo $* + +# The --enable-assembler simply does nothing on systems that does not +# support assembler speedups. +sh -c "CFLAGS=\"$CFLAGS\" \ + CXXFLAGS=\"$CXXFLAGS\" \ + AM_CPPFLAGS=\"$AM_CPPFLAGS\" \ + LDFLAGS=\"$LDFLAGS\" \ + ./configure \ + $* \ + --enable-assembler \ + --enable-local-infile \ + --with-mysqld-user=%{mysqld_user} \ + --with-unix-socket-path=/var/lib/mysql/mysql.sock \ + --with-pic \ + --prefix=/ \ +%if %{CLUSTER_BUILD} + --with-extra-charsets=all \ +%else + --with-extra-charsets=complex \ +%endif +%if %{YASSL_BUILD} + --with-ssl \ +%else + --without-ssl \ +%endif + --exec-prefix=%{_exec_prefix} \ + --libexecdir=%{_sbindir} \ + --libdir=%{_libdir} \ + --sysconfdir=%{_sysconfdir} \ + --datadir=%{_datadir} \ + --localstatedir=%{mysqldatadir} \ + --infodir=%{_infodir} \ + --includedir=%{_includedir} \ + --mandir=%{_mandir} \ + --enable-thread-safe-client \ + --enable-profiling \ +%if %{?ndbug_comment:1}0 + $OPT_COMMENT \ +%endif + $OPT_DEBUG \ +%if %{commercial} + --with-libedit \ +%else + --with-readline \ +%endif + ; make " +} +# end of function definition "BuildMySQL" + + +BuildServer() { +BuildMySQL "--enable-shared \ +%if %{?server_suffix:1}0 + --with-server-suffix='%{server_suffix}' \ +%endif +%if %{CLUSTER_BUILD} + --with-plugin-ndbcluster \ +%else + --without-plugin-ndbcluster \ +%endif +%if %{MARIA_BUILD} + --with-plugin-maria \ + --with-maria-tmp-tables \ +%else + --without-plugin-maria \ +%endif +%if %{INNODB_BUILD} + --with-plugin-innobase \ + --without-plugin-innodb_plugin \ +%else + --without-plugin-innobase \ + --without-plugin-innodb_plugin \ +%endif +%if %{PARTITION_BUILD} + --with-plugin-partition \ +%else + --without-plugin-partition \ +%endif + --with-plugin-csv \ + --with-plugin-archive \ + --with-plugin-blackhole \ + --with-plugin-federated \ +%if %{EMBEDDED_BUILD} + --with-embedded-server \ +%else + --without-embedded-server \ +%endif + --without-bench \ + --with-zlib-dir=bundled \ + --with-big-tables" + +if [ -n "$MYSQL_CONFLOG_DEST" ] ; then + cp -fp config.log "$MYSQL_CONFLOG_DEST" +fi + +#if [ -f sql/.libs/mysqld ] ; then +# nm --numeric-sort sql/.libs/mysqld > sql/mysqld.sym +#else +# nm --numeric-sort sql/mysqld > sql/mysqld.sym +#fi +} +# end of function definition "BuildServer" + + +RBR=$RPM_BUILD_ROOT +MBD=$RPM_BUILD_DIR/%{src_dir} + +# Clean up the BuildRoot first +[ "$RBR" != "/" ] && [ -d $RBR ] && rm -rf $RBR; +mkdir -p $RBR%{_libdir}/mysql $RBR%{_sbindir} + +# Use gcc for C and C++ code (to avoid a dependency on libstdc++ and +# including exceptions into the code +if [ -z "$CXX" -a -z "$CC" ] ; then + export CC="gcc" CXX="gcc" +fi + +if [ "%{redhatversion}" = "5" ] ; then +export CFLAGS="-static-libgcc -O2 -fno-omit-frame-pointer -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -mtune=generic" +export CXXFLAGS="-static-libgcc -O2 -fno-omit-frame-pointer -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -mtune=generic" +fi + +if [ "%{redhatversion}" != "5" ] ; then +export CFLAGS="-static-libgcc -O2 -g -fno-omit-frame-pointer -pipe " +export CXXFLAGS="-static-libgcc -O2 -g -fno-omit-frame-pointer -pipe " +fi + + +# Create the shared libs seperately to avoid a dependency for the client utilities +DEBUG=0 +BuildMySQL "--enable-shared" + +# Install shared libraries +cp -av libmysql/.libs/*.so* $RBR/%{_libdir} +cp -av libmysql_r/.libs/*.so* $RBR/%{_libdir} +mkdir -p $RBR%{_libdir}/mysql/plugin +cp -av storage/innobase/.libs/*.so* $RBR%{_libdir}/mysql/plugin +cp -av storage/innobase/scripts/install_innodb_plugins.sql $RBR%{_libdir}/mysql/plugin + +pushd $RBR%{_libdir}/mysql +tar cfz percona-xtradb-%{pluginversion}-%{xtradbversion}-%{mysqlversion}.$RPM_ARCH.tar.gz plugin +mv percona-xtradb-%{pluginversion}-%{xtradbversion}-%{mysqlversion}.$RPM_ARCH.tar.gz %{_topdir} +popd + +############################################################################## + +# Include libgcc.a in the devel subpackage (BUG 4921) +%if %{WITH_LIBGCC} +libgcc=`$CC $CFLAGS --print-libgcc-file` +install -m 644 "$libgcc" $RBR%{_libdir}/mysql/libmygcc.a +%endif + +############################################################################## + +# Now create a debug server +%if %{BUILD_DEBUG} +DEBUG=1 +make clean + +( BuildServer ) # subshell, so that CFLAGS + CXXFLAGS are modified only locally + +if [ "$MYSQL_RPMBUILD_TEST" != "no" ] ; then + MTR_BUILD_THREAD=auto make %{DEBUG_TEST_MODE} +fi + +# Get the debug server and its .sym file from the build tree +#if [ -f sql/.libs/mysqld ] ; then +# cp sql/.libs/mysqld $RBR%{_sbindir}/mysqld-debug +#else +# cp sql/mysqld $RBR%{_sbindir}/mysqld-debug +#fi +#cp libmysqld/libmysqld.a $RBR%{_libdir}/mysql/libmysqld-debug.a +#cp sql/mysqld.sym $RBR%{_libdir}/mysql/mysqld-debug.sym + +%endif + +# Now, the default server +DEBUG=0 +make clean + +BuildServer + +if [ "$MYSQL_RPMBUILD_TEST" != "no" ] ; then + MTR_BUILD_THREAD=auto make %{NORMAL_TEST_MODE} +fi + +# Now, build plugin +#BUILDSO=0 +#make clean + +#BuildServer + +#if [ "$MYSQL_RPMBUILD_TEST" != "no" ] ; then +# MTR_BUILD_THREAD=auto make %{NORMAL_TEST_MODE} +#fi + +%install +RBR=$RPM_BUILD_ROOT +MBD=$RPM_BUILD_DIR/%{src_dir} + +# Ensure that needed directories exists +install -d $RBR%{_sysconfdir}/{logrotate.d,init.d} +install -d $RBR%{mysqldatadir}/mysql +install -d $RBR%{_datadir}/mysql-test +install -d $RBR%{_datadir}/mysql/SELinux/RHEL4 +install -d $RBR%{_includedir} +install -d $RBR%{_libdir} +install -d $RBR%{_mandir} +install -d $RBR%{_sbindir} +install -d $RBR%{_libdir}/mysql/plugin + +make DESTDIR=$RBR benchdir_root=%{_datadir} install + +# install symbol files ( for stack trace resolution) +#install -m644 $MBD/sql/mysqld.sym $RBR%{_libdir}/mysql/mysqld.sym + +# Install logrotate and autostart +install -m644 $MBD/support-files/mysql-log-rotate \ + $RBR%{_sysconfdir}/logrotate.d/mysql +install -m755 $MBD/support-files/mysql.server \ + $RBR%{_sysconfdir}/init.d/mysql + +# in RPMs, it is unlikely that anybody should use "sql-bench" +rm -fr $RBR%{_datadir}/sql-bench + +# Create a symlink "rcmysql", pointing to the init.script. SuSE users +# will appreciate that, as all services usually offer this. +ln -s %{_sysconfdir}/init.d/mysql $RBR%{_sbindir}/rcmysql + +# Touch the place where the my.cnf config file and mysqlmanager.passwd +# (MySQL Instance Manager password file) might be located +# Just to make sure it's in the file list and marked as a config file +touch $RBR%{_sysconfdir}/my.cnf +touch $RBR%{_sysconfdir}/mysqlmanager.passwd + +# Install SELinux files in datadir +install -m600 $MBD/support-files/RHEL4-SElinux/mysql.{fc,te} \ + $RBR%{_datadir}/mysql/SELinux/RHEL4 + + +############################################################################## +# Post processing actions, i.e. when installed +############################################################################## + +%pre -n Percona-XtraDB-server%{package_suffix} +# Check if we can safely upgrade. An upgrade is only safe if it's from one +# of our RPMs in the same version family. + +installed=`rpm -q --whatprovides mysql-server 2> /dev/null` +if [ $? -eq 0 -a -n "$installed" ]; then + vendor=`rpm -q --queryformat='%{VENDOR}' "$installed" 2>&1` + version=`rpm -q --queryformat='%{VERSION}' "$installed" 2>&1` + myvendor='%{mysql_vendor}' + myversion='%{mysqlversion}' + + old_family=`echo $version | sed -n -e 's,^$[1-9][0-9]*\.[0-9][0-9]*$\..*$,\1,p'` + new_family=`echo $myversion | sed -n -e 's,^$[1-9][0-9]*\.[0-9][0-9]*$\..*$,\1,p'` + + [ -z "$vendor" ] && vendor='<unknown>' + [ -z "$old_family" ] && old_family="<unrecognized version $version>" + [ -z "$new_family" ] && new_family="<bad package specification: version $myversion>" + + error_text= +# if [ "$vendor" != "$myvendor" ]; then +# error_text="$error_text +#The current MySQL server package is provided by a different +#vendor ($vendor) than $myvendor. Some files may be installed +#to different locations, including log files and the service +#startup script in %{_sysconfdir}/init.d/. +#" +# fi + + if [ "$old_family" != "$new_family" ]; then + error_text="$error_text +Upgrading directly from MySQL $old_family to MySQL $new_family may not +be safe in all cases. A manual dump and restore using mysqldump is +recommended. It is important to review the MySQL manual's Upgrading +section for version-specific incompatibilities. +" + fi + + if [ -n "$error_text" ]; then + cat <<HERE >&2 + +****************************************************************** +A MySQL server package ($installed) is installed. +$error_text +A manual upgrade is required. + +- Ensure that you have a complete, working backup of your data and my.cnf + files +- Shut down the MySQL server cleanly +- Remove the existing MySQL packages. Usually this command will + list the packages you should remove: + rpm -qa | grep -i '^mysql-' + + You may choose to use 'rpm --nodeps -ev <package-name>' to remove + the package which contains the mysqlclient shared library. The + library will be reinstalled by the MySQL-shared-compat package. +- Install the new MySQL packages supplied by $myvendor +- Ensure that the MySQL server is started +- Run the 'mysql_upgrade' program + +This is a brief description of the upgrade process. Important details +can be found in the MySQL manual, in the Upgrading section. +****************************************************************** +HERE + exit 1 + fi +fi + +# Shut down a previously installed server first +if [ -x %{_sysconfdir}/init.d/mysql ] ; then + %{_sysconfdir}/init.d/mysql stop > /dev/null 2>&1 + echo "Giving mysqld 5 seconds to exit nicely" + sleep 5 +fi + +%post -n Percona-XtraDB-server%{package_suffix} +mysql_datadir=%{mysqldatadir} + +# ---------------------------------------------------------------------- +# Create data directory +# ---------------------------------------------------------------------- +mkdir -p $mysql_datadir/{mysql,test} + +# ---------------------------------------------------------------------- +# Make MySQL start/shutdown automatically when the machine does it. +# ---------------------------------------------------------------------- +if [ -x /sbin/chkconfig ] ; then + /sbin/chkconfig --add mysql +fi + +# ---------------------------------------------------------------------- +# Create a MySQL user and group. Do not report any problems if it already +# exists. +# ---------------------------------------------------------------------- +groupadd -r %{mysqld_group} 2> /dev/null || true +useradd -M -r -d $mysql_datadir -s /bin/bash -c "MySQL server" -g %{mysqld_group} %{mysqld_user} 2> /dev/null || true +# The user may already exist, make sure it has the proper group nevertheless (BUG#12823) +usermod -g %{mysqld_group} %{mysqld_user} 2> /dev/null || true + +# ---------------------------------------------------------------------- +# Change permissions so that the user that will run the MySQL daemon +# owns all database files. +# ---------------------------------------------------------------------- +chown -R %{mysqld_user}:%{mysqld_group} $mysql_datadir + +# ---------------------------------------------------------------------- +# Initiate databases +# ---------------------------------------------------------------------- +%{_bindir}/mysql_install_db --rpm --user=%{mysqld_user} + +# ---------------------------------------------------------------------- +# FIXME upgrade databases if needed would go here - but it cannot be +# automated yet +# ---------------------------------------------------------------------- + +# ---------------------------------------------------------------------- +# Change permissions again to fix any new files. +# ---------------------------------------------------------------------- +chown -R %{mysqld_user}:%{mysqld_group} $mysql_datadir + +# ---------------------------------------------------------------------- +# Fix permissions for the permission database so that only the user +# can read them. +# ---------------------------------------------------------------------- +chmod -R og-rw $mysql_datadir/mysql + +# ---------------------------------------------------------------------- +# install SELinux files - but don't override existing ones +# ---------------------------------------------------------------------- +SETARGETDIR=/etc/selinux/targeted/src/policy +SEDOMPROG=$SETARGETDIR/domains/program +SECONPROG=$SETARGETDIR/file_contexts/program +if [ -f /etc/redhat-release ] && \ + (grep -q "Red Hat Enterprise Linux .. release 4" /etc/redhat-release \ + || grep -q "CentOS release 4" /etc/redhat-release) ; then + echo + echo + echo 'Notes regarding SELinux on this platform:' + echo '=========================================' + echo + echo 'The default policy might cause server startup to fail because it is ' + echo 'not allowed to access critical files. In this case, please update ' + echo 'your installation. ' + echo + echo 'The default policy might also cause inavailability of SSL related ' + echo 'features because the server is not allowed to access /dev/random ' + echo 'and /dev/urandom. If this is a problem, please do the following: ' + echo + echo ' 1) install selinux-policy-targeted-sources from your OS vendor' + echo ' 2) add the following two lines to '$SEDOMPROG/mysqld.te':' + echo ' allow mysqld_t random_device_t:chr_file read;' + echo ' allow mysqld_t urandom_device_t:chr_file read;' + echo ' 3) cd to '$SETARGETDIR' and issue the following command:' + echo ' make load' + echo + echo +fi + +if [ -x sbin/restorecon ] ; then + sbin/restorecon -R var/lib/mysql +fi + +# Restart in the same way that mysqld will be started normally. +if [ -x %{_sysconfdir}/init.d/mysql ] ; then + %{_sysconfdir}/init.d/mysql start + echo "Giving mysqld 2 seconds to start" + sleep 2 +fi + +# Allow mysqld_safe to start mysqld and print a message before we exit +sleep 2 + +%if %{CLUSTER_BUILD} +%post -n MySQL%{cluster_package_prefix}storage%{package_suffix} +# Create cluster directory if needed +mkdir -p /var/lib/mysql-cluster +%endif + +%preun -n Percona-XtraDB-server%{package_suffix} +if [ $1 = 0 ] ; then + # Stop MySQL before uninstalling it + if [ -x %{_sysconfdir}/init.d/mysql ] ; then + %{_sysconfdir}/init.d/mysql stop > /dev/null + # Don't start it automatically anymore + if [ -x /sbin/chkconfig ] ; then + /sbin/chkconfig --del mysql + fi + fi +fi + +# We do not remove the mysql user since it may still own a lot of +# database files. + +# ---------------------------------------------------------------------- +# Clean up the BuildRoot after build is done +# ---------------------------------------------------------------------- +%clean +[ "$RPM_BUILD_ROOT" != "/" ] && [ -d $RPM_BUILD_ROOT ] && rm -rf $RPM_BUILD_ROOT; + +############################################################################## +# Files section +############################################################################## + +%files -n Percona-XtraDB-server%{package_suffix} +%defattr(-,root,root,0755) + +%doc %{lic_files} +%doc support-files/my-*.cnf +%if %{CLUSTER_BUILD} +%doc support-files/ndb-*.ini +%endif + +%doc %attr(644, root, root) %{_infodir}/mysql.info* + +%if %{INNODB_BUILD} +%doc %attr(644, root, man) %{_mandir}/man1/innochecksum.1* +%endif +%doc %attr(644, root, man) %{_mandir}/man1/my_print_defaults.1* +%doc %attr(644, root, man) %{_mandir}/man1/myisam_ftdump.1* +%doc %attr(644, root, man) %{_mandir}/man1/myisamchk.1* +%doc %attr(644, root, man) %{_mandir}/man1/myisamlog.1* +%doc %attr(644, root, man) %{_mandir}/man1/myisampack.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_convert_table_format.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_fix_extensions.1* +%doc %attr(644, root, man) %{_mandir}/man8/mysqld.8* +%doc %attr(644, root, man) %{_mandir}/man1/mysqld_multi.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqld_safe.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_fix_privilege_tables.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_install_db.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_secure_installation.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_setpermission.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_upgrade.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlhotcopy.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlman.1* +%doc %attr(644, root, man) %{_mandir}/man8/mysqlmanager.8* +%doc %attr(644, root, man) %{_mandir}/man1/mysql.server.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqltest.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_tzinfo_to_sql.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_zap.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlbug.1* +%doc %attr(644, root, man) %{_mandir}/man1/perror.1* +%doc %attr(644, root, man) %{_mandir}/man1/replace.1* +%doc %attr(644, root, man) %{_mandir}/man1/resolve_stack_dump.1* +%doc %attr(644, root, man) %{_mandir}/man1/resolveip.1* + +%ghost %config(noreplace,missingok) %{_sysconfdir}/my.cnf +%ghost %config(noreplace,missingok) %{_sysconfdir}/mysqlmanager.passwd + +%if %{INNODB_BUILD} +%attr(755, root, root) %{_bindir}/innochecksum +%endif +%attr(755, root, root) %{_bindir}/my_print_defaults +%attr(755, root, root) %{_bindir}/myisam_ftdump +%attr(755, root, root) %{_bindir}/myisamchk +%attr(755, root, root) %{_bindir}/myisamlog +%attr(755, root, root) %{_bindir}/myisampack +%attr(755, root, root) %{_bindir}/mysql_convert_table_format +%attr(755, root, root) %{_bindir}/mysql_fix_extensions +%attr(755, root, root) %{_bindir}/mysql_fix_privilege_tables +%attr(755, root, root) %{_bindir}/mysql_install_db +%attr(755, root, root) %{_bindir}/mysql_secure_installation +%attr(755, root, root) %{_bindir}/mysql_setpermission +%attr(755, root, root) %{_bindir}/mysql_tzinfo_to_sql +%attr(755, root, root) %{_bindir}/mysql_upgrade +%attr(755, root, root) %{_bindir}/mysql_zap +%attr(755, root, root) %{_bindir}/mysqlbug +%attr(755, root, root) %{_bindir}/mysqld_multi +%attr(755, root, root) %{_bindir}/mysqld_safe +%attr(755, root, root) %{_bindir}/mysqldumpslow +%attr(755, root, root) %{_bindir}/mysqlhotcopy +%attr(755, root, root) %{_bindir}/mysqltest +%attr(755, root, root) %{_bindir}/perror +%attr(755, root, root) %{_bindir}/replace +%attr(755, root, root) %{_bindir}/resolve_stack_dump +%attr(755, root, root) %{_bindir}/resolveip + +%attr(755, root, root) %{_sbindir}/mysqld +%if %{BUILD_DEBUG} +%attr(755, root, root) %{_sbindir}/mysqld-debug +%endif +%attr(755, root, root) %{_sbindir}/mysqlmanager +%attr(755, root, root) %{_sbindir}/rcmysql +#%attr(644, root, root) %{_libdir}/mysql/mysqld.sym +%if %{BUILD_DEBUG} +#%attr(644, root, root) %{_libdir}/mysql/mysqld-debug.sym +%endif + +%attr(644, root, root) %config(noreplace,missingok) %{_sysconfdir}/logrotate.d/mysql +%attr(755, root, root) %{_sysconfdir}/init.d/mysql + +%attr(755, root, root) %{_datadir}/mysql/ + +%files -n Percona-XtraDB-client%{package_suffix} +%defattr(-, root, root, 0755) +%attr(755, root, root) %{_bindir}/msql2mysql +%attr(755, root, root) %{_bindir}/mysql +%attr(755, root, root) %{_bindir}/mysql_find_rows +%attr(755, root, root) %{_bindir}/mysql_waitpid +%attr(755, root, root) %{_bindir}/mysqlaccess +%attr(755, root, root) %{_bindir}/mysqladmin +%attr(755, root, root) %{_bindir}/mysqlbinlog +%attr(755, root, root) %{_bindir}/mysqlcheck +%attr(755, root, root) %{_bindir}/mysqldump +%attr(755, root, root) %{_bindir}/mysqlimport +%attr(755, root, root) %{_bindir}/mysqlshow +%attr(755, root, root) %{_bindir}/mysqlslap + +%doc %attr(644, root, man) %{_mandir}/man1/msql2mysql.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_find_rows.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_waitpid.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlaccess.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqladmin.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlbinlog.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlcheck.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqldump.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlimport.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlshow.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysqlslap.1* + +%post -n Percona-XtraDB-shared%{package_suffix} +/sbin/ldconfig + +%postun -n Percona-XtraDB-shared%{package_suffix} +/sbin/ldconfig + +%if %{CLUSTER_BUILD} +%files -n MySQL%{cluster_package_prefix}storage%{package_suffix} +%defattr(-,root,root,0755) +%attr(755, root, root) %{_sbindir}/ndbd +%doc %attr(644, root, man) %{_mandir}/man8/ndbd.8* + +%files -n MySQL%{cluster_package_prefix}management%{package_suffix} +%defattr(-,root,root,0755) +%attr(755, root, root) %{_sbindir}/ndb_mgmd +%doc %attr(644, root, man) %{_mandir}/man8/ndb_mgmd.8* + +%files -n MySQL%{cluster_package_prefix}tools%{package_suffix} +%defattr(-,root,root,0755) +%attr(755, root, root) %{_bindir}/ndb_config +%attr(755, root, root) %{_bindir}/ndb_desc +%attr(755, root, root) %{_bindir}/ndb_error_reporter +%attr(755, root, root) %{_bindir}/ndb_mgm +%attr(755, root, root) %{_bindir}/ndb_restore +%attr(755, root, root) %{_bindir}/ndb_select_all +%attr(755, root, root) %{_bindir}/ndb_select_count +%attr(755, root, root) %{_bindir}/ndb_show_tables +%attr(755, root, root) %{_bindir}/ndb_size.pl +%attr(755, root, root) %{_bindir}/ndb_test_platform +%attr(755, root, root) %{_bindir}/ndb_waiter +%doc %attr(644, root, man) %{_mandir}/man1/ndb_config.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_desc.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_error_reporter.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_mgm.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_restore.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_select_all.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_select_count.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_show_tables.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_size.pl.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_waiter.1* + +%files -n MySQL%{cluster_package_prefix}extra%{package_suffix} +%defattr(-,root,root,0755) +%attr(755, root, root) %{_bindir}/ndb_delete_all +%attr(755, root, root) %{_bindir}/ndb_drop_index +%attr(755, root, root) %{_bindir}/ndb_drop_table +%attr(755, root, root) %{_sbindir}/ndb_cpcd +%doc %attr(644, root, man) %{_mandir}/man1/ndb_delete_all.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_drop_index.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_drop_table.1* +%doc %attr(644, root, man) %{_mandir}/man1/ndb_cpcd.1* +%endif + +%files -n Percona-XtraDB-devel%{package_suffix} +%defattr(-, root, root, 0755) +%if %{commercial} +%else +%doc EXCEPTIONS-CLIENT +%endif +%doc %attr(644, root, man) %{_mandir}/man1/comp_err.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql_config.1* +%attr(755, root, root) %{_bindir}/mysql_config +%dir %attr(755, root, root) %{_libdir}/mysql +%{_includedir}/mysql +%{_datadir}/aclocal/mysql.m4 +%{_libdir}/mysql/libdbug.a +%{_libdir}/mysql/libheap.a +%if %{WITH_LIBGCC} +%{_libdir}/mysql/libmygcc.a +%endif +%{_libdir}/mysql/libmyisam.a +%{_libdir}/mysql/libmyisammrg.a +%{_libdir}/mysql/libmysqlclient.a +%{_libdir}/mysql/libmysqlclient.la +%{_libdir}/mysql/libmysqlclient_r.a +%{_libdir}/mysql/libmysqlclient_r.la +%{_libdir}/mysql/libmystrings.a +%{_libdir}/mysql/libmysys.a +%{_libdir}/mysql/libvio.a +%{_libdir}/mysql/libz.a +%{_libdir}/mysql/libz.la +%if %{CLUSTER_BUILD} +%{_libdir}/mysql/libndbclient.a +%{_libdir}/mysql/libndbclient.la +%endif + +%files -n Percona-XtraDB-shared%{package_suffix} +%defattr(-, root, root, 0755) +# Shared libraries (omit for architectures that don't support them) +%{_libdir}/*.so* + +%files -n Percona-XtraDB-test%{package_suffix} +%defattr(-, root, root, 0755) +%{_datadir}/mysql-test +%attr(755, root, root) %{_bindir}/mysql_client_test +%doc %attr(644, root, man) %{_mandir}/man1/mysql_client_test.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql-stress-test.pl.1* +%doc %attr(644, root, man) %{_mandir}/man1/mysql-test-run.pl.1* + +%files -n Percona-XtraDB-%{pluginversion}-%{xtradbversion} +%defattr(-, root, root, 0755) +%attr(644, root, root) %{_libdir}/mysql/plugin/ha_innodb.so* +%attr(644, root, root) %{_libdir}/mysql/plugin/install_innodb_plugins.sql + +############################################################################## +# The spec file changelog only includes changes made to the spec file +# itself - note that they must be ordered by date (important when +# merging BK trees) +############################################################################## +%changelog +* Thu Feb 11 2010 Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> + +Package name changed to Percona-XtraDB + +* Tue Jan 05 2010 Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> + +- Corrected emails +- -m64 is removed from CFLAGS + +* Tue Apr 21 2009 Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> + +- Adoption for XtraDB Storage Engine + +* Fri Nov 07 2008 Joerg Bruehe <joerg(a)mysql.com> + +- Modify CFLAGS and CXXFLAGS such that a debug build is not optimized. + This should cover both gcc and icc flags. Fixes bug#40546. + +* Mon Aug 18 2008 Joerg Bruehe <joerg(a)mysql.com> + +- Get rid of the "warning: Installed (but unpackaged) file(s) found:" + Some generated files aren't needed in RPMs: + - the "sql-bench/" subdirectory + Some files were missing: + - /usr/share/aclocal/mysql.m4 ("devel" subpackage) + - Manuals for embedded tests ("test" subpackage) + - Manual "mysqlbug" ("server" subpackage) + - Manual "mysql_find_rows" ("client" subpackage) + +* Wed Jun 11 2008 Kent Boortz <kent(a)mysql.com> + +- Removed the Example storage engine, it is not to be in products + +* Fri Apr 04 2008 Daniel Fischer <df(a)mysql.com> + +- Added Cluster+InnoDB product + +* Mon Mar 31 2008 Kent Boortz <kent(a)mysql.com> + +- Made the "Federated" storage engine an option + +* Tue Mar 11 2008 Joerg Bruehe <joerg(a)mysql.com> + +- Cleanup: Remove manual file "mysql_tableinfo.1". + +* Mon Feb 18 2008 Timothy Smith <tim(a)mysql.com> + +- Require a manual upgrade if the alread-installed mysql-server is + from another vendor, or is of a different major version. + +* Fri Dec 14 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Add the "%doc" directive for all man pages and other documentation; + also, some re-ordering to reduce differences between spec files. + +* Fri Dec 14 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Added "client/mysqlslap" (bug#32077) + +* Wed Oct 31 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Explicitly handle InnoDB using its own variable and "--with"/"--without" + options, because the "configure" default is "yes". + Also, fix the specification of "community" to include "partitioning". + +* Mon Sep 03 2007 Kent Boortz <kent(a)mysql.com> + +- Let libmygcc be included unless "--without libgcc" is given. + +* Sun Sep 02 2007 Kent Boortz <kent(a)mysql.com> + +- Changed SSL flag given to configure to "--with-ssl" +- Removed symbolic link "safe_mysqld" +- Removed script and man page for "mysql_explain_log" +- Removed scripts "mysql_tableinfo" and "mysql_upgrade_shell" +- Removed "comp_err" from list to install +- Removed duplicates of "libndbclient.a" and "libndbclient.la" + +* Tue Jul 17 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Add the man page for "mysql-stress-test.pl" to the "test" RPM + (consistency in fixing bug#21023, the script is handled by "Makefile.am") + +* Wed Jul 11 2007 Daniel Fischer <df(a)mysql.com> + +- Change the way broken SELinux policies on RHEL4 and CentOS 4 + are handled to be more likely to actually work + +* Thu Jun 05 2007 kent Boortz <kent(a)mysql.com> + +- Enabled the CSV engine in all builds + +* Thu May 3 2007 Mads Martin Joergensen <mmj(a)mysql.com> + +- Spring cleanup + +* Thu Apr 19 2007 Mads Martin Joergensen <mmj(a)mysql.com> + +- If sbin/restorecon exists then run it + +* Wed Apr 18 2007 Kent Boortz <kent(a)mysql.com> + +- Packed unpacked files + + /usr/sbin/ndb_cpcd + /usr/bin/mysql_upgrade_shell + /usr/bin/innochecksum + /usr/share/man/man1/ndb_cpcd.1.gz + /usr/share/man/man1/innochecksum.1.gz + /usr/share/man/man1/mysql_fix_extensions.1.gz + /usr/share/man/man1/mysql_secure_installation.1.gz + /usr/share/man/man1/mysql_tableinfo.1.gz + /usr/share/man/man1/mysql_waitpid.1.gz + +- Commands currently not installed but that has man pages + + /usr/share/man/man1/make_win_bin_dist.1.gz + /usr/share/man/man1/make_win_src_distribution.1.gz + /usr/share/man/man1/mysql-stress-test.pl.1.gz + /usr/share/man/man1/ndb_print_backup_file.1.gz + /usr/share/man/man1/ndb_print_schema_file.1.gz + /usr/share/man/man1/ndb_print_sys_file.1.gz + +* Thu Mar 22 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Add "comment" options to the test runs, for better log analysis. + +* Wed Mar 21 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Add even more man pages. + +* Fri Mar 16 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Build the server twice, once as "mysqld-debug" and once as "mysqld"; + test them both, and include them in the resulting file. +- Consequences of the fix for bug#20166: + Remove "mysql_create_system_tables", + new "mysql_fix_privilege_tables.sql" is included implicitly. + +* Wed Mar 14 2007 Daniel Fischer <df(a)mysql.com> + +- Adjust compile options some more and change naming of community + cluster RPMs to explicitly say 'cluster'. + +* Mon Mar 12 2007 Daniel Fischer <df(a)mysql.com> + +- Adjust compile options and other settings for 5.0 community builds. + +* Fri Mar 02 2007 Joerg Bruehe <joerg(a)mysql.com> + +- Add several man pages which are now created. + +* Mon Jan 29 2007 Mads Martin Joergensen <mmj(a)mysql.com> + +- Make sure SELinux works correctly. Files from Colin Charles. + +* Fri Jan 05 2007 Kent Boortz <kent(a)mysql.com> + +- Add CFLAGS to gcc call with --print-libgcc-file, to make sure the + correct "libgcc.a" path is returned for the 32/64 bit architecture. + +* Tue Dec 19 2006 Joerg Bruehe <joerg(a)mysql.com> + +- The man page for "mysqld" is now in section 8. + +* Thu Dec 14 2006 Joerg Bruehe <joerg(a)mysql.com> + +- Include the new man pages for "my_print_defaults" and "mysql_tzinfo_to_sql" + in the server RPM. +- The "mysqlmanager" man page was relocated to section 8, reflect that. + +* Fri Nov 17 2006 Mads Martin Joergensen <mmj(a)mysql.com> + +- Really fix obsoletes/provides for community -> this +- Make it possible to not run test by setting + MYSQL_RPMBUILD_TEST to "no" + +* Wed Nov 15 2006 Joerg Bruehe <joerg(a)mysql.com> + +- Switch from "make test*" to explicit calls of the test suite, + so that "report features" can be used. + +* Wed Nov 15 2006 Kent Boortz <kent(a)mysql.com> + +- Added "--with cluster" and "--define cluster{_gpl}" + +* Tue Oct 24 2006 Mads Martin Joergensen <mmj(a)mysql.com> + +- Shared need to Provide/Obsolete mysql-shared + +* Mon Oct 23 2006 Mads Martin Joergensen <mmj(a)mysql.com> + +- Run sbin/restorecon after db init (Bug#12676) + +* Thu Jul 06 2006 Joerg Bruehe <joerg(a)mysql.com> + +- Correct a typing error in my previous change. + +* Tue Jul 04 2006 Joerg Bruehe <joerg(a)mysql.com> + +- Use the Perl script to run the tests, because it will automatically check + whether the server is configured with SSL. + +* Wed Jun 28 2006 Joerg Bruehe <joerg(a)mysql.com> + +- Revert all previous attempts to call "mysql_upgrade" during RPM upgrade, + there are some more aspects which need to be solved before this is possible. + For now, just ensure the binary "mysql_upgrade" is delivered and installed. + +* Wed Jun 28 2006 Joerg Bruehe <joerg(a)mysql.com> + +- Move "mysqldumpslow" from the client RPM to the server RPM (bug#20216). + +* Wed Jun 21 2006 Joerg Bruehe <joerg(a)mysql.com> + +- To run "mysql_upgrade", we need a running server; + start it in isolation and skip password checks. + +* Sat May 23 2006 Kent Boortz <kent(a)mysql.com> + +- Always compile for PIC, position independent code. + +* Fri Apr 28 2006 Kent Boortz <kent(a)mysql.com> + +- Install and run "mysql_upgrade" + +* Sat Apr 01 2006 Kent Boortz <kent(a)mysql.com> + +- Allow to override $LDFLAGS + +* Fri Jan 06 2006 Lenz Grimmer <lenz(a)mysql.com> + +- added a MySQL-test subpackage (BUG#16070) + +* Tue Dec 27 2005 Joerg Bruehe <joerg(a)mysql.com> + +- Some minor alignment with the 4.1 version + +* Wed Dec 14 2005 Rodrigo Novo <rodrigo(a)mysql.com> + +- Cosmetic changes: source code location & rpm packager +- Protect "nm -D" against libtool weirdness +- Add libz.a & libz.la to the list of files for subpackage -devel +- moved --with-zlib-dir=bundled out of BuildMySQL, as it doesn't makes + sense for the shared package + +* Tue Nov 22 2005 Joerg Bruehe <joerg(a)mysql.com> + +- Extend the file existence check for "init.d/mysql" on un-install + to also guard the call to "insserv"/"chkconfig". + +* Wed Nov 16 2005 Lenz Grimmer <lenz(a)mysql.com> + +- added mysql_client_test to the "client" subpackage (BUG#14546) + +* Tue Nov 15 2005 Lenz Grimmer <lenz(a)mysql.com> + +- changed default definitions to build a standard GPL release when not + defining anything else +- install the shared libs more elegantly by using "make install" + +* Wed Oct 19 2005 Kent Boortz <kent(a)mysql.com> + +- Made yaSSL support an option (off by default) + +* Wed Oct 19 2005 Kent Boortz <kent(a)mysql.com> + +- Enabled yaSSL support + +* Thu Oct 13 2005 Lenz Grimmer <lenz(a)mysql.com> + +- added a usermod call to assign a potential existing mysql user to the + correct user group (BUG#12823) +- added a separate macro "mysqld_group" to be able to define the + user group of the mysql user seperately, if desired. + +* Fri Oct 1 2005 Kent Boortz <kent(a)mysql.com> + +- Copy the config.log file to location outside + the build tree + +* Fri Sep 30 2005 Lenz Grimmer <lenz(a)mysql.com> + +- don't use install-strip to install the binaries (strip segfaults on + icc-compiled binaries on IA64) + +* Thu Sep 22 2005 Lenz Grimmer <lenz(a)mysql.com> + +- allow overriding the CFLAGS (needed for Intel icc compiles) +- replace the CPPFLAGS=-DBIG_TABLES with "--with-big-tables" configure option + +* Fri Aug 19 2005 Joerg Bruehe <joerg(a)mysql.com> + +- Protect against failing tests. + +* Thu Aug 04 2005 Lenz Grimmer <lenz(a)mysql.com> + +- Fixed the creation of the mysql user group account in the postinstall + section (BUG 12348) + +* Fri Jul 29 2005 Lenz Grimmer <lenz(a)mysql.com> + +- Fixed external RPM Requirements to better suit the target distribution + (BUG 12233) + +* Fri Jul 15 2005 Lenz Grimmer <lenz(a)mysql.com> + +- create a "mysql" user group and assign the mysql user account to that group + in the server postinstall section. (BUG 10984) + +* Wed Jun 01 2005 Lenz Grimmer <lenz(a)mysql.com> + +- use "mysqldatadir" variable instead of hard-coding the path multiple times +- use the "mysqld_user" variable on all occasions a user name is referenced +- removed (incomplete) Brazilian translations +- removed redundant release tags from the subpackage descriptions + +* Fri May 27 2005 Lenz Grimmer <lenz(a)mysql.com> + +- fixed file list (removed libnisam.a and libmerge.a from the devel subpackage) +- force running the test suite + +* Wed Apr 20 2005 Lenz Grimmer <lenz(a)mysql.com> + +- Enabled the "blackhole" storage engine for the Max RPM + +* Wed Apr 13 2005 Lenz Grimmer <lenz(a)mysql.com> + +- removed the MySQL manual files (html/ps/texi) - they have been removed + from the MySQL sources and are now available seperately. + +* Mon Apr 4 2005 Petr Chardin <petr(a)mysql.com> + +- old mysqlmanager, mysqlmanagerc and mysqlmanager-pwger renamed into + mysqltestmanager, mysqltestmanager and mysqltestmanager-pwgen respectively + +* Fri Mar 18 2005 Lenz Grimmer <lenz(a)mysql.com> + +- Disabled RAID in the Max binaries once and for all (it has finally been + removed from the source tree) + +* Sun Feb 20 2005 Petr Chardin <petr(a)mysql.com> + +- Install MySQL Instance Manager together with mysqld, touch mysqlmanager + password file + +* Mon Feb 14 2005 Lenz Grimmer <lenz(a)mysql.com> + +- Fixed the compilation comments and moved them into the separate build sections + for Max and Standard + +* Mon Feb 7 2005 Tomas Ulin <tomas(a)mysql.com> + +- enabled the "Ndbcluster" storage engine for the max binary +- added extra make install in ndb subdir after Max build to get ndb binaries +- added packages for ndbcluster storage engine + +* Fri Jan 14 2005 Lenz Grimmer <lenz(a)mysql.com> + +- replaced obsoleted "BuildPrereq" with "BuildRequires" instead + +* Thu Jan 13 2005 Lenz Grimmer <lenz(a)mysql.com> + +- enabled the "Federated" storage engine for the max binary + +* Tue Jan 04 2005 Petr Chardin <petr(a)mysql.com> + +- ISAM and merge storage engines were purged. As well as appropriate + tools and manpages (isamchk and isamlog) + +* Thu Dec 31 2004 Lenz Grimmer <lenz(a)mysql.com> + +- enabled the "Archive" storage engine for the max binary +- enabled the "CSV" storage engine for the max binary +- enabled the "Example" storage engine for the max binary + +* Thu Aug 26 2004 Lenz Grimmer <lenz(a)mysql.com> + +- MySQL-Max now requires MySQL-server instead of MySQL (BUG 3860) + +* Fri Aug 20 2004 Lenz Grimmer <lenz(a)mysql.com> + +- do not link statically on IA64/AMD64 as these systems do not have + a patched glibc installed + +* Tue Aug 10 2004 Lenz Grimmer <lenz(a)mysql.com> + +- Added libmygcc.a to the devel subpackage (required to link applications + against the the embedded server libmysqld.a) (BUG 4921) + +* Mon Aug 09 2004 Lenz Grimmer <lenz(a)mysql.com> + +- Added EXCEPTIONS-CLIENT to the "devel" package + +* Thu Jul 29 2004 Lenz Grimmer <lenz(a)mysql.com> + +- disabled OpenSSL in the Max binaries again (the RPM packages were the + only exception to this anyway) (BUG 1043) + +* Wed Jun 30 2004 Lenz Grimmer <lenz(a)mysql.com> + +- fixed server postinstall (mysql_install_db was called with the wrong + parameter) + +* Thu Jun 24 2004 Lenz Grimmer <lenz(a)mysql.com> + +- added mysql_tzinfo_to_sql to the server subpackage +- run "make clean" instead of "make distclean" + +* Mon Apr 05 2004 Lenz Grimmer <lenz(a)mysql.com> + +- added ncurses-devel to the build prerequisites (BUG 3377) + +* Thu Feb 12 2004 Lenz Grimmer <lenz(a)mysql.com> + +- when using gcc, _always_ use CXX=gcc +- replaced Copyright with License field (Copyright is obsolete) + +* Tue Feb 03 2004 Lenz Grimmer <lenz(a)mysql.com> + +- added myisam_ftdump to the Server package + +* Tue Jan 13 2004 Lenz Grimmer <lenz(a)mysql.com> + +- link the mysql client against libreadline instead of libedit (BUG 2289) + +* Mon Dec 22 2003 Lenz Grimmer <lenz(a)mysql.com> + +- marked /etc/logrotate.d/mysql as a config file (BUG 2156) + +* Fri Dec 13 2003 Lenz Grimmer <lenz(a)mysql.com> + +- fixed file permissions (BUG 1672) + +* Thu Dec 11 2003 Lenz Grimmer <lenz(a)mysql.com> + +- made testing for gcc3 a bit more robust + +* Fri Dec 05 2003 Lenz Grimmer <lenz(a)mysql.com> + +- added missing file mysql_create_system_tables to the server subpackage + +* Fri Nov 21 2003 Lenz Grimmer <lenz(a)mysql.com> + +- removed dependency on MySQL-client from the MySQL-devel subpackage + as it is not really required. (BUG 1610) + +* Fri Aug 29 2003 Lenz Grimmer <lenz(a)mysql.com> + +- Fixed BUG 1162 (removed macro names from the changelog) +- Really fixed BUG 998 (disable the checking for installed but + unpackaged files) + +* Tue Aug 05 2003 Lenz Grimmer <lenz(a)mysql.com> + +- Fixed BUG 959 (libmysqld not being compiled properly) +- Fixed BUG 998 (RPM build errors): added missing files to the + distribution (mysql_fix_extensions, mysql_tableinfo, mysqldumpslow, + mysql_fix_privilege_tables.1), removed "-n" from install section. + +* Wed Jul 09 2003 Lenz Grimmer <lenz(a)mysql.com> + +- removed the GIF Icon (file was not included in the sources anyway) +- removed unused variable shared_lib_version +- do not run automake before building the standard binary + (should not be necessary) +- add server suffix '-standard' to standard binary (to be in line + with the binary tarball distributions) +- Use more RPM macros (_exec_prefix, _sbindir, _libdir, _sysconfdir, + _datadir, _includedir) throughout the spec file. +- allow overriding CC and CXX (required when building with other compilers) + +* Fri May 16 2003 Lenz Grimmer <lenz(a)mysql.com> + +- re-enabled RAID again + +* Wed Apr 30 2003 Lenz Grimmer <lenz(a)mysql.com> + +- disabled MyISAM RAID (--with-raid) - it throws an assertion which + needs to be investigated first. + +* Mon Mar 10 2003 Lenz Grimmer <lenz(a)mysql.com> + +- added missing file mysql_secure_installation to server subpackage + (BUG 141) + +* Tue Feb 11 2003 Lenz Grimmer <lenz(a)mysql.com> + +- re-added missing pre- and post(un)install scripts to server subpackage +- added config file /etc/my.cnf to the file list (just for completeness) +- make sure to create the datadir with 755 permissions + +* Mon Jan 27 2003 Lenz Grimmer <lenz(a)mysql.com> + +- removed unused CC and CXX variables +- CFLAGS and CXXFLAGS should honor RPM_OPT_FLAGS + +* Fri Jan 24 2003 Lenz Grimmer <lenz(a)mysql.com> + +- renamed package "MySQL" to "MySQL-server" +- fixed Copyright tag +- added mysql_waitpid to client subpackage (required for mysql-test-run) + +* Wed Nov 27 2002 Lenz Grimmer <lenz(a)mysql.com> + +- moved init script from /etc/rc.d/init.d to /etc/init.d (the majority of + Linux distributions now support this scheme as proposed by the LSB either + directly or via a compatibility symlink) +- Use new "restart" init script action instead of starting and stopping + separately +- Be more flexible in activating the automatic bootup - use insserv (on + older SuSE versions) or chkconfig (Red Hat, newer SuSE versions and + others) to create the respective symlinks + +* Wed Sep 25 2002 Lenz Grimmer <lenz(a)mysql.com> + +- MySQL-Max now requires MySQL >= 4.0 to avoid version mismatches + (mixing 3.23 and 4.0 packages) + +* Fri Aug 09 2002 Lenz Grimmer <lenz(a)mysql.com> + +- Turn off OpenSSL in MySQL-Max for now until it works properly again +- enable RAID for the Max binary instead +- added compatibility link: safe_mysqld -> mysqld_safe to ease the + transition from 3.23 + +* Thu Jul 18 2002 Lenz Grimmer <lenz(a)mysql.com> + +- Reworked the build steps a little bit: the Max binary is supposed + to include OpenSSL, which cannot be linked statically, thus trying + to statically link against a special glibc is futile anyway +- because of this, it is not required to make yet another build run + just to compile the shared libs (saves a lot of time) +- updated package description of the Max subpackage +- clean up the BuildRoot directory afterwards + +* Mon Jul 15 2002 Lenz Grimmer <lenz(a)mysql.com> + +- Updated Packager information +- Fixed the build options: the regular package is supposed to + include InnoDB and linked statically, while the Max package + should include BDB and SSL support + +* Fri May 03 2002 Lenz Grimmer <lenz(a)mysql.com> + +- Use more RPM macros (e.g. infodir, mandir) to make the spec + file more portable +- reorganized the installation of documentation files: let RPM + take care of this +- reorganized the file list: actually install man pages along + with the binaries of the respective subpackage +- do not include libmysqld.a in the devel subpackage as well, if we + have a special "embedded" subpackage +- reworked the package descriptions + +* Mon Oct 8 2001 Monty + +- Added embedded server as a separate RPM + +* Fri Apr 13 2001 Monty + +- Added mysqld-max to the distribution + +* Tue Jan 2 2001 Monty + +- Added mysql-test to the bench package + +* Fri Aug 18 2000 Tim Smith <tim(a)mysql.com> + +- Added separate libmysql_r directory; now both a threaded + and non-threaded library is shipped. + +* Wed Sep 28 1999 David Axmark <davida(a)mysql.com> + +- Added the support-files/my-example.cnf to the docs directory. + +- Removed devel dependency on base since it is about client + development. + +* Wed Sep 8 1999 David Axmark <davida(a)mysql.com> + +- Cleaned up some for 3.23. + +* Thu Jul 1 1999 David Axmark <davida(a)mysql.com> + +- Added support for shared libraries in a separate sub + package. Original fix by David Fox (dsfox(a)cogsci.ucsd.edu) + +- The --enable-assembler switch is now automatically disables on + platforms there assembler code is unavailable. This should allow + building this RPM on non i386 systems. + +* Mon Feb 22 1999 David Axmark <david(a)detron.se> + +- Removed unportable cc switches from the spec file. The defaults can + now be overridden with environment variables. This feature is used + to compile the official RPM with optimal (but compiler version + specific) switches. + +- Removed the repetitive description parts for the sub rpms. Maybe add + again if RPM gets a multiline macro capability. + +- Added support for a pt_BR translation. Translation contributed by + Jorge Godoy <jorge(a)bestway.com.br>. + +* Wed Nov 4 1998 David Axmark <david(a)detron.se> + +- A lot of changes in all the rpm and install scripts. This may even + be a working RPM :-) + +* Sun Aug 16 1998 David Axmark <david(a)detron.se> + +- A developers changelog for MySQL is available in the source RPM. And + there is a history of major user visible changed in the Reference + Manual. Only RPM specific changes will be documented here. === modified file 'storage/xtradb/handler/ha_innodb.cc' --- a/storage/xtradb/handler/ha_innodb.cc 2010-04-28 12:52:24 +0000 +++ b/storage/xtradb/handler/ha_innodb.cc 2010-04-28 13:53:04 +0000 @@ -125,9 +125,6 @@ extern ib_int64_t trx_sys_mysql_relay_lo # ifndef MYSQL_PLUGIN_IMPORT # define MYSQL_PLUGIN_IMPORT /* nothing */ # endif /* MYSQL_PLUGIN_IMPORT */ -/* This is needed because of Bug #3596. Let us hope that pthread_mutex_t -is defined the same in both builds: the MySQL server and the InnoDB plugin. */ -extern MYSQL_PLUGIN_IMPORT pthread_mutex_t LOCK_thread_count; #if MYSQL_VERSION_ID < 50124 /* this is defined in mysql_priv.h inside #ifdef MYSQL_SERVER @@ -920,36 +917,6 @@ convert_error_code_to_mysql( } /*************************************************************//** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. */ -extern "C" UNIV_INTERN -void -innobase_mysql_prepare_print_arbitrary_thd(void) -/*============================================*/ -{ - ut_ad(!mutex_own(&kernel_mutex)); - VOID(pthread_mutex_lock(&LOCK_thread_count)); -} - -/*************************************************************//** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -In the InnoDB latching order, the mutex sits right above the -kernel_mutex. In debug builds, we assert that the kernel_mutex is -released before this function is invoked. */ -extern "C" UNIV_INTERN -void -innobase_mysql_end_print_arbitrary_thd(void) -/*========================================*/ -{ - ut_ad(!mutex_own(&kernel_mutex)); - VOID(pthread_mutex_unlock(&LOCK_thread_count)); -} - -/*************************************************************//** Prints info of a THD object (== user session thread) to the given file. */ extern "C" UNIV_INTERN void === modified file 'storage/xtradb/include/ha_prototypes.h' --- a/storage/xtradb/include/ha_prototypes.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ha_prototypes.h 2010-02-11 13:52:03 +0000 @@ -153,28 +153,6 @@ get_innobase_type_from_mysql_type( const void* field) /*!< in: MySQL Field */ __attribute__((nonnull)); -/*************************************************************//** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. */ -UNIV_INTERN -void -innobase_mysql_prepare_print_arbitrary_thd(void); -/*============================================*/ - -/*************************************************************//** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -In the InnoDB latching order, the mutex sits right above the -kernel_mutex. In debug builds, we assert that the kernel_mutex is -released before this function is invoked. */ -UNIV_INTERN -void -innobase_mysql_end_print_arbitrary_thd(void); -/*========================================*/ - /******************************************************************//** Get the variable length bounds of the given character set. */ UNIV_INTERN === modified file 'storage/xtradb/include/univ.i' --- a/storage/xtradb/include/univ.i 2010-04-21 00:25:59 +0000 +++ b/storage/xtradb/include/univ.i 2010-04-28 13:53:04 +0000 @@ -47,7 +47,7 @@ Created 1/20/1994 Heikki Tuuri #define INNODB_VERSION_MAJOR 1 #define INNODB_VERSION_MINOR 0 #define INNODB_VERSION_BUGFIX 6 -#define PERCONA_INNODB_VERSION 9 +#define PERCONA_INNODB_VERSION 9.1 /* The following is the InnoDB version as shown in SELECT plugin_version FROM information_schema.plugins; === modified file 'storage/xtradb/lock/lock0lock.c' --- a/storage/xtradb/lock/lock0lock.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/lock/lock0lock.c 2010-03-18 15:25:47 +0000 @@ -3364,23 +3364,26 @@ lock_deadlock_recursive( bit_no = lock_rec_find_set_bit(wait_lock); ut_a(bit_no != ULINT_UNDEFINED); - } + + /* get the starting point for the search for row level locks + since we are scanning from the front of the list */ + lock = lock_rec_get_first_on_page_addr(wait_lock->un_member.rec_lock.space, + wait_lock->un_member.rec_lock.page_no); + } + else { + /* table level locks use a two-way linked list so scanning backwards is OK */ + lock = UT_LIST_GET_PREV(un_member.tab_lock.locks, + lock); + } /* Look at the locks ahead of wait_lock in the lock queue */ for (;;) { - if (lock_get_type_low(lock) & LOCK_TABLE) { - lock = UT_LIST_GET_PREV(un_member.tab_lock.locks, - lock); - } else { - ut_ad(lock_get_type_low(lock) == LOCK_REC); - ut_a(bit_no != ULINT_UNDEFINED); - - lock = (lock_t*) lock_rec_get_prev(lock, bit_no); - } - if (lock == NULL) { + /* reached the original lock in the queue for row level locks + or past beginning of the list for table level locks */ + if (lock == NULL || lock == wait_lock) { /* We can mark this subtree as searched */ trx->deadlock_mark = 1; @@ -3505,6 +3508,17 @@ lock_deadlock_recursive( } } } + + /* next lock to check */ + if (lock_get_type_low(lock) & LOCK_TABLE) { + lock = UT_LIST_GET_PREV(un_member.tab_lock.locks, + lock); + } else { + ut_ad(lock_get_type_low(lock) == LOCK_REC); + ut_a(bit_no != ULINT_UNDEFINED); + + lock = (lock_t*) lock_rec_get_next(bit_no, lock); + } }/* end of the 'for (;;)'-loop */ } @@ -4336,11 +4350,6 @@ lock_print_info_summary( /*====================*/ FILE* file) /*!< in: file where to print */ { - /* We must protect the MySQL thd->query field with a MySQL mutex, and - because the MySQL mutex must be reserved before the kernel_mutex of - InnoDB, we call innobase_mysql_prepare_print_arbitrary_thd() here. */ - - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); if (lock_deadlock_found) { @@ -4423,7 +4432,6 @@ loop: if (trx == NULL) { lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); ut_ad(lock_validate()); @@ -4507,7 +4515,6 @@ loop: } lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); mtr_start(&mtr); @@ -4518,7 +4525,6 @@ loop: load_page_first = FALSE; - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); goto loop; === modified file 'storage/xtradb/trx/trx0i_s.c' --- a/storage/xtradb/trx/trx0i_s.c 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/trx/trx0i_s.c 2010-04-28 13:53:04 +0000 @@ -1205,9 +1205,6 @@ trx_i_s_possibly_fetch_data_into_cache( return(1); } - /* We are going to access trx->query in all transactions */ - innobase_mysql_prepare_print_arbitrary_thd(); - /* We need to read trx_sys and record/table lock queues */ mutex_enter(&kernel_mutex); @@ -1215,8 +1212,6 @@ trx_i_s_possibly_fetch_data_into_cache( mutex_exit(&kernel_mutex); - innobase_mysql_end_print_arbitrary_thd(); - return(0); } === modified file 'storage/xtradb/trx/trx0trx.c' --- a/storage/xtradb/trx/trx0trx.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/trx/trx0trx.c 2010-02-11 13:52:03 +0000 @@ -1695,9 +1695,7 @@ trx_mark_sql_stat_end( /**********************************************************************//** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ UNIV_INTERN void trx_print(

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2852)
by knielsen＠knielsen-hq.org 28 Apr '10

28 Apr '10

#At lp:maria 2852 knielsen(a)knielsen-hq.org 2010-04-28 [merge] Merged MySQL 5.1.46 GPLed documentation into MariaDB. modified: INSTALL-SOURCE INSTALL-WIN-SOURCE man/comp_err.1 man/innochecksum.1 man/make_win_bin_dist.1 man/msql2mysql.1 man/my_print_defaults.1 man/myisam_ftdump.1 man/myisamchk.1 man/myisamlog.1 man/myisampack.1 man/mysql-stress-test.pl.1 man/mysql-test-run.pl.1 man/mysql.1 man/mysql.server.1 man/mysql_client_test.1 man/mysql_config.1 man/mysql_convert_table_format.1 man/mysql_find_rows.1 man/mysql_fix_extensions.1 man/mysql_fix_privilege_tables.1 man/mysql_install_db.1 man/mysql_secure_installation.1 man/mysql_setpermission.1 man/mysql_tzinfo_to_sql.1 man/mysql_upgrade.1 man/mysql_waitpid.1 man/mysql_zap.1 man/mysqlaccess.1 man/mysqladmin.1 man/mysqlbinlog.1 man/mysqlbug.1 man/mysqlcheck.1 man/mysqld.8 man/mysqld_multi.1 man/mysqld_safe.1 man/mysqldump.1 man/mysqldumpslow.1 man/mysqlhotcopy.1 man/mysqlimport.1 man/mysqlmanager.8 man/mysqlshow.1 man/mysqlslap.1 man/mysqltest.1 man/ndbd.8 man/ndbd_redo_log_reader.1 man/ndbmtd.8 man/perror.1 man/replace.1 man/resolve_stack_dump.1 man/resolveip.1 scripts/fill_help_tables.sql === modified file 'INSTALL-SOURCE' --- a/INSTALL-SOURCE 2009-12-01 07:24:05 +0000 +++ b/INSTALL-SOURCE 2010-04-28 13:06:11 +0000 @@ -17,8 +17,7 @@ Chapter 2. Installing and Upgrading MySQ platform. Please note that not all platforms are equally suitable for running MySQL, and that not all platforms on which MySQL is - known to run are officially supported by Sun Microsystems, - Inc.: + known to run are officially supported by Oracle Corporation: 2. Choose which distribution to install. Several versions of MySQL are available, and most are @@ -77,12 +76,11 @@ Chapter 2. Installing and Upgrading MySQ Important - Sun Microsystems, Inc. does not necessarily provide official - support for all the platforms listed in this section. For - information about those platforms that are officially supported, - see MySQL Server Supported Platforms - (http://www.mysql.com/support/supportedplatforms.html) on the - MySQL Web site. + Oracle Corporation does not necessarily provide official support + for all the platforms listed in this section. For information + about those platforms that are officially supported, see + http://www.mysql.com/support/supportedplatforms.html on the MySQL + Web site. We use GNU Autoconf, so it is possible to port MySQL to all modern systems that have a C++ compiler and a working implementation of @@ -148,7 +146,7 @@ Important by the ability of the file system to deal with large files and dealing with them efficiently. - * Our level of expertise here at Sun Microsystems, Inc. with the + * Our level of expertise here at Oracle Corporation with the platform. If we know a platform well, we enable platform-specific optimizations and fixes at compile time. We can also provide advice on configuring your system optimally @@ -184,17 +182,16 @@ Important new features are being added that could affect stability. * MySQL 5.0 is the previous stable (production-quality) release - series. + series. MySQL 5.0 is now at the end of the product lifecycle. + Active development and support for this version has ended. + Extended support for MySQL 5.0 remains available. According to + the http://www.mysql.com/about/legal/lifecycle/, only Security + and Severity Level 1 issues are still being fixed for MySQL + 5.0. * MySQL 4.1, 4.0, and 3.23 are old stable (production-quality) - release series. MySQL 4.1 is now at the end of the product - lifecycle. Active development and support for these versions - has ended. - Extended support for MySQL 4.1 remains available. According to - the MySQL Lifecycle Policy - (http://www.mysql.com/about/legal/lifecycle/) only Security - and Severity Level 1 issues are still being fixed for MySQL - 4.1. + release series. Active development and support for these + versions has ended. We do not believe in a complete code freeze because this prevents us from making bugfixes and other fixes that must be done. By @@ -228,7 +225,7 @@ Important the code on which future releases are to be based. The MySQL naming scheme uses release names that consist of three - numbers and a suffix; for example, mysql-5.0.12-beta. The numbers + numbers and a suffix; for example, mysql-5.0.14-rc. The numbers within the release name are interpreted as follows: * The first number (5) is the major version and describes the @@ -238,7 +235,7 @@ Important the major version and release level constitute the release series number. - * The third number (12) is the version number within the release + * The third number (14) is the version number within the release series. This is incremented for each new release. Usually you want the latest version for the series you have chosen. @@ -307,11 +304,6 @@ Important actually made the code faster. See Section 7.1.3, "The MySQL Benchmark Suite." - * The crash-me test - This test tries to determine what features the database - supports and what its capabilities and limitations are. See - Section 7.1.3, "The MySQL Benchmark Suite." - We also test the newest MySQL version in our internal production environment, on at least one machine. We have more than 100GB of data to work with. @@ -475,8 +467,8 @@ Important shell> md5sum package_name Example: -shell> md5sum mysql-standard-5.1.41-linux-i686.tar.gz -aaab65abbec64d5e907dcd41b8699945 mysql-standard-5.1.41-linux-i686.ta +shell> md5sum mysql-standard-5.1.46-linux-i686.tar.gz +aaab65abbec64d5e907dcd41b8699945 mysql-standard-5.1.46-linux-i686.ta r.gz You should verify that the resulting checksum (the string of @@ -520,8 +512,7 @@ Note named build(a)mysql.com. Alternatively, you can cut and paste the key directly from the following text: -----BEGIN PGP PUBLIC KEY BLOCK----- -Version: GnuPG v1.0.6 (GNU/Linux) -Comment: For info see http://www.gnupg.org +Version: GnuPG v1.4.5 (GNU/Linux) mQGiBD4+owwRBAC14GIfUfCyEDSIePvEW3SAFUdJBtoQHH/nJKZyQT7h9bPlUWC3 RODjQReyCITRrdwyrKUGku2FmeVGwn2u2WmDMNABLnpprWPkBdCk96+OmSLN9brZ @@ -533,81 +524,26 @@ kYpXBACmWpP8NJTkamEnPCia2ZoOHODANwpUkP43 QJEXM6FSbi0LLtZciNlYsafwAPEOMDKpMqAK6IyisNtPvaLd8lH0bPAnWqcyefep rv0sxxqUEMcM3o7wwgfN83POkDasDbs3pjwPhxvhz6//62zQJ7Q7TXlTUUwgUGFj a2FnZSBzaWduaW5nIGtleSAod3d3Lm15c3FsLmNvbSkgPGJ1aWxkQG15c3FsLmNv -bT6IXQQTEQIAHQUCR6yUtAUJDTBYqAULBwoDBAMVAwIDFgIBAheAAAoJEIxxjTtQ -cuH1rpIAn38+BlBI815Dou9VXMIAsQEk4G3tAJ9+Cz69Y/Xwm611lzteJrCAA32+ -aYhMBBMRAgAMBQI+PqPRBYMJZgC7AAoJEElQ4SqycpHyJOEAn1mxHijft00bKXvu +bT6IXQQTEQIAHQULBwoDBAMVAwIDFgIBAheABQJLcC5lBQkQ8/JZAAoJEIxxjTtQ +cuH1oD4AoIcOQ4EoGsZvy06D0Ei5vcsWEy8dAJ4g46i3WEcdSWxMhcBSsPz65sh5 +lohMBBMRAgAMBQI+PqPRBYMJZgC7AAoJEElQ4SqycpHyJOEAn1mxHijft00bKXvu cSo/pECUmppiAJ41M9MRVj5VcdH/KN/KjRtW6tHFPYhMBBMRAgAMBQI+QoIDBYMJ YiKJAAoJELb1zU3GuiQ/lpEAoIhpp6BozKI8p6eaabzF5MlJH58pAKCu/ROofK8J -Eg2aLos+5zEYrB/LsohGBBARAgAGBQI/rOOvAAoJEK/FI0h4g3QP9pYAoNtSISDD -AAU2HafyAYlLD/yUC4hKAJ0czMsBLbo0M/xPaJ6Ox9Q5Hmw2uIhGBBARAgAGBQI/ -tEN3AAoJEIWWr6swc05mxsMAnRag9X61Ygu1kbfBiqDku4czTd9pAJ4q5W8KZ0+2 -ujTrEPN55NdWtnXj4YhGBBARAgAGBQJDW7PqAAoJEIvYLm8wuUtcf3QAnRCyqF0C -pMCTdIGc7bDO5I7CIMhTAJ0UTGx0O1d/VwvdDiKWj45N2tNbYIhGBBMRAgAGBQJE -8TMmAAoJEPZJxPRgk1MMCnEAoIm2pP0sIcVh9Yo0YYGAqORrTOL3AJwIbcy+e8HM -NSoNV5u51RnrVKie34hMBBARAgAMBQJBgcsBBYMGItmLAAoJEBhZ0B9ne6HsQo0A -nA/LCTQ3P5kvJvDhg1DsfVTFnJxpAJ49WFjg/kIcaN5iP1JfaBAITZI3H4hMBBAR -AgAMBQJBgcs0BYMGItlYAAoJEIHC9+viE7aSIiMAnRVTVVAfMXvJhV6D5uHfWeeD -046TAJ4kjwP2bHyd6DjCymq+BdEDz63axohMBBARAgAMBQJBgctiBYMGItkqAAoJ -EGtw7Nldw/RzCaoAmwWM6+Rj1zl4D/PIys5nW48Hql3hAJ0bLOBthv96g+7oUy9U -j09Uh41lF4hMBBARAgAMBQJB0JMkBYMF1BFoAAoJEH0lygrBKafCYlUAoIb1r5D6 -qMLMPMO1krHk3MNbX5b5AJ4vryx5fw6iJctC5GWJ+Y8ytXab34hMBBARAgAMBQJC -K1u6BYMFeUjSAAoJEOYbpIkV67mr8xMAoJMy+UJC0sqXMPSxh3BUsdcmtFS+AJ9+ -Z15LpoOnAidTT/K9iODXGViK6ohMBBIRAgAMBQJAKlk6BYMHektSAAoJEDyhHzSU -+vhhJlwAnA/gOdwOThjO8O+dFtdbpKuImfXJAJ0TL53QKp92EzscZSz49lD2YkoE -qohMBBIRAgAMBQJAPfq6BYMHZqnSAAoJEPLXXGPjnGWcst8AoLQ3MJWqttMNHDbl -xSyzXhFGhRU8AJ4ukRzfNJqElQHQ00ZM2WnCVNzOUIhMBBIRAgAMBQJBDgqEBYMG -lpoIAAoJEDnKK/Q9aopf/N0AniE2fcCKO1wDIwusuGVlC+JvnnWbAKDDoUSEYuNn -5qzRbrzWW5zBno/Nb4hMBBIRAgAMBQJCgKU0BYMFI/9YAAoJEAQNwIV8g5+o4yQA -nA9QOFLV5POCddyUMqB/fnctuO9eAJ4sJbLKP/Z3SAiTpKrNo+XZRxauqIhMBBMR -AgAMBQI+TU2EBYMJV1cIAAoJEC27dr+t1MkzBQwAoJU+RuTVSn+TI+uWxUpT82/d -s5NkAJ9bnNodffyMMK7GyMiv/TzifiTD+4hMBBMRAgAMBQJB14B2BYMFzSQWAAoJ -EGbv28jNgv0+P7wAn13uu8YkhwfNMJJhWdpK2/qM/4AQAJ40drnKW2qJ5EEIJwtx -pwapgrzWiYhMBBMRAgAMBQJCGIEOBYMFjCN+AAoJEHbBAxyiMW6hoO4An0Ith3Kx -5/sixbjZR9aEjoePGTNKAJ94SldLiESaYaJx2lGIlD9bbVoHQYhdBBMRAgAdBQJH -rJTPBQkNMFioBQsHCgMEAxUDAgMWAgECF4AACgkQjHGNO1By4fV0KgCgsLpG2wP0 -rc3s07Fync9g7MfairMAoIUefSNKrGTsTxvLeyH4DLzJW/QFiHsEMBECADsFAkJ3 -NfU0HQBPb3BzLi4uIHNob3VsZCBoYXZlIGJlZW4gbG9jYWwhIEknbSAqc28qIHN0 -dXBpZC4uLgAKCRA5yiv0PWqKX+9HAJ0WjTx/rqgouK4QCrOV/2IOU+jMQQCfYSC8 -JgsIIeN8aiyuStTdYrk0VWCIjwQwEQIATwUCRW8Av0gdAFNob3VsZCBoYXZlIGJl -ZW4gYSBsb2NhbCBzaWduYXR1cmUsIG9yIHNvbWV0aGluZyAtIFdURiB3YXMgSSB0 -aGlua2luZz8ACgkQOcor9D1qil+g+wCfcFWoo5qUl4XTE9K8tH3Q+xGWeYYAnjii -KxjtOXc0ls+BlqXxbfZ9uqBsiQIiBBABAgAMBQJBgcuFBYMGItkHAAoJEKrj5s5m -oURoqC8QAIISudocbJRhrTAROOPoMsReyp46Jdp3iL1oFDGcPfkZSBwWh8L+cJjh -dycIwwSeZ1D2h9S5Tc4EnoE0khsS6wBpuAuih5s//coRqIIiLKEdhTmNqulkCH5m -imCzc5zXWZDW0hpLr2InGsZMuh2QCwAkB4RTBM+r18cUXMLV4YHKyjIVaDhsiPP/ -MKUj6rJNsUDmDq1GiJdOjySjtCFjYADlQYSD7zcd1vpqQLThnZBESvEoCqumEfOP -xemNU6xAB0CL+pUpB40pE6Un6Krr5h6yZxYZ/N5vzt0Y3B5UUMkgYDSpjbulNvaU -TFiOxEU3gJvXc1+h0BsxM7FwBZnuMA8LEA+UdQb76YcyuFBcROhmcEUTiducLu84 -E2BZ2NSBdymRQKSinhvXsEWlH6Txm1gtJLynYsvPi4B4JxKbb+awnFPusL8W+gfz -jbygeKdyqzYgKj3M79R3geaY7Q75Kxl1UogiOKcbI5VZvg47OQCWeeERnejqEAdx -EQiwGA/ARhVOP/1l0LQA7jg2P1xTtrBqqC2ufDB+v+jhXaCXxstKSW1lTbv/b0d6 -454UaOUV7RisN39pE2zFvJvY7bwfiwbUJVmYLm4rWJAEOJLIDtDRtt2h8JahDObm -3CWkpadjw57S5v1c/mn+xV9yTgVx5YUfC/788L1HNKXfeVDq8zbAiQIiBBMBAgAM -BQJCnwocBYMFBZpwAAoJENjCCglaJFfPIT4P/25zvPp8ixqV85igs3rRqMBtBsj+ -5EoEW6DJnlGhoi26yf1nasC2frVasWG7i4JIm0U3WfLZERGDjR/nqlOCEqsP5gS3 -43N7r4UpDkBsYh0WxH/ZtST5llFK3zd7XgtxvqKL98l/OSgijH2W2SJ9DGpjtO+T -iegq7igtJzw7Vax9z/LQH2xhRQKZR9yernwMSYaJ72i9SyWbK3k0+e95fGnlR5pF -zlGq320rYHgD7v9yoQ2t1klsAxK6e3b7Z+RiJG6cAU8o8F0kGxjWzF4v8D1op7S+ -IoRdB0Bap01ko0KLyt3+g4/33/2UxsW50BtfqcvYNJvU4bZns1YSqAgDOOanBhg8 -Ip5XPlDxH6J/3997n5JNj/nk5ojfd8nYfe/5TjflWNiput6tZ7frEki1wl6pTNbv -V9C1eLUJMSXfDZyHtUXmiP9DKNpsucCUeBKWRKLqnsHLkLYydsIeUJ8+ciKc+EWh -FxEY+Ml72cXAaz5BuW9L8KHNzZZfez/ZJabiARQpFfjOwAnmhzJ9r++TEKRLEr96 -taUI9/8nVPvT6LnBpcM38Td6dJ639YvuH3ilAqmPPw50YvglIEe4BUYD5r52Seqc -8XQowouGOuBX4vs7zgWFuYA/s9ebfGaIw+uJd/56Xl9ll6q5CghqB/yt1EceFEnF -CAjQc2SeRo6qzx22uQINBD4+ox0QCADv4Yl/Fsx1jjCyU+eMf2sXg3ap9awQ3+XF -pmglhzdrozTZYKceXpqFPb+0ErbDVAjhgW15HjuAK+2Bvo7Ukd986jYd8uZENGJG -N3UNMIep7JfsIeFyCGP901GVbZnSXlAURyZX1TRWGndoV9YLhSN+zctT6GQBbMTv -NoPlwf0nvK//rG5lXDjXXHSHhSqxNxYy7SIzUHMQupfUNjsvCg8Rv871GRt/h+Yt -7XUTMhoJrg+oBFdBlzh2FKKcy3ordfgGtGwpN+jMG7vgXjsPwiVt/m9Jgdu4Tmn/ -WggPOeSD+nyRb7cXG5avJxyKoVNw3PbXnLJff0tcWeUvMpRv8XkbAAMFB/4vCqpr -wIatF+w4AnGKbrcId+3LmZRzmtRKdOyUZgQg4JHUF5Bq7I9ls8OwMP0xnVlpJp9q -cW/AUbouXH3GRTu3Or68ouhaSbi7nF/e+fnlWOdJ3VpD15CdRxeIvhycEahNs5Yj -f0RzLOCyXMF0L74w+NxBNwDunolRWw/qgAHcVBaDni25SjQRzxuwzxvcS/jYua5B -Pk10ocbAexdM+2XSSWThtCTg5qMeyLLUExqGlPbuNaMmUyIlz4hYnSaCGQoe33bq -z/KZ91/keR1DVzK+zPm2vJUjcXHvxd5Jh9C+67CqnYfXf2lcYSSDSfop1Q5611la -F7vRgY0/DXKNYlPUiEwEGBECAAwFAkeslPwFCQ0wWN8ACgkQjHGNO1By4fWlzgCf -Qj3rkfcljYZOuLOn50J7PFuF7FoAnjwWGhwVi9+Fm2B5RZvpo++BBkdP -=Xquv +Eg2aLos+5zEYrB/LsrkCDQQ+PqMdEAgA7+GJfxbMdY4wslPnjH9rF4N2qfWsEN/l +xaZoJYc3a6M02WCnHl6ahT2/tBK2w1QI4YFteR47gCvtgb6O1JHffOo2HfLmRDRi +Rjd1DTCHqeyX7CHhcghj/dNRlW2Z0l5QFEcmV9U0Vhp3aFfWC4Ujfs3LU+hkAWzE +7zaD5cH9J7yv/6xuZVw411x0h4UqsTcWMu0iM1BzELqX1DY7LwoPEb/O9Rkbf4fm +Le11EzIaCa4PqARXQZc4dhSinMt6K3X4BrRsKTfozBu74F47D8Ilbf5vSYHbuE5p +/1oIDznkg/p8kW+3FxuWrycciqFTcNz215yyX39LXFnlLzKUb/F5GwADBQf+Lwqq +a8CGrRfsOAJxim63CHfty5mUc5rUSnTslGYEIOCR1BeQauyPZbPDsDD9MZ1ZaSaf +anFvwFG6Llx9xkU7tzq+vKLoWkm4u5xf3vn55VjnSd1aQ9eQnUcXiL4cnBGoTbOW +I39EcyzgslzBdC++MPjcQTcA7p6JUVsP6oAB3FQWg54tuUo0Ec8bsM8b3Ev42Lmu +QT5NdKHGwHsXTPtl0klk4bQk4OajHsiy1BMahpT27jWjJlMiJc+IWJ0mghkKHt92 +6s/ymfdf5HkdQ1cyvsz5tryVI3Fx78XeSYfQvuuwqp2H139pXGEkg0n6KdUOetdZ +Whe70YGNPw1yjWJT1IhMBBgRAgAMBQI+PqMdBQkJZgGAAAoJEIxxjTtQcuH17p4A +n3r1QpVC9yhnW2cSAjq+kr72GX0eAJ4295kl6NxYEuFApmr1+0uUq/SlsQ== +=Mski + -----END PGP PUBLIC KEY BLOCK----- To import the build key into your personal public GPG keyring, use @@ -650,8 +586,8 @@ pg-signature.html signature, which also is available from the download page. The signature file has the same name as the distribution file with an .asc extension, as shown by the examples in the following table. - Distribution file mysql-standard-5.1.41-linux-i686.tar.gz - Signature file mysql-standard-5.1.41-linux-i686.tar.gz.asc + Distribution file mysql-standard-5.1.46-linux-i686.tar.gz + Signature file mysql-standard-5.1.46-linux-i686.tar.gz.asc Make sure that both files are stored in the same directory and then run the following command to verify the signature for the @@ -659,7 +595,7 @@ pg-signature.html shell> gpg --verify package_name.asc Example: -shell> gpg --verify mysql-standard-5.1.41-linux-i686.tar.gz.asc +shell> gpg --verify mysql-standard-5.1.46-linux-i686.tar.gz.asc gpg: Signature made Tue 12 Jul 2005 23:35:41 EST using DSA key ID 507 2E1F5 gpg: Good signature from "MySQL Package signing key (www.mysql.com) < @@ -679,8 +615,8 @@ build(a)mysql.com>" shell> rpm --checksig package_name.rpm Example: -shell> rpm --checksig MySQL-server-5.1.41-0.glibc23.i386.rpm -MySQL-server-5.1.41-0.glibc23.i386.rpm: md5 gpg OK +shell> rpm --checksig MySQL-server-5.1.46-0.glibc23.i386.rpm +MySQL-server-5.1.46-0.glibc23.i386.rpm: md5 gpg OK Note @@ -705,7 +641,7 @@ shell> rpm --import mysql_pubkey.asc This section describes the default layout of the directories created by installing binary or source distributions provided by - Sun Microsystems, Inc. A distribution provided by another vendor + Oracle Corporation. A distribution provided by another vendor might use a layout different from those shown here. Installations created from our Linux RPM distributions result in @@ -773,9 +709,7 @@ shell> rpm --import mysql_pubkey.asc This section covers the installation of MySQL binary distributions that are provided for various platforms in the form of compressed - tar files (files with a .tar.gz extension). See Section 2.2, - "Installing MySQL from Generic Binaries on Unix/Linux," for a - detailed list. + tar files (files with a .tar.gz extension). To obtain MySQL, see Section 2.1.3, "How to Get MySQL." @@ -793,7 +727,7 @@ shell> rpm --import mysql_pubkey.asc MySQL tar file binary distributions have names of the form mysql-VERSION-OS.tar.gz, where VERSION is a number (for example, - 5.1.41), and OS indicates the type of operating system for which + 5.1.46), and OS indicates the type of operating system for which the distribution is intended (for example, pc-linux-i686). In addition to these generic packages, we also offer binaries in @@ -818,7 +752,7 @@ shell> rpm --import mysql_pubkey.asc first. If you run into problems and need to file a bug report, please use - the instructions in Section 1.6, "How to Report Bugs or Problems." + the instructions in Section 1.7, "How to Report Bugs or Problems." The basic commands that you must execute to install and use a MySQL binary distribution are: @@ -987,7 +921,7 @@ Note MySQL source distributions are provided as compressed tar archives and have names of the form mysql-VERSION.tar.gz, where VERSION is - a number like 5.1.41. + a number like 5.1.46. You need the following tools to build and install MySQL from source: @@ -1005,15 +939,9 @@ Note systems with a deficient tar, you should install GNU tar first. - * A working ANSI C++ compiler. gcc 2.95.2 or later, SGI C++, and - SunPro C++ are some of the compilers that are known to work. - libg++ is not needed when using gcc. gcc 2.7.x has a bug that - makes it impossible to compile some perfectly legal C++ files, - such as sql/sql_base.cc. If you have only gcc 2.7.x, you must - upgrade your gcc to be able to compile MySQL. gcc 2.8.1 is - also known to have problems on some platforms, so it should be - avoided if a newer compiler exists for the platform. gcc - 2.95.2 or later is recommended. + * A working ANSI C++ compiler. GCC 3.2 or later, Sun Studio 10 + or later, Visual Studio 2005 or later, and many current + vendor-supplied compilers are known to work. * A good make program. GNU make is always recommended and is sometimes required. (BSD make fails, and vendor-provided make @@ -1035,7 +963,7 @@ CFLAGS="-O3" CXX=gcc CXXFLAGS="-O3 -feli On most systems, this gives you a fast and stable binary. If you run into problems and need to file a bug report, please use - the instructions in Section 1.6, "How to Report Bugs or Problems." + the instructions in Section 1.7, "How to Report Bugs or Problems." 2.3.1. Source Installation Overview @@ -1121,7 +1049,7 @@ shell> make from config.log that you think can help solve the problem. Also include the last couple of lines of output from configure. To file a bug report, please use the instructions - in Section 1.6, "How to Report Bugs or Problems." + in Section 1.7, "How to Report Bugs or Problems." If the compile fails, see Section 2.3.4, "Dealing with Problems Compiling MySQL," for help. @@ -1497,10 +1425,9 @@ shell> ./configure --with-charset=CHARSE cp1251, cp1256, cp1257, cp850, cp852, cp866, cp932, dec8, eucjpms, euckr, gb2312, gbk, geostd8, greek, hebrew, hp8, keybcs2, koi8r, koi8u, latin1, latin2, latin5, latin7, macce, - macroman, sjis, swe7, tis620, ucs2, ujis, utf8. See Section - 9.2, "The Character Set Used for Data and Sorting." - (Additional character sets might be available. Check the - output from ./configure --help for the current list.) + macroman, sjis, swe7, tis620, ucs2, ujis, utf8. (Additional + character sets might be available. Check the output from + ./configure --help for the current list.) The default collation may also be specified. MySQL uses the latin1_swedish_ci collation by default. To change this, use the --with-collation option: @@ -1602,7 +1529,7 @@ shell> ./configure --with-debug * When given with --enable-community-features, the --enable-profiling option enables the statement profiling capability exposed by the SHOW PROFILE and SHOW PROFILES - statements. (See Section 12.5.5.33, "SHOW PROFILES Syntax.") + statements. (See Section 12.4.5.33, "SHOW PROFILES Syntax.") This option was added in MySQL 5.1.24. It is enabled by default as of MySQL 5.1.28; to disable it, use --disable-profiling. @@ -1610,7 +1537,7 @@ shell> ./configure --with-debug * See Section 2.1, "General Installation Guidance," for options that pertain to particular operating systems. - * See Section 5.5.7.2, "Using SSL Connections," for options that + * See Section 5.5.6.2, "Using SSL Connections," for options that pertain to configuring MySQL to support secure (encrypted) connections. @@ -1664,12 +1591,12 @@ Caution (either a binary or source distribution). To obtain the most recent development source tree, you must have - Bazaar installed. You can obtain Bazaar from the Bazaar VCS - Website (http://bazaar-vcs.org) Bazaar is supported by any - platform that supports Python, and is therefore compatible with - any Linux, Unix, Windows or Mac OS X host. Instructions for - downloading and installing Bazaar on the different platforms are - available on the Bazaar website. + Bazaar installed. You can obtain Bazaar from the Bazaar VCS Web + site (http://bazaar-vcs.org) Bazaar is supported by any platform + that supports Python, and is therefore compatible with any Linux, + Unix, Windows or Mac OS X host. Instructions for downloading and + installing Bazaar on the different platforms are available on the + Bazaar Web site. All MySQL projects are hosted on Launchpad (http://launchpad.net/) MySQL projects, including MySQL server, @@ -1752,7 +1679,7 @@ shell> bzr log page. If you see diffs (changes) or code that you have a question about, do not hesitate to send email to the MySQL internals - mailing list. See Section 1.5.1, "MySQL Mailing Lists." Also, + mailing list. See Section 1.6.1, "MySQL Mailing Lists." Also, if you think you have a better idea on how to do something, send an email message to the list with a patch. @@ -1816,7 +1743,7 @@ shell> make 6. If you have gotten to the make stage, but the distribution does not compile, please enter the problem into our bugs - database using the instructions given in Section 1.6, "How to + database using the instructions given in Section 1.7, "How to Report Bugs or Problems." If you have installed the latest versions of the required GNU tools, and they crash trying to process our configuration files, please report that also. @@ -2092,7 +2019,7 @@ implicit declaration of function `int st * Before any upgrade, back up your databases, including the mysql database that contains the grant tables. See Section - 6.1, "Database Backup Methods." + 6.2, "Database Backup Methods." * Read all the notes in Section 2.4.1.1, "Upgrading from MySQL 5.0 to 5.1." These notes enable you to identify upgrade issues @@ -2117,7 +2044,7 @@ implicit declaration of function `int st * If you are running MySQL Server on Windows, see Section 2.5.7, "Upgrading MySQL on Windows." - * If you are using replication, see Section 16.3.3, "Upgrading a + * If you are using replication, see Section 16.4.3, "Upgrading a Replication Setup," for information on upgrading your replication setup. @@ -2251,7 +2178,7 @@ Note done before upgrading. Use of this statement with a version of MySQL different from the one used to create the table (that is, using it after upgrading) may damage the table. See - Section 12.5.2.6, "REPAIR TABLE Syntax." + Section 12.4.2.6, "REPAIR TABLE Syntax." * After you upgrade to a new version of MySQL, run mysql_upgrade (see Section 4.4.8, "mysql_upgrade --- Check Tables for MySQL @@ -2274,7 +2201,7 @@ Note * If you are running MySQL Server on Windows, see Section 2.5.7, "Upgrading MySQL on Windows." - * If you are using replication, see Section 16.3.3, "Upgrading a + * If you are using replication, see Section 16.4.3, "Upgrading a Replication Setup," for information on upgrading your replication setup. @@ -2322,13 +2249,13 @@ Note upgrading, and reload them into MySQL 5.1 after upgrading. * Known issue: The fix for - Bug#23491: http://bugs.mysql.com/23491 introduced a problem - with SHOW CREATE VIEW, which is used by mysqldump. This causes - an incompatibility when upgrading from versions affected by - that bug fix (MySQL 5.0.40 through 5.0.43, MySQL 5.1.18 - through 5.1.19): If you use mysqldump before upgrading from an - affected version and reload the data after upgrading to a - higher version, you must drop and recreate your views. + Bug#23491: http://bugs.mysql.com/bug.php?id=23491 introduced a + problem with SHOW CREATE VIEW, which is used by mysqldump. + This causes an incompatibility when upgrading from versions + affected by that bug fix (MySQL 5.0.40 through 5.0.43, MySQL + 5.1.18 through 5.1.19): If you use mysqldump before upgrading + from an affected version and reload the data after upgrading + to a higher version, you must drop and recreate your views. * Known issue: Dumps performed by using mysqldump to generate a dump file before the upgrade and reloading the file after @@ -2456,11 +2383,11 @@ RENAME TABLE table_b TO `table b`; * Incompatible change: MySQL 5.1 implements support for a plugin API that allows the loading and unloading of components at runtime, without restarting the server. Section 22.2, "The - MySQL Plugin Interface." The plugin API requires the - mysql.plugin table. After upgrading from an older version of - MySQL, you should run the mysql_upgrade command to create this - table. See Section 4.4.8, "mysql_upgrade --- Check Tables for - MySQL Upgrade." + MySQL Plugin API." The plugin API requires the mysql.plugin + table. After upgrading from an older version of MySQL, you + should run the mysql_upgrade command to create this table. See + Section 4.4.8, "mysql_upgrade --- Check Tables for MySQL + Upgrade." Plugins are installed in the directory named by the plugin_dir system variable. This variable also controls the location from which the server loads user-defined functions (UDFs), which is @@ -2718,7 +2645,7 @@ REPAIR TABLE tbl_name QUICK; specifies the locale that controls the language used to display day and month names and abbreviations. This variable affects the output from the DATE_FORMAT(), DAYNAME() and - MONTHNAME() functions. See Section 9.8, "MySQL Server Locale + MONTHNAME() functions. See Section 9.7, "MySQL Server Locale Support." * As of MySQL 5.1.9, mysqld_safe no longer implicitly invokes @@ -2749,7 +2676,7 @@ REPAIR TABLE tbl_name QUICK; to reload them into an upgraded server. Handlers that contain illegal label references will be rejected. For more information about condition handlers and writing them - to avoid invalid jumps, see Section 12.8.4.2, "DECLARE for + to avoid invalid jumps, see Section 12.7.4.2, "DECLARE for Handlers." * Incompatible change: The parser accepted statements that @@ -2758,13 +2685,13 @@ REPAIR TABLE tbl_name QUICK; contain unclosed /*-comments now are rejected with a syntax error. This fix has the potential to cause incompatibilities. Because - of Bug#26302: http://bugs.mysql.com/26302, which caused the - trailing */ to be truncated from comments in views, stored - routines, triggers, and events, it is possible that objects of - those types may have been stored with definitions that now - will be rejected as syntactically invalid. Such objects should - be dropped and re-created so that their definitions do not - contain truncated comments. + of Bug#26302: http://bugs.mysql.com/bug.php?id=26302, which + caused the trailing */ to be truncated from comments in views, + stored routines, triggers, and events, it is possible that + objects of those types may have been stored with definitions + that now will be rejected as syntactically invalid. Such + objects should be dropped and re-created so that their + definitions do not contain truncated comments. * Incompatible change: Multiple-table DELETE statements containing ambiguous aliases could have unintended side @@ -2851,21 +2778,20 @@ mysql> source /tmp/triggers.sql // mysqldump or mysqlhotcopy can be used as alternatives. * The LOAD DATA FROM MASTER and LOAD TABLE FROM MASTER - statements are deprecated. See Section 12.6.2.2, "LOAD DATA + statements are deprecated. See Section 12.5.2.2, "LOAD DATA FROM MASTER Syntax," for recommended alternatives. * The INSTALL PLUGIN and UNINSTALL PLUGIN statements that are used for the plugin API are new. So is the WITH PARSER clause for FULLTEXT index creation that associates a parser plugin - with a full-text index. Section 22.2, "The MySQL Plugin - Interface." + with a full-text index. Section 22.2, "The MySQL Plugin API." C API Changes: * Incompatible change: As of MySQL 5.1.7, the mysql_stmt_attr_get() C API function returns a boolean rather than an unsigned int for STMT_ATTR_UPDATE_MAX_LENGTH. - (Bug#16144: http://bugs.mysql.com/16144) + (Bug#16144: http://bugs.mysql.com/bug.php?id=16144) 2.4.2. Downgrading MySQL @@ -2930,10 +2856,10 @@ mysql> source /tmp/triggers.sql // 5. Reload the dump file into the older server. Your tables should be accessible. - It might also be the case that the structure of the system tables - in the mysql database has changed and that downgrading introduces - some loss of functionality or requires some adjustments. Here are - some examples: + It might also be the case that system tables in the mysql database + have changed and that downgrading introduces some loss of + functionality or requires some adjustments. Here are some + examples: * Trigger creation requires the TRIGGER privilege as of MySQL 5.1. In MySQL 5.0, there is no TRIGGER privilege and SUPER is @@ -2944,6 +2870,12 @@ mysql> source /tmp/triggers.sql // * Triggers were added in MySQL 5.0, so if you downgrade from 5.0 to 4.1, you cannot use triggers at all. + * The mysql.proc.comment column definition changed between MySQL + 5.1 and 5.5. After a downgrade from 5.5 to 5.1, this table is + seen as corrupt and in need of repair. To workaround this + problem, execute mysql_upgrade from the version of MySQL to + which you downgraded. + 2.4.2.1. Downgrading to MySQL 5.0 When downgrading to MySQL 5.0 from MySQL 5.1, you should keep in @@ -2979,9 +2911,10 @@ mysql> source /tmp/triggers.sql // --all-databases option). Instead, you should run mysqldump --routines prior to performing the downgrade and run the stored routines DDL statements following the downgrade. - See Bug#11986: http://bugs.mysql.com/11986, - Bug#30029: http://bugs.mysql.com/30029, and - Bug#30660: http://bugs.mysql.com/30660, for more information. + See Bug#11986: http://bugs.mysql.com/bug.php?id=11986, + Bug#30029: http://bugs.mysql.com/bug.php?id=30029, and + Bug#30660: http://bugs.mysql.com/bug.php?id=30660, for more + information. * Triggers. Trigger creation requires the TRIGGER privilege as of MySQL 5.1. In MySQL 5.0, there is no TRIGGER privilege and @@ -3060,10 +2993,10 @@ mysql> source /tmp/triggers.sql // report, the bug number is given. The list applies both for binary upgrades and downgrades. For - example, Bug#27877: http://bugs.mysql.com/27877 was fixed in MySQL - 5.1.24 and 5.4.0, so it applies to upgrades from versions older - than 5.1.24 to 5.1.24 or newer, and to downgrades from 5.1.24 or - newer to versions older than 5.1.24. + example, Bug#27877: http://bugs.mysql.com/bug.php?id=27877 was + fixed in MySQL 5.1.24 and 5.4.0, so it applies to upgrades from + versions older than 5.1.24 to 5.1.24 or newer, and to downgrades + from 5.1.24 or newer to versions older than 5.1.24. In many cases, you can use CHECK TABLE ... FOR UPGRADE to identify tables for which index rebuilding is required. (It will report: @@ -3073,33 +3006,36 @@ mysql> source /tmp/triggers.sql // TABLE. However, the use of CHECK TABLE applies only after upgrades, not downgrades. Also, CHECK TABLE is not applicable to all storage engines. For details about which storage engines CHECK - TABLE supports, see Section 12.5.2.3, "CHECK TABLE Syntax." + TABLE supports, see Section 12.4.2.3, "CHECK TABLE Syntax." Changes that cause index rebuilding to be necessary: - * MySQL 5.0.48, 5.1.21 (Bug#29461: http://bugs.mysql.com/29461) + * MySQL 5.0.48, 5.1.21 + (Bug#29461: http://bugs.mysql.com/bug.php?id=29461) Affects indexes for columns that use any of these character sets: eucjpms, euc_kr, gb2312, latin7, macce, ujis Affected tables can be detected by CHECK TABLE ... FOR UPGRADE as of MySQL 5.1.29, 5.4.0 (see - Bug#39585: http://bugs.mysql.com/39585) + Bug#39585: http://bugs.mysql.com/bug.php?id=39585) - * MySQL 5.0.48, 5.1.23 (Bug#27562: http://bugs.mysql.com/27562) + * MySQL 5.0.48, 5.1.23 + (Bug#27562: http://bugs.mysql.com/bug.php?id=27562) Affects indexes that use the ascii_general_ci collation for columns that contain any of these characters: '`' GRAVE ACCENT, '[' LEFT SQUARE BRACKET, '\' REVERSE SOLIDUS, ']' RIGHT SQUARE BRACKET, '~' TILDE Affected tables can be detected by CHECK TABLE ... FOR UPGRADE as of MySQL 5.1.29, 5.4.0 (see - Bug#39585: http://bugs.mysql.com/39585) + Bug#39585: http://bugs.mysql.com/bug.php?id=39585) - * MySQL 5.1.24, 5.4.0 (Bug#27877: http://bugs.mysql.com/27877) + * MySQL 5.1.24, 5.4.0 + (Bug#27877: http://bugs.mysql.com/bug.php?id=27877) Affects indexes that use the utf8_general_ci or ucs2_general_ci collation for columns that contain 'ß' LATIN SMALL LETTER SHARP S (German). Affected tables can be detected by CHECK TABLE ... FOR UPGRADE as of MySQL 5.1.30, 5.4.0 (see - Bug#40053: http://bugs.mysql.com/40053) + Bug#40053: http://bugs.mysql.com/bug.php?id=40053) 2.4.4. Rebuilding or Repairing Tables or Indexes @@ -3107,10 +3043,12 @@ mysql> source /tmp/triggers.sql // necessitated by changes to MySQL such as how data types are handled or changes to character set handling. For example, an error in a collation might have been corrected, necessitating a - table rebuild to rebuild the indexes for character columns that - use the collation. It might also be that a table repair or upgrade - should be done as indicated by a table check operation such as - that performed by CHECK TABLE, mysqlcheck, or mysql_upgrade. + table rebuild to update the indexes for character columns that use + the collation. (For examples, see Section 2.4.3, "Checking Whether + Tables or Indexes Must Be Rebuilt.") It might also be that a table + repair or upgrade should be done as indicated by a table check + operation such as that performed by CHECK TABLE, mysqlcheck, or + mysql_upgrade. Methods for rebuilding a table include dumping and reloading it, or using ALTER TABLE or REPAIR TABLE. @@ -3120,26 +3058,25 @@ Note If you are rebuilding tables because a different version of MySQL will not handle them after a binary (in-place) upgrade or downgrade, you must use the dump-and-reload method. Dump the - tables before upgrading or downgrading (using your original - version of MySQL), and reload the tables after upgrading or - downgrading (after installing the new version). + tables before upgrading or downgrading using your original version + of MySQL. Then reload the tables after upgrading or downgrading. If you use the dump-and-reload method of rebuilding tables only for the purpose of rebuilding indexes, you can perform the dump either before or after upgrading or downgrading. Reloading still must be done afterward. - To re-create a table by dumping and reloading it, use mysqldump to + To rebuild a table by dumping and reloading it, use mysqldump to create a dump file and mysql to reload the file: shell> mysqldump db_name t1 > dump.sql shell> mysql db_name < dump.sql - To recreate all the tables in a single database, specify the + To rebuild all the tables in a single database, specify the database name without any following table name: shell> mysqldump db_name > dump.sql shell> mysql db_name < dump.sql - To recreate all tables in all databases, use the --all-databases + To rebuild all tables in all databases, use the --all-databases option: shell> mysqldump --all-databases > dump.sql shell> mysql < dump.sql @@ -3165,7 +3102,7 @@ mysql> REPAIR TABLE t1; the file, as described earlier. For specifics about which storage engines REPAIR TABLE supports, - see Section 12.5.2.6, "REPAIR TABLE Syntax." + see Section 12.4.2.6, "REPAIR TABLE Syntax." mysqlcheck --repair provides command-line access to the REPAIR TABLE statement. This can be a more convenient means of repairing @@ -3447,7 +3384,7 @@ Note below for reference: * Windows Essentials --- this package has a file name similar to - mysql-essential-5.1.41-win32.msi and is supplied as a + mysql-essential-5.1.46-win32.msi and is supplied as a Microsoft Installer (MSI) package. The package includes the minimum set of files needed to install MySQL on Windows, including the MySQL Server Instance Config Wizard. This @@ -3458,7 +3395,7 @@ Note MySQL with the MSI Package." * Windows MSI Installer (Complete) --- this package has a file - name similar to mysql-5.1.41-win32.zip and contains all files + name similar to mysql-5.1.46-win32.zip and contains all files needed for a complete Windows installation, including the MySQL Server Instance Config Wizard. This package includes optional components such as the embedded server and benchmark @@ -3467,7 +3404,7 @@ Note MySQL with the MSI Package." * Without installer --- this package has a file name similar to - mysql-noinstall-5.1.41-win32.zip and contains all the files + mysql-noinstall-5.1.46-win32.zip and contains all the files found in the Complete install package, with the exception of the MySQL Server Instance Config Wizard. This package does not include an automated installer, and must be manually installed @@ -3618,7 +3555,7 @@ Note feedback of users like you. If you find that the MySQL Installation Wizard is lacking some feature important to you, or if you discover a bug, please report it in our bugs database using - the instructions given in Section 1.6, "How to Report Bugs or + the instructions given in Section 1.7, "How to Report Bugs or Problems." 2.5.3.1.1. Downloading and Starting the MySQL Installation Wizard @@ -3720,7 +3657,7 @@ Note directory. In a default installation it contains C:\Program Files\MySQL\MySQL Server 5.1\. The Version string contains the release number. For example, for an installation of MySQL Server - 5.1.41, the key contains a value of 5.1.41. + 5.1.46, the key contains a value of 5.1.46. These registry keys are used to help external tools identify the installed location of the MySQL server, preventing a complete scan @@ -3963,8 +3900,8 @@ shell> msiexec /x /quiet mysql-5.1.39.ms Apart from making changes to the my.ini file by running the MySQL Server Instance Config Wizard again, you can modify it by opening it with a text editor and making any necessary changes. You can - also modify the server configuration with the MySQL Administrator - (http://www.mysql.com/products/administrator/) utility. For more + also modify the server configuration with the + http://www.mysql.com/products/administrator/ utility. For more information about server configuration, see Section 5.1.2, "Server Command Options." @@ -4262,17 +4199,31 @@ Warning 2.5.4.11. The Security Options Dialog - It is strongly recommended that you set a root password for your - MySQL server, and the MySQL Server Instance Config Wizard requires - by default that you do so. If you do not wish to set a root - password, uncheck the box next to the Modify Security Settings - option. - MySQL Server Instance Config Wizard: Security - - To set the root password, enter the desired password into both the - New root password and Confirm boxes. If you are reconfiguring an - existing server, you need to enter the existing root password into - the Current root password box. + The content of the security options portion of the MySQL Server + Instance Configuration Wizard will depend on whether this is a new + installation, or modifying an existing installation. + + * Setting the root password for a new installation + It is strongly recommended that you set a root password for + your MySQL server, and the MySQL Server Instance Config Wizard + requires by default that you do so. If you do not wish to set + a root password, uncheck the box next to the Modify Security + Settings option. + MySQL Server Instance Config Wizard: Security + + * To set the root password, enter the desired password into both + the New root password and Confirm boxes. + Setting the root password for an existing installation + If you are modifying the configuration of an existing + configuration, or you are installing an upgrade and the MySQL + Server Instance Configuration Wizard has detected an existing + MySQL system, then you must enter the existing password for + root before changing the configuration information. + MySQL Server Instance Config Wizard: Security (Existing + Installation) + If you want to change the current root password, enter the + desired new password into both the New root password and + Confirm boxes. To allow root logins from across the network, check the box next to the Enable root access from remote machines option. This @@ -4718,7 +4669,7 @@ InnoDB: foreign key constraint system ta something like this, which indicates that the server is ready to service client connections: mysqld: ready for connections -Version: '5.1.41' socket: '' port: 3306 +Version: '5.1.46' socket: '' port: 3306 The server continues to write to the console any further diagnostic output it produces. You can open a new console window @@ -5104,7 +5055,7 @@ C:\> sc delete mysql Windows. 2. You should always back up your current MySQL installation - before performing an upgrade. See Section 6.1, "Database + before performing an upgrade. See Section 6.2, "Database Backup Methods." 3. Download the latest Windows distribution of MySQL from @@ -5389,7 +5340,7 @@ ROM db" mysql names that are compatible with the current ANSI code pages. For example, the following Japanese directory name will not work in the Western locale (code page 1252): -datadir="C:/维基百科关于中文维基百科" +datadir="C:/私たちのプロジェクトのデータ" The same limitation applies to directory and file names referred to in SQL statements, such as the data file path name in LOAD DATA INFILE. @@ -5451,10 +5402,9 @@ Note from the Bazaar tree. For production use, we do not advise using a MySQL server built by yourself from source. Normally, it is best to use precompiled binary distributions of MySQL that are built - specifically for optimal performance on Windows by Sun - Microsystems, Inc. Instructions for installing binary - distributions are available in Section 2.5, "Installing MySQL on - Windows." + specifically for optimal performance on Windows by Oracle + Corporation. Instructions for installing binary distributions are + available in Section 2.5, "Installing MySQL on Windows." To build MySQL on Windows from source, you must satisfy the following system, compiler, and resource requirements: @@ -5514,8 +5464,8 @@ Note You also need a MySQL source distribution for Windows, which can be obtained two ways: - * Obtain a source distribution packaged by Sun Microsystems, - Inc. These are available from http://dev.mysql.com/downloads/. + * Obtain a source distribution packaged by Oracle Corporation. + These are available from http://dev.mysql.com/downloads/. * Package a source distribution yourself from the latest Bazaar developer source tree. For instructions on pulling the latest @@ -5525,19 +5475,20 @@ Note If you find something not working as expected, or you have suggestions about ways to improve the current build process on Windows, please send a message to the win32 mailing list. See - Section 1.5.1, "MySQL Mailing Lists." + Section 1.6.1, "MySQL Mailing Lists." 2.5.10.1. Building MySQL from Source Using CMake and Visual Studio You can build MySQL on Windows by using a combination of cmake and Microsoft Visual Studio .NET 2003 (7.1), Microsoft Visual Studio - 2005 (8.0) or Microsoft Visual C++ 2005 Express Edition. You must - have the appropriate Microsoft Platform SDK installed. + 2005 (8.0), Microsoft Visual Studio 2008 (9.0) or Microsoft Visual + C++ 2005 Express Edition. You must have the appropriate Microsoft + Platform SDK installed. Note To compile from the source code on Windows you must use the - standard source distribution (for example, mysql-5.1.41.tar.gz). + standard source distribution (for example, mysql-5.1.46.tar.gz). You build from the same distribution as used to build MySQL on Unix, Linux and other platforms. Do not use the Windows Source distributions as they do not contain the necessary configuration @@ -5551,8 +5502,19 @@ Note tool that can read .zip files. This directory is the work directory in the following instructions. - 2. Using a command shell, navigate to the work directory and run - the following command: +Note + You must run the commands in the win directory from the + top-level source directory. Do not change into the win + directory, as the commands will not be executed correctly. + + 2. Start a command shell. If you have not configured the PATH and + other environment variables for all command shells, you may be + able to start a command shell from the Start Menu within the + Windows Visual Studio menu that contains the necessary + environment changes. + + 3. Within the command shell, navigate to the work directory and + run the following command: C:\workdir>win\configure.js options If you have associated the .js file extension with an application such as a text editor, then you may need to use @@ -5603,16 +5565,19 @@ C:\workdir>cscript win\configure.js opti C:\workdir>win\configure.js WITH_INNOBASE_STORAGE_ENGINE WITH_PARTITION_STORAGE_ENGINE MYSQL_SERVER_SUFFIX=-pro - 3. From the work directory, execute the win\build-vs8.bat or - win\build-vs71.bat file, depending on the version of Visual + 4. From the work directory, execute the win\build-vs9.bat + (Windows Visual Studio 2008), win\build-vs8.bat (Windows + Visual Studio 2005), or win\build-vs71.bat (Windows Visual + Stidion 2003) script, depending on the version of Visual Studio you have installed. The script invokes CMake, which generates the mysql.sln solution file. - You can also use win\build-vs8_x64.bat to build the 64-bit - version of MySQL. However, you cannot build the 64-bit version - with Visual Studio Express Edition. You must use Visual Studio - 2005 (8.0) or higher. + You can also use the corresponding 64-bit file (for example + win\build-vs8_x64.bat or win\build-vs9_x64.bat) to build the + 64-bit version of MySQL. However, you cannot build the 64-bit + version with Visual Studio Express Edition. You must use + Visual Studio 2005 (8.0) or higher. - 4. From the work directory, open the generated mysql.sln file + 5. From the work directory, open the generated mysql.sln file with Visual Studio and select the proper configuration using the Configuration menu. The menu provides Debug, Release, RelwithDebInfo, MinRelInfo options. Then select Solution > @@ -5621,7 +5586,7 @@ C:\workdir>win\configure.js WITH_INNOBAS important later when you run the test script because that script needs to know which configuration you used. - 5. Test the server. The server built using the preceding + 6. Test the server. The server built using the preceding instructions expects that the MySQL base directory and data directory are C:\mysql and C:\mysql\data by default. If you want to test your server using the source tree root directory @@ -5681,29 +5646,36 @@ C:\> mkdir C:\mysql\sql-bench Installation Notes." 2. From the work directory, copy into the C:\mysql directory the - following directories: + following files and directories: C:\> cd \workdir -C:\workdir> copy client_release\*.exe C:\mysql\bin -C:\workdir> copy client_debug\mysqld.exe C:\mysql\bin\mysqld-debug.ex -e +C:\workdir> mkdir C:\mysql +C:\workdir> mkdir C:\mysql\bin +C:\workdir> copy client\Release\*.exe C:\mysql\bin +C:\workdir> copy sql\Release\mysqld.exe C:\mysql\bin\mysqld.exe C:\workdir> xcopy scripts\*.* C:\mysql\scripts /E C:\workdir> xcopy share\*.* C:\mysql\share /E If you want to compile other clients and link them to MySQL, you should also copy several libraries and header files: -C:\workdir> copy lib_debug\mysqlclient.lib C:\mysql\lib\debug -C:\workdir> copy lib_debug\libmysql.* C:\mysql\lib\debug -C:\workdir> copy lib_debug\zlib.* C:\mysql\lib\debug -C:\workdir> copy lib_release\mysqlclient.lib C:\mysql\lib\opt -C:\workdir> copy lib_release\libmysql.* C:\mysql\lib\opt -C:\workdir> copy lib_release\zlib.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\debug +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\opt +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\opt C:\workdir> copy include\*.h C:\mysql\include C:\workdir> copy libmysql\libmysql.def C:\mysql\include + +Note + If you have compiled a Debug, rather than Release solution, + you can replace Release with Debug in the source file names + shown above. If you want to benchmark MySQL, you should also do this: C:\workdir> xcopy sql-bench\*.* C:\mysql\bench /E After installation, set up and start the server in the same way as - for binary Windows distributions. See Section 2.5, "Installing - MySQL on Windows." + for binary Windows distributions. This includes creating the + system tables by running mysql_install_db. For more information, + see Section 2.5, "Installing MySQL on Windows." 2.5.11. Compiling MySQL Clients on Windows @@ -6626,14 +6598,23 @@ DLTLIB LIB(MYSQLINST) Upgrading an existing MySQL instance - You need to execute the upgrade command, MYSQLINST/UPGMYSQL. You - must specify 6 parameters to perform an upgrade: + You need to execute the upgrade command, MYSQLINST/UPGMYSQL. + +Note + + You cannot use MYSQLINST/UPGMYSQL to upgrade between major + versions of MySQL (for example from 5.0 to 5.1). For information + and advice on migrating between major versions you can use the + advice provided in Section 2.4.1.1, "Upgrading from MySQL 5.0 to + 5.1." + + You must specify 6 parameters to perform an upgrade: * DIR('/QOpenSys/usr/local/') --- sets the installation location for the MySQL files. The directory will be created if it does not already exist. This is the directory that the MySQL server will be installed into, inside a directory with a name - matching the version and release. For example if installing + matching the version and release. For example, if installing MySQL 5.1.39 with the DIR set to /QOpenSys/usr/local/ would result in /QOpenSys/usr/local/mysql-5.1.39-i5os-power64 and a symbolic link to this directory will be created in @@ -7167,7 +7148,7 @@ shell> bin/mysqld_safe --user=mysql & logged in to the system as mysql, in which case you can omit the --user option from the command. Further instructions for running MySQL as an unprivileged user - are given in Section 5.3.5, "How to Run MySQL as a Normal + are given in Section 5.3.6, "How to Run MySQL as a Normal User." If you neglected to create the grant tables before proceeding to this step, the following message appears in the error log @@ -7185,10 +7166,10 @@ shell> bin/mysqladmin variables on your platform and version of MySQL, but should be similar to that shown here: shell> bin/mysqladmin version -mysqladmin Ver 14.12 Distrib 5.1.41, for pc-linux-gnu on i686 +mysqladmin Ver 14.12 Distrib 5.1.46, for pc-linux-gnu on i686 ... -Server version 5.1.41 +Server version 5.1.46 Protocol version 10 Connection Localhost via UNIX socket UNIX socket /var/lib/mysql/mysql.sock @@ -7292,7 +7273,7 @@ shell> mysql -vvf test < ./tests/auto_in The MySQL 5.1 installation procedure creates time zone tables in the mysql database. However, you must populate the tables manually - using the instructions in Section 9.7, "MySQL Server Time Zone + using the instructions in Section 9.6, "MySQL Server Time Zone Support." 2.13.1.1. Problems Running mysql_install_db @@ -7323,7 +7304,7 @@ mysqld ended carefully. The log should be located in the directory XXXXXX named by the error message and should indicate why mysqld didn't start. If you do not understand what happened, include - the log when you post a bug report. See Section 1.6, "How to + the log when you post a bug report. See Section 1.7, "How to Report Bugs or Problems." * There is a mysqld process running @@ -7972,7 +7953,7 @@ Note MYSQL_PS1 The command prompt to use in the mysql command-line client. MYSQL_PWD The default password when connecting to mysqld. Note - that using this is insecure. See Section 5.5.6.2, "End-User + that using this is insecure. See Section 5.3.2.2, "End-User Guidelines for Password Security." MYSQL_TCP_PORT The default TCP/IP port number. MYSQL_UNIX_PORT The default Unix socket file name; used for === modified file 'INSTALL-WIN-SOURCE' --- a/INSTALL-WIN-SOURCE 2009-12-01 07:24:05 +0000 +++ b/INSTALL-WIN-SOURCE 2010-04-28 13:06:11 +0000 @@ -13,10 +13,9 @@ Note from the Bazaar tree. For production use, we do not advise using a MySQL server built by yourself from source. Normally, it is best to use precompiled binary distributions of MySQL that are built - specifically for optimal performance on Windows by Sun - Microsystems, Inc. Instructions for installing binary - distributions are available in Section 2.5, "Installing MySQL on - Windows." + specifically for optimal performance on Windows by Oracle + Corporation. Instructions for installing binary distributions are + available in Section 2.5, "Installing MySQL on Windows." To build MySQL on Windows from source, you must satisfy the following system, compiler, and resource requirements: @@ -76,8 +75,8 @@ Note You also need a MySQL source distribution for Windows, which can be obtained two ways: - * Obtain a source distribution packaged by Sun Microsystems, - Inc. These are available from http://dev.mysql.com/downloads/. + * Obtain a source distribution packaged by Oracle Corporation. + These are available from http://dev.mysql.com/downloads/. * Package a source distribution yourself from the latest Bazaar developer source tree. For instructions on pulling the latest @@ -87,19 +86,20 @@ Note If you find something not working as expected, or you have suggestions about ways to improve the current build process on Windows, please send a message to the win32 mailing list. See - Section 1.5.1, "MySQL Mailing Lists." + Section 1.6.1, "MySQL Mailing Lists." 2.5.10.1. Building MySQL from Source Using CMake and Visual Studio You can build MySQL on Windows by using a combination of cmake and Microsoft Visual Studio .NET 2003 (7.1), Microsoft Visual Studio - 2005 (8.0) or Microsoft Visual C++ 2005 Express Edition. You must - have the appropriate Microsoft Platform SDK installed. + 2005 (8.0), Microsoft Visual Studio 2008 (9.0) or Microsoft Visual + C++ 2005 Express Edition. You must have the appropriate Microsoft + Platform SDK installed. Note To compile from the source code on Windows you must use the - standard source distribution (for example, mysql-5.1.41.tar.gz). + standard source distribution (for example, mysql-5.1.46.tar.gz). You build from the same distribution as used to build MySQL on Unix, Linux and other platforms. Do not use the Windows Source distributions as they do not contain the necessary configuration @@ -113,8 +113,19 @@ Note tool that can read .zip files. This directory is the work directory in the following instructions. - 2. Using a command shell, navigate to the work directory and run - the following command: +Note + You must run the commands in the win directory from the + top-level source directory. Do not change into the win + directory, as the commands will not be executed correctly. + + 2. Start a command shell. If you have not configured the PATH and + other environment variables for all command shells, you may be + able to start a command shell from the Start Menu within the + Windows Visual Studio menu that contains the necessary + environment changes. + + 3. Within the command shell, navigate to the work directory and + run the following command: C:\workdir>win\configure.js options If you have associated the .js file extension with an application such as a text editor, then you may need to use @@ -165,16 +176,19 @@ C:\workdir>cscript win\configure.js opti C:\workdir>win\configure.js WITH_INNOBASE_STORAGE_ENGINE WITH_PARTITION_STORAGE_ENGINE MYSQL_SERVER_SUFFIX=-pro - 3. From the work directory, execute the win\build-vs8.bat or - win\build-vs71.bat file, depending on the version of Visual + 4. From the work directory, execute the win\build-vs9.bat + (Windows Visual Studio 2008), win\build-vs8.bat (Windows + Visual Studio 2005), or win\build-vs71.bat (Windows Visual + Stidion 2003) script, depending on the version of Visual Studio you have installed. The script invokes CMake, which generates the mysql.sln solution file. - You can also use win\build-vs8_x64.bat to build the 64-bit - version of MySQL. However, you cannot build the 64-bit version - with Visual Studio Express Edition. You must use Visual Studio - 2005 (8.0) or higher. + You can also use the corresponding 64-bit file (for example + win\build-vs8_x64.bat or win\build-vs9_x64.bat) to build the + 64-bit version of MySQL. However, you cannot build the 64-bit + version with Visual Studio Express Edition. You must use + Visual Studio 2005 (8.0) or higher. - 4. From the work directory, open the generated mysql.sln file + 5. From the work directory, open the generated mysql.sln file with Visual Studio and select the proper configuration using the Configuration menu. The menu provides Debug, Release, RelwithDebInfo, MinRelInfo options. Then select Solution > @@ -183,7 +197,7 @@ C:\workdir>win\configure.js WITH_INNOBAS important later when you run the test script because that script needs to know which configuration you used. - 5. Test the server. The server built using the preceding + 6. Test the server. The server built using the preceding instructions expects that the MySQL base directory and data directory are C:\mysql and C:\mysql\data by default. If you want to test your server using the source tree root directory @@ -243,26 +257,33 @@ C:\> mkdir C:\mysql\sql-bench Installation Notes." 2. From the work directory, copy into the C:\mysql directory the - following directories: + following files and directories: C:\> cd \workdir -C:\workdir> copy client_release\*.exe C:\mysql\bin -C:\workdir> copy client_debug\mysqld.exe C:\mysql\bin\mysqld-debug.ex -e +C:\workdir> mkdir C:\mysql +C:\workdir> mkdir C:\mysql\bin +C:\workdir> copy client\Release\*.exe C:\mysql\bin +C:\workdir> copy sql\Release\mysqld.exe C:\mysql\bin\mysqld.exe C:\workdir> xcopy scripts\*.* C:\mysql\scripts /E C:\workdir> xcopy share\*.* C:\mysql\share /E If you want to compile other clients and link them to MySQL, you should also copy several libraries and header files: -C:\workdir> copy lib_debug\mysqlclient.lib C:\mysql\lib\debug -C:\workdir> copy lib_debug\libmysql.* C:\mysql\lib\debug -C:\workdir> copy lib_debug\zlib.* C:\mysql\lib\debug -C:\workdir> copy lib_release\mysqlclient.lib C:\mysql\lib\opt -C:\workdir> copy lib_release\libmysql.* C:\mysql\lib\opt -C:\workdir> copy lib_release\zlib.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\debug +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\opt +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\opt C:\workdir> copy include\*.h C:\mysql\include C:\workdir> copy libmysql\libmysql.def C:\mysql\include + +Note + If you have compiled a Debug, rather than Release solution, + you can replace Release with Debug in the source file names + shown above. If you want to benchmark MySQL, you should also do this: C:\workdir> xcopy sql-bench\*.* C:\mysql\bench /E After installation, set up and start the server in the same way as - for binary Windows distributions. See Section 2.5, "Installing - MySQL on Windows." + for binary Windows distributions. This includes creating the + system tables by running mysql_install_db. For more information, + see Section 2.5, "Installing MySQL on Windows." === modified file 'man/comp_err.1' --- a/man/comp_err.1 2009-12-01 07:24:05 +0000 +++ b/man/comp_err.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBcomp_err\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBCOMP_ERR\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBCOMP_ERR\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -63,7 +63,7 @@ shell> \fBcomp_err [\fR\fB\fIoptions\fR\ .\} .PP \fBcomp_err\fR -supports the options described in the following list\&. +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -254,7 +254,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/innochecksum.1' --- a/man/innochecksum.1 2009-12-01 07:24:05 +0000 +++ b/man/innochecksum.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBinnochecksum\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBINNOCHECKSUM\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBINNOCHECKSUM\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -59,7 +59,7 @@ shell> \fBinnochecksum [\fR\fB\fIoptions .\} .PP \fBinnochecksum\fR -supports the options described in the following list\&. For options that refer to page numbers, the numbers are zero\-based\&. +supports the following options\&. For options that refer to page numbers, the numbers are zero\-based\&. .sp .RS 4 .ie n \{\ @@ -141,7 +141,7 @@ Verbose mode; print a progress indicator .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/make_win_bin_dist.1' --- a/man/make_win_bin_dist.1 2009-12-01 07:24:05 +0000 +++ b/man/make_win_bin_dist.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmake_win_bin_dist\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMAKE_WIN_BIN_DIST" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMAKE_WIN_BIN_DIST" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -67,7 +67,7 @@ bin/mysqld\-max\&.exe=\&.\&./my\-max\-bu If you specify a directory, the entire directory will be copied\&. .PP \fBmake_win_bin_dist\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -169,7 +169,7 @@ directories)\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/msql2mysql.1' --- a/man/msql2mysql.1 2009-12-01 07:24:05 +0000 +++ b/man/msql2mysql.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmsql2mysql\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMSQL2MYSQL\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMSQL2MYSQL\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -60,7 +60,7 @@ utility to make the function name substi .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/my_print_defaults.1' --- a/man/my_print_defaults.1 2009-12-01 07:24:05 +0000 +++ b/man/my_print_defaults.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmy_print_defaults\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMY_PRINT_DEFAULTS" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMY_PRINT_DEFAULTS" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -53,7 +53,7 @@ shell> \fBmy_print_defaults mysqlcheck c The output consists of options, one per line, in the form that they would be specified on the command line\&. .PP \fBmy_print_defaults\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -195,7 +195,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisam_ftdump.1' --- a/man/myisam_ftdump.1 2009-12-01 07:24:05 +0000 +++ b/man/myisam_ftdump.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisam_ftdump\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAM_FTDUMP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAM_FTDUMP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -33,7 +33,13 @@ indexes in MyISAM tables\&. It reads the MyISAM -index file directly, so it must be run on the server host where the table is located +index file directly, so it must be run on the server host where the table is located\&. Before using +\fBmyisam_ftdump\fR, be sure to issue a +FLUSH TABLES +statement first if the server is running\&. +.PP +\fBmyisam_ftdump\fR +scans and dumps the entire index, which is not particularly fast\&. On the other hand, the distribution of words changes infrequently, so it need not be run often\&. .PP Invoke \fBmyisam_ftdump\fR @@ -120,6 +126,20 @@ shell> \fBmyisam_ftdump /usr/local/mysql .RE .\} .PP +You can use +\fBmyisam_ftdump\fR +to generate a list of index entries in order of frequency of occurrence like this: +.sp +.if n \{\ +.RS 4 +.\} +.nf +shell> \fBmyisam_ftdump \-c mytexttable 1 | sort \-r\fR +.fi +.if n \{\ +.RE +.\} +.PP \fBmyisam_ftdump\fR supports the following options: .sp @@ -222,7 +242,7 @@ Verbose mode\&. Print more output about .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisamchk.1' --- a/man/myisamchk.1 2009-12-01 07:24:05 +0000 +++ b/man/myisamchk.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisamchk\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAMCHK\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAMCHK\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -219,12 +219,16 @@ CHECK TABLE instead of \fBmyisamchk\fR to check tables\&. See -Section\ \&12.5.2.3, \(lqCHECK TABLE Syntax\(rq\&. +Section\ \&12.4.2.3, \(lqCHECK TABLE Syntax\(rq\&. .sp .5v .RE .PP \fBmyisamchk\fR -supports the options in the following table\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[myisamchk] +option file group\&. +\fBmyisamchk\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .SH "MYISAMCHK GENERAL OPTIONS" .\" options: myisamchk @@ -521,18 +525,7 @@ system variable\&. For more information, myisam_stats_method in Section\ \&5.1.4, \(lqServer System Variables\(rq, and -Section\ \&7.4.7, \(lqMyISAM Index Statistics Collection\(rq\&. For MySQL 5\&.1, -stats_method -was added in MySQL 5\&.0\&.14\&. For older versions, the statistics collection method is equivalent to -nulls_equal\&. -.PP -The -ft_min_word_len -and -ft_max_word_len -variables are available as of MySQL 4\&.0\&.0\&. -ft_stopword_file -is available as of MySQL 4\&.0\&.19\&. +Section\ \&7.4.7, \(lqMyISAM Index Statistics Collection\(rq\&. .PP ft_min_word_len and @@ -824,7 +817,7 @@ file as \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -981,7 +974,7 @@ and \fB\-\-quick\fR, \fB\-q\fR .sp -Achieve a faster repair by not modifying the data file\&. You can specify this option twice to force +Achieve a faster repair by modifying only the index file, not the data file\&. You can specify this option twice to force \fBmyisamchk\fR to modify the original data file in case of duplicate keys\&. .RE @@ -1469,7 +1462,11 @@ The format used to store table rows\&. T Fixed length\&. Other possible values are Compressed and -Packed\&. +Packed\&. (Packed +corresponds to what +SHOW TABLE STATUS +reports as +Dynamic\&.) .RE .sp .RS 4 @@ -1585,7 +1582,7 @@ The number of rows in the table\&. Deleted blocks .sp How many deleted blocks still have reserved space\&. You can optimize your table to minimize this space\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -1613,7 +1610,7 @@ Data records\&. Deleted data .sp How many bytes of unreclaimed deleted data there are\&. You can optimize your table to minimize this space\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -2100,7 +2097,7 @@ The number of rows in the table\&. Deleted blocks .sp How many deleted blocks still have reserved space\&. You can optimize your table to minimize this space\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -2238,7 +2235,7 @@ What percentage of the data file is unus Blocks/Record .sp Average number of blocks per row (that is, how many links a fragmented row is composed of)\&. This is always 1\&.0 for fixed\-format tables\&. This value should stay as close to 1\&.0 as possible\&. If it gets too large, you can reorganize the table\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -2447,7 +2444,7 @@ instead of .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisamlog.1' --- a/man/myisamlog.1 2009-12-01 07:24:05 +0000 +++ b/man/myisamlog.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisamlog\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAMLOG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAMLOG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -225,7 +225,7 @@ Display version information\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisampack.1' --- a/man/myisampack.1 2009-12-01 07:24:05 +0000 +++ b/man/myisampack.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisampack\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAMPACK\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAMPACK\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -100,7 +100,7 @@ to rebuild its indexes\&. \fBmyisamchk\fR(1)\&. .PP \fBmyisampack\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options\&. It also reads option files and supports the options for processing them described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -149,7 +149,7 @@ Make a backup of each table\'s data file \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -832,7 +832,7 @@ option to .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql-stress-test.pl.1' --- a/man/mysql-stress-test.pl.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql-stress-test.pl.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql-stress-test.pl\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQL\-STRESS\-TE" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQL\-STRESS\-TE" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -75,9 +75,9 @@ Display a help message and exit\&. .\} .\" mysql-stress-test.pl: abort-on-error option .\" abort-on-error option: mysql-stress-test.pl -\fB\-\-abort\-on\-error\fR +\fB\-\-abort\-on\-error=\fR\fB\fIN\fR\fR .sp -Unknown\&. +Causes the program to abort if an error with severity less than or equal to N was encountered\&. Set to 1 to abort on any error\&. .RE .sp .RS 4 @@ -169,7 +169,8 @@ program\&. .\" server-database option: mysql-stress-test.pl \fB\-\-server\-database=\fR\fB\fIdb_name\fR\fR .sp -The database to use for the tests\&. +The database to use for the tests\&. The default is +test\&. .RE .sp .RS 4 @@ -333,7 +334,7 @@ option\&. \fB\-\-stress\-init\-file[=\fR\fB\fIpath\fR\fR\fB]\fR .sp \fIfile_name\fR -is the location of the file that contains the list of tests\&. If missing, the default file is +is the location of the file that contains the list of tests to be run once to initialize the database for the testing\&. If missing, the default file is stress_init\&.txt in the test suite directory\&. .RE @@ -464,21 +465,6 @@ The duration of stress testing in second .sp -1 .IP \(bu 2.3 .\} -.\" mysql-stress-test.pl: test-suffix option -.\" test-suffix option: mysql-stress-test.pl -\fB\-\-test\-suffix=\fR\fB\fIstr\fR\fR -.sp -Unknown\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-stress-test.pl: threads option .\" threads option: mysql-stress-test.pl \fB\-\-threads=\fR\fB\fIN\fR\fR @@ -503,7 +489,7 @@ Verbose mode\&. Print more information a .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql-test-run.pl.1' --- a/man/mysql-test-run.pl.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql-test-run.pl.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql-test-run.pl\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQL\-TEST\-RUN\" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQL\-TEST\-RUN\" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -30,8 +30,7 @@ The \fBmysql\-test\-run\&.pl\fR Perl script is the main application used to run the MySQL test suite\&. It invokes \fBmysqltest\fR -to run individual test cases\&. (Prior to MySQL 4\&.1, a similar shell script, -\fBmysql\-test\-run\fR, can be used instead\&.) +to run individual test cases\&. .PP Invoke \fBmysql\-test\-run\&.pl\fR @@ -84,7 +83,7 @@ shell> \fBmysql\-test\-run\&.pl t/mytest .RE .\} .PP -As of MySQL 5\&.1\&.23, a suite name can be given as part of the test name\&. That is, the syntax for naming a test is: +A suite name can be given as part of the test name\&. That is, the syntax for naming a test is: .sp .if n \{\ .RS 4 @@ -98,7 +97,14 @@ As of MySQL 5\&.1\&.23, a suite name can .PP If a suite name is given, \fBmysql\-test\-run\&.pl\fR -looks in that suite for the test\&. With no suite name, +looks in that suite for the test\&. The test file corresponding to a test named +\fIsuite_name\&.test_name\fR +is found in +suite/\fIsuite_name\fR/t/\fItest_name\fR\&.test\&. There is also an implicit suite name +main +for the tests in the top +t +directory\&. With no suite name, \fBmysql\-test\-run\&.pl\fR looks in the default list of suites for a match and runs the test in any suites where it finds the test\&. Suppose that the default suite list is main, @@ -131,11 +137,11 @@ rpl)\&. \fB\-\-skip\-test\fR has the opposite effect of skipping test cases for which the names share a common prefix\&. .PP -As of MySQL 5\&.0\&.54/5\&.1\&.23/6\&.0\&.5, the argument for the +The argument for the \fB\-\-do\-test\fR and \fB\-\-skip\-test\fR -options allows more flexible specification of which tests to perform or skip\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, +options also allows more flexible specification of which tests to perform or skip\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, \fB\-\-do\-test=testa\fR matches tests that begin with testa, @@ -150,7 +156,7 @@ main followed by testa with anything in between\&. In the latter case, the pattern match is not anchored to the beginning of the test name, so it also matches names such as -xmainytestz\&. +xmainytesta\&. .PP To perform setup prior to running tests, \fBmysql\-test\-run\&.pl\fR @@ -160,8 +166,7 @@ with the \fB\-\-bootstrap\fR and \fB\-\-skip\-grant\-tables\fR -options (see -\m[blue]\fBTypical \fBconfigure\fR Options\fR\m[]\&\s-2\u[1]\d\s+2)\&. If MySQL was configured with the +options\&. If MySQL was configured with the \fB\-\-disable\-grant\-options\fR option, \fB\-\-bootstrap\fR, @@ -217,28 +222,11 @@ shell> \fB\&./mysqltest\-run\&.pl \-\-fo .RE .\} .PP -If you have a copy of -\fBmysqld\fR -running on the machine where you want to run the test suite, you do not have to stop it, as long as it is not using ports -9306 -or -9307\&. If either of those ports is taken, you should set the -MTR_BUILD_THREAD -environment variable to an appropriate value, and the test suite will use a different set of ports for master, slave, NDB, and Instance Manager)\&. For example: -.sp -.if n \{\ -.RS 4 -.\} -.nf -shell> \fBexport MTR_BUILD_THREAD=31\fR -shell> \fB\&./mysql\-test\-run\&.pl [\fR\fB\fIoptions\fR\fR\fB] [\fR\fB\fItest_name\fR\fR\fB]\fR -.fi -.if n \{\ -.RE -.\} -.PP \fBmysql\-test\-run\&.pl\fR -defines several environment variables\&. Some of them are listed in the following table\&. +uses several environment variables\&. Some of them are listed in the following table\&. Some of these are set from the outside and used by +\fBmysql\-test\-run\&.pl\fR, others are set by +\fBmysql\-test\-run\&.pl\fR +instead, and may be referred to in tests\&. .TS allbox tab(:); l l @@ -246,6 +234,12 @@ l l l l l l l l +l l +l l +l l +l l +l l +l l l l. T{ \fBVariable\fR @@ -253,15 +247,52 @@ T}:T{ \fBMeaning\fR T} T{ -MYSQL_TEST +MTR_VERSION T}:T{ -Path name to \fBmysqltest\fR binary +If set to 1, will run the older version 1 of + \fBmysql\-test\-run\&.pl\fR\&. This will affect + what functionailty is available and what command line + options are supported\&. T} T{ -MYSQLTEST_VARDIR +MTR_MEM T}:T{ -Path name to the var directory that is used for - logs, temporary files, and so forth +If set to anything, will run tests with files in "memory" using tmpfs or + ramdisk\&. Not available on Windows\&. Same as + \fB\-\-mem\fR option +T} +T{ +MTR_PARALLEL +T}:T{ +If set, defines number of parallel threads executing tests\&. Same as + \fB\-\-parallel\fR option +T} +T{ +MTR_BUILD_THREAD +T}:T{ +If set, defines which port number range is used for the server +T} +T{ +MTR_PORT_BASE +T}:T{ +If set, defines which port number range is used for the server +T} +T{ +MTR_\fINAME\fR_TIMEOUT +T}:T{ +Setting of a timeout in minutes or seconds, corresponding to command + line option + \fB\-\-\fR\fB\fIname\fR\fR\fB\-timeout\fR\&. + Avaliable timeout names are TESTCASE, + SUITE (both in minutes) and + START, SHUTDOWN + (both in seconds)\&. These variables are supported from + MySQL 5\&.1\&.44\&. +T} +T{ +MYSQL_TEST +T}:T{ +Path name to \fBmysqltest\fR binary T} T{ MYSQLD_BOOTSTRAP @@ -269,18 +300,31 @@ T}:T{ Full path name to \fBmysqld\fR that has all options enabled T} T{ -MASTER_MYPORT +MYSQLTEST_VARDIR +T}:T{ +Path name to the var directory that is used for + logs, temporary files, and so forth +T} +T{ +MYSQL_TEST_DIR T}:T{ -??? +Full path to the mysql\-test directory where tests + are being run from T} T{ -MASTER_MYSOCK +MYSQL_TMP_DIR T}:T{ -??? +Path to temp directory used for temporary files during tests T} .TE .sp 1 .PP +The variable +MTR_PORT_BASE +was added in MySQL 5\&.1\&.45 as a more logical replacement for +MTR_BUILD_THREAD\&. It gives the actual port number directly (will be rounded down to a multiple of 10)\&. If you use +MTR_BUILD_THREAD, the port number is found by multiplying this by 10 and adding 10000\&. +.PP Tests sometimes rely on certain environment variables being defined\&. For example, certain tests assume that MYSQL_TEST is defined so that @@ -288,16 +332,23 @@ is defined so that can invoke itself with exec $MYSQL_TEST\&. .PP +Other tests may refer to the last three variables listed in the preceeding table, to locate files to read or write\&. For example, tests that need to create files will typically put them in +$MYSQL_TMP_DIR/\fIfile_name\fR\&. +.PP +If you are running +\fBmysql\-test\-run\&.pl\fR +version 1 by setting +MTR_VERSION, note that this only affects the test driver, not the test client (and its language) or the tests themselves\&. +.PP +A few tests might not run with version 1 because they depend on some feature of version 2\&. You may have those tests skipped by adding the test name to the file +lib/v1/incompatible\&.tests\&. This feature is available from MySQL 5\&.1\&.40\&. +.PP \fBmysql\-test\-run\&.pl\fR supports the options in the following list\&. An argument of \fB\-\-\fR tells \fBmysql\-test\-run\&.pl\fR -not to process any following arguments as options\&. (A description of differences between the options supported by -\fBmysql\-test\-run\&.pl\fR -and -\fBmysql\-test\-run\fR -appears following the list\&.) +not to process any following arguments as options\&. .sp .RS 4 .ie n \{\ @@ -323,11 +374,16 @@ Display a help message and exit\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: bench option -.\" bench option: mysql-test-run.pl -\fB\-\-bench\fR +.\" mysql-test-run.pl: big-test option +.\" big-test option: mysql-test-run.pl +\fB\-\-big\-test\fR .sp -Run the benchmark suite\&. +Allow tests marked as "big" to run\&. Tests can be thus marked by including the line +\fB\-\-source include/big_test\&.inc\fR, and they will only be run if this option is given, or if the environment variable +BIG_TEST +is set to 1\&. +.sp +This is typically done for tests that take very long to run, or that use very much resources, so that they are not suitable for running as part of a normal test suite run\&. .RE .sp .RS 4 @@ -338,12 +394,25 @@ Run the benchmark suite\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: benchdir option -.\" benchdir option: mysql-test-run.pl -\fB\-\-benchdir=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: build-thread option +.\" build-thread option: mysql-test-run.pl +\fB\-\-build\-thread=\fR\fB\fInumber\fR\fR +.sp +Specify a number to calculate port numbers from\&. The formula is 10 * +\fIbuild_thread\fR ++ 10000\&. Instead of a number, it can be set to +auto, which is also the default value, in which case +\fBmysql\-test\-run\&.pl\fR +will allocate a number unique to this host\&. +.sp +The value (number or +auto) can also be set with the +MTR_BUILD_THREAD +environment variable\&. .sp -The directory where the benchmark suite is located\&. The default path is -\&.\&./\&.\&./mysql\-bench\&. +From MySQL 5\&.1\&.45, the more logical +\fB\-\-port\-base\fR +is supported as an alternative\&. .RE .sp .RS 4 @@ -354,14 +423,14 @@ The directory where the benchmark suite .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: big option -.\" big option: mysql-test-run.pl -\fB\-\-big\-test\fR +.\" mysql-test-run.pl: callgrind option +.\" callgrind option: mysql-test-run.pl +\fB\-\-callgrind\fR .sp -Pass the -\fB\-\-big\-test\fR -option to -\fBmysqltest\fR\&. +Instructs +\fBvalgrind\fR +to use +\fBcallgrind\fR\&. .RE .sp .RS 4 @@ -376,7 +445,7 @@ option to .\" check-testcases option: mysql-test-run.pl \fB\-\-check\-testcases\fR .sp -Check test cases for side effects\&. +Check test cases for side effects\&. This is done by checking system state before and after each test case; if there is any difference, a warning to that effect will be written, but the test case will not be marked as failed because of it\&. This check is enabled by default\&. .RE .sp .RS 4 @@ -389,9 +458,9 @@ Check test cases for side effects\&. .\} .\" mysql-test-run.pl: client-bindir option .\" client-bindir option: mysql-test-run.pl -\fB\-\-client\-bindir\fR +\fB\-\-client\-bindir=\fR\fB\fIpath\fR\fR .sp -The path to the directory where client binaries are located\&. This option was added in MySQL 5\&.0\&.66/5\&.1\&.27\&. +The path to the directory where client binaries are located\&. .RE .sp .RS 4 @@ -423,7 +492,7 @@ debugger\&. .\} .\" mysql-test-run.pl: client-debugger option .\" client-debugger option: mysql-test-run.pl -\fB\-\-client\-debugger\fR +\fB\-\-client\-debugger=\fR\fB\fIdebugger\fR\fR .sp Start \fBmysqltest\fR @@ -459,9 +528,9 @@ debugger\&. .\} .\" mysql-test-run.pl: client-libdir option .\" client-libdir option: mysql-test-run.pl -\fB\-\-client\-libdir\fR +\fB\-\-client\-libdir=\fR\fB\fIpath\fR\fR .sp -The path to the directory where client libraries are located\&. This option was added in MySQL 5\&.0\&.66/5\&.1\&.27\&. +The path to the directory where client libraries are located\&. .RE .sp .RS 4 @@ -493,8 +562,6 @@ is to create a combinations file in the suite directory\&. The file should contain a section of options for each test run\&. See Section\ \&4.9, \(lqPassing Options from mysql-test-run.pl to mysqld or mysqltest\(rq\&. -.sp -This option was added in MySQL 5\&.1\&.23/6\&.0\&.4\&. .RE .sp .RS 4 @@ -511,7 +578,8 @@ This option was added in MySQL 5\&.1\&.2 .sp Write \fIstr\fR -to the output\&. +to the output within lines filled with +#, as a form of banner\&. .RE .sp .RS 4 @@ -593,7 +661,7 @@ Dump trace output for all clients and se .\} .\" mysql-test-run.pl: debugger option .\" debugger option: mysql-test-run.pl -\fB\-\-debugger\fR +\fB\-\-debugger=\fR\fB\fIdebugger\fR\fR .sp Start \fBmysqld\fR @@ -627,7 +695,37 @@ does not fail if Debug Sync is not compi For information about using the Debug Sync facility for testing, see Section\ \&4.14, \(lqThread Synchronization in Test Cases\(rq\&. .sp -This option was added in MySQL 5\&.4\&.4/6\&.0\&.6\&. +This option was added in MySQL 5\&.1\&.41/5\&.5\&.0/6\&.0\&.6\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: defaults-file option +.\" default-file option: mysql-test-run.pl +\fB\-\-defaults\-file=\fR\fB\fIfile_name\fR\fR +.sp +Use the named file as fixed config file template for all tests\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: defaults_extra_file option +.\" default_extra_file option: mysql-test-run.pl +\fB\-\-defaults_extra_file=\fR\fB\fIfile_name\fR\fR +.sp +Add setting from the named file to all generated configs\&. .RE .sp .RS 4 @@ -646,9 +744,9 @@ Run all test cases having a name that be \fIprefix\fR value\&. This option provides a convenient way to run a family of similarly named tests\&. .sp -As of MySQL 5\&.0\&.54/5\&.1\&.23/6\&.0\&.5, the argument for the +The argument for the \fB\-\-do\-test\fR -option allows more flexible specification of which tests to perform\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, +option also allows more flexible specification of which tests to perform\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, \fB\-\-do\-test=testa\fR matches tests that begin with testa, @@ -691,6 +789,23 @@ built with the embedded server\&. .sp -1 .IP \(bu 2.3 .\} +.\" mysql-test-run.pl: enable-disabled option +.\" enable-disabled option: mysql-test-run.pl +\fB\-\-enable\-disabled\fR +.sp +Ignore any +disabled\&.def +file, and run also tests marked as disbaled\&. Success or failure of those tests will be reported the same way as other tests\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} .\" mysql-test-run.pl: experimental option .\" experimental option: mysql-test-run.pl \fB\-\-experimental=\fR\fB\fIfile_name\fR\fR @@ -699,7 +814,7 @@ Specify a file that contains a list of t [ exp\-fail ] code rather than [ fail ] -if they fail\&. This option was added in MySQL 5\&.1\&.33/6\&.0\&.11\&. +if they fail\&. This option was added in MySQL 5\&.1\&.33\&. .sp For an example of a file that might be specified via this option, see mysql\-test/collections/default\&.experimental\&. @@ -716,8 +831,25 @@ mysql\-test/collections/default\&.experi .\" mysql-test-run.pl: extern option .\" extern option: mysql-test-run.pl \fB\-\-extern\fR +\fIoption\fR=\fIvalue\fR .sp -Use an already running server\&. +Use an already running server\&. The option/value pair is what is needed by the +\fBmysql\fR +client to connect to the server\&. Each +\fB\-\-extern\fR +can only take one option/value pair as argument, so it you need more you need to repeat +\fB\-\-extern\fR +for each of them\&. Example: +.sp +.if n \{\ +.RS 4 +.\} +.nf + \&./mysql\-test\-run\&.pl \-\-extern socket=var/tmp/mysqld\&.1\&.sock alias +.fi +.if n \{\ +.RE +.\} .sp Note: If a test case has an \&.opt @@ -736,7 +868,8 @@ file that requires the server to be rest .\" fast option: mysql-test-run.pl \fB\-\-fast\fR .sp -Do not clean up from earlier test runs\&. +Do not perform controlled shutdown when servers need to be restarted or at the end of the test run\&. This is equivalent to using +\-\-shutdown\-timeout=0\&. .RE .sp .RS 4 @@ -809,6 +942,8 @@ debugger\&. Run tests with the \fBgprof\fR profiling tool\&. +\fB\-\-gprof\fR +was added in 5\&.1\&.45\&. .RE .sp .RS 4 @@ -819,12 +954,13 @@ profiling tool\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: im-mysqld1-port option -.\" im-mysqld1-port option: mysql-test-run.pl -\fB\-\-im\-mysqld1\-port\fR +.\" mysql-test-run.pl: manual-ddd option +.\" manual-ddd option: mysql-test-run.pl +\fB\-\-manual\-ddd\fR .sp -TCP/IP port number to use for the first -\fBmysqld\fR, controlled by Instance Manager\&. +Use a server that has already been started by the user in the +\fBddd\fR +debugger\&. .RE .sp .RS 4 @@ -835,12 +971,11 @@ TCP/IP port number to use for the first .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: im-mysqld2-port option -.\" im-mysqld2-port option: mysql-test-run.pl -\fB\-\-im\-mysqld2\-port\fR +.\" mysql-test-run.pl: manual-debug option +.\" manual-debug option: mysql-test-run.pl +\fB\-\-manual\-debug\fR .sp -TCP/IP port number to use for the second -\fBmysqld\fR, controlled by Instance Manager\&. +Use a server that has already been started by the user in a debugger\&. .RE .sp .RS 4 @@ -851,12 +986,13 @@ TCP/IP port number to use for the second .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: im-port option -.\" im-port option: mysql-test-run.pl -\fB\-\-im\-port\fR +.\" mysql-test-run.pl: manual-gdb option +.\" manual-gdb option: mysql-test-run.pl +\fB\-\-manual\-gdb\fR .sp -TCP/IP port number to use for -\fBmysqld\fR, controlled by Instance Manager\&. +Use a server that has already been started by the user in the +\fBgdb\fR +debugger\&. .RE .sp .RS 4 @@ -867,14 +1003,12 @@ TCP/IP port number to use for .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: log-warnings option -.\" log-warnings option: mysql-test-run.pl -\fB\-\-log\-warnings\fR +.\" mysql-test-run.pl: mark-progress option +.\" mark-progress option: mysql-test-run.pl +\fB\-\-mark\-progress\fR .sp -Pass the -\fB\-\-log\-warnings\fR -option to -\fBmysqld\fR\&. +Marks progress with timing (in milliseconds) and line number in +var/log/\fItestname\fR\&.progress\&. .RE .sp .RS 4 @@ -885,11 +1019,14 @@ option to .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: manual-debug option -.\" manual-debug option: mysql-test-run.pl -\fB\-\-manual\-debug\fR +.\" mysql-test-run.pl: max-connections option +.\" max-connections option: mysql-test-run.pl +\fB\-\-max\-connections=\fR\fB\fInum\fR\fR .sp -Use a server that has already been started by the user in a debugger\&. +The maximum number of simultaneous server connections that may be used per test\&. If not set, the maximum is 128\&. Minimum allowed limit is 8, maximum is 5120\&. Corresponds to the same option for +\fBmysqltest\fR\&. +.sp +This option is available from MySQL 5\&.1\&.45\&. .RE .sp .RS 4 @@ -900,13 +1037,12 @@ Use a server that has already been start .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: manual-gdb option -.\" manual-gdb option: mysql-test-run.pl -\fB\-\-manual\-gdb\fR +.\" mysql-test-run.pl: max-save-core option +.\" max-save-core option: mysql-test-run.pl +\fB\-\-max\-save\-core=\fR\fB\fIN\fR\fR .sp -Use a server that has already been started by the user in the -\fBgdb\fR -debugger\&. +Limit the number of core files saved, to avoid filling up disks in case of a frequently crashing server\&. Defaults to 5, set to 0 for no limit\&. May also be set with the environment variable +MTR_MAX_SAVE_CORE .RE .sp .RS 4 @@ -917,13 +1053,12 @@ debugger\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: master-binary option -.\" master-binary option: mysql-test-run.pl -\fB\-\-master\-binary=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: max-save-datadir option +.\" max-save-datadir option: mysql-test-run.pl +\fB\-\-max\-save\-datadir=\fR\fB\fIN\fR\fR .sp -Specify the path of the -\fBmysqld\fR -binary to use for master servers\&. +Limit the number of data directories saved after failed tests, to avoid filling up disks in case of frequent failures\&. Defaults to 20, set to 0 for no limit\&. May also be set with the environment variable +MTR_MAX_SAVE_DATADIR .RE .sp .RS 4 @@ -934,11 +1069,12 @@ binary to use for master servers\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: master_port option -.\" master_port option: mysql-test-run.pl -\fB\-\-master_port=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: max-test-fail option +.\" max-test-fail option: mysql-test-run.pl +\fB\-\-max\-test\-fail=\fR\fB\fIN\fR\fR .sp -Specify the TCP/IP port number for the first master server to use\&. Observe that the option name has an underscore and not a dash\&. +Stop execution after the specified number of tests have failed, to avoid using up resources (and time) in case of massive failures\&. retries are noe counted, nor are failures of tests marked experimental\&. Defaults to 10, set to 0 for no limit\&. May also be set with the environment variable +MTR_MAX_TEST_FAIL .RE .sp .RS 4 @@ -953,7 +1089,9 @@ Specify the TCP/IP port number for the f .\" mem option: mysql-test-run.pl \fB\-\-mem\fR .sp -Run the test suite in memory, using tmpfs or ramdisk\&. This can decrease test times significantly\&. +This option is not supported on Windows\&. +.sp +Run the test suite in memory, using tmpfs or ramdisk\&. This can decrease test times significantly, in particular if you would otherwise be running over a remote file system\&. \fBmysql\-test\-run\&.pl\fR attempts to find a suitable location using a built\-in list of standard locations for tmpfs and puts the var @@ -966,7 +1104,14 @@ MTR_MEM[=\fIdir_name\fR]\&. If \fIdir_name\fR is given, it is added to the beginning of the list of locations to search, so it takes precedence over any built\-in locations\&. .sp -This option was added in MySQL 4\&.1\&.22, 5\&.0\&.30, and 5\&.1\&.13\&. +Once you have run tests with +\fB\-\-mem\fR +within a +mysql\-testdirectory, a soflink +var +will have been set up to the temporary directory, and this will be re\-used the next time, until the soflink is deleted\&. Thus, you do not have to repeat the +\fB\-\-mem\fR +option next time\&. .RE .sp .RS 4 @@ -996,25 +1141,6 @@ Section\ \&4.9, \(lqPassing Options from .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: mysqltest option -.\" mysqltest option: mysql-test-run.pl -\fB\-\-mysqltest=\fR\fB\fIvalue\fR\fR -.sp -Extra options to pass to -\fBmysqltest\fR\&. The value should consist of one or more comma\-separated -\fBmysqltest\fR -options\&. See -Section\ \&4.9, \(lqPassing Options from mysql-test-run.pl to mysqld or mysqltest\(rq\&. This option was added in MySQL 6\&.0\&.6\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: ndb-connectstring option .\" ndb-connectstring option: mysql-test-run.pl \fB\-\-ndb\-connectstring=\fR\fB\fIstr\fR\fR @@ -1034,15 +1160,13 @@ from starting a cluster\&. It is assumed .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndb-connectstring-slave option -.\" ndb-connectstring-slave option: mysql-test-run.pl -\fB\-\-ndb\-connectstring\-slave=\fR\fB\fIstr\fR\fR +.\" mysql-test-run.pl: nocheck-testcases option +.\" nocheck-testcases option: mysql-test-run.pl +\fB\-\-nocheck\-testcases\fR .sp -Pass -\fB\-\-ndb\-connectstring=\fR\fB\fIstr\fR\fR -to slave MySQL servers\&. This option also prevents -\fBmysql\-test\-run\&.pl\fR -from starting a cluster\&. It is assumed that there is already a cluster running to which the server can connect with the given connectstring\&. +Disable the check for test case side effects; see +\fB\-\-check\-testcases\fR +for a description\&. .RE .sp .RS 4 @@ -1053,11 +1177,11 @@ from starting a cluster\&. It is assumed .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndb-extra-test option -.\" ndb-extra-test option: mysql-test-run.pl -\fB\-\-ndb\-extra\-test\fR +.\" mysql-test-run.pl: noreorder option +.\" noreorder option: mysql-test-run.pl +\fB\-\-noreorder\fR .sp -Unknown\&. +Do not reorder tests to reduce number of restarts, but run them in exactly the order given\&. If a whole suite is to be run, the tests are run in alphabetical order, though similiar combinations will be grouped together\&. If more than one suite is listed, the tests are run one suite at a time, in the order listed\&. .RE .sp .RS 4 @@ -1068,14 +1192,13 @@ Unknown\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndbcluster-port option -.\" ndbcluster-port option: mysql-test-run.pl -.\" mysql-test-run.pl: ndbcluster_port option -.\" ndbcluster_port option: mysql-test-run.pl -\fB\-\-ndbcluster\-port=\fR\fB\fIport_num\fR\fR, -\fB\-\-ndbcluster_port=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: notimer option +.\" notimer option: mysql-test-run.pl +\fB\-\-notimer\fR .sp -Specify the TCP/IP port number that NDB Cluster should use\&. +Cause +\fBmysqltest\fR +not to generate a timing file\&. The effect of this is that the report from each test case does not include the timing in milliseconds as it normally does\&. .RE .sp .RS 4 @@ -1086,11 +1209,11 @@ Specify the TCP/IP port number that NDB .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndbcluster-port-slave option -.\" ndbcluster-port-slave option: mysql-test-run.pl -\fB\-\-ndbcluster\-port\-slave=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: nowarnings option +.\" nowarnings option: mysql-test-run.pl +\fB\-\-nowarnings\fR .sp -Specify the TCP/IP port number that the slave NDB Cluster should use\&. +Do not look for and report errors and warning in the server logs\&. .RE .sp .RS 4 @@ -1101,13 +1224,16 @@ Specify the TCP/IP port number that the .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: netware option -.\" netware option: mysql-test-run.pl -\fB\-\-netware\fR +.\" mysql-test-run.pl: parallel option +.\" parallel option: mysql-test-run.pl +\fB\-\-parallel={\fR\fB\fIN\fR\fR\fB|auto}\fR .sp -Run -\fBmysqld\fR -with options needed on NetWare\&. +Run tests using +\fIN\fR +parallel threads\&. By default, 1 thread is used\&. Use +\fB\-\-parallel=auto\fR +for auto\-setting of +\fIN\fR\&. The auto value was added in MySQL 5\&.1\&.36\&. .RE .sp .RS 4 @@ -1118,13 +1244,24 @@ with options needed on NetWare\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: notimer option -.\" notimer option: mysql-test-run.pl -\fB\-\-notimer\fR +.\" mysql-test-run.pl: port-base option +.\" port-base option: mysql-test-run.pl +\fB\-\-port\-base=\fR\fB\fIP\fR\fR .sp -Cause -\fBmysqltest\fR -not to generate a timing file\&. +Specify base of port numbers to be used; a block of 10 will be allocated\&. +\fIP\fR +should be divisible by 10; if it is not, it will be rounded down\&. If running with more than one parallel test thread, thread 2 will use the next block of 10 and so on\&. +.sp +If the port number is given as +auto, which is also the default, +\fBmysql\-test\-run\&.pl\fRwill allocate a number unique to this host\&. The value may also be given with the environment variable +MTR_PORT_BASE\&. +.sp +\fB\-\-port\-base\fR +was added in MySQL 5\&.1\&.45 as a more logical alternative to +\fB\-\-build\-thread\fR\&. If both are used, +\fB\-\-port\-base\fR +takes presedence\&. .RE .sp .RS 4 @@ -1135,16 +1272,11 @@ not to generate a timing file\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: parallel option -.\" parallel option: mysql-test-run.pl -\fB\-\-parallel={\fR\fB\fIN\fR\fR\fB|auto}\fR +.\" mysql-test-run.pl: print-testcases option +.\" print-testcases option: mysql-test-run.pl +\fB\-\-print\-testcases\fR .sp -Run tests using -\fIN\fR -parallel threads\&. By default, 1 thread is used\&. Use -\fB\-\-parallel=auto\fR -for auto\-setting of -\fIN\fR\&. This option was added in MySQL 5\&.1\&.36\&. +Do not run any tests, but print details about all tests, in the order they would have been run\&. .RE .sp .RS 4 @@ -1195,7 +1327,24 @@ option to .\" reorder option: mysql-test-run.pl \fB\-\-reorder\fR .sp -Reorder tests to minimize the number of server restarts needed\&. +Reorder tests to minimize the number of server restarts needed\&. This is the default behavior\&. There is no guarantee that a particular set of tests will always end up in the same order\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: repeat option +.\" repeat option: mysql-test-run.pl +\fB\-\-repeat=\fR\fB\fIN\fR\fR +.sp +Run each test +\fIN\fR +number of times\&. .RE .sp .RS 4 @@ -1214,8 +1363,33 @@ Display the output of SHOW ENGINES and SHOW VARIABLES\&. This can be used to verify that binaries are built with all required features\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: retry option +.\" retry option: mysql-test-run.pl +\fB\-\-retry=\fR\fB\fIN\fR\fR .sp -This option was added in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.14\&. +If a test fails, it is retried up to a maximum of +\fIN\fR +runs, but will terminate after 2 failures\&. Default is 3, set to 1 or 0 for no retries\&. This option has no effect unless +\fB\-\-force\fR +is also used; without it, test execution will terminate after the first failure\&. +.sp +The +\fB\-\-retry\fR +and +\fB\-\-retry\-failure\fR +options do not affect how many times a test repeated with +\fB\-\-repeat\fR +may fail in total, as each repetition is considered a new test case, which may in turn be retried if it fails\&. .RE .sp .RS 4 @@ -1226,13 +1400,11 @@ This option was added in MySQL 4\&.1\&.2 .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: script-debug option -.\" script-debug option: mysql-test-run.pl -\fB\-\-script\-debug\fR +.\" mysql-test-run.pl: retry-failure option +.\" retry-failure option: mysql-test-run.pl +\fB\-\-retry\-failure=\fR\fB\fIN\fR\fR .sp -Enable debug output for -\fBmysql\-test\-run\&.pl\fR -itself\&. +Allow a failed and retried test to fail more than the default 2 times before giving it up\&. Setting it to 0 or 1 effectively turns off retries .RE .sp .RS 4 @@ -1243,11 +1415,11 @@ itself\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: skip-im option -.\" skip-im option: mysql-test-run.pl -\fB\-\-skip\-im\fR +.\" mysql-test-run.pl: shutdown-timeout option +.\" shutdown-timeout option: mysql-test-run.pl +\fB\-\-shutdown\-timeout=\fR\fB\fISECONDS\fR\fR .sp -Do not start Instance Manager; skip Instance Manager test cases\&. +Max number of seconds to wait for servers to do controlled shutdown before killing them\&. Default is 10\&. .RE .sp .RS 4 @@ -1258,11 +1430,11 @@ Do not start Instance Manager; skip Inst .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: skip-master-binlog option -.\" skip-master-binlog option: mysql-test-run.pl -\fB\-\-skip\-master\-binlog\fR +.\" mysql-test-run.pl: skip-combinations option +.\" skip-combinations option: mysql-test-run.pl +\fB\-\-skip\-combinations\fR .sp -Do not enable master server binary logging\&. +Do not apply combinations; ignore combinations file or option\&. .RE .sp .RS 4 @@ -1324,21 +1496,6 @@ Skip replication test cases\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: skip-slave-binlog option -.\" skip-slave-binlog option: mysql-test-run.pl -\fB\-\-skip\-slave\-binlog\fR -.sp -Do not enable master server binary logging\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: skip-ssl option .\" skip-ssl option: mysql-test-run.pl \fB\-\-skip\-ssl\fR @@ -1362,7 +1519,7 @@ with support for SSL connections\&. .sp Specify a regular expression to be applied to test case names\&. Cases with names that match the expression are skipped\&. tests to skip\&. .sp -As of MySQL 5\&.0\&.54/5\&.1\&.23/6\&.0\&.5, the argument for the +The argument for the \fB\-\-skip\-test\fR option allows more flexible specification of which tests to skip\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. See the description of the \fB\-\-do\-test\fR @@ -1393,13 +1550,14 @@ are passed to the master server\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: slave-binary option -.\" slave-binary option: mysql-test-run.pl -\fB\-\-slave\-binary=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: sleep option +.\" sleep option: mysql-test-run.pl +\fB\-\-sleep=\fR\fB\fIN\fR\fR .sp -Specify the path of the -\fBmysqld\fR -binary to use for slave servers\&. +Pass +\fB\-\-sleep=\fR\fB\fIN\fR\fR +to +\fBmysqltest\fR\&. .RE .sp .RS 4 @@ -1410,82 +1568,14 @@ binary to use for slave servers\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: slave_port option -.\" slave_port option: mysql-test-run.pl -\fB\-\-slave_port=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: sp-protocol option +.\" sp-protocol option: mysql-test-run.pl +\fB\-\-sp\-protocol\fR .sp -Specify the TCP/IP port number for the first master server to use\&. Observe that the option name has an underscore and not a dash\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: sleep option -.\" sleep option: mysql-test-run.pl -\fB\-\-sleep=\fR\fB\fIN\fR\fR -.sp -Pass -\fB\-\-sleep=\fR\fB\fIN\fR\fR -to -\fBmysqltest\fR\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: small-bench option -.\" small-bench option: mysql-test-run.pl -\fB\-\-small\-bench\fR -.sp -Run the benchmarks with the -\fB\-\-small\-tests\fR -and -\fB\-\-small\-tables\fR -options\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: socket option -.\" socket option: mysql-test-run.pl -\fB\-\-socket=\fR\fB\fIfile_name\fR\fR -.sp -For connections to -localhost, the Unix socket file to use, or, on Windows, the name of the named pipe to use\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: sp-protocol option -.\" sp-protocol option: mysql-test-run.pl -\fB\-\-sp\-protocol\fR -.sp -Pass the -\fB\-\-sp\-protocol\fR -option to -\fBmysqltest\fR\&. +Pass the +\fB\-\-sp\-protocol\fR +option to +\fBmysqltest\fR\&. .RE .sp .RS 4 @@ -1520,39 +1610,11 @@ Couldn\'t find support for SSL .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: start option -.\" start option: mysql-test-run.pl -\fB\-\-start\fR -.sp -Initialize and start servers with the startup settings for the first specified test case\&. For example: -.sp -.if n \{\ -.RS 4 -.\} -.nf -shell> \fBcd mysql\-test\fR -shell> \fB\&./mysql\-test\-run\&.pl \-\-start alias &\fR -.fi -.if n \{\ -.RE -.\} -.sp -This option was added in MySQL 5\&.1\&.32/6\&.0\&.11\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: start-and-exit option .\" start-and-exit option: mysql-test-run.pl -\fB\-\-start\-and\-exit\fR +\fB\-\-start\fR .sp -Initialize and start servers with the startup settings for the specified test case or cases, if any, and then exit\&. You can use this option to start a server to which you can connect later\&. For example, after building a source distribution you can start a server and connect to it with the +Initialize and start servers with the startup settings for the specified test case\&. You can use this option to start a server to which you can connect later\&. For example, after building a source distribution you can start a server and connect to it with the \fBmysql\fR client like this: .sp @@ -1561,12 +1623,19 @@ client like this: .\} .nf shell> \fBcd mysql\-test\fR -shell> \fB\&./mysql\-test\-run\&.pl \-\-start\-and\-exit\fR +shell> \fB\&./mysql\-test\-run\&.pl \-\-start alias &\fR shell> \fB\&.\&./mysql \-S \&./var/tmp/master\&.sock \-h localhost \-u root\fR .fi .if n \{\ .RE .\} +.sp +If no tests are named on the command line, the server(s) will be started with settings for the first test that would have been run without the +\fB\-\-start\fR +option\&. +.sp +\fBmysql\-test\-run\&.pl\fR +will stop once the server has been started, but will terminate if the server dies\&. If killed, it will also shut down the server\&. .RE .sp .RS 4 @@ -1581,7 +1650,8 @@ shell> \fB\&.\&./mysql \-S \&./var/tmp/m .\" start-dirty option: mysql-test-run.pl \fB\-\-start\-dirty\fR .sp -Start servers (without initialization) for the specified test case or cases, if any, and then exit\&. You can then manually run the test cases\&. +This is similar to +\fB\-\-start\fR, but will skip the database initialization phase and assume that database files are already available\&. Usually this means you must have run another test first\&. .RE .sp .RS 4 @@ -1627,161 +1697,6 @@ output for .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: stress option -.\" stress option: mysql-test-run.pl -\fB\-\-stress\fR -.sp -Run the stress test\&. The other -\fB\-\-stress\-\fR\fB\fIxxx\fR\fR -options apply in this case\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-init-file option -.\" stress-init-file option: mysql-test-run.pl -\fB\-\-stress\-init\-file=\fR\fB\fIfile_name\fR\fR -.sp -\fIfile_name\fR -is the location of the file that contains the list of tests\&. The default file is -stress_init\&.txt -in the test suite directory\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-loop-count option -.\" stress-loop-count option: mysql-test-run.pl -\fB\-\-stress\-loop\-count=\fR\fB\fIN\fR\fR -.sp -In sequential stress\-test mode, the number of loops to execute before exiting\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-mode option -.\" stress-mode option: mysql-test-run.pl -\fB\-\-stress\-mode=\fR\fB\fImode\fR\fR -.sp -This option indicates the test order in stress\-test mode\&. The -\fImode\fR -value is either -random -to select tests in random order or -seq -to run tests in each thread in the order specified in the test list file\&. The default mode is -random\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-suite option -.\" stress-suite option: mysql-test-run.pl -\fB\-\-stress\-suite=\fR\fB\fIsuite_name\fR\fR -.sp -The name of the test suite to use for stress testing\&. The default suite name is -main -(the regular test suite located in the -mysql\-test -directory)\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-test-count option -.\" stress-test-count option: mysql-test-run.pl -\fB\-\-stress\-test\-count=\fR\fB\fIN\fR\fR -.sp -For stress testing, the number of tests to execute before exiting\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-test-duration option -.\" stress-test-duration option: mysql-test-run.pl -\fB\-\-stress\-test\-duration=\fR\fB\fIN\fR\fR -.sp -For stress testing, the duration of stress testing in seconds\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-test-file option -.\" stress-test-file option: mysql-test-run.pl -\fB\-\-stress\-test\-file=\fR\fB\fIfile_name\fR\fR -.sp -The file that contains the list of tests to use in stress testing\&. The tests should be named without the -\&.test -extension\&. The default file is -stress_tests\&.txt -in the test suite directory\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-threads option -.\" stress-threads option: mysql-test-run.pl -\fB\-\-stress\-threads=\fR\fB\fIN\fR\fR -.sp -The number of threads to use in stress testing\&. The default is 5\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: suite option .\" suite option: mysql-test-run.pl \fB\-\-suite=\fR\fB\fIsuite_name\fR\fR @@ -1831,30 +1746,12 @@ Specify the maximum test case runtime\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: timer option -.\" timer option: mysql-test-run.pl -\fB\-\-timer\fR -.sp -Cause -\fBmysqltest\fR -to generate a timing file\&. The default file is named -\&./var/log/timer\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: tmpdir option -.\" tmpdir option: mysql-test-run.pl -\fB\-\-tmpdir=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: timediff option +.\" timediff option: mysql-test-run.pl +\fB\-\-timediff\fR .sp -The directory where temporary file are stored\&. The default location is -\&./var/tmp\&. +Adds to each test report for a test case, the total time in sconds and milliseconds passed since the preceding test ended\&. This option can only be used together with +\fB\-\-timestamp\fR, and has no effect without it\&. .RE .sp .RS 4 @@ -1865,12 +1762,14 @@ The directory where temporary file are s .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: unified-diff option -.\" unified-diff option: mysql-test-run.pl -\fB\-\-unified\-diff\fR, -\fB\-\-udiff\fR +.\" mysql-test-run.pl: timer option +.\" timer option: mysql-test-run.pl +\fB\-\-timer\fR .sp -Use unified diff format when presenting differences between expected and actual test case results\&. +Cause +\fBmysqltest\fR +to generate a timing file\&. The default file is named +\&./var/log/timer\&. .RE .sp .RS 4 @@ -1881,11 +1780,11 @@ Use unified diff format when presenting .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: use-old-data option -.\" use-old-data option: mysql-test-run.pl -\fB\-\-use\-old\-data\fR +.\" mysql-test-run.pl: timestamp option +.\" timestamp option: mysql-test-run.pl +\fB\-\-timestamp\fR .sp -Do not install the test databases\&. (Use existing ones\&.) +Prints a timestamp before the test case name in each test report line, showing when the test ended\&. .RE .sp .RS 4 @@ -1896,11 +1795,14 @@ Do not install the test databases\&. (Us .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: user-test option -.\" user-test option: mysql-test-run.pl -\fB\-\-user\-test=\fR\fB\fIval\fR\fR +.\" mysql-test-run.pl: tmpdir option +.\" tmpdir option: mysql-test-run.pl +\fB\-\-tmpdir=\fR\fB\fIpath\fR\fR .sp -Unused\&. +The directory where temporary file are stored\&. The default location is +\&./var/tmp\&. The environment variable +MYSQL_TMP_DIR +will be set to the path for this directory, whether it has the default value or has been set explicitly\&. This may be referred to in tests\&. .RE .sp .RS 4 @@ -1935,7 +1837,11 @@ Run and \fBmysqld\fR with -\fBvalgrind\fR\&. +\fBvalgrind\fR\&. Thiks and the following +\fB\-\-valgrind\fR +options require that the executables have been build with +\fBvalgrind\fR +support\&. .RE .sp .RS 4 @@ -1946,16 +1852,13 @@ with .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: valgrind-all option -.\" valgrind-all option: mysql-test-run.pl -\fB\-\-valgrind\-all\fR +.\" mysql-test-run.pl: valgrind-mysqld option +.\" valgrind-mysqld option: mysql-test-run.pl +\fB\-\-valgrind\-mysqld\fR .sp -Like -\fB\-\-valgrind\fR, but passes the -\fB\-\-verbose\fR -and -\fB\-\-show\-reachable\fR -options to +Run the +\fBmysqld\fR +server with \fBvalgrind\fR\&. .RE .sp @@ -1985,30 +1888,9 @@ with .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: valgrind-mysqltest-all option -.\" valgrind-mysqltest-all option: mysql-test-run.pl -\fB\-\-valgrind\-mysqltest\-all\fR -.sp -Like -\fB\-\-valgrind\-mysqltest\fR, but passes the -\fB\-\-verbose\fR -and -\fB\-\-show\-reachable\fR -options to -\fBvalgrind\fR\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: valgrind-options option .\" valgrind-options option: mysql-test-run.pl -\fB\-\-valgrind\-options=\fR\fB\fIstr\fR\fR +\fB\-\-valgrind\-option=\fR\fB\fIstr\fR\fR .sp Extra options to pass to \fBvalgrind\fR\&. @@ -2044,7 +1926,9 @@ executable\&. \fB\-\-vardir=\fR\fB\fIpath\fR\fR .sp Specify the path where files generated during the test run are stored\&. The default location is -\&./var\&. +\&./var\&. The environment variable +MYSQLTEST_VARDIR +will be set to the path for this directory, whether it has the default value or has been set explicitly\&. This may be referred to in tests\&. .RE .sp .RS 4 @@ -2055,14 +1939,11 @@ Specify the path where files generated d .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: view-protocol option -.\" view-protocol option: mysql-test-run.pl -\fB\-\-view\-protocol\fR +.\" mysql-test-run.pl: verbose option +.\" verbose option: mysql-test-run.pl +\fB\-\-verbose\fR .sp -Pass the -\fB\-\-view\-protocol\fR -option to -\fBmysqltest\fR\&. +Give more verbose output regarding test execution\&. Use the option twice to get even more output\&. Note that the output generated within each test case is not affected\&. .RE .sp .RS 4 @@ -2073,13 +1954,11 @@ option to .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: vs-config option -.\" vs-config option: mysql-test-run.pl -\fB\-\-vs\-config=\fR\fB\fIconfig_val\fR\fR +.\" mysql-test-run.pl: verbose-restart option +.\" verbose-restart option: mysql-test-run.pl +\fB\-\-verbose\-restart\fR .sp -Specify the configuration used to build MySQL (for example, -\fB\-\-vs\-config=debug\fR -\fB\-\-vs\-config=release\fR)\&. This option is for Windows only\&. It is available as of MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.14\&. +Write when and why servers are restarted between test cases\&. .RE .sp .RS 4 @@ -2090,11 +1969,14 @@ Specify the configuration used to build .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: wait-timeout option -.\" wait-timeout option: mysql-test-run.pl -\fB\-\-wait\-timeout=\fR\fB\fIN\fR\fR +.\" mysql-test-run.pl: view-protocol option +.\" view-protocol option: mysql-test-run.pl +\fB\-\-view\-protocol\fR .sp -Unused? +Pass the +\fB\-\-view\-protocol\fR +option to +\fBmysqltest\fR\&. .RE .sp .RS 4 @@ -2105,12 +1987,13 @@ Unused? .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: warnings option -.\" warnings option: mysql-test-run.pl -\fB\-\-warnings\fR +.\" mysql-test-run.pl: vs-config option +.\" vs-config option: mysql-test-run.pl +\fB\-\-vs\-config=\fR\fB\fIconfig_val\fR\fR .sp -This option is a synonym for -\fB\-\-log\-warnings\fR\&. +Specify the configuration used to build MySQL (for example, +\fB\-\-vs\-config=debug\fR +\fB\-\-vs\-config=release\fR)\&. This option is for Windows only\&. .RE .sp .RS 4 @@ -2121,11 +2004,17 @@ This option is a synonym for .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: with-ndbcluster option -.\" with-ndbcluster option: mysql-test-run.pl -\fB\-\-with\-ndbcluster\fR +.\" mysql-test-run.pl: wait-all option +.\" wait-all option: mysql-test-run.pl +\fB\-\-wait\-all\fR .sp -Use NDB Cluster and enable test cases that require it\&. +If +\fB\-\-start\fR +or +\fB\-\-start\-dirty\fR +is used, wait for all servers to exit before termination\&. Otherise, it will terminate if one (of several) servers is restarted\&. +.sp +This option was added in MySQL 5\&.1\&.36\&. .RE .sp .RS 4 @@ -2136,11 +2025,12 @@ Use NDB Cluster and enable test cases th .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: with-ndbcluster-all option -.\" with-ndbcluster-all option: mysql-test-run.pl -\fB\-\-with\-ndbcluster\-all\fR +.\" mysql-test-run.pl: warnings option +.\" warnings option: mysql-test-run.pl +\fB\-\-warnings\fR .sp -Use NDB Cluster in all tests\&. +Search the server log for errors or warning after each test and report any suspicious ones; if any are found, the test will be marked as failed\&. This is the default behavior, it may be turned off with +\fB\-\-nowarnings\fR\&. .RE .sp .RS 4 @@ -2159,100 +2049,10 @@ Run only test cases that have ndb in their name\&. .RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: with-ndbcluster-slave option -.\" with-ndbcluster-slave option: mysql-test-run.pl -\fB\-\-with\-ndbcluster\-slave\fR -.sp -Unknown\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: with-openssl option -.\" with-openssl option: mysql-test-run.pl -\fB\-\-with\-openssl\fR -.sp -This option is a synonym for -\fB\-\-ssl\fR\&. -.RE -.if n \{\ -.sp -.\} -.RS 4 -.it 1 an-trap -.nr an-no-space-flag 1 -.nr an-break-flag 1 -.br -.ps +1 -\fBNote\fR -.ps -1 -.br -.PP -\fBmysql\-test\-run\fR -supports the following options not supported by -\fBmysql\-test\-run\&.pl\fR: -\fB\-\-local\fR, -\fB\-\-local\-master\fR, -\fB\-\-ndb\-verbose\fR, -\fB\-\-ndb_mgm\-extra\-opts\fR, -\fB\-\-ndb_mgmd\-extra\-opts\fR, -\fB\-\-ndbd\-extra\-opts\fR, -\fB\-\-old\-master\fR, -\fB\-\-purify\fR, -\fB\-\-use\-old\-data\fR, -\fB\-\-valgrind\-mysqltest\-all\fR\&. -.PP -Conversely, -\fBmysql\-test\-run\&.pl\fR -supports the following options not supported by -\fBmysql\-test\-run\fR: -\fB\-\-benchdir\fR, -\fB\-\-check\-testcases\fR, -\fB\-\-client\-ddd\fR, -\fB\-\-client\-debugger\fR, -\fB\-\-cursor\-protocol\fR, -\fB\-\-debugger\fR, -\fB\-\-im\-mysqld1\-port\fR, -\fB\-\-im\-mysqld2\-port\fR, -\fB\-\-im\-port\fR, -\fB\-\-manual\-debug\fR, -\fB\-\-netware\fR, -\fB\-\-notimer\fR, -\fB\-\-reorder\fR, -\fB\-\-script\-debug\fR, -\fB\-\-skip\-im\fR, -\fB\-\-skip\-ssl\fR, -\fB\-\-sp\-protocol\fR, -\fB\-\-start\-dirty\fR, -\fB\-\-suite\fR, -\fB\-\-suite\-timeout\fR, -\fB\-\-testcase\-timeout\fR, -\fB\-\-udiff\fR, -\fB\-\-unified\-diff\fR,, -\fB\-\-valgrind\-path\fR, -\fB\-\-vardir\fR, -\fB\-\-view\-protocol\fR\&. -.sp .5v -.RE .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -2260,12 +2060,6 @@ This documentation is distributed in the .PP You should have received a copy of the GNU General Public License along with the program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA or see http://www.gnu.org/licenses/. .sp -.SH "NOTES" -.IP " 1." 4 -Typical \fBconfigure\fR Options -.RS 4 -\%http://dev.mysql.com/doc/refman/5.1/en/configure-options.html -.RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, which may already be installed locally and which is also available === modified file 'man/mysql.1' --- a/man/mysql.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -103,7 +103,13 @@ shell> \fBmysql \fR\fB\fIdb_name\fR\fR\f .\" startup parameters: mysql .PP \fBmysql\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysql] +and +[client] +option file groups\&. +\fBmysql\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -182,7 +188,7 @@ option\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -345,7 +351,7 @@ latin1 character set by default\&. You can usually fix such issues by using this option to force the client to use the system character set instead\&. .sp See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq, for more information\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq, for more information\&. .RE .sp .RS 4 @@ -550,10 +556,8 @@ the section called \(lqMYSQL COMMANDS\(r \fB\-\-no\-auto\-rehash\fR, \fB\-A\fR .sp -Deprecated form of -\fB\-skip\-auto\-rehash\fR\&. Use -\fB\-\-disable\-auto\-rehash\fR -instead\&. See the description for +This has the same effect as +\fB\-skip\-auto\-rehash\fR\&. See the description for \fB\-\-auto\-rehash\fR\&. .RE .sp @@ -589,6 +593,8 @@ Do not beep when errors occur\&. Deprecated, use \fB\-\-disable\-named\-commands\fR instead\&. +\fB\-\-no\-named\-commands\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -607,6 +613,8 @@ Deprecated form of \fB\-\-skip\-pager\fR\&. See the \fB\-\-pager\fR option\&. +\fB\-\-no\-pager\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -621,8 +629,12 @@ option\&. .\" no-tee option: mysql \fB\-\-no\-tee\fR .sp -Do not copy output to a file\&. -the section called \(lqMYSQL COMMANDS\(rq, discusses tee files further\&. +Deprecated form of +\fB\-\-skip\-tee\fR\&. See the +\fB\-\-tee\fR +option\&. +\fB\-\-no\-tee\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -684,10 +696,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysql\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -703,7 +717,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -950,9 +964,7 @@ option\&. \fB\-\-skip\-column\-names\fR, \fB\-N\fR .sp -Do not write column names in results\&. The short format, -\fB\-N\fR -is deprecated, use the long format instead\&. +Do not write column names in results\&. .RE .sp .RS 4 @@ -968,9 +980,7 @@ is deprecated, use the long format inste \fB\-\-skip\-line\-numbers\fR, \fB\-L\fR .sp -Do not write line numbers for errors\&. Useful when you want to compare result files that include error messages\&. The short format, -\fB\-L\fR -is deprecated, use the long format instead\&. +Do not write line numbers for errors\&. Useful when you want to compare result files that include error messages\&. .RE .sp .RS 4 @@ -1005,7 +1015,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -1229,7 +1239,7 @@ shell> \fBmysql \-\-xml \-uroot \-e "SHO You can also set the following variables by using \fB\-\-\fR\fB\fIvar_name\fR\fR\fB=\fR\fB\fIvalue\fR\fR\&. The \fB\-\-set\-variable\fR -format is deprecated\&. +format is deprecated and is removed in MySQL 5\&.5\&. .sp .RS 4 .ie n \{\ @@ -1325,7 +1335,7 @@ environment variable\&. The \&.mysql_history should be protected with a restrictive access mode because sensitive information might be written to it, such as the text of SQL statements that contain passwords\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. .PP If you do not want to maintain a history file, first remove \&.mysql_history @@ -2805,7 +2815,7 @@ Section\ \&21.9.11, \(lqControlling Auto .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -2817,7 +2827,7 @@ You should have received a copy of the G .IP " 1." 4 Bug#25946 .RS 4 -\%http://bugs.mysql.com/25946 +\%http://bugs.mysql.com/bug.php?id=25946 .RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, === modified file 'man/mysql.server.1' --- a/man/mysql.server.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql.server.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql.server\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL\&.SERVER\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL\&.SERVER\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -62,7 +62,7 @@ sections, although you should rename suc when using MySQL 5\&.1\&. .PP \fBmysql\&.server\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -176,7 +176,7 @@ The login user name to use for running .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_client_test.1' --- a/man/mysql_client_test.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_client_test.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_client_test\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQL_CLIENT_TEST" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQL_CLIENT_TEST" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -40,10 +40,15 @@ and its test language\&. is similar but used for testing the embedded server\&. Both programs are run as part of the test suite\&. .PP The source code for the programs can be found in in -test/mysql_client_test\&.c +tests/mysql_client_test\&.c in a source distribution\&. The program serves as a good source of examples illustrating how to use various features of the client API\&. .PP \fBmysql_client_test\fR +is used in a test by the same name in the main tests suite of +\fBmysql\-test\-run\&.pl\fR +but may also be run directly\&. Unlike the other programs listed here, it does not read an external description of what tests to run\&. Instead, all tests are coded into the program, which is written to cover all aspects of the C language API\&. +.PP +\fBmysql_client_test\fR supports the following options: .sp .RS 4 @@ -70,10 +75,10 @@ Display a help message and exit\&. .sp -1 .IP \(bu 2.3 .\} -\fB\-b \fR\fB\fIdir_name\fR\fR, +\fB\-\-basedir=\fR\fB\fIdir_name\fR\fR, .\" mysql_client_test: basedir option .\" basedir option: mysql_client_test -\fB\-\-basedir=\fR\fB\fIdir_name\fR\fR +\fB\-b \fR\fB\fIdir_name\fR\fR .sp The base directory for the tests\&. .RE @@ -86,10 +91,10 @@ The base directory for the tests\&. .sp -1 .IP \(bu 2.3 .\} -\fB\-t \fR\fB\fIcount\fR\fR, +\fB\-\-count=\fR\fB\fIcount\fR\fR, .\" mysql_client_test: count option .\" count option: mysql_client_test -\fB\-\-count=\fR\fB\fIcount\fR\fR +\fB\-t \fR\fB\fIcount\fR\fR .sp The number of times to execute the tests\&. .RE @@ -137,10 +142,10 @@ value is .sp -1 .IP \(bu 2.3 .\} -\fB\-g \fR\fB\fIoption\fR\fR, +\fB\-\-getopt\-ll\-test=\fR\fB\fIoption\fR\fR, .\" mysql_client_test: getopt-ll-test option .\" getopt-ll-test option: mysql_client_test -\fB\-\-getopt\-ll\-test=\fR\fB\fIoption\fR\fR +\fB\-g \fR\fB\fIoption\fR\fR .sp Option to use for testing bugs in the getopt @@ -213,10 +218,10 @@ The TCP/IP port number to use for the co .sp -1 .IP \(bu 2.3 .\} -\fB\-A \fR\fB\fIarg\fR\fR, +\fB\-\-server\-arg=\fR\fB\fIarg\fR\fR, .\" mysql_client_test: server-arg option .\" server-arg option: mysql_client_test -\fB\-\-server\-arg=\fR\fB\fIarg\fR\fR +\fB\-A \fR\fB\fIarg\fR\fR .sp Argument to send to the embedded server\&. .RE @@ -229,8 +234,8 @@ Argument to send to the embedded server\ .sp -1 .IP \(bu 2.3 .\} -\fB\-T\fR, -\fB\-\-show\-tests\fR +\fB\-\-show\-tests\fR, +\fB\-T\fR .sp Show all test names\&. .RE @@ -277,12 +282,13 @@ localhost .sp -1 .IP \(bu 2.3 .\} -\fB\-c\fR, -\fB\-\-testcase\fR +\fB\-\-testcase\fR, +\fB\-c\fR .sp -The option may disable some code when run as a -\fBmysql\-test\-run\&.pl\fR -test case\&. +The option is used when called from +\fBmysql\-test\-run\&.pl\fR, so that +\fBmysql_client_test\fR +may optionally behave in a different way than if called manually, for example by skipping some tests\&. Currently, there is no difference in behavior but the option is included in order to make this possible\&. .RE .sp .RS 4 @@ -320,7 +326,7 @@ mysql\-test/var\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_config.1' --- a/man/mysql_config.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_config.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_config\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_CONFIG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_CONFIG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -30,7 +30,7 @@ mysql_config \- get compile options for provides you with useful information for compiling your MySQL client and connecting it to MySQL\&. .PP \fBmysql_config\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -224,7 +224,7 @@ this way, be sure to invoke it within ba .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_convert_table_format.1' --- a/man/mysql_convert_table_format.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_convert_table_format.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_convert_table_format\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_CONVERT_TAB" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_CONVERT_TAB" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -115,10 +115,10 @@ Connect to the MySQL server on the given .\" password option: mysql_convert_table_format \fB\-\-password=\fR\fB\fIpassword\fR\fR .sp -The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. You can use an option file to avoid giving the password on the command line\&. +The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -216,7 +216,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_find_rows.1' --- a/man/mysql_find_rows.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_find_rows.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_find_rows\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_FIND_ROWS\F" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_FIND_ROWS\F" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -151,7 +151,7 @@ Start output from this row\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_fix_extensions.1' --- a/man/mysql_fix_extensions.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_fix_extensions.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_fix_extensions\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_FIX_EXTENSI" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_FIX_EXTENSI" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -61,7 +61,7 @@ shell> \fBmysql_fix_extensions \fR\fB\fI .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_fix_privilege_tables.1' --- a/man/mysql_fix_privilege_tables.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_fix_privilege_tables.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_fix_privilege_tables\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_FIX_PRIVILE" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_FIX_PRIVILE" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -131,7 +131,9 @@ mysql> \fBSOURCE share/mysql_fix_privile .ps -1 .br .PP -Prior to version 5\&.1\&.17, this script is found in the +Prior to version 5\&.1\&.17, the +mysql_fix_privilege_tables\&.sql +script is found in the scripts directory\&. .sp .5v @@ -157,7 +159,7 @@ After running the script, stop the serve .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_install_db.1' --- a/man/mysql_install_db.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_install_db.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_install_db\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_INSTALL_DB\" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_INSTALL_DB\" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -94,8 +94,12 @@ environment variable to the full path na will use that server\&. .PP \fBmysql_install_db\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at -Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. +supports the following options, which can be specified on the command line or in the +[mysql_install_db] +and (if they are common to +\fBmysqld\fR) +[mysqld] +option file groups\&. .sp .RS 4 .ie n \{\ @@ -248,7 +252,7 @@ For internal use\&. This option is used .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_secure_installation.1' --- a/man/mysql_secure_installation.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_secure_installation.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_secure_installation\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_SECURE_INST" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_SECURE_INST" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -96,7 +96,7 @@ The script will prompt you to determine .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_setpermission.1' --- a/man/mysql_setpermission.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_setpermission.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_setpermission\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_SETPERMISSI" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_SETPERMISSI" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -109,10 +109,10 @@ Connect to the MySQL server on the given .\" password option: mysql_setpermission \fB\-\-password=\fR\fB\fIpassword\fR\fR .sp -The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. You can use an option file to avoid giving the password on the command line\&. +The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -163,7 +163,7 @@ The MySQL user name to use when connecti .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_tzinfo_to_sql.1' --- a/man/mysql_tzinfo_to_sql.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_tzinfo_to_sql.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_tzinfo_to_sql\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_TZINFO_TO_S" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_TZINFO_TO_S" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -33,11 +33,11 @@ program loads the time zone tables in th mysql database\&. It is used on systems that have a zoneinfo -database (the set of files describing time zones)\&. Examples of such systems are Linux, FreeBSD, Sun Solaris, and Mac OS X\&. One likely location for these files is the +database (the set of files describing time zones)\&. Examples of such systems are Linux, FreeBSD, Solaris, and Mac OS X\&. One likely location for these files is the /usr/share/zoneinfo directory (/usr/share/lib/zoneinfo on Solaris)\&. If your system does not have a zoneinfo database, you can use the downloadable package described in -Section\ \&9.7, \(lqMySQL Server Time Zone Support\(rq\&. +Section\ \&9.6, \(lqMySQL Server Time Zone Support\(rq\&. .PP \fBmysql_tzinfo_to_sql\fR can be invoked several ways: @@ -113,7 +113,7 @@ After running .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_upgrade.1' --- a/man/mysql_upgrade.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_upgrade.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_upgrade\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_UPGRADE\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_UPGRADE\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -52,6 +52,24 @@ for manual table repair strategies\&. .nr an-break-flag 1 .br .ps +1 +\fBNote\fR +.ps -1 +.br +.PP +On Windows Server 2008 and Windows Vista, you must run +\fBmysql_upgrade\fR +with administrator privileges\&. You can do this by running a Command Prompt as Administrator and running the command\&. Failure to do so may result in the upgrade failing to execute correctly\&. +.sp .5v +.RE +.if n \{\ +.sp +.\} +.RS 4 +.it 1 an-trap +.nr an-no-space-flag 1 +.nr an-break-flag 1 +.br +.ps +1 \fBCaution\fR .ps -1 .br @@ -59,7 +77,7 @@ for manual table repair strategies\&. You should always back up your current MySQL installation \fIbefore\fR performing an upgrade\&. See -Section\ \&6.1, \(lqDatabase Backup Methods\(rq\&. +Section\ \&6.2, \(lqDatabase Backup Methods\(rq\&. .PP Some upgrade incompatibilities may require special handling before you upgrade your MySQL installation and run \fBmysql_upgrade\fR\&. See @@ -132,7 +150,7 @@ FOR UPGRADE option of the CHECK TABLE statement (see -Section\ \&12.5.2.3, \(lqCHECK TABLE Syntax\(rq)\&. +Section\ \&12.4.2.3, \(lqCHECK TABLE Syntax\(rq)\&. .RE .sp .RS 4 @@ -144,7 +162,7 @@ Section\ \&12.5.2.3, \(lqCHECK TABLE Syn .IP \(bu 2.3 .\} \fIfix_priv_tables\fR -represents a script generated interally by +represents a script generated internally by \fBmysql_upgrade\fR that contains SQL statements to upgrade the tables in the mysql @@ -198,15 +216,17 @@ was added as a shell script and worked o is an executable binary and is available on all systems\&. .PP \fBmysql_upgrade\fR -supports the options in the following list\&. It also reads option files (the +supports the following options, which can be specified on the command line or in the [mysql_upgrade] and [client] -groups) and supports the options for processing them described at -Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. Other options are passed to +option file groups\&. Other options are passed to \fBmysqlcheck\fR\&. For example, it might be necessary to specify the \fB\-\-password[=\fR\fB\fIpassword\fR\fR\fB]\fR option\&. +\fBmysql_upgrade\fR +also supports the options for processing option files described at +Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 .ie n \{\ @@ -375,7 +395,7 @@ This option was introduced in MySQL 5\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_waitpid.1' --- a/man/mysql_waitpid.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_waitpid.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_waitpid\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_WAITPID\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_WAITPID\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -120,7 +120,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_zap.1' --- a/man/mysql_zap.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_zap.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_zap\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_ZAP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_ZAP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -116,7 +116,7 @@ Test mode\&. Display information about e .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlaccess.1' --- a/man/mysqlaccess.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlaccess.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlaccess\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLACCESS\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLACCESS\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -54,7 +54,7 @@ shell> \fBmysqlaccess [\fR\fB\fIhost_nam .\} .PP \fBmysqlaccess\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -221,10 +221,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlaccess\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. .RE .sp .RS 4 @@ -319,13 +321,15 @@ Undo the most recent changes to the temp The password to use when connecting to the server as the superuser\&. If you omit the \fIpassword\fR value following the -\fB\-\-password\fR +\fB\-\-spassword\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlaccess\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. .RE .sp .RS 4 @@ -419,7 +423,7 @@ error will occur when you run .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqladmin.1' --- a/man/mysqladmin.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqladmin.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqladmin\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLADMIN\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLADMIN\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -46,7 +46,7 @@ shell> \fBmysqladmin [\fR\fB\fIoptions\f .\} .PP \fBmysqladmin\fR -supports the commands described in the following list\&. Some of the commands take an argument following the command name\&. +supports the following commands\&. Some of the commands take an argument following the command name\&. .sp .RS 4 .ie n \{\ @@ -211,7 +211,7 @@ old\-password \fInew\-password\fR This is like the password command but stores the password using the old (pre\-4\&.1) password\-hashing format\&. (See -Section\ \&5.5.6.3, \(lqPassword Hashing in MySQL\(rq\&.) +Section\ \&5.3.2.3, \(lqPassword Hashing in MySQL\(rq\&.) .RE .sp .RS 4 @@ -304,7 +304,7 @@ statement\&. If the \fB\-\-verbose\fR option is given, the output is like that of SHOW FULL PROCESSLIST\&. (See -Section\ \&12.5.5.31, \(lqSHOW PROCESSLIST Syntax\(rq\&.) +Section\ \&12.4.5.31, \(lqSHOW PROCESSLIST Syntax\(rq\&.) .RE .sp .RS 4 @@ -586,7 +586,13 @@ waits until the server\'s process ID fil .\" startup parameters: mysqladmin .PP \fBmysqladmin\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqladmin] +and +[client] +option file groups\&. +\fBmysqladmin\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -618,7 +624,7 @@ Display a help message and exit\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -720,7 +726,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -794,10 +800,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqladmin\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -813,7 +821,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -940,7 +948,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -1030,7 +1038,7 @@ You can also set the following variables \fB\-\-\fR\fB\fIvar_name\fR\fR\fB=\fR\fB\fIvalue\fR\fR The \fB\-\-set\-variable\fR -format is deprecated\&. syntax: +format is deprecated and is removed in MySQL 5\&.5\&. syntax: .sp .RS 4 .ie n \{\ @@ -1064,7 +1072,7 @@ The maximum number of seconds to wait fo .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlbinlog.1' --- a/man/mysqlbinlog.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlbinlog.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlbinlog\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLBINLOG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLBINLOG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -34,7 +34,7 @@ utility\&. You can also use \fBmysqlbinlog\fR to display the contents of relay log files written by a slave server in a replication setup because relay logs have the same format as binary logs\&. The binary log and relay log are discussed further in Section\ \&5.2.4, \(lqThe Binary Log\(rq, and -Section\ \&16.4.2, \(lqReplication Relay and Status Files\(rq\&. +Section\ \&16.2.2, \(lqReplication Relay and Status Files\(rq\&. .PP Invoke \fBmysqlbinlog\fR @@ -64,18 +64,52 @@ shell> \fBmysqlbinlog binlog\&.0000003\f .\} .PP The output includes events contained in -binlog\&.000003\&. Event information includes the statement, the ID of the server on which it was executed, the timestamp when the statement was executed, how much time it took, and so forth\&. +binlog\&.000003\&. For statement\-based logging, event information includes the SQL statement, the ID of the server on which it was executed, the timestamp when the statement was executed, how much time it took, and so forth\&. For row\-based logging, the event indicates a row change rather than an SQL statement\&. See +Section\ \&16.1.2, \(lqReplication Formats\(rq, for information about logging modes\&. +.PP +Events are preceded by header comments that provide additional information\&. For example: +.sp +.if n \{\ +.RS 4 +.\} +.nf +# at 141 +#100309 9:28:36 server id 123 end_log_pos 245 + Query thread_id=3350 exec_time=11 error_code=0 +.fi +.if n \{\ +.RE +.\} +.PP +In the first line, the number following +at +indicates the starting position of the event in the binary log file\&. +.PP +The second line starts with a date and time indicating when the statement started on the server where the event originated\&. For replication, this timestamp is propagated to slave servers\&. +server id +is the +server_id +value of the server where the event originated\&. +end_log_pos +indicates where the next event starts (that is, it is the end position of the current event + 1)\&. +thread_id +indicates which thread executed the event\&. +exec_time +is the time spent executing the event, on a master server\&. On a slave, it is the difference of the end execution time on the slave minus the beginning execution time on the master\&. The difference serves as an indicator of how much replication lags behind the master\&. +error_code +indicates the result from executing the event\&. Zero means that no error occurred\&. .PP The output from \fBmysqlbinlog\fR can be re\-executed (for example, by using it as input to -\fBmysql\fR) to reapply the statements in the log\&. This is useful for recovery operations after a server crash\&. For other usage examples, see the discussion later in this section\&. +\fBmysql\fR) to redo the statements in the log\&. This is useful for recovery operations after a server crash\&. For other usage examples, see the discussion later in this section and +Section\ \&6.5, \(lqPoint-in-Time (Incremental) Recovery Using the Binary Log\(rq\&. .PP Normally, you use \fBmysqlbinlog\fR to read binary log files directly and apply them to the local MySQL server\&. It is also possible to read binary logs from a remote server by using the \fB\-\-read\-from\-remote\-server\fR -option\&. When you read remote binary logs, the connection parameter options can be given to indicate how to connect to the server\&. These options are +option\&. To read remote binary logs, the connection parameter options can be given to indicate how to connect to the server\&. These options are \fB\-\-host\fR, \fB\-\-password\fR, \fB\-\-port\fR, @@ -86,7 +120,13 @@ option\&. When you read remote binary lo option\&. .PP \fBmysqlbinlog\fR -supports the following options\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlbinlog] +and +[client] +option file groups\&. +\fBmysqlbinlog\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -248,7 +288,7 @@ the section called \(lqMYSQLBINLOG ROW E \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -264,15 +304,140 @@ Section\ \&9.2, \(lqThe Character Set Us \fB\-\-database=\fR\fB\fIdb_name\fR\fR, \fB\-d \fR\fB\fIdb_name\fR\fR .sp -List entries for just this database (local log only)\&. You can only specify one database with this option \- if you specify multiple +This option causes +\fBmysqlbinlog\fR +to output entries from the binary log (local log only) that occur while +\fIdb_name\fR +is been selected as the default database by +USE\&. +.sp +The +\fB\-\-database\fR +option for +\fBmysqlbinlog\fR +is similar to the +\fB\-\-binlog\-do\-db\fR +option for +\fBmysqld\fR, but can be used to specify only one database\&. If +\fB\-\-database\fR +is given multiple times, only the last instance is used\&. +.sp +The effects of this option depend on whether the statement\-based or row\-based logging format is in use, in the same way that the effects of +\fB\-\-binlog\-do\-db\fR +depend on whether statement\-based or row\-based logging is in use\&. +.PP +\fBStatement-based logging\fR. The \fB\-\-database\fR -options, only the last one is used\&. This option forces +option works as follows: +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +While +\fIdb_name\fR +is the default database, statements are output whether they modify tables in +\fIdb_name\fR +or a different database\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +Unless +\fIdb_name\fR +is selected as the default database, statements are not output, even if they modify tables in +\fIdb_name\fR\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +There is an exception for +CREATE DATABASE, +ALTER DATABASE, and +DROP DATABASE\&. The database being +\fIcreated, altered, or dropped\fR +is considered to be the default database when determining whether to output the statement\&. +.RE +.RS 4 +Suppose that the binary log was created by executing these statements using statement\-based\-logging: +.sp +.if n \{\ +.RS 4 +.\} +.nf +INSERT INTO test\&.t1 (i) VALUES(100); +INSERT INTO db2\&.t2 (j) VALUES(200); +USE test; +INSERT INTO test\&.t1 (i) VALUES(101); +INSERT INTO t1 (i) VALUES(102); +INSERT INTO db2\&.t2 (j) VALUES(201); +USE db2; +INSERT INTO test\&.t1 (i) VALUES(103); +INSERT INTO db2\&.t2 (j) VALUES(202); +INSERT INTO t2 (j) VALUES(203); +.fi +.if n \{\ +.RE +.\} +.sp +\fBmysqlbinlog \-\-database=test\fR +does not output the first two +INSERT +statements because there is no default database\&. It outputs the three +INSERT +statements following +USE test, but not the three +INSERT +statements following +USE db2\&. +.sp +\fBmysqlbinlog \-\-database=db2\fR +does not output the first two +INSERT +statements because there is no default database\&. It does not output the three +INSERT +statements following +USE test, but does output the three +INSERT +statements following +USE db2\&. +.PP +\fBRow-based logging\fR. +\fBmysqlbinlog\fR +outputs only entries that change tables belonging to +\fIdb_name\fR\&. The default database has no effect on this\&. Suppose that the binary log just described was created using row\-based logging rather than statement\-based logging\&. +\fBmysqlbinlog \-\-database=test\fR +outputs only those entries that modify +t1 +in the test database, regardless of whether +USE +was issued or what the default database is\&. +If a server is running with +binlog_format +set to +MIXED +and you want it to be possible to use \fBmysqlbinlog\fR -to output entries from the binary log where the default database (that is, the one selected by -USE) is -\fIdb_name\fR\&. Note that this does not replicate cross\-database statements such as -UPDATE \fIsome_db\&.some_table\fR SET foo=\'bar\' -while having selected a different database or no database\&. +with the +\fB\-\-database\fR +option, you must ensure that tables that are modified are in the database selected by +USE\&. (In particular, no cross\-database updates should be used\&.) .if n \{\ .sp .\} @@ -406,7 +571,7 @@ stops if it reads such an event\&. \fB\-H\fR .sp Display a hex dump of the log in comments, as described in -the section called \(lqMYSQLBINLOG HEX DUMP FORMAT\(rq\&. This output can be helpful for replication debugging\&. This option was added in MySQL 5\&.1\&.2\&. +the section called \(lqMYSQLBINLOG HEX DUMP FORMAT\(rq\&. The hex output can be helpful for replication debugging\&. This option was added in MySQL 5\&.1\&.2\&. .RE .sp .RS 4 @@ -482,10 +647,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlbinlog\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -514,12 +681,13 @@ The TCP/IP port number to use for connec .\} .\" mysqlbinlog: position option .\" position option: mysqlbinlog -\fB\-\-position=\fR\fB\fIN\fR\fR, -\fB\-j \fR\fB\fIN\fR\fR +\fB\-\-position=\fR\fB\fIN\fR\fR .sp Deprecated\&. Use \fB\-\-start\-position\fR instead\&. +\fB\-\-position\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -590,7 +758,7 @@ Direct output to the given file\&. .\" server-id option: mysqlbinlog \fB\-\-server\-id=\fR\fB\fIid\fR\fR .sp -Extract only those events created by the server having the given server ID\&. This option is available as of MySQL 5\&.1\&.4\&. +Display only those events created by the server having the given server ID\&. This option is available as of MySQL 5\&.1\&.4\&. .RE .sp .RS 4 @@ -677,7 +845,7 @@ shell> \fBmysqlbinlog \-\-start\-datetim .\} .sp This option is useful for point\-in\-time recovery\&. See -Section\ \&6.2, \(lqExample Backup and Recovery Strategy\(rq\&. +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -690,10 +858,14 @@ Section\ \&6.2, \(lqExample Backup and R .\} .\" mysqlbinlog: start-position option .\" start-position option: mysqlbinlog -\fB\-\-start\-position=\fR\fB\fIN\fR\fR +\fB\-\-start\-position=\fR\fB\fIN\fR\fR, +\fB\-j \fR\fB\fIN\fR\fR .sp Start reading the binary log at the first event having a position equal to or greater than \fIN\fR\&. This option applies to the first log file named on the command line\&. +.sp +This option is useful for point\-in\-time recovery\&. See +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -715,6 +887,9 @@ argument\&. This option is useful for po option for information about the \fIdatetime\fR value\&. +.sp +This option is useful for point\-in\-time recovery\&. See +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -731,6 +906,9 @@ value\&. .sp Stop reading the binary log at the first event having a position equal to or greater than \fIN\fR\&. This option applies to the last log file named on the command line\&. +.sp +This option is useful for point\-in\-time recovery\&. See +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -779,7 +957,7 @@ The MySQL user name to use when connecti \fB\-\-verbose\fR, \fB\-v\fR .sp -Reconstruct row events and display them as commented SQL statements\&. If given twice, the output includes comments to indicate column data types and some metadata\&. This option was added in MySQL 5\&.1\&.28\&. +Reconstruct row events and display them as commented SQL statements\&. If this option is given twice, the output includes comments to indicate column data types and some metadata\&. This option was added in MySQL 5\&.1\&.28\&. .sp For examples that show the effect of \fB\-\-base64\-output\fR @@ -804,33 +982,6 @@ the section called \(lqMYSQLBINLOG ROW E .sp Display version information and exit\&. .RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysqlbinlog: write-binlog option -.\" write-binlog option: mysqlbinlog -\fB\-\-write\-binlog\fR -.sp -This option is enabled by default, so that -ANALYZE TABLE, -OPTIMIZE TABLE, and -REPAIR TABLE -statements generated by -\fBmysqlcheck\fR -are written to the binary log\&. Use -\fB\-\-skip\-write\-binlog\fR -to cause -NO_WRITE_TO_BINLOG -to be added to the statements so that they are not logged\&. Use the -\fB\-\-skip\-write\-binlog\fR -when these statements should not be sent to replication slaves or run when using the binary logs for recovery from backup\&. This option was added in MySQL 5\&.1\&.18\&. -.RE .PP You can also set the following variable by using \fB\-\-\fR\fB\fIvar_name\fR\fR\fB=\fR\fB\fIvalue\fR\fR @@ -854,14 +1005,14 @@ You can pipe the output of \fBmysqlbinlog\fR into the \fBmysql\fR -client to execute the statements contained in the binary log\&. This is used to recover from a crash when you have an old backup (see -Section\ \&6.1, \(lqDatabase Backup Methods\(rq)\&. For example: +client to execute the events contained in the binary log\&. This technique is used to recover from a crash when you have an old backup (see +Section\ \&6.5, \(lqPoint-in-Time (Incremental) Recovery Using the Binary Log\(rq)\&. For example: .sp .if n \{\ .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.000001 | mysql\fR +shell> \fBmysqlbinlog binlog\&.000001 | mysql \-u root \-p\fR .fi .if n \{\ .RE @@ -873,7 +1024,7 @@ Or: .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.[0\-9]* | mysql\fR +shell> \fBmysqlbinlog binlog\&.[0\-9]* | mysql \-u root \-p\fR .fi .if n \{\ .RE @@ -883,12 +1034,25 @@ You can also redirect the output of \fBmysqlbinlog\fR to a text file instead, if you need to modify the statement log first (for example, to remove statements that you do not want to execute for some reason)\&. After editing the file, execute the statements that it contains by using it as input to the \fBmysql\fR -program\&. +program: +.sp +.if n \{\ +.RS 4 +.\} +.nf +shell> \fBmysqlbinlog binlog\&.000001 > tmpfile\fR +shell> \&.\&.\&. \fIedit tmpfile\fR \&.\&.\&. +shell> \fBmysql \-u root \-p < tmpfile\fR +.fi +.if n \{\ +.RE +.\} .PP +When \fBmysqlbinlog\fR -has the +is invoked with the \fB\-\-start\-position\fR -option, which prints only those statements with an offset in the binary log greater than or equal to a given position (the given position must match the start of one event)\&. It also has options to stop and start when it sees an event with a given date and time\&. This enables you to perform point\-in\-time recovery using the +option, it displays only those events with an offset in the binary log greater than or equal to a given position (the given position must match the start of one event)\&. It also has options to stop and start when it sees an event with a given date and time\&. This enables you to perform point\-in\-time recovery using the \fB\-\-stop\-datetime\fR option (to be able to say, for example, \(lqroll forward my databases to how they were today at 10:30 a\&.m\&.\(rq)\&. @@ -900,8 +1064,8 @@ If you have more than one binary log to .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.000001 | mysql # DANGER!!\fR -shell> \fBmysqlbinlog binlog\&.000002 | mysql # DANGER!!\fR +shell> \fBmysqlbinlog binlog\&.000001 | mysql \-u root \-p # DANGER!!\fR +shell> \fBmysqlbinlog binlog\&.000002 | mysql \-u root \-p # DANGER!!\fR .fi .if n \{\ .RE @@ -918,13 +1082,14 @@ process attempts to use the table, the s .PP To avoid problems like this, use a \fIsingle\fR -connection to execute the contents of all binary logs that you want to process\&. Here is one way to do so: +\fBmysql\fR +process to execute the contents of all binary logs that you want to process\&. Here is one way to do so: .sp .if n \{\ .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.000001 binlog\&.000002 | mysql\fR +shell> \fBmysqlbinlog binlog\&.000001 binlog\&.000002 | mysql \-u root \-p\fR .fi .if n \{\ .RE @@ -938,7 +1103,7 @@ Another approach is to write all the log .nf shell> \fBmysqlbinlog binlog\&.000001 > /tmp/statements\&.sql\fR shell> \fBmysqlbinlog binlog\&.000002 >> /tmp/statements\&.sql\fR -shell> \fBmysql \-e "source /tmp/statements\&.sql"\fR +shell> \fBmysql \-u root \-p \-e "source /tmp/statements\&.sql"\fR .fi .if n \{\ .RE @@ -962,10 +1127,10 @@ LOAD DATA INFILE statements to LOAD DATA LOCAL INFILE statements (that is, it adds -LOCAL), both the client and the server that you use to process the statements must be configured to allow +LOCAL), both the client and the server that you use to process the statements must be configured with the LOCAL -capability\&. See -Section\ \&5.3.4, \(lqSecurity Issues with LOAD DATA LOCAL\(rq\&. +capability enabled\&. See +Section\ \&5.3.5, \(lqSecurity Issues with LOAD DATA LOCAL\(rq\&. .if n \{\ .sp .\} @@ -991,7 +1156,9 @@ automatically deleted because they are n .PP The \fB\-\-hexdump\fR -option produces a hex dump of the log contents: +option causes +\fBmysqlbinlog\fR +to produce a hex dump of the binary log contents: .sp .if n \{\ .RS 4 @@ -1029,7 +1196,8 @@ ROLLBACK; .RE .\} .PP -Hex dump output currently contains the following elements\&. This format is subject to change\&. +Hex dump output currently contains the elements in the following list\&. This format is subject to change\&. (For more information about binary log format, see +\m[blue]\fB\%http://forge.mysql.com/wiki/MySQL_Internals_Binary_Log\fR\m[]\&.) .sp .RS 4 .ie n \{\ @@ -1796,7 +1964,7 @@ option can be used to prevent this heade .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -1808,7 +1976,7 @@ You should have received a copy of the G .IP " 1." 4 Bug#42941 .RS 4 -\%http://bugs.mysql.com/42941 +\%http://bugs.mysql.com/bug.php?id=42941 .RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, === modified file 'man/mysqlbug.1' --- a/man/mysqlbug.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlbug.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlbug\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLBUG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLBUG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -26,7 +26,7 @@ mysqlbug \- generate bug report \fBmysqlbug\fR .SH "DESCRIPTION" .PP -This program enables you to generate a bug report and send it to Sun Microsystems, Inc\&. It is a shell script and runs on Unix\&. +This program enables you to generate a bug report and send it to Oracle Corporation\&. It is a shell script and runs on Unix\&. .PP The normal way to report bugs is to visit \m[blue]\fB\%http://bugs.mysql.com/\fR\m[], which is the address for our bugs database\&. This database is public and can be browsed and searched by anyone\&. If you log in to the system, you can enter new reports\&. If you have no Web access, you can generate a bug report by using the @@ -62,7 +62,7 @@ will send the report by email\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlcheck.1' --- a/man/mysqlcheck.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlcheck.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlcheck\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLCHECK\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLCHECK\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -37,7 +37,7 @@ client performs table maintenance: It ch Each table is locked and therefore unavailable to other sessions while it is being processed, although for check operations, the table is locked with a READ lock only (see -Section\ \&12.4.5, \(lqLOCK TABLES and UNLOCK TABLES Syntax\(rq, for more information about +Section\ \&12.3.5, \(lqLOCK TABLES and UNLOCK TABLES Syntax\(rq, for more information about READ and WRITE @@ -72,7 +72,7 @@ REPAIR TABLE, ANALYZE TABLE, and OPTIMIZE TABLE in a convenient way for the user\&. It determines which statements to use for the operation you want to perform, and then sends the statements to the server to be executed\&. For details about which storage engines each statement works with, see the descriptions for those statements in -Section\ \&12.5.2, \(lqTable Maintenance Statements\(rq\&. +Section\ \&12.4.2, \(lqTable Maintenance Statements\(rq\&. .PP The MyISAM @@ -135,8 +135,8 @@ There are three general ways to invoke .RS 4 .\} .nf -shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItables\fR\fR\fB]\fR -shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name1\fR\fR\fB [\fR\fB\fIdb_name2\fR\fR\fB \fR\fB\fIdb_name3\fR\fR\fB\&.\&.\&.]\fR +shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItbl_name\fR\fR\fB \&.\&.\&.]\fR +shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name\fR\fR\fB \&.\&.\&.\fR shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \-\-all\-databases\fR .fi .if n \{\ @@ -188,7 +188,13 @@ T} .sp 1 .PP \fBmysqlcheck\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlcheck] +and +[client] +option file groups\&. +\fBmysqlcheck\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -285,7 +291,7 @@ If a checked table is corrupted, automat \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -444,7 +450,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -598,10 +604,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlcheck\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -617,7 +625,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -734,7 +742,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -824,10 +832,37 @@ Verbose mode\&. Print information about .sp Display version information and exit\&. .RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysqlcheck: write-binlog option +.\" write-binlog option: mysqlcheck +\fB\-\-write\-binlog\fR +.sp +This option is enabled by default, so that +ANALYZE TABLE, +OPTIMIZE TABLE, and +REPAIR TABLE +statements generated by +\fBmysqlcheck\fR +are written to the binary log\&. Use +\fB\-\-skip\-write\-binlog\fR +to cause +NO_WRITE_TO_BINLOG +to be added to the statements so that they are not logged\&. Use the +\fB\-\-skip\-write\-binlog\fR +when these statements should not be sent to replication slaves or run when using the binary logs for recovery from backup\&. This option was added in MySQL 5\&.1\&.18\&. +.RE .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqld.8' --- a/man/mysqld.8 2009-12-01 07:24:05 +0000 +++ b/man/mysqld.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqld\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLD\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLD\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -53,7 +53,7 @@ Chapter\ \&2, Installing and Upgrading M .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqld_multi.1' --- a/man/mysqld_multi.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqld_multi.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqld_multi\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLD_MULTI\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLD_MULTI\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -227,7 +227,7 @@ pass the options to instances, so these techniques are inapplicable\&. .PP \fBmysqld_multi\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -258,6 +258,8 @@ Display a help message and exit\&. .sp As of MySQL 5\&.1\&.18, this option is deprecated\&. If given, it is treated the same way as \fB\-\-defaults\-extra\-file\fR, described earlier\&. +\fB\-\-config\-file\fR +is removed in MySQL 5\&.5\&. .sp Before MySQL 5\&.1\&.18, this option specifies the name of an extra option file\&. It affects where \fBmysqld_multi\fR @@ -534,7 +536,7 @@ use the Unix account for this, unless you \fIknow\fR what you are doing\&. See -Section\ \&5.3.5, \(lqHow to Run MySQL as a Normal User\(rq\&. +Section\ \&5.3.6, \(lqHow to Run MySQL as a Normal User\(rq\&. .sp .5v .RE .RE @@ -726,7 +728,7 @@ Section\ \&4.2.3.3, \(lqUsing Option Fil .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqld_safe.1' --- a/man/mysqld_safe.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqld_safe.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqld_safe\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLD_SAFE\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLD_SAFE\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -842,7 +842,7 @@ file in the data directory\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqldump.1' --- a/man/mysqldump.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqldump.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqldump\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLDUMP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLDUMP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -50,8 +50,8 @@ There are three general ways to invoke .RS 4 .\} .nf -shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItables\fR\fR\fB]\fR -shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name1\fR\fR\fB [\fR\fB\fIdb_name2\fR\fR\fB \fR\fB\fIdb_name3\fR\fR\fB\&.\&.\&.]\fR +shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItbl_name\fR\fR\fB \&.\&.\&.]\fR +shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name\fR\fR\fB \&.\&.\&.\fR shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \-\-all\-databases\fR .fi .if n \{\ @@ -69,36 +69,70 @@ option, entire databases are dumped\&. \fBmysqldump\fR does not dump the INFORMATION_SCHEMA -database\&. If you name that database explicitly on the command line, +database by default\&. As of MySQL 5\&.1\&.38, \fBmysqldump\fR -silently ignores it\&. +dumps +INFORMATION_SCHEMA +if you name it explicitly on the command line, although currently you must also use the +\fB\-\-skip\-lock\-tables\fR +option\&. Before 5\&.1\&.38, +\fBmysqldump\fR +silently ignores +INFORMATION_SCHEMA +even if you name it explicitly on the command line\&. .PP -To get a list of the options your version of +To see a list of the options your version of \fBmysqldump\fR supports, execute \fBmysqldump \-\-help\fR\&. .PP Some \fBmysqldump\fR -options are shorthand for groups of other options\&. -\fB\-\-opt\fR -and -\fB\-\-compact\fR -fall into this category\&. For example, use of +options are shorthand for groups of other options: +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +Use of \fB\-\-opt\fR is the same as specifying -\fB\-\-add\-drop\-table\fR -\fB\-\-add\-locks\fR -\fB\-\-create\-options\fR -\fB\-\-disable\-keys\fR -\fB\-\-extended\-insert\fR -\fB\-\-lock\-tables\fR -\fB\-\-quick\fR -\fB\-\-set\-charset\fR\&. Note that all of the options that +\fB\-\-add\-drop\-table\fR, +\fB\-\-add\-locks\fR, +\fB\-\-create\-options\fR, +\fB\-\-disable\-keys\fR, +\fB\-\-extended\-insert\fR, +\fB\-\-lock\-tables\fR, +\fB\-\-quick\fR, and +\fB\-\-set\-charset\fR\&. All of the options that \fB\-\-opt\fR stands for also are on by default because \fB\-\-opt\fR is on by default\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +Use of +\fB\-\-compact\fR +is the same as specifying +\fB\-\-skip\-add\-drop\-table\fR, +\fB\-\-skip\-add\-locks\fR, +\fB\-\-skip\-comments\fR, +\fB\-\-skip\-disable\-keys\fR, and +\fB\-\-skip\-set\-charset\fR +options\&. +.RE .PP To reverse the effect of a group option, uses its \fB\-\-skip\-\fR\fB\fIxxx\fR\fR @@ -118,10 +152,10 @@ To select the effect of \fB\-\-opt\fR except for some features, use the \fB\-\-skip\fR -option for each feature\&. For example, to disable extended inserts and memory buffering, use +option for each feature\&. To disable extended inserts and memory buffering, use \fB\-\-opt\fR \fB\-\-skip\-extended\-insert\fR -\fB\-\-skip\-quick\fR\&. (As of MySQL 5\&.1, +\fB\-\-skip\-quick\fR\&. (Actually, \fB\-\-skip\-extended\-insert\fR \fB\-\-skip\-quick\fR is sufficient because @@ -161,7 +195,7 @@ option (or \fB\-\-quick\fR)\&. The \fB\-\-opt\fR option (and hence -\fB\-\-quick\fR) is enabled by default in MySQL 5\&.1; to enable memory buffering, use +\fB\-\-quick\fR) is enabled by default, so to enable memory buffering, use \fB\-\-skip\-quick\fR\&. .PP If you are using a recent version of @@ -187,12 +221,18 @@ instead\&. .br .PP \fBmysqldump\fR -from the MySQL 5\&.1\&.21 distribution cannot be used to create dumps from MySQL server versions 5\&.1\&.20 and older\&. This issue is fixed in MySQL 5\&.1\&.22\&. (\m[blue]\fBBug#30123\fR\m[]\&\s-2\u[1]\d\s+2) +from MySQL 5\&.1\&.21 cannot be used to create dumps from MySQL server 5\&.1\&.20 and older\&. This issue is fixed in MySQL 5\&.1\&.22\&. (\m[blue]\fBBug#30123\fR\m[]\&\s-2\u[1]\d\s+2) .sp .5v .RE .PP \fBmysqldump\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqldump] +and +[client] +option file groups\&. +\fBmysqldump\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -227,7 +267,13 @@ Add a DROP DATABASE statement before each CREATE DATABASE -statement\&. +statement\&. This option is typically used in conjunction with the +\fB\-\-all\-databases\fR +or +\fB\-\-databases\fR +option because no +CREATE DATABASE +statements are written unless one of those options is specified\&. .RE .sp .RS 4 @@ -336,7 +382,7 @@ Allow creation of column names that are \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -368,7 +414,7 @@ Write additional information in the dump .\" compact option: mysqldump \fB\-\-compact\fR .sp -Produce less verbose output\&. This option enables the +Produce more compact output\&. This option enables the \fB\-\-skip\-add\-drop\-table\fR, \fB\-\-skip\-add\-locks\fR, \fB\-\-skip\-comments\fR, @@ -387,7 +433,7 @@ options\&. \fBNote\fR .ps -1 .br -Prior to release 5\&.1\&.21, this option did not create valid SQL if the database dump contained views\&. The recreation of views requires the creation and removal of temporary tables and this option suppressed the removal of those temporary tables\&. As a workaround, use +Prior to MySQL 5\&.1\&.21, this option did not create valid SQL if the database dump contained views\&. The recreation of views requires the creation and removal of temporary tables and this option suppressed the removal of those temporary tables\&. As a workaround, use \fB\-\-compact\fR with the \fB\-\-add\-drop\-table\fR @@ -409,7 +455,7 @@ option and then manually adjust the dump \fB\-\-compatible=\fR\fB\fIname\fR\fR .sp Produce output that is more compatible with other database systems or with older MySQL servers\&. The value of -name +\fIname\fR can be ansi, mysql323, @@ -569,7 +615,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. If no character set is specified, +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. If no character set is specified, \fBmysqldump\fR uses utf8, and earlier versions use @@ -611,7 +657,9 @@ statements\&. .\" delete-master-logs option: mysqldump \fB\-\-delete\-master\-logs\fR .sp -On a master replication server, delete the binary logs after performing the dump operation\&. This option automatically enables +On a master replication server, delete the binary logs by sending a +PURGE BINARY LOGS +statement to the server after performing the dump operation\&. This option automatically enables \fB\-\-master\-data\fR\&. .RE .sp @@ -651,12 +699,23 @@ tables\&. .\" dump-date option: mysqldump \fB\-\-dump\-date\fR .sp +If the +\fB\-\-comments\fR +option is given, \fBmysqldump\fR -produces a +produces a comment at the end of the dump of the following form: +.sp +.if n \{\ +.RS 4 +.\} +.nf \-\- Dump completed on \fIDATE\fR -comment at the end of the dump if the -\fB\-\-comments\fR -option is given\&. However, the date causes dump files for identical data take at different times to appear to be different\&. +.fi +.if n \{\ +.RE +.\} +.sp +However, the date causes dump files taken at different times to appear to be different, even if the data are otherwise identical\&. \fB\-\-dump\-date\fR and \fB\-\-skip\-dump\-date\fR @@ -680,7 +739,7 @@ suppresses date printing\&. This option \fB\-\-events\fR, \fB\-E\fR .sp -Dump events from the dumped databases\&. This option was added in MySQL 5\&.1\&.8\&. +Include Event Scheduler events for the dumped databases in the output\&. This option was added in MySQL 5\&.1\&.8\&. .RE .sp .RS 4 @@ -725,8 +784,10 @@ lists\&. This results in a smaller dump \fB\-\-fields\-escaped\-by=\&.\&.\&.\fR .sp These options are used with the -\fB\-T\fR -option and have the same meaning as the corresponding clauses for +\fB\-\-tab\fR +option and have the same meaning as the corresponding +FIELDS +clauses for LOAD DATA INFILE\&. See Section\ \&12.2.6, \(lqLOAD DATA INFILE Syntax\(rq\&. .RE @@ -741,11 +802,13 @@ Section\ \&12.2.6, \(lqLOAD DATA INFILE .\} .\" mysqldump: first-slave option .\" first-slave option: mysqldump -\fB\-\-first\-slave\fR, -\fB\-x\fR +\fB\-\-first\-slave\fR .sp -Deprecated\&. Now renamed to -\fB\-\-lock\-all\-tables\fR\&. +Deprecated\&. Use +\fB\-\-lock\-all\-tables\fR +instead\&. +\fB\-\-first\-slave\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -763,10 +826,9 @@ Deprecated\&. Now renamed to .sp Flush the MySQL server log files before starting the dump\&. This option requires the RELOAD -privilege\&. Note that if you use this option in combination with the +privilege\&. If you use this option in combination with the \fB\-\-all\-databases\fR -(or -\fB\-A\fR) option, the logs are flushed +option, the logs are flushed \fIfor each database dumped\fR\&. The exception is when using \fB\-\-lock\-all\-tables\fR or @@ -790,9 +852,9 @@ or .\" flush-privileges option: mysqldump \fB\-\-flush\-privileges\fR .sp -Emit a +Send a FLUSH PRIVILEGES -statement after dumping the +statement to the server after dumping the mysql database\&. This option should be used any time the dump contains the mysql @@ -861,8 +923,9 @@ Dump binary columns using hexadecimal no becomes 0x616263)\&. The affected data types are BINARY, -VARBINARY, -BLOB, and +VARBINARY, the +BLOB +types, and BIT\&. .RE .sp @@ -894,10 +957,10 @@ Do not dump the given table, which must \fB\-\-insert\-ignore\fR .sp Write +INSERT IGNORE +statements rather than INSERT -statements with the -IGNORE -option\&. +statements\&. .RE .sp .RS 4 @@ -913,8 +976,10 @@ option\&. \fB\-\-lines\-terminated\-by=\&.\&.\&.\fR .sp This option is used with the -\fB\-T\fR -option and has the same meaning as the corresponding clause for +\fB\-\-tab\fR +option and has the same meaning as the corresponding +LINES +clause for LOAD DATA INFILE\&. See Section\ \&12.2.6, \(lqLOAD DATA INFILE Syntax\(rq\&. .RE @@ -951,18 +1016,20 @@ and \fB\-\-lock\-tables\fR, \fB\-l\fR .sp -Lock all tables before dumping them\&. The tables are locked with +For each dumped database, lock all tables to be dumped before dumping them\&. The tables are locked with READ LOCAL to allow concurrent inserts in the case of MyISAM tables\&. For transactional tables such as InnoDB, \fB\-\-single\-transaction\fR -is a much better option, because it does not need to lock the tables at all\&. +is a much better option than +\fB\-\-lock\-tables\fR +because it does not need to lock the tables at all\&. .sp -Please note that when dumping multiple databases, +Because \fB\-\-lock\-tables\fR -locks tables for each database separately\&. Therefore, this option does not guarantee that the tables in the dump file are logically consistent between databases\&. Tables in different databases may be dumped in completely different states\&. +locks tables for each database separately, this option does not guarantee that the tables in the dump file are logically consistent between databases\&. Tables in different databases may be dumped in completely different states\&. .RE .sp .RS 4 @@ -977,7 +1044,7 @@ locks tables for each database separatel .\" log-error option: mysqldump \fB\-\-log\-error=\fR\fB\fIfile_name\fR\fR .sp -Append warnings and errors to the named file\&. This option was added in MySQL 5\&.1\&.18\&. +Log warnings and errors by appending them to the named file\&. The default is to do no logging\&. This option was added in MySQL 5\&.1\&.18\&. .RE .sp .RS 4 @@ -994,11 +1061,11 @@ Append warnings and errors to the named .sp Use this option to dump a master replication server to produce a dump file that can be used to set up another server as a slave of the master\&. It causes the dump output to include a CHANGE MASTER TO -statement that indicates the binary log coordinates (file name and position) of the dumped server\&. These are the master server coordinates from which the slave should start replicating\&. +statement that indicates the binary log coordinates (file name and position) of the dumped server\&. These are the master server coordinates from which the slave should start replicating after you load the dump file into the slave\&. .sp If the option value is 2, the CHANGE MASTER TO -statement is written as an SQL comment, and thus is informative only; it has no effect when the dump file is reloaded\&. If the option value is 1, the statement takes effect when the dump file is reloaded\&. If the option value is not specified, the default value is 1\&. +statement is written as an SQL comment, and thus is informative only; it has no effect when the dump file is reloaded\&. If the option value is 1, the statement is not written as a comment and takes effect when the dump file is reloaded\&. If no option value is specified, the default value is 1\&. .sp This option requires the RELOAD @@ -1045,7 +1112,16 @@ mysql> \fBSHOW SLAVE STATUS;\fR .sp -1 .IP " 2." 4.2 .\} -From the output of the SHOW SLAVE STATUS statement, get the binary log coordinates of the master server from which the new slave should start replicating\&. These coordinates are the values of the Relay_Master_Log_File and Exec_Master_Log_Pos values\&. Denote those values as file_name and file_pos\&. +From the output of the +SHOW SLAVE STATUS +statement, the binary log coordinates of the master server from which the new slave should start replicating are the values of the +Relay_Master_Log_File +and +Exec_Master_Log_Pos +fields\&. Denote those values as +\fIfile_name\fR +and +\fIfile_pos\fR\&. .RE .sp .RS 4 @@ -1098,7 +1174,7 @@ mysql> \fBSTART SLAVE;\fR .sp -1 .IP " 5." 4.2 .\} -On the new slave, reload the dump file: +On the new slave, load the dump file: .sp .if n \{\ .RS 4 @@ -1126,7 +1202,7 @@ On the new slave, set the replication co .\} .nf mysql> \fBCHANGE MASTER TO\fR - \-> \fBMASTER_LOG_FILE = \'file_name\', MASTER_LOG_POS = file_pos;\fR + \-> \fBMASTER_LOG_FILE = \'\fR\fB\fIfile_name\fR\fR\fB\', MASTER_LOG_POS = \fR\fB\fIfile_pos\fR\fR\fB;\fR .fi .if n \{\ .RE @@ -1214,9 +1290,9 @@ statements that re\-create each dumped t \fB\-\-no\-data\fR, \fB\-d\fR .sp -Do not write any table row information (that is, do not dump table contents)\&. This is very useful if you want to dump only the +Do not write any table row information (that is, do not dump table contents)\&. This is useful if you want to dump only the CREATE TABLE -statement for the table\&. +statement for the table (for example, to create an empty copy of the table by loading the dump file)\&. .RE .sp .RS 4 @@ -1229,11 +1305,11 @@ statement for the table\&. .\} .\" mysqldump: no-set-names option .\" no-set-names option: mysqldump -\fB\-\-no\-set\-names\fR +\fB\-\-no\-set\-names\fR, +\fB\-N\fR .sp -This option is deprecated\&. Use -\fB\-\-skip\-set\-charset\fR -instead\&. +This has the same effect as +\fB\-\-skip\-set\-charset\fR\&. .RE .sp .RS 4 @@ -1248,7 +1324,7 @@ instead\&. .\" opt option: mysqldump \fB\-\-opt\fR .sp -This option is shorthand; it is the same as specifying +This option is shorthand\&. It is the same as specifying \fB\-\-add\-drop\-table\fR \fB\-\-add\-locks\fR \fB\-\-create\-options\fR @@ -1259,7 +1335,7 @@ This option is shorthand; it is the same \fB\-\-set\-charset\fR\&. It should give you a fast dump operation and produce a dump file that can be reloaded into a MySQL server quickly\&. .sp \fIThe \fR\fI\fB\-\-opt\fR\fR\fI option is enabled by default\&. Use \fR\fI\fB\-\-skip\-opt\fR\fR\fI to disable it\&.\fR -See the discussion at the beginning of this section for information about selectively enabling or disabling certain of the options affected by +See the discussion at the beginning of this section for information about selectively enabling or disabling a subset of the options affected by \fB\-\-opt\fR\&. .RE .sp @@ -1275,11 +1351,11 @@ See the discussion at the beginning of t .\" order-by-primary option: mysqldump \fB\-\-order\-by\-primary\fR .sp -Sort each table\'s rows by its primary key, or by its first unique index, if such an index exists\&. This is useful when dumping a +Dump each table\'s rows sorted by its primary key, or by its first unique index, if such an index exists\&. This is useful when dumping a MyISAM table to be loaded into an InnoDB -table, but will make the dump itself take considerably longer\&. +table, but will make the dump operation take considerably longer\&. .RE .sp .RS 4 @@ -1303,10 +1379,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqldump\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -1322,7 +1400,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -1388,11 +1466,11 @@ to retrieve rows for a table from the se \fB\-\-quote\-names\fR, \fB\-Q\fR .sp -Quote database, table, and column names within +Quote identifiers (such as database, table, and column names) within \(lq`\(rq characters\&. If the ANSI_QUOTES -SQL mode is enabled, names are quoted within +SQL mode is enabled, identifiers are quoted within \(lq"\(rq characters\&. This option is enabled by default\&. It can be disabled with \fB\-\-skip\-quote\-names\fR, but this option should be given after any option such as @@ -1417,7 +1495,7 @@ Write REPLACE statements rather than INSERT -statements\&. Available as of MySQL 5\&.1\&.3\&. +statements\&. This option was added in MySQL 5\&.1\&.3\&. .RE .sp .RS 4 @@ -1437,7 +1515,7 @@ Direct output to a given file\&. This op \(lq\en\(rq characters from being converted to \(lq\er\en\(rq -carriage return/newline sequences\&. The result file is created and its contents overwritten, even if an error occurs while generating the dump\&. The previous contents are lost\&. +carriage return/newline sequences\&. The result file is created and its previous contents overwritten, even if an error occurs while generating the dump\&. .RE .sp .RS 4 @@ -1453,7 +1531,7 @@ carriage return/newline sequences\&. The \fB\-\-routines\fR, \fB\-R\fR .sp -Dump stored routines (procedures and functions) from the dumped databases\&. Use of this option requires the +Included stored routines (procedures and functions) for the dumped databases in the output\&. Use of this option requires the SELECT privilege for the mysql\&.proc @@ -1511,9 +1589,9 @@ statement, use .\" single-transaction option: mysqldump \fB\-\-single\-transaction\fR .sp -This option issues a -BEGIN -SQL statement before dumping data from the server\&. It is useful only with transactional tables such as +This option sends a +START TRANSACTION +SQL statement to the server before dumping data\&. It is useful only with transactional tables such as InnoDB, because then it dumps the consistent state of the database at the time when BEGIN was issued without blocking any applications\&. @@ -1528,16 +1606,25 @@ tables dumped while using this option ma .sp While a \fB\-\-single\-transaction\fR -dump is in process, to ensure a valid dump file (correct table contents and binary log position), no other connection should use the following statements: +dump is in process, to ensure a valid dump file (correct table contents and binary log coordinates), no other connection should use the following statements: ALTER TABLE, +CREATE TABLE, DROP TABLE, RENAME TABLE, TRUNCATE TABLE\&. A consistent read is not isolated from those statements, so use of them on a table to be dumped can cause the SELECT -performed by +that is performed by \fBmysqldump\fR to retrieve the table contents to obtain incorrect contents or fail\&. .sp +The +\fB\-\-single\-transaction\fR +option and the +\fB\-\-lock\-tables\fR +option are mutually exclusive because +LOCK TABLES +causes any pending transactions to be committed implicitly\&. +.sp This option is not supported for MySQL Cluster tables; the results cannot be guaranteed to be consistent due to the fact that the NDBCLUSTER storage engine supports only the @@ -1546,15 +1633,9 @@ transaction isolation level\&. You shoul NDB backup and restore instead\&. .sp -The +To dump large tables, you should combine the \fB\-\-single\-transaction\fR -option and the -\fB\-\-lock\-tables\fR -option are mutually exclusive, because -LOCK TABLES -causes any pending transactions to be committed implicitly\&. -.sp -To dump large tables, you should combine this option with +option with \fB\-\-quick\fR\&. .RE .sp @@ -1624,7 +1705,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -1640,29 +1721,15 @@ Section\ \&5.5.7.3, \(lqSSL Command Opti \fB\-\-tab=\fR\fB\fIpath\fR\fR, \fB\-T \fR\fB\fIpath\fR\fR .sp -Produce tab\-separated data files\&. For each dumped table, +Produce tab\-separated text\-format data files\&. For each dumped table, \fBmysqldump\fR creates a \fItbl_name\fR\&.sql file that contains the CREATE TABLE -statement that creates the table, and a +statement that creates the table, and the server writes a \fItbl_name\fR\&.txt file that contains its data\&. The option value is the directory in which to write the files\&. -.sp -By default, the -\&.txt -data files are formatted using tab characters between column values and a newline at the end of each line\&. The format can be specified explicitly using the -\fB\-\-fields\-\fR\fB\fIxxx\fR\fR -and -\fB\-\-lines\-terminated\-by\fR -options\&. -.sp -As of MySQL 5\&.1\&.38, column values are written converted to the character set specified by the -\fB\-\-default\-character\-set\fR -option\&. Prior to 5\&.1\&.38 or if no such option is present, values are dumped using the -binary -character set\&. In effect, there is no character set conversion\&. If a table contains columns in several character sets, the output data file will as well and you may not be able to reload the file correctly\&. .if n \{\ .sp .\} @@ -1684,6 +1751,19 @@ FILE privilege, and the server must have permission to write files in the directory that you specify\&. .sp .5v .RE +By default, the +\&.txt +data files are formatted using tab characters between column values and a newline at the end of each line\&. The format can be specified explicitly using the +\fB\-\-fields\-\fR\fB\fIxxx\fR\fR +and +\fB\-\-lines\-terminated\-by\fR +options\&. +.sp +As of MySQL 5\&.1\&.38, column values are converted to the character set specified by the +\fB\-\-default\-character\-set\fR +option\&. Prior to 5\&.1\&.38 or if no such option is present, values are dumped using the +binary +character set\&. In effect, there is no character set conversion\&. If a table contains columns in several character sets, the output data file will as well and you may not be able to reload the file correctly\&. .RE .sp .RS 4 @@ -1719,7 +1799,7 @@ regards all name arguments following the .\" triggers option: mysqldump \fB\-\-triggers\fR .sp -Dump triggers for each dumped table\&. This option is enabled by default; disable it with +Include triggers for each dumped table in the output\&. This option is enabled by default; disable it with \fB\-\-skip\-triggers\fR\&. .RE .sp @@ -1743,7 +1823,7 @@ sets its connection time zone to UTC and SET TIME_ZONE=\'+00:00\' to the dump file\&. Without this option, TIMESTAMP -columns are dumped and reloaded in the time zones local to the source and destination servers, which can cause the values to change\&. +columns are dumped and reloaded in the time zones local to the source and destination servers, which can cause the values to change if the servers are in different time zones\&. \fB\-\-tz\-utc\fR also protects against changes due to daylight saving time\&. \fB\-\-tz\-utc\fR @@ -1846,7 +1926,7 @@ Examples: .sp Write dump output as well\-formed XML\&. .sp -\fBNULL\fR\fB, \fR\fB\'NULL\'\fR\fB, and Empty Values\fR: For some column named +\fBNULL\fR\fB, \fR\fB\'NULL\'\fR\fB, and Empty Values\fR: For a column named \fIcolumn_name\fR, the NULL value, an empty string, and the string value @@ -1884,7 +1964,7 @@ Beginning with MySQL 5\&.1\&.12, the out \fBmysql\fR client when run using the \fB\-\-xml\fR -option also follows these rules\&. (See +option also follows the preceding rules\&. (See the section called \(lqMYSQL OPTIONS\(rq\&.) .sp Beginning with MySQL 5\&.1\&.18, XML output from @@ -1905,11 +1985,13 @@ shell> \fBmysqldump \-\-xml \-u root wor <field Field="CountryCode" Type="char(3)" Null="NO" Key="" Default="" Extra="" /> <field Field="District" Type="char(20)" Null="NO" Key="" Default="" Extra="" /> <field Field="Population" Type="int(11)" Null="NO" Key="" Default="0" Extra="" /> -<key Table="City" Non_unique="0" Key_name="PRIMARY" Seq_in_index="1" Column_name="ID" Collation="A" Cardinality="4079" -Null="" Index_type="BTREE" Comment="" /> -<options Name="City" Engine="MyISAM" Version="10" Row_format="Fixed" Rows="4079" Avg_row_length="67" Data_length="27329 -3" Max_data_length="18858823439613951" Index_length="43008" Data_free="0" Auto_increment="4080" Create_time="2007\-03\-31 01:47:01" Updat -e_time="2007\-03\-31 01:47:02" Collation="latin1_swedish_ci" Create_options="" Comment="" /> +<key Table="City" Non_unique="0" Key_name="PRIMARY" Seq_in_index="1" Column_name="ID" +Collation="A" Cardinality="4079" Null="" Index_type="BTREE" Comment="" /> +<options Name="City" Engine="MyISAM" Version="10" Row_format="Fixed" Rows="4079" +Avg_row_length="67" Data_length="273293" Max_data_length="18858823439613951" +Index_length="43008" Data_free="0" Auto_increment="4080" +Create_time="2007\-03\-31 01:47:01" Update_time="2007\-03\-31 01:47:02" +Collation="latin1_swedish_ci" Create_options="" Comment="" /> </table_structure> <table_data name="City"> <row> @@ -1964,10 +2046,13 @@ The maximum size of the buffer for clien .\} net_buffer_length .sp -The initial size of the buffer for client/server communication\&. When creating multiple\-row\-insert statements (as with option +The initial size of the buffer for client/server communication\&. When creating multiple\-row +INSERT +statements (as with the \fB\-\-extended\-insert\fR or -\fB\-\-opt\fR), +\fB\-\-opt\fR +option), \fBmysqldump\fR creates rows up to net_buffer_length @@ -1976,9 +2061,9 @@ net_buffer_length variable in the MySQL server is at least this large\&. .RE .PP -The most common use of +A common use of \fBmysqldump\fR -is probably for making a backup of an entire database: +is for making a backup of an entire database: .sp .if n \{\ .RS 4 @@ -1990,7 +2075,7 @@ shell> \fBmysqldump \fR\fB\fIdb_name\fR\ .RE .\} .PP -You can read the dump file back into the server like this: +You can load the dump file back into the server like this: .sp .if n \{\ .RS 4 @@ -2072,7 +2157,7 @@ shell> \fBmysqldump \-\-all\-databases \ This backup acquires a global read lock on all tables (using FLUSH TABLES WITH READ LOCK) at the beginning of the dump\&. As soon as this lock has been acquired, the binary log coordinates are read and the lock is released\&. If long updating statements are running when the FLUSH -statement is issued, the MySQL server may get stalled until those statements finish\&. After that, the dump becomes lock\-free and does not disturb reads and writes on the tables\&. If the update statements that the MySQL server receives are short (in terms of execution time), the initial lock period should not be noticeable, even with many updates\&. +statement is issued, the MySQL server may get stalled until those statements finish\&. After that, the dump becomes lock free and does not disturb reads and writes on the tables\&. If the update statements that the MySQL server receives are short (in terms of execution time), the initial lock period should not be noticeable, even with many updates\&. .PP For point\-in\-time recovery (also known as \(lqroll\-forward,\(rq @@ -2106,13 +2191,13 @@ The \fB\-\-master\-data\fR and \fB\-\-single\-transaction\fR -options can be used simultaneously, which provides a convenient way to make an online backup suitable for point\-in\-time recovery if tables are stored using the +options can be used simultaneously, which provides a convenient way to make an online backup suitable for use prior to point\-in\-time recovery if tables are stored using the InnoDB storage engine\&. .PP For more information on making backups, see -Section\ \&6.1, \(lqDatabase Backup Methods\(rq, and -Section\ \&6.2, \(lqExample Backup and Recovery Strategy\(rq\&. +Section\ \&6.2, \(lqDatabase Backup Methods\(rq, and +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .\" mysqldump: views .\" mysqldump: problems .\" mysqldump: workarounds @@ -2122,7 +2207,7 @@ Section\ \&D.4, \(lqRestrictions on View .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -2134,7 +2219,7 @@ You should have received a copy of the G .IP " 1." 4 Bug#30123 .RS 4 -\%http://bugs.mysql.com/30123 +\%http://bugs.mysql.com/bug.php?id=30123 .RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, === modified file 'man/mysqldumpslow.1' --- a/man/mysqldumpslow.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqldumpslow.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqldumpslow\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLDUMPSLOW\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLDUMPSLOW\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -60,7 +60,7 @@ shell> \fBmysqldumpslow [\fR\fB\fIoption .\} .PP \fBmysqldumpslow\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -309,7 +309,7 @@ Count: 3 Time=2\&.13s (6s) Lock=0\&.00 .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlhotcopy.1' --- a/man/mysqlhotcopy.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlhotcopy.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlhotcopy\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLHOTCOPY\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLHOTCOPY\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -32,18 +32,28 @@ mysqlhotcopy \- a database backup progra .PP \fBmysqlhotcopy\fR is a Perl script that was originally written and contributed by Tim Bunce\&. It uses -LOCK TABLES, -FLUSH TABLES, and +FLUSH TABLES, +LOCK TABLES, and cp or scp -to make a database backup quickly\&. It is the fastest way to make a backup of the database or single tables, but it can be run only on the same machine where the database directories are located\&. +to make a database backup\&. It is a fast way to make a backup of the database or single tables, but it can be run only on the same machine where the database directories are located\&. \fBmysqlhotcopy\fR works only for backing up MyISAM and ARCHIVE tables\&. It runs on Unix and NetWare\&. +.PP +To use +\fBmysqlhotcopy\fR, you must have read access to the files for the tables that you are backing up, the +SELECT +privilege for those tables, the +RELOAD +privilege (to be able to execute +FLUSH TABLES), and the +LOCK TABLES +privilege (to be able to lock the tables)\&. .sp .if n \{\ .RS 4 @@ -90,7 +100,11 @@ shell> \fBmysqlhotcopy \fR\fB\fIdb_name\ .\} .PP \fBmysqlhotcopy\fR -supports the following options: +supports the following options, which can be specified on the command line or in the +[mysqlhotcopy] +and +[client] +option file groups\&. .sp .RS 4 .ie n \{\ @@ -275,7 +289,8 @@ Do not delete previous (renamed) target .sp The method for copying files (cp or -scp)\&. +scp)\&. The default is +cp\&. .RE .sp .RS 4 @@ -290,7 +305,9 @@ scp)\&. .\" noindices option: mysqlhotcopy \fB\-\-noindices\fR .sp -Do not include full index files in the backup\&. This makes the backup smaller and faster\&. The indexes for reloaded tables can be reconstructed later with +Do not include full index files for +MyISAM +tables in the backup\&. This makes the backup smaller and faster\&. The indexes for reloaded tables can be reconstructed later with \fBmyisamchk \-rq\fR\&. .RE .sp @@ -307,10 +324,10 @@ Do not include full index files in the b \fB\-\-password=\fR\fB\fIpassword\fR\fR, \fB\-p\fR\fB\fIpassword\fR\fR .sp -The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. You can use an option file to avoid giving the password on the command line\&. +The password to use when connecting to the server\&. The password value is not optional for this option, unlike for other MySQL programs\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -423,7 +440,8 @@ file after locking all the tables\&. \fB\-\-socket=\fR\fB\fIpath\fR\fR, \fB\-S \fR\fB\fIpath\fR\fR .sp -The Unix socket file to use for the connection\&. +The Unix socket file to use for connections to +localhost\&. .RE .sp .RS 4 @@ -438,7 +456,7 @@ The Unix socket file to use for the conn .\" suffix option: mysqlhotcopy \fB\-\-suffix=\fR\fB\fIstr\fR\fR .sp -The suffix for names of copied databases\&. +The suffix to use for names of copied databases\&. .RE .sp .RS 4 @@ -473,23 +491,6 @@ The temporary directory\&. The default i The MySQL user name to use when connecting to the server\&. .RE .PP -\fBmysqlhotcopy\fR -reads the -[client] -and -[mysqlhotcopy] -option groups from option files\&. -.PP -To execute -\fBmysqlhotcopy\fR, you must have access to the files for the tables that you are backing up, the -SELECT -privilege for those tables, the -RELOAD -privilege (to be able to execute -FLUSH TABLES), and the -LOCK TABLES -privilege (to be able to lock the tables)\&. -.PP Use perldoc for additional @@ -512,7 +513,7 @@ shell> \fBperldoc mysqlhotcopy\fR .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlimport.1' --- a/man/mysqlimport.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlimport.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlimport\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLIMPORT\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLIMPORT\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -64,8 +64,18 @@ patient all would be imported into a table named patient\&. .PP -\fBmysqlimport\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +For additional information about +\fBmysqldump\fR, see +Section\ \&6.4, \(lqUsing mysqldump for Backups\(rq\&. +.PP +\fBmysqldump\fR +supports the following options, which can be specified on the command line or in the +[mysqldump] +and +[client] +option file groups\&. +\fBmysqldump\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -97,7 +107,7 @@ Display a help message and exit\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -197,7 +207,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -379,8 +389,9 @@ tables for writing before processing any .sp Use LOW_PRIORITY -when loading the table\&. This affects only storage engines that use only table\-level locking (MyISAM, -MEMORY, +when loading the table\&. This affects only storage engines that use only table\-level locking (such as +MyISAM, +MEMORY, and MERGE)\&. .RE .sp @@ -405,10 +416,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlimport\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -424,7 +437,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -529,7 +542,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -633,7 +646,7 @@ shell> \fBmysql \-e \'SELECT * FROM impt .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlmanager.8' --- a/man/mysqlmanager.8 2009-12-01 07:24:05 +0000 +++ b/man/mysqlmanager.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlmanager\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLMANAGER\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLMANAGER\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -153,7 +153,7 @@ in the directory where Instance Manager option\&. .PP \fBmysqlmanager\fR -supports the options described in the following list\&. The options for managing entries in the password file are described further in +supports the following options\&. The options for managing entries in the password file are described further in the section called \(lqINSTANCE MANAGER USER AND PASSWORD MANAGEMENT\(rq\&. .sp .RS 4 @@ -2062,7 +2062,7 @@ Query OK, 0 rows affected (0\&.04 sec) .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlshow.1' --- a/man/mysqlshow.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlshow.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlshow\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLSHOW\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLSHOW\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -39,7 +39,7 @@ client can be used to quickly see which provides a command\-line interface to several SQL SHOW statements\&. See -Section\ \&12.5.5, \(lqSHOW Syntax\(rq\&. The same information can be obtained by using those statements directly\&. For example, you can issue them from the +Section\ \&12.4.5, \(lqSHOW Syntax\(rq\&. The same information can be obtained by using those statements directly\&. For example, you can issue them from the \fBmysql\fR client program\&. .PP @@ -112,7 +112,13 @@ shows you only the table names that matc last on the command line as a separate argument\&. .PP \fBmysqlshow\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlshow] +and +[client] +option file groups\&. +\fBmysqlshow\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -144,7 +150,7 @@ Display a help message and exit\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -244,7 +250,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -300,10 +306,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlshow\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -319,7 +327,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -406,7 +414,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -475,7 +483,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlslap.1' --- a/man/mysqlslap.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlslap.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlslap\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLSLAP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLSLAP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -151,7 +151,13 @@ mysqlslap \-\-concurrency=5 \e .\} .PP \fBmysqlslap\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlslap] +and +[client] +option file groups\&. +\fBmysqlslap\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -615,7 +621,24 @@ is specified\&. .\" number-of-queries option: mysqlslap \fB\-\-number\-of\-queries=\fR\fB\fIN\fR\fR .sp -Limit each client to approximately this number of queries\&. This option was added in MySQL 5\&.1\&.5\&. +Limit each client to approximately this many queries\&. Query counting takes into account the statement delimiter\&. For example, if you invoke +\fBmysqlslap\fR +as follows, the +; +delimiter is recognized so that each instance of the query string counts as two queries\&. As a result, 5 rows (not 10) are inserted\&. +.sp +.if n \{\ +.RS 4 +.\} +.nf +shell> \fBmysqlslap \-\-delimiter=";" \-\-number\-of\-queries=10\fR + \fB\-\-query="use test;insert into t values(null)"\fR +.fi +.if n \{\ +.RE +.\} +.sp +This option was added in MySQL 5\&.1\&.5\&. .RE .sp .RS 4 @@ -653,15 +676,15 @@ The password to use when connecting to t have a space between the option and the password\&. If you omit the \fIpassword\fR value following the -.\" mysqlslap: password option -.\" password option: mysqlslap \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlslap\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -677,7 +700,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -897,7 +920,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -971,7 +994,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqltest.1' --- a/man/mysqltest.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqltest.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqltest\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQLTEST\FR" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQLTEST\FR" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -175,31 +175,11 @@ The base directory for tests\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysqltest: big-test option -.\" big-test option: mysqltest -\fB\-\-big\-test\fR, -\fB\-B\fR -.sp -Define the -\fBmysqltest\fR -variable -$BIG_TEST -as 1\&. This option was removed in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.13\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysqltest: character-sets-dir option .\" character-sets-dir option: mysqltest \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp -The directory where character sets are installed\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.32, and 5\&.1\&.13\&. +The directory where character sets are installed\&. .RE .sp .RS 4 @@ -226,12 +206,11 @@ Compress all information sent between th .sp -1 .IP \(bu 2.3 .\} -.\" mysqltest: cursor-protocol option +.\" mysqltest: currsor-protocol option .\" cursor-protocol option: mysqltest \fB\-\-cursor\-protocol\fR .sp -Use cursors for prepared statements (implies -\fB\-\-ps\-protocol\fR)\&. This option was added in MySQL 5\&.0\&.19\&. +Use cursors for prepared statements\&. .RE .sp .RS 4 @@ -281,7 +260,7 @@ value is .\" debug-check option: mysqltest \fB\-\-debug\-check\fR .sp -Print some debugging information when the program exits\&. This option was added in MySQL 5\&.1\&.21\&. +Print some debugging information when the program exits\&. .RE .sp .RS 4 @@ -296,7 +275,7 @@ Print some debugging information when th .\" debug-info option: mysqltest \fB\-\-debug\-info\fR .sp -Print debugging information and memory and CPU usage statistics when the program exits\&. This option was added in MySQL 5\&.1\&.14\&. +Print debugging information and memory and CPU usage statistics when the program exits\&. .RE .sp .RS 4 @@ -332,7 +311,7 @@ Include the contents of the given file b \fBmysqltest\fR test files\&. This option has the same effect as putting a \-\-source \fIfile_name\fR -command as the first line of the test file\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.7\&. +command as the first line of the test file\&. .RE .sp .RS 4 @@ -347,7 +326,7 @@ command as the first line of the test fi .\" logdir option: mysqltest \fB\-\-logdir=\fR\fB\fIdir_name\fR\fR .sp -The directory to use for log files\&. This option was added in MySQL 5\&.1\&.14\&. +The directory to use for log files\&. .RE .sp .RS 4 @@ -363,7 +342,7 @@ The directory to use for log files\&. Th \fB\-\-mark\-progress\fR .sp Write the line number and elapsed time to -\fItest_file\fR\&.progress\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.12\&. +\fItest_file\fR\&.progress\&. .RE .sp .RS 4 @@ -378,7 +357,24 @@ Write the line number and elapsed time t .\" max-connect-retries option: mysqltest \fB\-\-max\-connect\-retries=\fR\fB\fInum\fR\fR .sp -The maximum number of connection attempts when connecting to server\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.23, and 5\&.1\&.11\&. +The maximum number of connection attempts when connecting to server\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysqltest: max-connections option +.\" max-connections option: mysqltest +\fB\-\-max\-connections=\fR\fB\fInum\fR\fR +.sp +The maximum number of simultaneous server connections per client (that is, per test)\&. If not set, the maximum is 128\&. Minimum allowed limit is 8, maximum is 5120\&. +.sp +This option is available from MySQL 5\&.1\&.45\&. .RE .sp .RS 4 @@ -393,7 +389,7 @@ The maximum number of connection attempt .\" no-defaults option: mysqltest \fB\-\-no\-defaults\fR .sp -Do not read default options from any option files\&. +Do not read default options from any option files\&. If used, this must be the first option\&. .RE .sp .RS 4 @@ -484,7 +480,8 @@ Suppress all normal output\&. This is a .sp Record the output that results from running the test file into the file named by the \fB\-\-result\-file\fR -option, if that option is given\&. +option, if that option is given\&. It is an error to use this option without also using +\fB\-\-result\-file\fR\&. .RE .sp .RS 4 @@ -516,7 +513,9 @@ treats the test actual and expected resu .\} If the test produces no results, \fBmysqltest\fR -exits with an error message to that effect\&. +exits with an error message to that effect, unless +\fB\-\-result\-file\fR +is given and the named file is an empty file\&. .RE .sp .RS 4 @@ -551,7 +550,7 @@ reads the expected results from the give \fBmysqltest\fR writes a \&.reject -file in the same directory as the result file and exits with an error\&. +file in the same directory as the result file, outputs a diff of the two files, and exits with an error\&. .RE .sp .RS 4 @@ -695,7 +694,22 @@ localhost .sp Execute DML statements within a stored procedure\&. For every DML statement, \fBmysqltest\fR -creates and invokes a stored procedure that executes the statement rather than executing the statement directly\&. This option was added in MySQL 5\&.0\&.19\&. +creates and invokes a stored procedure that executes the statement rather than executing the statement directly\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysqltest: tail-lines option +.\" tail-lines option: mysqltest +\fB\-\-tail\-lines=\fR\fB\fInn\fR\fR +.sp +Specify how many lines of the result to include in the output if the test fails because an SQL statement fails\&. The default is 0, meaning no lines of result printed\&. .RE .sp .RS 4 @@ -727,7 +741,9 @@ Read test input from this file\&. The de \fB\-\-timer\-file=\fR\fB\fIfile_name\fR\fR, \fB\-m \fR\fB\fIfile_name\fR\fR .sp -The file where the timing in microseconds is written\&. +If given, the number of millisecond spent running the test will be written to this file\&. This is used by +\fBmysql\-test\-run\&.pl\fR +for its reporting\&. .RE .sp .RS 4 @@ -743,7 +759,7 @@ The file where the timing in microsecond \fB\-\-tmpdir=\fR\fB\fIdir_name\fR\fR, \fB\-t \fR\fB\fIdir_name\fR\fR .sp -The temporary directory where socket files are put\&. +The temporary directory where socket files are created\&. .RE .sp .RS 4 @@ -775,7 +791,7 @@ The MySQL user name to use when connecti \fB\-\-verbose\fR, \fB\-v\fR .sp -Verbose mode\&. Print out more information what the program does\&. +Verbose mode\&. Print out more information about what the program does\&. .RE .sp .RS 4 @@ -813,7 +829,7 @@ statement is wrapped inside a view\&. Th .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/ndbd.8' --- a/man/ndbd.8 2009-12-01 07:24:05 +0000 +++ b/man/ndbd.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBndbd\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBNDBD\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBNDBD\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -93,7 +93,7 @@ T}:T{ 5\&.1\&.12 T} T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-bind\-address=name T} @@ -137,7 +137,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-daemon T} @@ -189,7 +189,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-initial T} @@ -298,7 +298,7 @@ Backup files that have already been crea .IP \(bu 2.3 .\} MySQL Cluster Disk Data files (see -Section\ \&17.5.9, \(lqMySQL Cluster Disk Data Tables\(rq)\&. +Section\ \&17.5.10, \(lqMySQL Cluster Disk Data Tables\(rq)\&. .RE .RS 4 .sp @@ -334,7 +334,7 @@ T}:T{ 5\&.1\&.11 T} T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-initial\-start T} @@ -358,20 +358,27 @@ T} This option is used when performing a partial initial start of the cluster\&. Each node should be started with this option, as well as \fB\-\-nowait\-nodes\fR\&. .sp -For example, suppose you have a 4\-node cluster whose data nodes have the IDs 2, 3, 4, and 5, and you wish to perform a partial initial start using only nodes 2, 4, and 5 \(em that is, omitting node 3: +Suppose that you have a 4\-node cluster whose data nodes have the IDs 2, 3, 4, and 5, and you wish to perform a partial initial start using only nodes 2, 4, and 5 \(em that is, omitting node 3: .sp .if n \{\ .RS 4 .\} .nf -ndbd \-\-ndbd\-nodeid=2 \-\-nowait\-nodes=3 \-\-initial\-start -ndbd \-\-ndbd\-nodeid=4 \-\-nowait\-nodes=3 \-\-initial\-start -ndbd \-\-ndbd\-nodeid=5 \-\-nowait\-nodes=3 \-\-initial\-start +shell> \fBndbd \-\-ndb\-nodeid=2 \-\-nowait\-nodes=3 \-\-initial\-start\fR +shell> \fBndbd \-\-ndb\-nodeid=4 \-\-nowait\-nodes=3 \-\-initial\-start\fR +shell> \fBndbd \-\-ndb\-nodeid=5 \-\-nowait\-nodes=3 \-\-initial\-start\fR .fi .if n \{\ .RE .\} .sp +Prior to MySQL 5\&.1\&.19, it was not possible to perform DDL operations involving Disk Data tables on a partially started cluster\&. (See +\m[blue]\fBBug#24631\fR\m[]\&\s-2\u[1]\d\s+2\&.) +.sp +When using this option, you must also specify the node ID for the data node being started with the +\fB\-\-ndb\-nodeid\fR +option\&. +.sp This option was added in MySQL 5\&.1\&.11\&. .if n \{\ .sp @@ -385,8 +392,11 @@ This option was added in MySQL 5\&.1\&.1 \fBImportant\fR .ps -1 .br -Prior to MySQL 5\&.1\&.19, it was not possible to perform DDL operations involving Disk Data tables on a partially started cluster\&. (See -\m[blue]\fBBug#24631\fR\m[]\&\s-2\u[1]\d\s+2\&.) +Do not confuse this option with the +\fB\-\-nowait\-nodes\fR +option added for +\fBndb_mgmd\fR +in MySQL Cluster NDB 7\&.0\&.10, which can be used to allow a cluster configured with multiple management servers to be started without all management servers being online\&. .sp .5v .RE .RE @@ -412,10 +422,10 @@ l l s T{ \fBVersion Introduced\fR T}:T{ -5\&.1\&.11 +5\&.1\&.9 T} T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-nowait\-nodes=list T} @@ -470,9 +480,12 @@ allbox tab(:); l l s l l s ^ l l +^ l l +l l s +^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-nodaemon T} @@ -491,6 +504,21 @@ T} T}:T{ FALSE T} +T{ +\ \& +T}:T{ +\fBPermitted Values \fR +T} +:T{ +\fBType\fR (windows) +T}:T{ +boolean +T} +:T{ +\fBDefault\fR +T}:T{ +TRUE +T} .TE .sp 1 Instructs @@ -527,7 +555,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-nostart T} @@ -759,7 +787,7 @@ Section\ \&17.1.5, \(lqKnown Limitations .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -771,12 +799,12 @@ You should have received a copy of the G .IP " 1." 4 Bug#24631 .RS 4 -\%http://bugs.mysql.com/24631 +\%http://bugs.mysql.com/bug.php?id=24631 .RE .IP " 2." 4 Bug#45588 .RS 4 -\%http://bugs.mysql.com/45588 +\%http://bugs.mysql.com/bug.php?id=45588 .RE .IP " 3." 4 ndbd Error Messages === modified file 'man/ndbd_redo_log_reader.1' --- a/man/ndbd_redo_log_reader.1 2009-12-01 07:24:05 +0000 +++ b/man/ndbd_redo_log_reader.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBndbd_redo_log_reader\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBNDBD_REDO_LOG_REA" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBNDBD_REDO_LOG_REA" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -77,7 +77,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-noprint T} @@ -116,7 +116,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-nocheck T} @@ -154,7 +154,7 @@ must be run on a cluster data node, sinc .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/ndbmtd.8' --- a/man/ndbmtd.8 2009-12-01 07:24:05 +0000 +++ b/man/ndbmtd.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBndbmtd\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBNDBMTD\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBNDBMTD\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -74,7 +74,7 @@ Prior to MySQL Cluster NDB 7\&.0\&.6, th \fBndbmtd\fR with MySQL Cluster Disk Data tables\&. If you wish to use multi\-threaded data nodes with disk\-based NDB -tables, you should insure that you are running MySQL Cluster NDB 7\&.0\&.6 or later\&. (\m[blue]\fBBug#41915\fR\m[]\&\s-2\u[1]\d\s+2, +tables, you should ensure that you are running MySQL Cluster NDB 7\&.0\&.6 or later\&. (\m[blue]\fBBug#41915\fR\m[]\&\s-2\u[1]\d\s+2, \m[blue]\fBBug#44915\fR\m[]\&\s-2\u[2]\d\s+2) .PP Using @@ -356,7 +356,7 @@ concurrently on different data nodes in .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -368,12 +368,12 @@ You should have received a copy of the G .IP " 1." 4 Bug#41915 .RS 4 -\%http://bugs.mysql.com/41915 +\%http://bugs.mysql.com/bug.php?id=41915 .RE .IP " 2." 4 Bug#44915 .RS 4 -\%http://bugs.mysql.com/44915 +\%http://bugs.mysql.com/bug.php?id=44915 .RE .IP " 3." 4 ndbd Error Messages === modified file 'man/perror.1' --- a/man/perror.1 2009-12-01 07:24:05 +0000 +++ b/man/perror.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBperror\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBPERROR\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBPERROR\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -96,7 +96,7 @@ shell> \fBperror \-\-ndb \fR\fB\fIerrorc Note that the meaning of system error messages may be dependent on your operating system\&. A given error code may mean different things on different operating systems\&. .PP \fBperror\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -181,7 +181,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/replace.1' --- a/man/replace.1 2009-12-01 07:24:05 +0000 +++ b/man/replace.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBreplace\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBREPLACE\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBREPLACE\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -90,7 +90,7 @@ program is used by \fBmsql2mysql\fR(1)\&. .PP \fBreplace\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -160,7 +160,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/resolve_stack_dump.1' --- a/man/resolve_stack_dump.1 2009-12-01 07:24:05 +0000 +++ b/man/resolve_stack_dump.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBresolve_stack_dump\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBRESOLVE_STACK_DUM" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBRESOLVE_STACK_DUM" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -49,7 +49,7 @@ command\&. The numeric dump file should \fBmysqld\fR\&. If no numeric dump file is named on the command line, the stack trace is read from the standard input\&. .PP \fBresolve_stack_dump\fR -supports the options described in the following list\&. +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -117,7 +117,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/resolveip.1' --- a/man/resolveip.1 2009-12-01 07:24:05 +0000 +++ b/man/resolveip.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBresolveip\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBRESOLVEIP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBRESOLVEIP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -45,7 +45,7 @@ shell> \fBresolveip [\fR\fB\fIoptions\fR .\} .PP \fBresolveip\fR -supports the options described in the following list\&. +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -99,7 +99,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'scripts/fill_help_tables.sql' --- a/scripts/fill_help_tables.sql 2009-12-01 07:24:05 +0000 +++ b/scripts/fill_help_tables.sql 2010-04-28 13:06:11 +0000 @@ -1,4 +1,4 @@ --- Copyright (C) 2000-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc. +-- Copyright (c) 2003, &year, Oracle and/or its affiliates. All rights reserved. -- -- This program is free software; you can redistribute it and/or modify -- it under the terms of the GNU General Public License as published by @@ -11,7 +11,7 @@ -- -- You should have received a copy of the GNU General Public License -- along with this program; if not, write to the Free Software --- Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +-- Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -- DO NOT EDIT THIS FILE. It is generated automatically. @@ -87,24 +87,24 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (18,25,'SHOW CREATE PROCEDURE','Syntax:\nSHOW CREATE PROCEDURE proc_name\n\nThis statement is a MySQL extension. It returns the exact string that\ncan be used to re-create the named stored procedure. A similar\nstatement, SHOW CREATE FUNCTION, displays information about stored\nfunctions (see [HELP SHOW CREATE FUNCTION]).\n\nBoth statements require that you be the owner of the routine or have\nSELECT access to the mysql.proc table. If you do not have privileges\nfor the routine itself, the value displayed for the Create Procedure or\nCreate Function field will be NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-create-procedure.html\n\n','mysql> SHOW CREATE PROCEDURE test.simpleproc\\G\n*************************** 1. row ***************************\n Procedure: simpleproc\n sql_mode:\n Create Procedure: CREATE PROCEDURE `simpleproc`(OUT param1 INT)\n BEGIN\n SELECT COUNT(*) INTO param1 FROM t;\n END\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n\nmysql> SHOW CREATE FUNCTION test.hello\\G\n*************************** 1. row ***************************\n Function: hello\n sql_mode:\n Create Function: CREATE FUNCTION `hello`(s CHAR(20))\n RETURNS CHAR(50)\n RETURN CONCAT(\'Hello, \',s,\'!\')\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n','http://dev.mysql.com/doc/refman/5.1/en/show-create-procedure.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (19,20,'INTEGER','INTEGER[(M)] [UNSIGNED] [ZEROFILL]\n\nThis type is a synonym for INT.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (20,35,'LOWER','Syntax:\nLOWER(str)\n\nReturns the string str with all characters changed to lowercase\naccording to the current character set mapping. The default is latin1\n(cp1252 West European).\n\nmysql> SELECT LOWER(\'QUADRATICALLY\');\n -> \'quadratically\'\n\nLOWER() (and UPPER()) are ineffective when applied to binary strings\n(BINARY, VARBINARY, BLOB). To perform lettercase conversion, convert\nthe string to a nonbinary string:\n\nmysql> SET @str = BINARY \'New York\';\nmysql> SELECT LOWER(@str), LOWER(CONVERT(@str USING latin1));\n+-------------+-----------------------------------+\n| LOWER(@str) | LOWER(CONVERT(@str USING latin1)) |\n+-------------+-----------------------------------+\n| New York | new york |\n+-------------+-----------------------------------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (21,25,'SHOW COLUMNS','Syntax:\nSHOW [FULL] COLUMNS {FROM | IN} tbl_name [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW COLUMNS displays information about the columns in a given table.\nIt also works for views. The LIKE clause, if present, indicates which\ncolumn names to match. The WHERE clause can be given to select rows\nusing more general conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nmysql> SHOW COLUMNS FROM City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nIf the data types differ from what you expect them to be based on a\nCREATE TABLE statement, note that MySQL sometimes changes data types\nwhen you create or alter a table. The conditions under which this\noccurs are described in\nhttp://dev.mysql.com/doc/refman/5.1/en/silent-column-changes.html.\n\nT… FULL keyword causes the output to include the column collation and\ncomments, as well as the privileges you have for each column.\n\nYou can use db_name.tbl_name as an alternative to the tbl_name FROM\ndb_name syntax. In other words, these two statements are equivalent:\n\nmysql> SHOW COLUMNS FROM mytable FROM mydb;\nmysql> SHOW COLUMNS FROM mydb.mytable;\n\nSHOW COLUMNS displays the following values for each table column:\n\nField indicates the column name.\n\nType indicates the column data type.\n\nCollation indicates the collation for nonbinary string columns, or NULL\nfor other columns. This value is displayed only if you use the FULL\nkeyword.\n\nThe Null field contains YES if NULL values can be stored in the column,\nNO if not.\n\nThe Key field indicates whether the column is indexed:\n\no If Key is empty, the column either is not indexed or is indexed only\n as a secondary column in a multiple-column, nonunique index.\n\no If Key is PRI, the column is a PRIMARY KEY or is one of the columns\n in a multiple-column PRIMARY KEY.\n\no If Key is UNI, the column is the first column of a unique-valued\n index that cannot contain NULL values.\n\no If Key is MUL, multiple occurrences of a given value are allowed\n within the column. The column is the first column of a nonunique\n index or a unique-valued index that can contain NULL values.\n\nIf more than one of the Key values applies to a given column of a\ntable, Key displays the one with the highest priority, in the order\nPRI, UNI, MUL.\n\nA UNIQUE index may be displayed as PRI if it cannot contain NULL values\nand there is no PRIMARY KEY in the table. A UNIQUE index may display as\nMUL if several columns form a composite UNIQUE index; although the\ncombination of the columns is unique, each column can still hold\nmultiple occurrences of a given value.\n\nThe Default field indicates the default value that is assigned to the\ncolumn.\n\nThe Extra field contains any additional information that is available\nabout a given column. The value is nonempty in these cases:\nauto_increment for columns that have the AUTO_INCREMENT attribute; as\nof MySQL 5.1.23, on update CURRENT_TIMESTAMP for TIMESTAMP columns that\nhave the ON UPDATE CURRENT_TIMESTAMP attribute.\n\nPrivileges indicates the privileges you have for the column. This value\nis displayed only if you use the FULL keyword.\n\nComment indicates any comment the column has. This value is displayed\nonly if you use the FULL keyword.\n\nSHOW FIELDS is a synonym for SHOW COLUMNS. You can also list a table\'s\ncolumns with the mysqlshow db_name tbl_name command.\n\nThe DESCRIBE statement provides information similar to SHOW COLUMNS.\nSee [HELP DESCRIBE].\n\nThe SHOW CREATE TABLE, SHOW TABLE STATUS, and SHOW INDEX statements\nalso provide information about tables. See [HELP SHOW].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-columns.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-columns.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (22,37,'CREATE TRIGGER','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n TRIGGER trigger_name trigger_time trigger_event\n ON tbl_name FOR EACH ROW trigger_stmt\n\nThis statement creates a new trigger. A trigger is a named database\nobject that is associated with a table, and that activates when a\nparticular event occurs for the table. The trigger becomes associated\nwith the table named tbl_name, which must refer to a permanent table.\nYou cannot associate a trigger with a TEMPORARY table or a view.\n\nCREATE TRIGGER requires the TRIGGER privilege for the table associated\nwith the trigger. (Before MySQL 5.1.6, this statement requires the\nSUPER privilege.)\n\nThe DEFINER clause determines the security context to be used when\nchecking access privileges at trigger activation time.\n\ntrigger_time is the trigger action time. It can be BEFORE or AFTER to\nindicate that the trigger activates before or after each row to be\nmodified.\n\ntrigger_event indicates the kind of statement that activates the\ntrigger. The trigger_event can be one of the following:\n\no INSERT: The trigger is activated whenever a new row is inserted into\n the table; for example, through INSERT, LOAD DATA, and REPLACE\n statements.\n\no UPDATE: The trigger is activated whenever a row is modified; for\n example, through UPDATE statements.\n\no DELETE: The trigger is activated whenever a row is deleted from the\n table; for example, through DELETE and REPLACE statements. However,\n DROP TABLE and TRUNCATE statements on the table do not activate this\n trigger, because they do not use DELETE. Dropping a partition does\n not activate DELETE triggers, either. See [HELP TRUNCATE TABLE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (21,25,'SHOW COLUMNS','Syntax:\nSHOW [FULL] COLUMNS {FROM | IN} tbl_name [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW COLUMNS displays information about the columns in a given table.\nIt also works for views. The LIKE clause, if present, indicates which\ncolumn names to match. The WHERE clause can be given to select rows\nusing more general conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nmysql> SHOW COLUMNS FROM City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nIf the data types differ from what you expect them to be based on a\nCREATE TABLE statement, note that MySQL sometimes changes data types\nwhen you create or alter a table. The conditions under which this\noccurs are described in\nhttp://dev.mysql.com/doc/refman/5.1/en/silent-column-changes.html.\n\nT… FULL keyword causes the output to include the column collation and\ncomments, as well as the privileges you have for each column.\n\nYou can use db_name.tbl_name as an alternative to the tbl_name FROM\ndb_name syntax. In other words, these two statements are equivalent:\n\nmysql> SHOW COLUMNS FROM mytable FROM mydb;\nmysql> SHOW COLUMNS FROM mydb.mytable;\n\nSHOW COLUMNS displays the following values for each table column:\n\nField indicates the column name.\n\nType indicates the column data type.\n\nCollation indicates the collation for nonbinary string columns, or NULL\nfor other columns. This value is displayed only if you use the FULL\nkeyword.\n\nThe Null field contains YES if NULL values can be stored in the column,\nNO if not.\n\nThe Key field indicates whether the column is indexed:\n\no If Key is empty, the column either is not indexed or is indexed only\n as a secondary column in a multiple-column, nonunique index.\n\no If Key is PRI, the column is a PRIMARY KEY or is one of the columns\n in a multiple-column PRIMARY KEY.\n\no If Key is UNI, the column is the first column of a UNIQUE index. (A\n UNIQUE index allows multiple NULL values, but you can tell whether\n the column allows NULL by checking the Null field.)\n\no If Key is MUL, the column is the first column of a nonunique index in\n which multiple occurrences of a given value are allowed within the\n column.\n\nIf more than one of the Key values applies to a given column of a\ntable, Key displays the one with the highest priority, in the order\nPRI, UNI, MUL.\n\nA UNIQUE index may be displayed as PRI if it cannot contain NULL values\nand there is no PRIMARY KEY in the table. A UNIQUE index may display as\nMUL if several columns form a composite UNIQUE index; although the\ncombination of the columns is unique, each column can still hold\nmultiple occurrences of a given value.\n\nThe Default field indicates the default value that is assigned to the\ncolumn.\n\nThe Extra field contains any additional information that is available\nabout a given column. The value is nonempty in these cases:\nauto_increment for columns that have the AUTO_INCREMENT attribute; as\nof MySQL 5.1.23, on update CURRENT_TIMESTAMP for TIMESTAMP columns that\nhave the ON UPDATE CURRENT_TIMESTAMP attribute.\n\nPrivileges indicates the privileges you have for the column. This value\nis displayed only if you use the FULL keyword.\n\nComment indicates any comment the column has. This value is displayed\nonly if you use the FULL keyword.\n\nSHOW FIELDS is a synonym for SHOW COLUMNS. You can also list a table\'s\ncolumns with the mysqlshow db_name tbl_name command.\n\nThe DESCRIBE statement provides information similar to SHOW COLUMNS.\nSee [HELP DESCRIBE].\n\nThe SHOW CREATE TABLE, SHOW TABLE STATUS, and SHOW INDEX statements\nalso provide information about tables. See [HELP SHOW].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-columns.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-columns.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (22,37,'CREATE TRIGGER','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n TRIGGER trigger_name trigger_time trigger_event\n ON tbl_name FOR EACH ROW trigger_stmt\n\nThis statement creates a new trigger. A trigger is a named database\nobject that is associated with a table, and that activates when a\nparticular event occurs for the table. The trigger becomes associated\nwith the table named tbl_name, which must refer to a permanent table.\nYou cannot associate a trigger with a TEMPORARY table or a view.\n\nCREATE TRIGGER requires the TRIGGER privilege for the table associated\nwith the trigger. If binary logging is enabled, the CREATE TRIGGER\nstatement might also require the SUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\nS… may also be required depending on the DEFINER value, as described\nlater. (Before MySQL 5.1.6, there is no TRIGGER privilege and this\nstatement requires the SUPER privilege in all cases.)\n\nThe DEFINER clause determines the security context to be used when\nchecking access privileges at trigger activation time.\n\ntrigger_time is the trigger action time. It can be BEFORE or AFTER to\nindicate that the trigger activates before or after each row to be\nmodified.\n\ntrigger_event indicates the kind of statement that activates the\ntrigger. The trigger_event can be one of the following:\n\no INSERT: The trigger is activated whenever a new row is inserted into\n the table; for example, through INSERT, LOAD DATA, and REPLACE\n statements.\n\no UPDATE: The trigger is activated whenever a row is modified; for\n example, through UPDATE statements.\n\no DELETE: The trigger is activated whenever a row is deleted from the\n table; for example, through DELETE and REPLACE statements. However,\n DROP TABLE and TRUNCATE TABLE statements on the table do not activate\n this trigger, because they do not use DELETE. Dropping a partition\n does not activate DELETE triggers, either. See [HELP TRUNCATE TABLE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (23,30,'MONTH','Syntax:\nMONTH(date)\n\nReturns the month for date, in the range 1 to 12 for January to\nDecember, or 0 for dates such as \'0000-00-00\' or \'2008-00-00\' that have\na zero month part.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MONTH(\'2008-02-03\');\n -> 2\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (24,20,'TINYINT','TINYINT[(M)] [UNSIGNED] [ZEROFILL]\n\nA very small integer. The signed range is -128 to 127. The unsigned\nrange is 0 to 255.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (25,25,'SHOW TRIGGERS','Syntax:\nSHOW TRIGGERS [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW TRIGGERS lists the triggers currently defined for tables in a\ndatabase (the default database unless a FROM clause is given). This\nstatement requires the TRIGGER privilege (prior to MySQL 5.1.22, it\nrequires the SUPER privilege). The LIKE clause, if present, indicates\nwhich table names to match and causes the statement to display triggers\nfor those tables. The WHERE clause can be given to select rows using\nmore general conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nFor the trigger ins_sum as defined in\nhttp://dev.mysql.com/doc/refman/5.1/en/triggers.html, the output of\nthis statement is as shown here:\n\nmysql> SHOW TRIGGERS LIKE \'acc%\'\\G\n*************************** 1. row ***************************\n Trigger: ins_sum\n Event: INSERT\n Table: account\n Statement: SET @sum = @sum + NEW.amount\n Timing: BEFORE\n Created: NULL\n sql_mode:\n Definer: myname@localhost\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n\ncharacter_set_client is the session value of the character_set_client\nsystem variable when the trigger was created. collation_connection is\nthe session value of the collation_connection system variable when the\ntrigger was created. Database Collation is the collation of the\ndatabase with which the trigger is associated. These columns were added\nin MySQL 5.1.21.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-triggers.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-triggers.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (26,14,'MASTER_POS_WAIT','Syntax:\nMASTER_POS_WAIT(log_name,log_pos[,timeout])\n\nThis function is useful for control of master/slave synchronization. It\nblocks until the slave has read and applied all updates up to the\nspecified position in the master log. The return value is the number of\nlog events the slave had to wait for to advance to the specified\nposition. The function returns NULL if the slave SQL thread is not\nstarted, the slave\'s master information is not initialized, the\narguments are incorrect, or an error occurs. It returns -1 if the\ntimeout has been exceeded. If the slave SQL thread stops while\nMASTER_POS_WAIT() is waiting, the function returns NULL. If the slave\nis past the specified position, the function returns immediately.\n\nIf a timeout value is specified, MASTER_POS_WAIT() stops waiting when\ntimeout seconds have elapsed. timeout must be greater than 0; a zero or\nnegative timeout means no timeout.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (27,35,'REGEXP','Syntax:\nexpr REGEXP pat, expr RLIKE pat\n\nPerforms a pattern match of a string expression expr against a pattern\npat. The pattern can be an extended regular expression. The syntax for\nregular expressions is discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/regexp.html. Returns 1 if expr\nmatches pat; otherwise it returns 0. If either expr or pat is NULL, the\nresult is NULL. RLIKE is a synonym for REGEXP, provided for mSQL\ncompatibility.\n\nThe pattern need not be a literal string. For example, it can be\nspecified as a string expression or table column.\n\n*Note*: Because MySQL uses the C escape syntax in strings (for example,\n"\\n" to represent the newline character), you must double any "\\" that\nyou use in your REGEXP strings.\n\nREGEXP is not case sensitive, except when used with binary strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/regexp.html\n\n','mysql> SELECT \'Monty!\' REGEXP \'m%y%%\';\n -> 0\nmysql> SELECT \'Monty!\' REGEXP \'.*\';\n -> 1\nmysql> SELECT \'new*\\n*line\' REGEXP \'new\\\\*.\\\\*line\';\n -> 1\nmysql> SELECT \'a\' REGEXP \'A\', \'a\' REGEXP BINARY \'A\';\n -> 1 0\nmysql> SELECT \'a\' REGEXP \'^[a-d]\';\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/regexp.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (27,35,'REGEXP','Syntax:\nexpr REGEXP pat, expr RLIKE pat\n\nPerforms a pattern match of a string expression expr against a pattern\npat. The pattern can be an extended regular expression. The syntax for\nregular expressions is discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/regexp.html. Returns 1 if expr\nmatches pat; otherwise it returns 0. If either expr or pat is NULL, the\nresult is NULL. RLIKE is a synonym for REGEXP, provided for mSQL\ncompatibility.\n\nThe pattern need not be a literal string. For example, it can be\nspecified as a string expression or table column.\n\n*Note*: Because MySQL uses the C escape syntax in strings (for example,\n"\\n" to represent the newline character), you must double any "\\" that\nyou use in your REGEXP strings.\n\nREGEXP is not case sensitive, except when used with binary strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/regexp.html\n\n','mysql> SELECT \'Monty!\' REGEXP \'.*\';\n -> 1\nmysql> SELECT \'new*\\n*line\' REGEXP \'new\\\\*.\\\\*line\';\n -> 1\nmysql> SELECT \'a\' REGEXP \'A\', \'a\' REGEXP BINARY \'A\';\n -> 1 0\nmysql> SELECT \'a\' REGEXP \'^[a-d]\';\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/regexp.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (28,22,'IF STATEMENT','Syntax:\nIF search_condition THEN statement_list\n [ELSEIF search_condition THEN statement_list] ...\n [ELSE statement_list]\nEND IF\n\nIF implements a basic conditional construct. If the search_condition\nevaluates to true, the corresponding SQL statement list is executed. If\nno search_condition matches, the statement list in the ELSE clause is\nexecuted. Each statement_list consists of one or more statements.\n\n*Note*: There is also an IF() function, which differs from the IF\nstatement described here. See\nhttp://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html.\n\…: http://dev.mysql.com/doc/refman/5.1/en/if-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/if-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (29,18,'^','Syntax:\n^\n\nBitwise XOR:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html\n\n','mysql> SELECT 1 ^ 1;\n -> 0\nmysql> SELECT 1 ^ 0;\n -> 1\nmysql> SELECT 11 ^ 3;\n -> 8\n','http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (30,37,'DROP VIEW','Syntax:\nDROP VIEW [IF EXISTS]\n view_name [, view_name] ...\n [RESTRICT | CASCADE]\n\nDROP VIEW removes one or more views. You must have the DROP privilege\nfor each view. If any of the views named in the argument list do not\nexist, MySQL returns an error indicating by name which nonexisting\nviews it was unable to drop, but it also drops all of the views in the\nlist that do exist.\n\nThe IF EXISTS clause prevents an error from occurring for views that\ndon\'t exist. When this clause is given, a NOTE is generated for each\nnonexistent view. See [HELP SHOW WARNINGS].\n\nRESTRICT and CASCADE, if given, are parsed and ignored.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-view.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-view.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (31,29,'WITHIN','Within(g1,g2)\n\nReturns 1 or 0 to indicate whether g1 is spatially within g2. This\ntests the opposite relationship as Contains().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (32,30,'WEEK','Syntax:\nWEEK(date[,mode])\n\nThis function returns the week number for date. The two-argument form\nof WEEK() allows you to specify whether the week starts on Sunday or\nMonday and whether the return value should be in the range from 0 to 53\nor from 1 to 53. If the mode argument is omitted, the value of the\ndefault_week_format system variable is used. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT WEEK(\'2008-02-20\');\n -> 7\nmysql> SELECT WEEK(\'2008-02-20\',0);\n -> 7\nmysql> SELECT WEEK(\'2008-02-20\',1);\n -> 8\nmysql> SELECT WEEK(\'2008-12-31\',1);\n -> 53\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (33,25,'SHOW PLUGINS','Syntax:\nSHOW PLUGINS\n\nSHOW PLUGINS displays information about known plugins.\n\nmysql> SHOW PLUGINS;\n+------------+--------+----------------+---------+\n| Name | Status | Type | Library |\n+------------+--------+----------------+---------+\n| MEMORY | ACTIVE | STORAGE ENGINE | NULL |\n| MyISAM | ACTIVE | STORAGE ENGINE | NULL |\n| InnoDB | ACTIVE | STORAGE ENGINE | NULL |\n| ARCHIVE | ACTIVE | STORAGE ENGINE | NULL |\n| CSV | ACTIVE | STORAGE ENGINE | NULL |\n| BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL |\n| FEDERATED | ACTIVE | STORAGE ENGINE | NULL |\n| MRG_MYISAM | ACTIVE | STORAGE ENGINE | NULL |\n+------------+--------+----------------+---------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (33,25,'SHOW PLUGINS','Syntax:\nSHOW PLUGINS\n\nSHOW PLUGINS displays information about server plugins.\n\nmysql> SHOW PLUGINS\\G\n*************************** 1. row ***************************\n Name: binlog\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n*************************** 2. row ***************************\n Name: CSV\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n*************************** 3. row ***************************\n Name: MEMORY\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n*************************** 4. row ***************************\n Name: MyISAM\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n...\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (34,21,'DROP FUNCTION UDF','Syntax:\nDROP FUNCTION function_name\n\nThis statement drops the user-defined function (UDF) named\nfunction_name.\n\nTo drop a function, you must have the DELETE privilege for the mysql\ndatabase. This is because DROP FUNCTION removes a row from the\nmysql.func system table that records the function\'s name, type, and\nshared library name.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-function-udf.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-function-udf.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (35,25,'PREPARE','Syntax:\nPREPARE stmt_name FROM preparable_stmt\n\nThe PREPARE statement prepares a statement and assigns it a name,\nstmt_name, by which to refer to the statement later. Statement names\nare not case sensitive. preparable_stmt is either a string literal or a\nuser variable that contains the text of the statement. The text must\nrepresent a single SQL statement, not multiple statements. Within the\nstatement, "?" characters can be used as parameter markers to indicate\nwhere data values are to be bound to the query later when you execute\nit. The "?" characters should not be enclosed within quotes, even if\nyou intend to bind them to string values. Parameter markers can be used\nonly where data values should appear, not for SQL keywords,\nidentifiers, and so forth.\n\nIf a prepared statement with the given name already exists, it is\ndeallocated implicitly before the new statement is prepared. This means\nthat if the new statement contains an error and cannot be prepared, an\nerror is returned and no statement with the given name exists.\n\nA prepared statement is executed with EXECUTE and released with\nDEALLOCATE PREPARE.\n\nThe scope of a prepared statement is the session within which it is\ncreated. Other sessions cannot see it.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/prepare.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/prepare.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (36,8,'LOCK','Syntax:\nLOCK TABLES\n tbl_name [[AS] alias] lock_type\n [, tbl_name [[AS] alias] lock_type] ...\n\nlock_type:\n READ [LOCAL]\n | [LOW_PRIORITY] WRITE\n\nUNLOCK TABLES\n\nMySQL enables client sessions to acquire table locks explicitly for the\npurpose of cooperating with other sessions for access to tables, or to\nprevent other sessions from modifying tables during periods when a\nsession requires exclusive access to them. A session can acquire or\nrelease locks only for itself. One session cannot acquire locks for\nanother session or release locks held by another session.\n\nLocks may be used to emulate transactions or to get more speed when\nupdating tables. This is explained in more detail later in this\nsection.\n\nLOCK TABLES explicitly acquires table locks for the current client\nsession. Table locks can be acquired for base tables or views. You must\nhave the LOCK TABLES privilege, and the SELECT privilege for each\nobject to be locked.\n\nFor view locking, LOCK TABLES adds all base tables used in the view to\nthe set of tables to be locked and locks them automatically. If you\nlock a table explicitly with LOCK TABLES, any tables used in triggers\nare also locked implicitly, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/lock-tables-and-triggers.html.\n… TABLES explicitly releases any table locks held by the current\nsession.\n\nAnother use for UNLOCK TABLES is to release the global read lock\nacquired with the FLUSH TABLES WITH READ LOCK statement, which enables\nyou to lock all tables in all databases. See [HELP FLUSH]. (This is a\nvery convenient way to get backups if you have a file system such as\nVeritas that can take snapshots in time.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/lock-tables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/lock-tables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (37,35,'UPDATEXML','Syntax:\nUpdateXML(xml_target, xpath_expr, new_xml)\n\nThis function replaces a single portion of a given fragment of XML\nmarkup xml_target with a new XML fragment new_xml, and then returns the\nchanged XML. The portion of xml_target that is replaced matches an\nXPath expression xpath_expr supplied by the user. If no expression\nmatching xpath_expr is found, or if multiple matches are found, the\nfunction returns the original xml_target XML fragment. All three\narguments should be strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html\n\n','mysql> SELECT\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'/a\', \'<e>fff</e>\') AS val1,\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'/b\', \'<e>fff</e>\') AS val2,\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'//b\', \'<e>fff</e>\') AS val3,\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'/a/d\', \'<e>fff</e>\') AS val4,\n -> UpdateXML(\'<a><d></d>ccc<d></d></a>\', \'/a/d\', \'<e>fff</e>\') AS val5\n -> \\G\n\n*************************** 1. row ***************************\nval1: <e>fff</e>\nval2: <a>ccc<d></d></a>\nval3: <a><e>fff</e><d></d></a>\nval4: <a>ccc<e>fff</e></a>\nval5: <a><d></d>ccc<d></d></a>\n','http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (38,25,'RESET SLAVE','Syntax:\nRESET SLAVE\n\nRESET SLAVE makes the slave forget its replication position in the\nmaster\'s binary logs. This statement is meant to be used for a clean\nstart: It deletes the master.info and relay-log.info files, all the\nrelay logs, and starts a new relay log.\n\n*Note*: All relay logs are deleted, even if they have not been\ncompletely executed by the slave SQL thread. (This is a condition\nlikely to exist on a replication slave if you have issued a STOP SLAVE\nstatement or if the slave is highly loaded.)\n\nConnection information stored in the master.info file is immediately\nreset using any values specified in the corresponding startup options.\nThis information includes values such as master host, master port,\nmaster user, and master password. If the slave SQL thread was in the\nmiddle of replicating temporary tables when it was stopped, and RESET\nSLAVE is issued, these replicated temporary tables are deleted on the\nslave.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (38,25,'RESET SLAVE','Syntax:\nRESET SLAVE\n\nRESET SLAVE makes the slave forget its replication position in the\nmaster\'s binary log. This statement is meant to be used for a clean\nstart: It deletes the master.info and relay-log.info files, all the\nrelay log files, and starts a new relay log file. To use RESET SLAVE,\nthe slave replication threads must be stopped (use STOP SLAVE if\nnecessary).\n\n*Note*: All relay log files are deleted, even if they have not been\ncompletely executed by the slave SQL thread. (This is a condition\nlikely to exist on a replication slave if you have issued a STOP SLAVE\nstatement or if the slave is highly loaded.)\n\nConnection information stored in the master.info file is immediately\nreset using any values specified in the corresponding startup options.\nThis information includes values such as master host, master port,\nmaster user, and master password. If the slave SQL thread was in the\nmiddle of replicating temporary tables when it was stopped, and RESET\nSLAVE is issued, these replicated temporary tables are deleted on the\nslave.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (39,25,'SHOW BINARY LOGS','Syntax:\nSHOW BINARY LOGS\nSHOW MASTER LOGS\n\nLists the binary log files on the server. This statement is used as\npart of the procedure described in [HELP PURGE BINARY LOGS], that shows\nhow to determine which logs can be purged.\n\nmysql> SHOW BINARY LOGS;\n+---------------+-----------+\n| Log_name | File_size |\n+---------------+-----------+\n| binlog.000015 | 724935 |\n| binlog.000016 | 733481 |\n+---------------+-----------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-binary-logs.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-binary-logs.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (40,23,'POLYGON','Polygon(ls1,ls2,...)\n\nConstructs a Polygon value from a number of LineString or WKB\nLineString arguments. If any argument does not represent a LinearRing\n(that is, not a closed and simple LineString), the return value is\nNULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (41,30,'MINUTE','Syntax:\nMINUTE(time)\n\nReturns the minute for time, in the range 0 to 59.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MINUTE(\'2008-02-03 10:05:03\');\n -> 5\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); @@ -118,7 +118,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (49,4,'ROUND','Syntax:\nROUND(X), ROUND(X,D)\n\nRounds the argument X to D decimal places. The rounding algorithm\ndepends on the data type of X. D defaults to 0 if not specified. D can\nbe negative to cause D digits left of the decimal point of the value X\nto become zero.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT ROUND(-1.23);\n -> -1\nmysql> SELECT ROUND(-1.58);\n -> -2\nmysql> SELECT ROUND(1.58);\n -> 2\nmysql> SELECT ROUND(1.298, 1);\n -> 1.3\nmysql> SELECT ROUND(1.298, 0);\n -> 1\nmysql> SELECT ROUND(23.298, -1);\n -> 20\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (50,7,'NULLIF','Syntax:\nNULLIF(expr1,expr2)\n\nReturns NULL if expr1 = expr2 is true, otherwise returns expr1. This is\nthe same as CASE WHEN expr1 = expr2 THEN NULL ELSE expr1 END.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html\n\n','mysql> SELECT NULLIF(1,1);\n -> NULL\nmysql> SELECT NULLIF(1,2);\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (51,22,'CLOSE','Syntax:\nCLOSE cursor_name\n\nThis statement closes a previously opened cursor.\n\nIf not closed explicitly, a cursor is closed at the end of the compound\nstatement in which it was declared.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/close.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/close.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (52,25,'STOP SLAVE','Syntax:\nSTOP SLAVE [thread_type [, thread_type] ... ]\n\nthread_type: IO_THREAD | SQL_THREAD\n\nStops the slave threads. STOP SLAVE requires the SUPER privilege.\n\nLike START SLAVE, this statement may be used with the IO_THREAD and\nSQL_THREAD options to name the thread or threads to be stopped.\n\n*Note*: The transactional behavior of STOP SLAVE changed in MySQL\n5.1.35. Previously, it took effect immediately; beginning with MySQL\n5.1.35, it waits until the current replication event group (if any) has\nfinished executing, or until the user issues a KILL QUERY or KILL\nCONNECTION statement. (Bug#319 (http://bugs.mysql.com/319) Bug#38205\n(http://bugs.mysql.com/38205))\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (52,25,'STOP SLAVE','Syntax:\nSTOP SLAVE [thread_type [, thread_type] ... ]\n\nthread_type: IO_THREAD | SQL_THREAD\n\nStops the slave threads. STOP SLAVE requires the SUPER privilege.\n\nLike START SLAVE, this statement may be used with the IO_THREAD and\nSQL_THREAD options to name the thread or threads to be stopped.\n\n*Note*: The transactional behavior of STOP SLAVE changed in MySQL\n5.1.35. Previously, it took effect immediately. Beginning with MySQL\n5.1.35, it waits until any current replication event group affecting\none or more non-transactional tables has finished executing (if there\nis any such replication group), or until the user issues a KILL QUERY\nor KILL CONNECTION statement. (Bug#319\n(http://bugs.mysql.com/bug.php?id=319), Bug#38205\n(http://bugs.mysql.com/bug.php?id=38205))\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (53,30,'TIMEDIFF','Syntax:\nTIMEDIFF(expr1,expr2)\n\nTIMEDIFF() returns expr1 - expr2 expressed as a time value. expr1 and\nexpr2 are time or date-and-time expressions, but both must be of the\nsame type.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMEDIFF(\'2000:01:01 00:00:00\',\n -> \'2000:01:01 00:00:00.000001\');\n -> \'-00:00:00.000001\'\nmysql> SELECT TIMEDIFF(\'2008-12-31 23:59:59.000001\',\n -> \'2008-12-30 01:01:01.000002\');\n -> \'46:58:57.999999\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (54,35,'REPLACE FUNCTION','Syntax:\nREPLACE(str,from_str,to_str)\n\nReturns the string str with all occurrences of the string from_str\nreplaced by the string to_str. REPLACE() performs a case-sensitive\nmatch when searching for from_str.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT REPLACE(\'www.mysql.com\', \'w\', \'Ww\');\n -> \'WwWwWw.mysql.com\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (55,27,'USE','Syntax:\nUSE db_name\n\nThe USE db_name statement tells MySQL to use the db_name database as\nthe default (current) database for subsequent statements. The database\nremains the default until the end of the session or another USE\nstatement is issued:\n\nUSE db1;\nSELECT COUNT(*) FROM mytable; # selects from db1.mytable\nUSE db2;\nSELECT COUNT(*) FROM mytable; # selects from db2.mytable\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/use.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/use.html'); @@ -147,13 +147,13 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (78,35,'LCASE','Syntax:\nLCASE(str)\n\nLCASE() is a synonym for LOWER().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (79,17,'<=','Syntax:\n<=\n\nLess than or equal:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 0.1 <= 2;\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (80,25,'SHOW PROFILES','Syntax:\nSHOW PROFILE [type [, type] ... ]\n [FOR QUERY n]\n [LIMIT row_count [OFFSET offset]]\n\ntype:\n ALL\n | BLOCK IO\n | CONTEXT SWITCHES\n | CPU\n | IPC\n | MEMORY\n | PAGE FAULTS\n | SOURCE\n | SWAPS\n\nThe SHOW PROFILES and SHOW PROFILE statements display profiling\ninformation that indicates resource usage for statements executed\nduring the course of the current session.\n\nProfiling is controlled by the profiling session variable, which has a\ndefault value of 0 (OFF). Profiling is enabled by setting profiling to\n1 or ON:\n\nmysql> SET profiling = 1;\n\nSHOW PROFILES displays a list of the most recent statements sent to the\nmaster. The size of the list is controlled by the\nprofiling_history_size session variable, which has a default value of\n15. The maximum value is 100. Setting the value to 0 has the practical\neffect of disabling profiling.\n\nAll statements are profiled except SHOW PROFILES and SHOW PROFILE, so\nyou will find neither of those statements in the profile list.\nMalformed statements are profiled. For example, SHOW PROFILING is an\nillegal statement, and a syntax error occurs if you try to execute it,\nbut it will show up in the profiling list.\n\nSHOW PROFILE displays detailed information about a single statement.\nWithout the FOR QUERY n clause, the output pertains to the most\nrecently executed statement. If FOR QUERY n is included, SHOW PROFILE\ndisplays information for statement n. The values of n correspond to the\nQuery_ID values displayed by SHOW PROFILES.\n\nThe LIMIT row_count clause may be given to limit the output to\nrow_count rows. If LIMIT is given, OFFSET offset may be added to begin\nthe output offset rows into the full set of rows.\n\nBy default, SHOW PROFILE displays Status and Duration columns. The\nStatus values are like the State values displayed by SHOW PROCESSLIST,\nalthought there might be some minor differences in interpretion for the\ntwo statements for some status values (see\nhttp://dev.mysql.com/doc/refman/5.1/en/thread-information.html).\n\nOp… type values may be specified to display specific additional\ntypes of information:\n\no ALL displays all information\n\no BLOCK IO displays counts for block input and output operations\n\no CONTEXT SWITCHES displays counts for voluntary and involuntary\n context switches\n\no CPU displays user and system CPU usage times\n\no IPC displays counts for messages sent and received\n\no MEMORY is not currently implemented\n\no PAGE FAULTS displays counts for major and minor page faults\n\no SOURCE displays the names of functions from the source code, together\n with the name and line number of the file in which the function\n occurs\n\no SWAPS displays swap counts\n\nProfiling is enabled per session. When a session ends, its profiling\ninformation is lost.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-profiles.html\n\n','mysql> SELECT @@profiling;\n+-------------+\n| @@profiling |\n+-------------+\n| 0 |\n+-------------+\n1 row in set (0.00 sec)\n\nmysql> SET profiling = 1;\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> DROP TABLE IF EXISTS t1;\nQuery OK, 0 rows affected, 1 warning (0.00 sec)\n\nmysql> CREATE TABLE T1 (id INT);\nQuery OK, 0 rows affected (0.01 sec)\n\nmysql> SHOW PROFILES;\n+----------+----------+--------------------------+\n| Query_ID | Duration | Query |\n+----------+----------+--------------------------+\n| 0 | 0.000088 | SET PROFILING = 1 |\n| 1 | 0.000136 | DROP TABLE IF EXISTS t1 |\n| 2 | 0.011947 | CREATE TABLE t1 (id INT) |\n+----------+----------+--------------------------+\n3 rows in set (0.00 sec)\n\nmysql> SHOW PROFILE;\n+----------------------+----------+\n| Status | Duration |\n+----------------------+----------+\n| checking permissions | 0.000040 |\n| creating table | 0.000056 |\n| After create | 0.011363 |\n| query end | 0.000375 |\n| freeing items | 0.000089 |\n| logging slow query | 0.000019 |\n| cleaning up | 0.000005 |\n+----------------------+----------+\n7 rows in set (0.00 sec)\n\nmysql> SHOW PROFILE FOR QUERY 1;\n+--------------------+----------+\n| Status | Duration |\n+--------------------+----------+\n| query end | 0.000107 |\n| freeing items | 0.000008 |\n| logging slow query | 0.000015 |\n| cleaning up | 0.000006 |\n+--------------------+----------+\n4 rows in set (0.00 sec)\n\nmysql> SHOW PROFILE CPU FOR QUERY 2;\n+----------------------+----------+----------+------------+\n| Status | Duration | CPU_user | CPU_system |\n+----------------------+----------+----------+------------+\n| checking permissions | 0.000040 | 0.000038 | 0.000002 |\n| creating table | 0.000056 | 0.000028 | 0.000028 |\n| After create | 0.011363 | 0.000217 | 0.001571 |\n| query end | 0.000375 | 0.000013 | 0.000028 |\n| freeing items | 0.000089 | 0.000010 | 0.000014 |\n| logging slow query | 0.000019 | 0.000009 | 0.000010 |\n| cleaning up | 0.000005 | 0.000003 | 0.000002 |\n+----------------------+----------+----------+------------+\n7 rows in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/show-profiles.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (81,26,'UPDATE','Syntax:\nSingle-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_reference\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n [ORDER BY ...]\n [LIMIT row_count]\n\nMultiple-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_references\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n\nFor the single-table syntax, the UPDATE statement updates columns of\nexisting rows in the named table with new values. The SET clause\nindicates which columns to modify and the values they should be given.\nEach value can be given as an expression, or the keyword DEFAULT to set\na column explicitly to its default value. The WHERE clause, if given,\nspecifies the conditions that identify which rows to update. With no\nWHERE clause, all rows are updated. If the ORDER BY clause is\nspecified, the rows are updated in the order that is specified. The\nLIMIT clause places a limit on the number of rows that can be updated.\n\nFor the multiple-table syntax, UPDATE updates rows in each table named\nin table_references that satisfy the conditions. In this case, ORDER BY\nand LIMIT cannot be used.\n\nwhere_condition is an expression that evaluates to true for each row to\nbe updated.\n\ntable_references and where_condition are is specified as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nYou need the UPDATE privilege only for columns referenced in an UPDATE\nthat are actually updated. You need only the SELECT privilege for any\ncolumns that are read but not modified.\n\nThe UPDATE statement supports the following modifiers:\n\no If you use the LOW_PRIORITY keyword, execution of the UPDATE is\n delayed until no other clients are reading from the table. This\n affects only storage engines that use only table-level locking\n (MyISAM, MEMORY, MERGE).\n\no If you use the IGNORE keyword, the update statement does not abort\n even if errors occur during the update. Rows for which duplicate-key\n conflicts occur are not updated. Rows for which columns are updated\n to values that would cause data conversion errors are updated to the\n closest valid values instead.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/update.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/update.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (81,26,'UPDATE','Syntax:\nSingle-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_reference\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n [ORDER BY ...]\n [LIMIT row_count]\n\nMultiple-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_references\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n\nFor the single-table syntax, the UPDATE statement updates columns of\nexisting rows in the named table with new values. The SET clause\nindicates which columns to modify and the values they should be given.\nEach value can be given as an expression, or the keyword DEFAULT to set\na column explicitly to its default value. The WHERE clause, if given,\nspecifies the conditions that identify which rows to update. With no\nWHERE clause, all rows are updated. If the ORDER BY clause is\nspecified, the rows are updated in the order that is specified. The\nLIMIT clause places a limit on the number of rows that can be updated.\n\nFor the multiple-table syntax, UPDATE updates rows in each table named\nin table_references that satisfy the conditions. In this case, ORDER BY\nand LIMIT cannot be used.\n\nwhere_condition is an expression that evaluates to true for each row to\nbe updated.\n\ntable_references and where_condition are is specified as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nYou need the UPDATE privilege only for columns referenced in an UPDATE\nthat are actually updated. You need only the SELECT privilege for any\ncolumns that are read but not modified.\n\nThe UPDATE statement supports the following modifiers:\n\no If you use the LOW_PRIORITY keyword, execution of the UPDATE is\n delayed until no other clients are reading from the table. This\n affects only storage engines that use only table-level locking (such\n as MyISAM, MEMORY, and MERGE).\n\no If you use the IGNORE keyword, the update statement does not abort\n even if errors occur during the update. Rows for which duplicate-key\n conflicts occur are not updated. Rows for which columns are updated\n to values that would cause data conversion errors are updated to the\n closest valid values instead.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/update.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/update.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (82,17,'IS NOT NULL','Syntax:\nIS NOT NULL\n\nTests whether a value is not NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 1 IS NOT NULL, 0 IS NOT NULL, NULL IS NOT NULL;\n -> 1, 1, 0\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (83,22,'CASE STATEMENT','Syntax:\nCASE case_value\n WHEN when_value THEN statement_list\n [WHEN when_value THEN statement_list] ...\n [ELSE statement_list]\nEND CASE\n\nOr:\n\nCASE\n WHEN search_condition THEN statement_list\n [WHEN search_condition THEN statement_list] ...\n [ELSE statement_list]\nEND CASE\n\nThe CASE statement for stored programs implements a complex conditional\nconstruct. If a search_condition evaluates to true, the corresponding\nSQL statement list is executed. If no search condition matches, the\nstatement list in the ELSE clause is executed. Each statement_list\nconsists of one or more statements.\n\nIf no when_value or search_condition matches the value tested and the\nCASE statement contains no ELSE clause, a Case not found for CASE\nstatement error results.\n\nEach statement_list consists of one or more statements; an empty\nstatement_list is not allowed. To handle situations where no value is\nmatched by any WHEN clause, use an ELSE containing an empty BEGIN ...\nEND block, as shown in this example: DELIMITER | CREATE PROCEDURE p()\nBEGIN DECLARE v INT DEFAULT 1; CASE v WHEN 2 THEN SELECT v; WHEN 3 THEN\nSELECT 0; ELSE BEGIN END; END CASE; END; | (The indentation used here\nin the ELSE clause is for purposes of clarity only, and is not\notherwise significant.)\n\n*Note*: The syntax of the CASE statement used inside stored programs\ndiffers slightly from that of the SQL CASE expression described in\nhttp://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html. The\nCASE statement cannot have an ELSE NULL clause, and it is terminated\nwith END CASE instead of END.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/case-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/case-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (84,25,'EXECUTE STATEMENT','Syntax:\nEXECUTE stmt_name\n [USING @var_name [, @var_name] ...]\n\nAfter preparing a statement with PREPARE, you execute it with an\nEXECUTE statement that refers to the prepared statement name. If the\nprepared statement contains any parameter markers, you must supply a\nUSING clause that lists user variables containing the values to be\nbound to the parameters. Parameter values can be supplied only by user\nvariables, and the USING clause must name exactly as many variables as\nthe number of parameter markers in the statement.\n\nYou can execute a given prepared statement multiple times, passing\ndifferent variables to it or setting the variables to different values\nbefore each execution.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/execute.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/execute.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (85,37,'DROP INDEX','Syntax:\nDROP [ONLINE|OFFLINE] INDEX index_name ON tbl_name\n\nDROP INDEX drops the index named index_name from the table tbl_name.\nThis statement is mapped to an ALTER TABLE statement to drop the index.\nSee [HELP ALTER TABLE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-index.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-index.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (86,35,'MATCH AGAINST','Syntax:\nMATCH (col1,col2,...) AGAINST (expr [search_modifier])\n\nMySQL has support for full-text indexing and searching:\n\no A full-text index in MySQL is an index of type FULLTEXT.\n\no Full-text indexes can be used only with MyISAM tables, and can be\n created only for CHAR, VARCHAR, or TEXT columns.\n\no A FULLTEXT index definition can be given in the CREATE TABLE\n statement when a table is created, or added later using ALTER TABLE\n or CREATE INDEX.\n\no For large data sets, it is much faster to load your data into a table\n that has no FULLTEXT index and then create the index after that, than\n to load data into a table that has an existing FULLTEXT index.\n\nFull-text searching is performed using MATCH() ... AGAINST syntax.\nMATCH() takes a comma-separated list that names the columns to be\nsearched. AGAINST takes a string to search for, and an optional\nmodifier that indicates what type of search to perform. The search\nstring must be a literal string, not a variable or a column name. There\nare three types of full-text searches:\n\no A boolean search interprets the search string using the rules of a\n special query language. The string contains the words to search for.\n It can also contain operators that specify requirements such that a\n word must be present or absent in matching rows, or that it should be\n weighted higher or lower than usual. Common words such as "some" or\n "then" are stopwords and do not match if present in the search\n string. The IN BOOLEAN MODE modifier specifies a boolean search. For\n more information, see\n http://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html.\n\no A natural language search interprets the search string as a phrase in\n natural human language (a phrase in free text). There are no special\n operators. The stopword list applies. In addition, words that are\n present in 50% or more of the rows are considered common and do not\n match. Full-text searches are natural language searches if the IN\n NATURAL LANGUAGE MODE modifier is given or if no modifier is given.\n\no A query expansion search is a modification of a natural language\n search. The search string is used to perform a natural language\n search. Then words from the most relevant rows returned by the search\n are added to the search string and the search is done again. The\n query returns the rows from the second search. The IN NATURAL\n LANGUAGE MODE WITH QUERY EXPANSION or WITH QUERY EXPANSION modifier\n specifies a query expansion search. For more information, see\n http://dev.mysql.com/doc/refman/5.1/en/fulltext-query-expansion.html.\n\nThe IN NATURAL LANGUAGE MODE and IN NATURAL LANGUAGE MODE WITH QUERY\nEXPANSION modifiers were added in MySQL 5.1.7.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html\n\n','mysql> SELECT id, body, MATCH (title,body) AGAINST\n -> (\'Security implications of running MySQL as root\'\n -> IN NATURAL LANGUAGE MODE) AS score\n -> FROM articles WHERE MATCH (title,body) AGAINST\n -> (\'Security implications of running MySQL as root\'\n -> IN NATURAL LANGUAGE MODE);\n+----+-------------------------------------+-----------------+\n| id | body | score |\n+----+-------------------------------------+-----------------+\n| 4 | 1. Never run mysqld as root. 2. ... | 1.5219271183014 |\n| 6 | When configured properly, MySQL ... | 1.3114095926285 |\n+----+-------------------------------------+-----------------+\n2 rows in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (87,37,'CREATE EVENT','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n EVENT\n [IF NOT EXISTS]\n event_name\n ON SCHEDULE schedule\n [ON COMPLETION [NOT] PRESERVE]\n [ENABLE | DISABLE | DISABLE ON SLAVE]\n [COMMENT \'comment\']\n DO sql_statement;\n\nschedule:\n AT timestamp [+ INTERVAL interval] ...\n | EVERY interval\n [STARTS timestamp [+ INTERVAL interval] ...]\n [ENDS timestamp [+ INTERVAL interval] ...]\n\ninterval:\n quantity {YEAR | QUARTER | MONTH | DAY | HOUR | MINUTE |\n WEEK | SECOND | YEAR_MONTH | DAY_HOUR | DAY_MINUTE |\n DAY_SECOND | HOUR_MINUTE | HOUR_SECOND | MINUTE_SECOND}\n\nThis statement creates and schedules a new event. It requires the EVENT\nprivilege for the schema in which the event is to be created.\n\nThe minimum requirements for a valid CREATE EVENT statement are as\nfollows:\n\no The keywords CREATE EVENT plus an event name, which uniquely\n identifies the event within a database schema. (Prior to MySQL\n 5.1.12, the event name needed to be unique only among events created\n by the same user within a schema.)\n\no An ON SCHEDULE clause, which determines when and how often the event\n executes.\n\no A DO clause, which contains the SQL statement to be executed by an\n event.\n\nThis is an example of a minimal CREATE EVENT statement:\n\nCREATE EVENT myevent\n ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 HOUR\n DO\n UPDATE myschema.mytable SET mycol = mycol + 1;\n\nThe previous statement creates an event named myevent. This event\nexecutes once --- one hour following its creation --- by running an SQL\nstatement that increments the value of the myschema.mytable table\'s\nmycol column by 1.\n\nThe event_name must be a valid MySQL identifier with a maximum length\nof 64 characters. Event names are not case sensitive, so you cannot\nhave two events named myevent and MyEvent in the same schema. In\ngeneral, the rules governing event names are the same as those for\nnames of stored routines. See\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\nAn event is associated with a schema. If no schema is indicated as part\nof event_name, the default (current) schema is assumed. To create an\nevent in a specific schema, qualify the event name with a schema using\nschema_name.event_name syntax.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-event.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-event.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (87,37,'CREATE EVENT','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n EVENT\n [IF NOT EXISTS]\n event_name\n ON SCHEDULE schedule\n [ON COMPLETION [NOT] PRESERVE]\n [ENABLE | DISABLE | DISABLE ON SLAVE]\n [COMMENT \'comment\']\n DO sql_statement;\n\nschedule:\n AT timestamp [+ INTERVAL interval] ...\n | EVERY interval\n [STARTS timestamp [+ INTERVAL interval] ...]\n [ENDS timestamp [+ INTERVAL interval] ...]\n\ninterval:\n quantity {YEAR | QUARTER | MONTH | DAY | HOUR | MINUTE |\n WEEK | SECOND | YEAR_MONTH | DAY_HOUR | DAY_MINUTE |\n DAY_SECOND | HOUR_MINUTE | HOUR_SECOND | MINUTE_SECOND}\n\nThis statement creates and schedules a new event. It requires the EVENT\nprivilege for the schema in which the event is to be created (and\nperhaps SUPER depending on the DEFINER value, as described later). The\nevent will not run unless the Event Scheduler is enabled. For\ninformation about checking Event Scheduler status and enabling it if\nnecessary, see\nhttp://dev.mysql.com/doc/refman/5.1/en/events-configuration.html.\n\nT… minimum requirements for a valid CREATE EVENT statement are as\nfollows:\n\no The keywords CREATE EVENT plus an event name, which uniquely\n identifies the event within a database schema. (Prior to MySQL\n 5.1.12, the event name needed to be unique only among events created\n by the same user within a schema.)\n\no An ON SCHEDULE clause, which determines when and how often the event\n executes.\n\no A DO clause, which contains the SQL statement to be executed by an\n event.\n\nThis is an example of a minimal CREATE EVENT statement:\n\nCREATE EVENT myevent\n ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 HOUR\n DO\n UPDATE myschema.mytable SET mycol = mycol + 1;\n\nThe previous statement creates an event named myevent. This event\nexecutes once --- one hour following its creation --- by running an SQL\nstatement that increments the value of the myschema.mytable table\'s\nmycol column by 1.\n\nThe event_name must be a valid MySQL identifier with a maximum length\nof 64 characters. Event names are not case sensitive, so you cannot\nhave two events named myevent and MyEvent in the same schema. In\ngeneral, the rules governing event names are the same as those for\nnames of stored routines. See\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\nAn event is associated with a schema. If no schema is indicated as part\nof event_name, the default (current) schema is assumed. To create an\nevent in a specific schema, qualify the event name with a schema using\nschema_name.event_name syntax.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-event.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-event.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (88,4,'ABS','Syntax:\nABS(X)\n\nReturns the absolute value of X.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT ABS(2);\n -> 2\nmysql> SELECT ABS(-32);\n -> 32\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (89,31,'POLYFROMWKB','PolyFromWKB(wkb[,srid]), PolygonFromWKB(wkb[,srid])\n\nConstructs a POLYGON value using its WKB representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (90,35,'NOT LIKE','Syntax:\nexpr NOT LIKE pat [ESCAPE \'escape_char\']\n\nThis is the same as NOT (expr LIKE pat [ESCAPE \'escape_char\']).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-comparison-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-comparison-functions.html'); @@ -170,7 +170,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (101,11,'COMPRESS','Syntax:\nCOMPRESS(string_to_compress)\n\nCompresses a string and returns the result as a binary string. This\nfunction requires MySQL to have been compiled with a compression\nlibrary such as zlib. Otherwise, the return value is always NULL. The\ncompressed string can be uncompressed with UNCOMPRESS().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','mysql> SELECT LENGTH(COMPRESS(REPEAT(\'a\',1000)));\n -> 21\nmysql> SELECT LENGTH(COMPRESS(\'\'));\n -> 0\nmysql> SELECT LENGTH(COMPRESS(\'a\'));\n -> 13\nmysql> SELECT LENGTH(COMPRESS(REPEAT(\'a\',16)));\n -> 15\n','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (102,26,'INSERT','Syntax:\nINSERT [LOW_PRIORITY | DELAYED | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name [(col_name,...)]\n {VALUES | VALUE} ({expr | DEFAULT},...),(...),...\n [ ON DUPLICATE KEY UPDATE\n col_name=expr\n [, col_name=expr] ... ]\n\nOr:\n\nINSERT [LOW_PRIORITY | DELAYED | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name\n SET col_name={expr | DEFAULT}, ...\n [ ON DUPLICATE KEY UPDATE\n col_name=expr\n [, col_name=expr] ... ]\n\nOr:\n\nINSERT [LOW_PRIORITY | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name [(col_name,...)]\n SELECT ...\n [ ON DUPLICATE KEY UPDATE\n col_name=expr\n [, col_name=expr] ... ]\n\nINSERT inserts new rows into an existing table. The INSERT ... VALUES\nand INSERT ... SET forms of the statement insert rows based on\nexplicitly specified values. The INSERT ... SELECT form inserts rows\nselected from another table or tables. INSERT ... SELECT is discussed\nfurther in [HELP INSERT SELECT].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/insert.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/insert.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (103,16,'COUNT','Syntax:\nCOUNT(expr)\n\nReturns a count of the number of non-NULL values of expr in the rows\nretrieved by a SELECT statement. The result is a BIGINT value.\n\nCOUNT() returns 0 if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','mysql> SELECT student.student_name,COUNT(*)\n -> FROM student,course\n -> WHERE student.student_id=course.student_id\n -> GROUP BY student_name;\n','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (104,26,'HANDLER','Syntax:\nHANDLER tbl_name OPEN [ [AS] alias]\nHANDLER tbl_name READ index_name { = | >= | <= | < } (value1,value2,...)\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ index_name { FIRST | NEXT | PREV | LAST }\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ { FIRST | NEXT }\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name CLOSE\n\nThe HANDLER statement provides direct access to table storage engine\ninterfaces. It is available for MyISAM and InnoDB tables.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/handler.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/handler.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (104,26,'HANDLER','Syntax:\nHANDLER tbl_name OPEN [ [AS] alias]\n\nHANDLER tbl_name READ index_name { = | <= | >= | < | > } (value1,value2,...)\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ index_name { FIRST | NEXT | PREV | LAST }\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ { FIRST | NEXT }\n [ WHERE where_condition ] [LIMIT ... ]\n\nHANDLER tbl_name CLOSE\n\nThe HANDLER statement provides direct access to table storage engine\ninterfaces. It is available for MyISAM and InnoDB tables.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/handler.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/handler.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (105,3,'MLINEFROMTEXT','MLineFromText(wkt[,srid]), MultiLineStringFromText(wkt[,srid])\n\nConstructs a MULTILINESTRING value using its WKT representation and\nSRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (106,31,'GEOMCOLLFROMWKB','GeomCollFromWKB(wkb[,srid]), GeometryCollectionFromWKB(wkb[,srid])\n\nConstructs a GEOMETRYCOLLECTION value using its WKB representation and\nSRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (107,37,'RENAME TABLE','Syntax:\nRENAME TABLE tbl_name TO new_tbl_name\n [, tbl_name2 TO new_tbl_name2] ...\n\nThis statement renames one or more tables.\n\nThe rename operation is done atomically, which means that no other\nsession can access any of the tables while the rename is running. For\nexample, if you have an existing table old_table, you can create\nanother table new_table that has the same structure but is empty, and\nthen replace the existing table with the empty one as follows (assuming\nthat backup_table does not already exist):\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/rename-table.html\n\n','CREATE TABLE new_table (...);\nRENAME TABLE old_table TO backup_table, new_table TO old_table;\n','http://dev.mysql.com/doc/refman/5.1/en/rename-table.html'); @@ -181,9 +181,9 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (112,19,'OPTIMIZE TABLE','Syntax:\nOPTIMIZE [NO_WRITE_TO_BINLOG | LOCAL] TABLE\n tbl_name [, tbl_name] ...\n\nOPTIMIZE TABLE should be used if you have deleted a large part of a\ntable or if you have made many changes to a table with variable-length\nrows (tables that have VARCHAR, VARBINARY, BLOB, or TEXT columns).\nDeleted rows are maintained in a linked list and subsequent INSERT\noperations reuse old row positions. You can use OPTIMIZE TABLE to\nreclaim the unused space and to defragment the data file.\n\nThis statement requires SELECT and INSERT privileges for the table.\n\nBeginning with MySQL 5.1.27, OPTIMIZE TABLE is also supported for\npartitioned tables. Also beginning with MySQL 5.1.27, you can use ALTER\nTABLE ... OPTIMIZE PARTITION to optimize one or more partitions; for\nmore information, see [HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/optimize-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/optimize-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (113,11,'DECODE','Syntax:\nDECODE(crypt_str,pass_str)\n\nDecrypts the encrypted string crypt_str using pass_str as the password.\ncrypt_str should be a string returned from ENCODE().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (114,17,'<=>','Syntax:\n<=>\n\nNULL-safe equal. This operator performs an equality comparison like the\n= operator, but returns 1 rather than NULL if both operands are NULL,\nand 0 rather than NULL if one operand is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 1 <=> 1, NULL <=> NULL, 1 <=> NULL;\n -> 1, 1, 0\nmysql> SELECT 1 = 1, NULL = NULL, 1 = NULL;\n -> 1, NULL, NULL\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (115,25,'LOAD DATA FROM MASTER','Syntax:\nLOAD DATA FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated in\nversions 4.1 of MySQL and above. We will introduce a more advanced\ntechnique (called "online backup") in a future version. That technique\nwill have the additional advantage of working with more storage\nengines.\n\nFor MySQL 5.1 and earlier, the recommended alternative solution to\nusing LOAD DATA FROM MASTER or LOAD TABLE FROM MASTER is using\nmysqldump or mysqlhotcopy. The latter requires Perl and two Perl\nmodules (DBI and DBD:mysql) and works for MyISAM and ARCHIVE tables\nonly. With mysqldump, you can create SQL dumps on the master and pipe\n(or copy) these to a mysql client on the slave. This has the advantage\nof working for all storage engines, but can be quite slow, since it\nworks using SELECT.\n\nThis statement takes a snapshot of the master and copies it to the\nslave. It updates the values of MASTER_LOG_FILE and MASTER_LOG_POS so\nthat the slave starts replicating from the correct position. Any table\nand database exclusion rules specified with the --replicate-*-do-* and\n--replicate-*-ignore-* options are honored. --replicate-rewrite-db is\nnot taken into account because a user could use this option to set up a\nnonunique mapping such as --replicate-rewrite-db="db1->db3" and\n--replicate-rewrite-db="db2->db3", which would confuse the slave when\nloading tables from the master.\n\nUse of this statement is subject to the following conditions:\n\no It works only for MyISAM tables. Attempting to load a non-MyISAM\n table results in the following error:\n\nERROR 1189 (08S01): Net error reading from master\n\no It acquires a global read lock on the master while taking the\n snapshot, which prevents updates on the master during the load\n operation.\n\nIf you are loading large tables, you might have to increase the values\nof net_read_timeout and net_write_timeout on both the master and slave\nservers. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html.\n… that LOAD DATA FROM MASTER does not copy any tables from the mysql\ndatabase. This makes it easy to have different users and privileges on\nthe master and the slave.\n\nTo use LOAD DATA FROM MASTER, the replication account that is used to\nconnect to the master must have the RELOAD and SUPER privileges on the\nmaster and the SELECT privilege for all master tables you want to load.\nAll master tables for which the user does not have the SELECT privilege\nare ignored by LOAD DATA FROM MASTER. This is because the master hides\nthem from the user: LOAD DATA FROM MASTER calls SHOW DATABASES to know\nthe master databases to load, but SHOW DATABASES returns only databases\nfor which the user has some privilege. See [HELP SHOW DATABASES]. On\nthe slave side, the user that issues LOAD DATA FROM MASTER must have\nprivileges for dropping and creating the databases and tables that are\ncopied.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (116,25,'RESET','Syntax:\nRESET reset_option [, reset_option] ...\n\nThe RESET statement is used to clear the state of various server\noperations. You must have the RELOAD privilege to execute RESET.\n\nRESET acts as a stronger version of the FLUSH statement. See [HELP\nFLUSH].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (117,27,'HELP STATEMENT','Syntax:\nHELP \'search_string\'\n\nThe HELP statement returns online information from the MySQL Reference\nmanual. Its proper operation requires that the help tables in the mysql\ndatabase be initialized with help topic information (see\nhttp://dev.mysql.com/doc/refman/5.1/en/server-side-help-support.html).… HELP statement searches the help tables for the given search string\nand displays the result of the search. The search string is not case\nsensitive.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/help.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/help.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (115,27,'HELP STATEMENT','Syntax:\nHELP \'search_string\'\n\nThe HELP statement returns online information from the MySQL Reference\nmanual. Its proper operation requires that the help tables in the mysql\ndatabase be initialized with help topic information (see\nhttp://dev.mysql.com/doc/refman/5.1/en/server-side-help-support.html).… HELP statement searches the help tables for the given search string\nand displays the result of the search. The search string is not case\nsensitive.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/help.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/help.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (116,25,'LOAD DATA FROM MASTER','Syntax:\nLOAD DATA FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated as\nof MySQL 4.1 and removed in MySQL 5.5.\n\nThe recommended alternative solution to using LOAD DATA FROM MASTER or\nLOAD TABLE FROM MASTER is using mysqldump or mysqlhotcopy. The latter\nrequires Perl and two Perl modules (DBI and DBD:mysql) and works for\nMyISAM and ARCHIVE tables only. With mysqldump, you can create SQL\ndumps on the master and pipe (or copy) these to a mysql client on the\nslave. This has the advantage of working for all storage engines, but\ncan be quite slow, since it works using SELECT.\n\nThis statement takes a snapshot of the master and copies it to the\nslave. It updates the values of MASTER_LOG_FILE and MASTER_LOG_POS so\nthat the slave starts replicating from the correct position. Any table\nand database exclusion rules specified with the --replicate-*-do-* and\n--replicate-*-ignore-* options are honored. --replicate-rewrite-db is\nnot taken into account because a user could use this option to set up a\nnonunique mapping such as --replicate-rewrite-db="db1->db3" and\n--replicate-rewrite-db="db2->db3", which would confuse the slave when\nloading tables from the master.\n\nUse of this statement is subject to the following conditions:\n\no It works only for MyISAM tables. Attempting to load a non-MyISAM\n table results in the following error:\n\nERROR 1189 (08S01): Net error reading from master\n\no It acquires a global read lock on the master while taking the\n snapshot, which prevents updates on the master during the load\n operation.\n\nIf you are loading large tables, you might have to increase the values\nof net_read_timeout and net_write_timeout on both the master and slave\nservers. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html.\n… that LOAD DATA FROM MASTER does not copy any tables from the mysql\ndatabase. This makes it easy to have different users and privileges on\nthe master and the slave.\n\nTo use LOAD DATA FROM MASTER, the replication account that is used to\nconnect to the master must have the RELOAD and SUPER privileges on the\nmaster and the SELECT privilege for all master tables you want to load.\nAll master tables for which the user does not have the SELECT privilege\nare ignored by LOAD DATA FROM MASTER. This is because the master hides\nthem from the user: LOAD DATA FROM MASTER calls SHOW DATABASES to know\nthe master databases to load, but SHOW DATABASES returns only databases\nfor which the user has some privilege. See [HELP SHOW DATABASES]. On\nthe slave side, the user that issues LOAD DATA FROM MASTER must have\nprivileges for dropping and creating the databases and tables that are\ncopied.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (117,25,'RESET','Syntax:\nRESET reset_option [, reset_option] ...\n\nThe RESET statement is used to clear the state of various server\noperations. You must have the RELOAD privilege to execute RESET.\n\nRESET acts as a stronger version of the FLUSH statement. See [HELP\nFLUSH].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (118,14,'GET_LOCK','Syntax:\nGET_LOCK(str,timeout)\n\nTries to obtain a lock with a name given by the string str, using a\ntimeout of timeout seconds. Returns 1 if the lock was obtained\nsuccessfully, 0 if the attempt timed out (for example, because another\nclient has previously locked the name), or NULL if an error occurred\n(such as running out of memory or the thread was killed with mysqladmin\nkill). If you have a lock obtained with GET_LOCK(), it is released when\nyou execute RELEASE_LOCK(), execute a new GET_LOCK(), or your\nconnection terminates (either normally or abnormally). Locks obtained\nwith GET_LOCK() do not interact with transactions. That is, committing\na transaction does not release any such locks obtained during the\ntransaction.\n\nThis function can be used to implement application locks or to simulate\nrecord locks. Names are locked on a server-wide basis. If a name has\nbeen locked by one client, GET_LOCK() blocks any request by another\nclient for a lock with the same name. This allows clients that agree on\na given lock name to use the name to perform cooperative advisory\nlocking. But be aware that it also allows a client that is not among\nthe set of cooperating clients to lock a name, either inadvertently or\ndeliberately, and thus prevent any of the cooperating clients from\nlocking that name. One way to reduce the likelihood of this is to use\nlock names that are database-specific or application-specific. For\nexample, use lock names of the form db_name.str or app_name.str.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','mysql> SELECT GET_LOCK(\'lock1\',10);\n -> 1\nmysql> SELECT IS_FREE_LOCK(\'lock2\');\n -> 1\nmysql> SELECT GET_LOCK(\'lock2\',10);\n -> 1\nmysql> SELECT RELEASE_LOCK(\'lock2\');\n -> 1\nmysql> SELECT RELEASE_LOCK(\'lock1\');\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (119,35,'UCASE','Syntax:\nUCASE(str)\n\nUCASE() is a synonym for UPPER().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (120,25,'SHOW BINLOG EVENTS','Syntax:\nSHOW BINLOG EVENTS\n [IN \'log_name\'] [FROM pos] [LIMIT [offset,] row_count]\n\nShows the events in the binary log. If you do not specify \'log_name\',\nthe first binary log is displayed.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-binlog-events.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-binlog-events.html'); @@ -199,7 +199,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (130,25,'SHOW OPEN TABLES','Syntax:\nSHOW OPEN TABLES [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW OPEN TABLES lists the non-TEMPORARY tables that are currently open\nin the table cache. See\nhttp://dev.mysql.com/doc/refman/5.1/en/table-cache.html. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nThe FROM and LIKE clauses may be used beginning with MySQL 5.1.24. The\nLIKE clause, if present, indicates which table names to match. The FROM\nclause, if present, restricts the tables shown to those present in the\ndb_name database.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-open-tables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-open-tables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (131,30,'EXTRACT','Syntax:\nEXTRACT(unit FROM date)\n\nThe EXTRACT() function uses the same kinds of unit specifiers as\nDATE_ADD() or DATE_SUB(), but extracts parts from the date rather than\nperforming date arithmetic.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT EXTRACT(YEAR FROM \'2009-07-02\');\n -> 2009\nmysql> SELECT EXTRACT(YEAR_MONTH FROM \'2009-07-02 01:02:03\');\n -> 200907\nmysql> SELECT EXTRACT(DAY_MINUTE FROM \'2009-07-02 01:02:03\');\n -> 20102\nmysql> SELECT EXTRACT(MICROSECOND\n -> FROM \'2003-01-02 10:30:00.000123\');\n -> 123\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (132,11,'ENCRYPT','Syntax:\nENCRYPT(str[,salt])\n\nEncrypts str using the Unix crypt() system call and returns a binary\nstring. The salt argument should be a string with at least two\ncharacters. If no salt argument is given, a random value is used.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','mysql> SELECT ENCRYPT(\'hello\');\n -> \'VxuFAJXVARROc\'\n','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (133,25,'SHOW STATUS','Syntax:\nSHOW [GLOBAL | SESSION] STATUS\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW STATUS provides server status information. This information also\ncan be obtained using the mysqladmin extended-status command. The LIKE\nclause, if present, indicates which variable names to match. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\nThis statement does not require any privilege. It requires only the\nability to connect to the server.\nWith a LIKE clause, the statement displays only rows for those\nvariables with names that match the pattern:\n\nmysql> SHOW STATUS LIKE \'Key%\';\n+--------------------+----------+\n| Variable_name | Value |\n+--------------------+----------+\n| Key_blocks_used | 14955 |\n| Key_read_requests | 96854827 |\n| Key_reads | 162040 |\n| Key_write_requests | 7589728 |\n| Key_writes | 3813196 |\n+--------------------+----------+\n\nWith the GLOBAL modifier, SHOW STATUS displays the status values for\nall connections to MySQL. With SESSION, it displays the status values\nfor the current connection. If no modifier is present, the default is\nSESSION. LOCAL is a synonym for SESSION.\n\nSome status variables have only a global value. For these, you get the\nsame value for both GLOBAL and SESSION. The scope for each status\nvariable is listed at\nhttp://dev.mysql.com/doc/refman/5.1/en/server-status-variables.html.\n\…: http://dev.mysql.com/doc/refman/5.1/en/show-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-status.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (133,25,'SHOW STATUS','Syntax:\nSHOW [GLOBAL | SESSION] STATUS\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW STATUS provides server status information. This information also\ncan be obtained using the mysqladmin extended-status command. The LIKE\nclause, if present, indicates which variable names to match. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\nThis statement does not require any privilege. It requires only the\nability to connect to the server.\nWith a LIKE clause, the statement displays only rows for those\nvariables with names that match the pattern:\n\nmysql> SHOW STATUS LIKE \'Key%\';\n+--------------------+----------+\n| Variable_name | Value |\n+--------------------+----------+\n| Key_blocks_used | 14955 |\n| Key_read_requests | 96854827 |\n| Key_reads | 162040 |\n| Key_write_requests | 7589728 |\n| Key_writes | 3813196 |\n+--------------------+----------+\n\nWith the GLOBAL modifier, SHOW STATUS displays the status values for\nall connections to MySQL. With SESSION, it displays the status values\nfor the current connection. If no modifier is present, the default is\nSESSION. LOCAL is a synonym for SESSION.\n\nSome status variables have only a global value. For these, you get the\nsame value for both GLOBAL and SESSION. The scope for each status\nvariable is listed at\nhttp://dev.mysql.com/doc/refman/5.1/en/server-status-variables.html.\n\… invocation of the SHOW STATUS statement uses an internal temporary\ntable and increments the global Created_tmp_tables value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-status.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (134,35,'EXTRACTVALUE','Syntax:\nExtractValue(xml_frag, xpath_expr)\n\nExtractValue() takes two string arguments, a fragment of XML markup\nxml_frag and an XPath expression xpath_expr (also known as a locator);\nit returns the text (CDATA) of the first text node which is a child of\nthe element(s) matched by the XPath expression. It is the equivalent of\nperforming a match using the xpath_expr after appending /text(). In\nother words, ExtractValue(\'<a>Sakila</a>\', \'/a/b\') and\nExtractValue(\'<a>Sakila</a>\', \'/a/b/text()\') produce the same\nresult.\n\nIf multiple matches are found, then the content of the first child text\nnode of each matching element is returned (in the order matched) as a\nsingle, space-delimited string.\n\nIf no matching text node is found for the expression (including the\nimplicit /text()) --- for whatever reason, as long as xpath_expr is\nvalid, and xml_frag consists of elements which are properly nested and\nclosed --- an empty string is returned. No distinction is made between\na match on an empty element and no match at all. This is by design.\n\nIf you need to determine whether no matching element was found in\nxml_frag or such an element was found but contained no child text\nnodes, you should test the result of an expression that uses the XPath\ncount() function. For example, both of these statements return an empty\nstring, as shown here:\n\nmysql> SELECT ExtractValue(\'<a></a>\', \'/a/b\');\n+-------------------------------------+\n| ExtractValue(\'<a></a>\', \'/a/b\') |\n+-------------------------------------+\n| |\n+-------------------------------------+\n1 row in set (0.00 sec)\n\nmysql> SELECT ExtractValue(\'<a><c/></a>\', \'/a/b\');\n+-------------------------------------+\n| ExtractValue(\'<a><c/></a>\', \'/a/b\') |\n+-------------------------------------+\n| |\n+-------------------------------------+\n1 row in set (0.00 sec)\n\nHowever, you can determine whether there was actually a matching\nelement using the following:\n\nmysql> SELECT ExtractValue(\'<a></a>\', \'count(/a/b)\');\n+-------------------------------------+\n| ExtractValue(\'<a></a>\', \'count(/a/b)\') |\n+-------------------------------------+\n| 1 |\n+-------------------------------------+\n1 row in set (0.00 sec)\n\nmysql> SELECT ExtractValue(\'<a><c/></a>\', \'count(/a/b)\');\n+-------------------------------------+\n| ExtractValue(\'<a><c/></a>\', \'count(/a/b)\') |\n+-------------------------------------+\n| 0 |\n+-------------------------------------+\n1 row in set (0.01 sec)\n\n*Important*: ExtractValue() returns only CDATA, and does not return any\ntags that might be contained within a matching tag, nor any of their\ncontent (see the result returned as val1 in the following example).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html\n\n','mysql> SELECT\n -> ExtractValue(\'<a>cccddd</a>\', \'/a\') AS val1,\n -> ExtractValue(\'<a>cccddd</a>\', \'/a/b\') AS val2,\n -> ExtractValue(\'<a>cccddd</a>\', \'//b\') AS val3,\n -> ExtractValue(\'<a>cccddd</a>\', \'/b\') AS val4,\n -> ExtractValue(\'<a>cccdddeee</a>\', \'//b\') AS val5;\n\n+------+------+------+------+---------+\n| val1 | val2 | val3 | val4 | val5 |\n+------+------+------+------+---------+\n| ccc | ddd | ddd | | ddd eee |\n+------+------+------+------+---------+\n','http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (135,11,'OLD_PASSWORD','Syntax:\nOLD_PASSWORD(str)\n\nOLD_PASSWORD() was added to MySQL when the implementation of PASSWORD()\nwas changed to improve security. OLD_PASSWORD() returns the value of\nthe old (pre-4.1) implementation of PASSWORD() as a binary string, and\nis intended to permit you to reset passwords for any pre-4.1 clients\nthat need to connect to your version 5.1 MySQL server without locking\nthem out. See\nhttp://dev.mysql.com/doc/refman/5.1/en/password-hashing.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (136,22,'SET VARIABLE','Syntax:\nSET var_name = expr [, var_name = expr] ...\n\nThe SET statement in stored programs is an extended version of the\ngeneral SET statement (see [HELP SET]). Each var_name may refer to a\nlocal variable declared inside a stored program, a system variable, or\na user-defined variable.\n\nThe SET statement in stored programs is implemented as part of the\npre-existing SET syntax. This allows an extended syntax of SET a=x,\nb=y, ... where different variable types (locally declared variables,\nglobal and session system variables, user-defined variables) can be\nmixed. This also allows combinations of local variables and some\noptions that make sense only for system variables; in that case, the\noptions are recognized but ignored.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-statement.html'); @@ -239,31 +239,31 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (170,30,'DAYOFYEAR','Syntax:\nDAYOFYEAR(date)\n\nReturns the day of the year for date, in the range 1 to 366.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DAYOFYEAR(\'2007-02-03\');\n -> 34\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (171,20,'LONGTEXT','LONGTEXT [CHARACTER SET charset_name] [COLLATE collation_name]\n\nA TEXT column with a maximum length of 4,294,967,295 or 4GB (232 - 1)\ncharacters. The effective maximum length is less if the value contains\nmulti-byte characters. The effective maximum length of LONGTEXT columns\nalso depends on the configured maximum packet size in the client/server\nprotocol and available memory. Each LONGTEXT value is stored using a\nfour-byte length prefix that indicates the number of bytes in the\nvalue.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (172,4,'%','Syntax:\nN % M\n\nModulo operation. Returns the remainder of N divided by M. For more\ninformation, see the description for the MOD() function in\nhttp://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html.\n\n…: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (173,25,'KILL','Syntax:\nKILL [CONNECTION | QUERY] thread_id\n\nEach connection to mysqld runs in a separate thread. You can see which\nthreads are running with the SHOW PROCESSLIST statement and kill a\nthread with the KILL thread_id statement.\n\nKILL allows the optional CONNECTION or QUERY modifier:\n\no KILL CONNECTION is the same as KILL with no modifier: It terminates\n the connection associated with the given thread_id.\n\no KILL QUERY terminates the statement that the connection is currently\n executing, but leaves the connection itself intact.\n\nIf you have the PROCESS privilege, you can see all threads. If you have\nthe SUPER privilege, you can kill all threads and statements.\nOtherwise, you can see and kill only your own threads and statements.\n\nYou can also use the mysqladmin processlist and mysqladmin kill\ncommands to examine and kill threads.\n\n*Note*: You cannot use KILL with the Embedded MySQL Server library,\nbecause the embedded server merely runs inside the threads of the host\napplication. It does not create any connection threads of its own.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/kill.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/kill.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (173,25,'KILL','Syntax:\nKILL [CONNECTION | QUERY] thread_id\n\nEach connection to mysqld runs in a separate thread. You can see which\nthreads are running with the SHOW PROCESSLIST statement and kill a\nthread with the KILL thread_id statement.\n\nKILL allows the optional CONNECTION or QUERY modifier:\n\no KILL CONNECTION is the same as KILL with no modifier: It terminates\n the connection associated with the given thread_id.\n\no KILL QUERY terminates the statement that the connection is currently\n executing, but leaves the connection itself intact.\n\nIf you have the PROCESS privilege, you can see all threads. If you have\nthe SUPER privilege, you can kill all threads and statements.\nOtherwise, you can see and kill only your own threads and statements.\n\nYou can also use the mysqladmin processlist and mysqladmin kill\ncommands to examine and kill threads.\n\n*Note*: You cannot use KILL with the Embedded MySQL Server library\nbecause the embedded server merely runs inside the threads of the host\napplication. It does not create any connection threads of its own.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/kill.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/kill.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (174,29,'DISJOINT','Disjoint(g1,g2)\n\nReturns 1 or 0 to indicate whether g1 is spatially disjoint from (does\nnot intersect) g2.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (175,3,'ASTEXT','AsText(g), AsWKT(g)\n\nConverts a value in internal geometry format to its WKT representation\nand returns the string result.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…','mysql> SET @g = \'LineString(1 1,2 2,3 3)\';\nmysql> SELECT AsText(GeomFromText(@g));\n+--------------------------+\n| AsText(GeomFromText(@g)) |\n+--------------------------+\n| LINESTRING(1 1,2 2,3 3) |\n+--------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (176,35,'LPAD','Syntax:\nLPAD(str,len,padstr)\n\nReturns the string str, left-padded with the string padstr to a length\nof len characters. If str is longer than len, the return value is\nshortened to len characters.\n\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT LPAD(\'hi\',4,\'??\');\n -> \'??hi\'\nmysql> SELECT LPAD(\'hi\',1,\'??\');\n -> \'h\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (177,19,'RESTORE TABLE','Syntax:\nRESTORE TABLE tbl_name [, tbl_name] ... FROM \'/path/to/backup/directory\'\n\nRESTORE TABLE restores the table or tables from a backup that was made\nwith BACKUP TABLE. The directory should be specified as a full path\nname.\n\nExisting tables are not overwritten; if you try to restore over an\nexisting table, an error occurs. Just as for BACKUP TABLE, RESTORE\nTABLE currently works only for MyISAM tables. Restored tables are not\nreplicated from master to slave.\n\nThe backup for each table consists of its .frm format file and .MYD\ndata file. The restore operation restores those files, and then uses\nthem to rebuild the .MYI index file. Restoring takes longer than\nbacking up due to the need to rebuild the indexes. The more indexes the\ntable has, the longer it takes.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/restore-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/restore-table.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (178,22,'DECLARE CONDITION','Syntax:\nDECLARE condition_name CONDITION FOR condition_value\n\ncondition_value:\n SQLSTATE [VALUE] sqlstate_value\n | mysql_error_code\n\nThe DECLARE ... CONDITION statement defines a named error condition. It\nspecifies a condition that needs specific handling and associates a\nname with that condition. The name can be referred to in a subsequence\nDECLARE ... HANDLER statement. See [HELP DECLARE HANDLER].\n\nA condition_value for DECLARE ... CONDITION can be an SQLSTATE value (a\n5-character string literal) or a MySQL error code (a number). You\nshould not use SQLSTATE value \'00000\' or MySQL error code 0, because\nthose indicate sucess rather than an error condition. For a list of\nSQLSTATE values and MySQL error codes, see\nhttp://dev.mysql.com/doc/refman/5.1/en/error-messages-server.html.\n\n…: http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (177,19,'RESTORE TABLE','Syntax:\nRESTORE TABLE tbl_name [, tbl_name] ... FROM \'/path/to/backup/directory\'\n\n*Note*: This statement is deprecated and is removed in MySQL 5.5.\n\nRESTORE TABLE restores the table or tables from a backup that was made\nwith BACKUP TABLE. The directory should be specified as a full path\nname.\n\nExisting tables are not overwritten; if you try to restore over an\nexisting table, an error occurs. Just as for BACKUP TABLE, RESTORE\nTABLE currently works only for MyISAM tables. Restored tables are not\nreplicated from master to slave.\n\nThe backup for each table consists of its .frm format file and .MYD\ndata file. The restore operation restores those files, and then uses\nthem to rebuild the .MYI index file. Restoring takes longer than\nbacking up due to the need to rebuild the indexes. The more indexes the\ntable has, the longer it takes.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/restore-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/restore-table.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (178,22,'DECLARE CONDITION','Syntax:\nDECLARE condition_name CONDITION FOR condition_value\n\ncondition_value:\n SQLSTATE [VALUE] sqlstate_value\n | mysql_error_code\n\nThe DECLARE ... CONDITION statement defines a named error condition. It\nspecifies a condition that needs specific handling and associates a\nname with that condition. The name can be referred to in a subsequent\nDECLARE ... HANDLER statement. For an example, see [HELP DECLARE\nHANDLER].\n\nA condition_value for DECLARE ... CONDITION can be an SQLSTATE value (a\n5-character string literal) or a MySQL error code (a number). You\nshould not use SQLSTATE value \'00000\' or MySQL error code 0, because\nthose indicate success rather than an error condition. For a list of\nSQLSTATE values and MySQL error codes, see\nhttp://dev.mysql.com/doc/refman/5.1/en/error-messages-server.html.\n\n…: http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (179,29,'OVERLAPS','Overlaps(g1,g2)\n\nReturns 1 or 0 to indicate whether g1 spatially overlaps g2. The term\nspatially overlaps is used if two geometries intersect and their\nintersection results in a geometry of the same dimension but not equal\nto either of the given geometries.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (180,25,'SET GLOBAL SQL_SLAVE_SKIP_COUNTER','Syntax:\nSET GLOBAL SQL_SLAVE_SKIP_COUNTER = N\n\nThis statement skips the next N events from the master. This is useful\nfor recovering from replication stops caused by a statement.\n\nThis statement is valid only when the slave thread is not running.\nOtherwise, it produces an error.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…','','http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (180,25,'SET GLOBAL SQL_SLAVE_SKIP_COUNTER','Syntax:\nSET GLOBAL sql_slave_skip_counter = N\n\nThis statement skips the next N events from the master. This is useful\nfor recovering from replication stops caused by a statement.\n\nThis statement is valid only when the slave threads are not running.\nOtherwise, it produces an error.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…','','http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (181,24,'NUMGEOMETRIES','NumGeometries(gc)\n\nReturns the number of geometries in the GeometryCollection value gc.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#geo…','mysql> SET @gc = \'GeometryCollection(Point(1 1),LineString(2 2, 3 3))\';\nmysql> SELECT NumGeometries(GeomFromText(@gc));\n+----------------------------------+\n| NumGeometries(GeomFromText(@gc)) |\n+----------------------------------+\n| 2 |\n+----------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#geo…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (182,30,'MONTHNAME','Syntax:\nMONTHNAME(date)\n\nReturns the full name of the month for date. As of MySQL 5.1.12, the\nlanguage used for the name is controlled by the value of the\nlc_time_names system variable\n(http://dev.mysql.com/doc/refman/5.1/en/locale-support.html).\n\n…: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MONTHNAME(\'2008-02-03\');\n -> \'February\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (183,36,'PROCEDURE ANALYSE','Syntax:\nANALYSE([max_elements[,max_memory]])\n\nANALYSE() is defined in the sql/sql_analyse.cc source file, which\nserves as an example of how to create a procedure for use with the\nPROCEDURE clause of SELECT statements. ANALYSE() is built in and is\navailable by default; other procedures can be created using the format\ndemonstrated in the source file.\n\nANALYSE() examines the result from a query and returns an analysis of\nthe results that suggests optimal data types for each column that may\nhelp reduce table sizes. To obtain this analysis, append PROCEDURE\nANALYSE to the end of a SELECT statement:\n\nSELECT ... FROM ... WHERE ... PROCEDURE ANALYSE([max_elements,[max_memory]])\n\nFor example:\n\nSELECT col1, col2 FROM table1 PROCEDURE ANALYSE(10, 2000);\n\nThe results show some statistics for the values returned by the query,\nand propose an optimal data type for the columns. This can be helpful\nfor checking your existing tables, or after importing new data. You may\nneed to try different settings for the arguments so that PROCEDURE\nANALYSE() does not suggest the ENUM data type when it is not\nappropriate.\n\nThe arguments are optional and are used as follows:\n\no max_elements (default 256) is the maximum number of distinct values\n that ANALYSE() notices per column. This is used by ANALYSE() to check\n whether the optimal data type should be of type ENUM; if there are\n more than max_elements distinct values, then ENUM is not a suggested\n type.\n\no max_memory (default 8192) is the maximum amount of memory that\n ANALYSE() should allocate per column while trying to find all\n distinct values.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/procedure-analyse.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/procedure-analyse.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (184,25,'CHANGE MASTER TO','Syntax:\nCHANGE MASTER TO master_def [, master_def] ...\n\nmaster_def:\n MASTER_BIND = \'interface_name\'\n | MASTER_HOST = \'host_name\'\n | MASTER_USER = \'user_name\'\n | MASTER_PASSWORD = \'password\'\n | MASTER_PORT = port_num\n | MASTER_CONNECT_RETRY = interval\n | MASTER_HEARTBEAT_PERIOD = interval\n | MASTER_LOG_FILE = \'master_log_name\'\n | MASTER_LOG_POS = master_log_pos\n | RELAY_LOG_FILE = \'relay_log_name\'\n | RELAY_LOG_POS = relay_log_pos\n | MASTER_SSL = {0|1}\n | MASTER_SSL_CA = \'ca_file_name\'\n | MASTER_SSL_CAPATH = \'ca_directory_name\'\n | MASTER_SSL_CERT = \'cert_file_name\'\n | MASTER_SSL_KEY = \'key_file_name\'\n | MASTER_SSL_CIPHER = \'cipher_list\'\n | MASTER_SSL_VERIFY_SERVER_CERT = {0|1}\n\nCHANGE MASTER TO changes the parameters that the slave server uses for\nconnecting to and communicating with the master server. It also updates\nthe contents of the master.info and relay-log.info files.\n\nMASTER_USER, MASTER_PASSWORD, MASTER_SSL, MASTER_SSL_CA,\nMASTER_SSL_CAPATH, MASTER_SSL_CERT, MASTER_SSL_KEY, MASTER_SSL_CIPHER,\nand MASTER_SSL_VERIFY_SERVER_CERT provide information to the slave\nabout how to connect to its master. MASTER_SSL_VERIFY_SERVER_CERT was\nadded in MySQL 5.1.18. It is used as described for the\n--ssl-verify-server-cert option in\nhttp://dev.mysql.com/doc/refman/5.1/en/ssl-options.html.\n\nMASTER_CONN… specifies how many seconds to wait between connect\nretries. The default is 60. The number of reconnection attempts is\nlimited by the --master-retry-count server option; for more\ninformation, see\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-options.html.\n\nThe SSL options (MASTER_SSL, MASTER_SSL_CA, MASTER_SSL_CAPATH,\nMASTER_SSL_CERT, MASTER_SSL_KEY, MASTER_SSL_CIPHER), and\nMASTER_SSL_VERIFY_SERVER_CERT can be changed even on slaves that are\ncompiled without SSL support. They are saved to the master.info file,\nbut are ignored unless you use a server that has SSL support enabled.\n\nIf you do not specify a given parameter, it keeps its old value, except\nas indicated in the following discussion. For example, if the password\nto connect to your MySQL master has changed, you just need to issue\nthese statements to tell the slave about the new password:\n\nSTOP SLAVE; -- if replication was running\nCHANGE MASTER TO MASTER_PASSWORD=\'new3cret\';\nSTART SLAVE; -- if you want to restart replication\n\nThere is no need to specify the parameters that do not change (host,\nport, user, and so forth).\n\nMASTER_HOST and MASTER_PORT are the host name (or IP address) of the\nmaster host and its TCP/IP port.\n\nThe next two options (MASTER_BIND and MASTER_HEARTBEAT_PERIOD) are\navailable in MySQL Cluster NDB 6.3 and later, but are not supported in\nmainline MySQL 5.1:\n\no MASTER_BIND is for use on replication slaves having multiple network\n interfaces, and determines which of the slave\'s network interfaces is\n chosen for connecting to the master. It is also possible to determine\n which network interface is to be used in such cases by starting the\n slave mysqld process with the --master-bind option.\n\n The ability to bind a replication slave to specific network interface\n was added in MySQL Cluster NDB 6.3.4.\n\no MASTER_HEARTBEAT_PERIOD is used to set the interval in seconds\n between replication heartbeats. Whenever the master\'s binlog is\n updated with an event, the waiting period for the next heartbeat is\n reset. interval is a decimal value having the range 0 to 4294967\n seconds and a resolution to hundredths of a second; the smallest\n nonzero value is 0.001. Heartbeats are sent by the master only if\n there are no unsent events in the binlog file for a period longer\n than interval.\n\n Setting interval to 0 disables heartbeats altogether. The default\n value for interval is equal to the value of slave_net_timeout divided\n by 2.\n\n Setting @@global.slave_net_timeout to a value less than that of the\n current heartbeat interval results in a warning being issued. The\n effect of issuing RESET SLAVE on the heartbeat interval is to reset\n it to the default value.\n\n MASTER_HEARTBEAT_PERIOD was added in MySQL Cluster NDB 6.3.4.\n\n*Note*: Replication cannot use Unix socket files. You must be able to\nconnect to the master MySQL server using TCP/IP.\n\nIf you specify MASTER_HOST or MASTER_PORT, the slave assumes that the\nmaster server is different from before (even if you specify a host or\nport value that is the same as the current value.) In this case, the\nold values for the master binary log name and position are considered\nno longer applicable, so if you do not specify MASTER_LOG_FILE and\nMASTER_LOG_POS in the statement, MASTER_LOG_FILE=\'\' and\nMASTER_LOG_POS=4 are silently appended to it.\n\nSetting MASTER_HOST=\'\' --- that is, setting its value explicitly to an\nempty string --- is not the same as not setting it at all. Setting this\noption to an empty string causes START SLAVE subsequently to fail. This\nissue is addressed in MySQL 5.5. (Bug#28796\n(http://bugs.mysql.com/28796))\n\nMASTER_LOG_FILE and MASTER_LOG_POS are the coordinates at which the\nslave I/O thread should begin reading from the master the next time the\nthread starts. If you specify either of them, you cannot specify\nRELAY_LOG_FILE or RELAY_LOG_POS. If neither of MASTER_LOG_FILE or\nMASTER_LOG_POS are specified, the slave uses the last coordinates of\nthe slave SQL thread before CHANGE MASTER TO was issued. This ensures\nthat there is no discontinuity in replication, even if the slave SQL\nthread was late compared to the slave I/O thread, when you merely want\nto change, say, the password to use.\n\nCHANGE MASTER TO deletes all relay log files and starts a new one,\nunless you specify RELAY_LOG_FILE or RELAY_LOG_POS. In that case, relay\nlogs are kept; the relay_log_purge global variable is set silently to\n0.\n\nCHANGE MASTER TO is useful for setting up a slave when you have the\nsnapshot of the master and have recorded the log and the offset\ncorresponding to it. After loading the snapshot into the slave, you can\nrun CHANGE MASTER TO MASTER_LOG_FILE=\'log_name_on_master\',\nMASTER_LOG_POS=log_offset_on_master on the slave.\n\nThe following example changes the master and master\'s binary log\ncoordinates. This is used when you want to set up the slave to\nreplicate the master:\n\nCHANGE MASTER TO\n MASTER_HOST=\'master2.mycompany.com\',\n MASTER_USER=\'replication\',\n MASTER_PASSWORD=\'bigs3cret\',\n MASTER_PORT=3306,\n MASTER_LOG_FILE=\'master2-bin.001\',\n MASTER_LOG_POS=4,\n MASTER_CONNECT_RETRY=10;\n\nThe next example shows an operation that is less frequently employed.\nIt is used when the slave has relay logs that you want it to execute\nagain for some reason. To do this, the master need not be reachable.\nYou need only use CHANGE MASTER TO and start the SQL thread (START\nSLAVE SQL_THREAD):\n\nCHANGE MASTER TO\n RELAY_LOG_FILE=\'slave-relay-bin.006\',\n RELAY_LOG_POS=4025;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (184,25,'CHANGE MASTER TO','Syntax:\nCHANGE MASTER TO option [, option] ...\n\noption:\n MASTER_BIND = \'interface_name\'\n | MASTER_HOST = \'host_name\'\n | MASTER_USER = \'user_name\'\n | MASTER_PASSWORD = \'password\'\n | MASTER_PORT = port_num\n | MASTER_CONNECT_RETRY = interval\n | MASTER_HEARTBEAT_PERIOD = interval\n | MASTER_LOG_FILE = \'master_log_name\'\n | MASTER_LOG_POS = master_log_pos\n | RELAY_LOG_FILE = \'relay_log_name\'\n | RELAY_LOG_POS = relay_log_pos\n | MASTER_SSL = {0|1}\n | MASTER_SSL_CA = \'ca_file_name\'\n | MASTER_SSL_CAPATH = \'ca_directory_name\'\n | MASTER_SSL_CERT = \'cert_file_name\'\n | MASTER_SSL_KEY = \'key_file_name\'\n | MASTER_SSL_CIPHER = \'cipher_list\'\n | MASTER_SSL_VERIFY_SERVER_CERT = {0|1}\n\nCHANGE MASTER TO changes the parameters that the slave server uses for\nconnecting to the master server, for reading the master binary log, and\nreading the slave relay log. It also updates the contents of the\nmaster.info and relay-log.info files. To use CHANGE MASTER TO, the\nslave replication threads must be stopped (use STOP SLAVE if\nnecessary).\n\nOptions not specified retain their value, except as indicated in the\nfollowing discussion. Thus, in most cases, there is no need to specify\noptions that do not change. For example, if the password to connect to\nyour MySQL master has changed, you just need to issue these statements\nto tell the slave about the new password:\n\nSTOP SLAVE; -- if replication was running\nCHANGE MASTER TO MASTER_PASSWORD=\'new3cret\';\nSTART SLAVE; -- if you want to restart replication\n\nMASTER_HOST, MASTER_USER, MASTER_PASSWORD, and MASTER_PORT provide\ninformation to the slave about how to connect to its master:\n\no MASTER_HOST and MASTER_PORT are the host name (or IP address) of the\n master host and its TCP/IP port.\n\n *Note*: Replication cannot use Unix socket files. You must be able to\n connect to the master MySQL server using TCP/IP.\n\n If you specify the MASTER_HOST or MASTER_PORT option, the slave\n assumes that the master server is different from before (even if the\n option value is the same as its current value.) In this case, the old\n values for the master binary log file name and position are\n considered no longer applicable, so if you do not specify\n MASTER_LOG_FILE and MASTER_LOG_POS in the statement,\n MASTER_LOG_FILE=\'\' and MASTER_LOG_POS=4 are silently appended to it.\n\n Setting MASTER_HOST=\'\' (that is, setting its value explicitly to an\n empty string) is not the same as not setting MASTER_HOST at all.\n Setting this option to an empty string causes START SLAVE\n subsequently to fail. This issue is addressed in MySQL 5.5.\n (Bug#28796 (http://bugs.mysql.com/bug.php?id=28796))\n\no MASTER_USER and MASTER_PASSWORD are the user name and password of the\n account to use for connecting to the master.\n\nThe MASTER_SSL_xxx options provide information about using SSL for the\nconnection. They correspond to the --ssl-xxx options described in\nhttp://dev.mysql.com/doc/refman/5.1/en/ssl-options.html, and\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-solutions-ssl.html.… was added in MySQL 5.1.18. These options\ncan be changed even on slaves that are compiled without SSL support.\nThey are saved to the master.info file, but are ignored if the slave\ndoes not have SSL support enabled.\n\nMASTER_CONNECT_RETRY specifies how many seconds to wait between connect\nretries. The default is 60. The number of reconnection attempts is\nlimited by the --master-retry-count server option; for more\ninformation, see\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-options.html.\n\nThe next two options (MASTER_BIND and MASTER_HEARTBEAT_PERIOD) are\navailable in MySQL Cluster NDB 6.3 and later, but are not supported in\nmainline MySQL 5.1:\n\nMASTER_BIND is for use on replication slaves having multiple network\ninterfaces, and determines which of the slave\'s network interfaces is\nchosen for connecting to the master. It is also possible to determine\nwhich network interface is to be used in such cases by starting the\nslave mysqld process with the --master-bind option.\n\nThe ability to bind a replication slave to specific network interface\nwas added in MySQL Cluster NDB 6.3.4.\n\nMASTER_HEARTBEAT_PERIOD is used to set the interval in seconds between\nreplication heartbeats. Whenever the master\'s binary log is updated\nwith an event, the waiting period for the next heartbeat is reset.\ninterval is a decimal value having the range 0 to 4294967 seconds and a\nresolution to hundredths of a second; the smallest nonzero value is\n0.01. Heartbeats are sent by the master only if there are no unsent\nevents in the binary log file for a period longer than interval.\n\nSetting interval to 0 disables heartbeats altogether. The default value\nfor interval is equal to the value of slave_net_timeout divided by 2.\n\nSetting @@global.slave_net_timeout to a value less than that of the\ncurrent heartbeat interval results in a warning being issued. The\neffect of issuing RESET SLAVE on the heartbeat interval is to reset it\nto the default value.\n\nMASTER_HEARTBEAT_PERIOD was added in MySQL Cluster NDB 6.3.4.\n\nMASTER_LOG_FILE and MASTER_LOG_POS are the coordinates at which the\nslave I/O thread should begin reading from the master the next time the\nthread starts. RELAY_LOG_FILE and RELAY_LOG_POS are the coordinates at\nwhich the slave SQL thread should begin reading from the relay log the\nnext time the thread starts. If you specify either of MASTER_LOG_FILE\nor MASTER_LOG_POS, you cannot specify RELAY_LOG_FILE or RELAY_LOG_POS.\nIf neither of MASTER_LOG_FILE or MASTER_LOG_POS is specified, the slave\nuses the last coordinates of the slave SQL thread before CHANGE MASTER\nTO was issued. This ensures that there is no discontinuity in\nreplication, even if the slave SQL thread was late compared to the\nslave I/O thread, when you merely want to change, say, the password to\nuse.\n\nCHANGE MASTER TO deletes all relay log files and starts a new one,\nunless you specify RELAY_LOG_FILE or RELAY_LOG_POS. In that case, relay\nlog files are kept; the relay_log_purge global variable is set silently\nto 0.\n\nCHANGE MASTER TO is useful for setting up a slave when you have the\nsnapshot of the master and have recorded the master binary log\ncoordinates corresponding to the time of the snapshot. After loading\nthe snapshot into the slave to synchronize it to the slave, you can run\nCHANGE MASTER TO MASTER_LOG_FILE=\'log_name\', MASTER_LOG_POS=log_pos on\nthe slave to specify the coodinates at which the slave should begin\nreading the master binary log.\n\nThe following example changes the master server the slave uses and\nestablishes the master binary log coordinates from which the slave\nbegins reading. This is used when you want to set up the slave to\nreplicate the master:\n\nCHANGE MASTER TO\n MASTER_HOST=\'master2.mycompany.com\',\n MASTER_USER=\'replication\',\n MASTER_PASSWORD=\'bigs3cret\',\n MASTER_PORT=3306,\n MASTER_LOG_FILE=\'master2-bin.001\',\n MASTER_LOG_POS=4,\n MASTER_CONNECT_RETRY=10;\n\nThe next example shows an operation that is less frequently employed.\nIt is used when the slave has relay log files that you want it to\nexecute again for some reason. To do this, the master need not be\nreachable. You need only use CHANGE MASTER TO and start the SQL thread\n(START SLAVE SQL_THREAD):\n\nCHANGE MASTER TO\n RELAY_LOG_FILE=\'slave-relay-bin.006\',\n RELAY_LOG_POS=4025;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (185,37,'DROP DATABASE','Syntax:\nDROP {DATABASE | SCHEMA} [IF EXISTS] db_name\n\nDROP DATABASE drops all tables in the database and deletes the\ndatabase. Be very careful with this statement! To use DROP DATABASE,\nyou need the DROP privilege on the database. DROP SCHEMA is a synonym\nfor DROP DATABASE.\n\n*Important*: When a database is dropped, user privileges on the\ndatabase are not automatically dropped. See [HELP GRANT].\n\nIF EXISTS is used to prevent an error from occurring if the database\ndoes not exist.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-database.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-database.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (186,6,'MBREQUAL','MBREqual(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 are the same.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (187,30,'TIMESTAMP FUNCTION','Syntax:\nTIMESTAMP(expr), TIMESTAMP(expr1,expr2)\n\nWith a single argument, this function returns the date or datetime\nexpression expr as a datetime value. With two arguments, it adds the\ntime expression expr2 to the date or datetime expression expr1 and\nreturns the result as a datetime value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMESTAMP(\'2003-12-31\');\n -> \'2003-12-31 00:00:00\'\nmysql> SELECT TIMESTAMP(\'2003-12-31 12:00:00\',\'12:00:00\');\n -> \'2004-01-01 00:00:00\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (188,35,'CHARACTER_LENGTH','Syntax:\nCHARACTER_LENGTH(str)\n\nCHARACTER_LENGTH() is a synonym for CHAR_LENGTH().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (189,25,'SHOW GRANTS','Syntax:\nSHOW GRANTS [FOR user]\n\nThis statement lists the GRANT statement or statements that must be\nissued to duplicate the privileges that are granted to a MySQL user\naccount. The account is named using the same format as for the GRANT\nstatement; for example, \'jeffrey\'@\'localhost\'. If you specify only the\nuser name part of the account name, a host name part of \'%\' is used.\nFor additional information about specifying account names, see [HELP\nGRANT].\n\nmysql> SHOW GRANTS FOR \'root\'@\'localhost\';\n+---------------------------------------------------------------------+\n| Grants for root@localhost |\n+---------------------------------------------------------------------+\n| GRANT ALL PRIVILEGES ON *.* TO \'root\'@\'localhost\' WITH GRANT OPTION |\n+---------------------------------------------------------------------+\n\nTo list the privileges granted to the account that you are using to\nconnect to the server, you can use any of the following statements:\n\nSHOW GRANTS;\nSHOW GRANTS FOR CURRENT_USER;\nSHOW GRANTS FOR CURRENT_USER();\n\nAs of MySQL 5.1.12, if SHOW GRANTS FOR CURRENT_USER (or any of the\nequivalent syntaxes) is used in DEFINER context, such as within a\nstored procedure that is defined with SQL SECURITY DEFINER), the grants\ndisplayed are those of the definer and not the invoker.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-grants.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-grants.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (190,25,'SHOW PRIVILEGES','Syntax:\nSHOW PRIVILEGES\n\nSHOW PRIVILEGES shows the list of system privileges that the MySQL\nserver supports. The exact list of privileges depends on the version of\nyour server.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-privileges.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-privileges.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (191,37,'CREATE TABLESPACE','Syntax:\nCREATE TABLESPACE tablespace_name\n ADD DATAFILE \'file_name\'\n USE LOGFILE GROUP logfile_group\n [EXTENT_SIZE [=] extent_size]\n [INITIAL_SIZE [=] initial_size]\n [AUTOEXTEND_SIZE [=] autoextend_size]\n [MAX_SIZE [=] max_size]\n [NODEGROUP [=] nodegroup_id]\n [WAIT]\n [COMMENT [=] comment_text]\n ENGINE [=] engine_name\n\nThis statement is used to create a tablespace, which can contain one or\nmore data files, providing storage space for tables. One data file is\ncreated and added to the tablespace using this statement. Additional\ndata files may be added to the tablespace by using the ALTER TABLESPACE\nstatement (see [HELP ALTER TABLESPACE]). For rules covering the naming\nof tablespaces, see\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and a log file group with the same name, or a\ntablespace and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/31770))\n\nA log file group of one or more UNDO log files must be assigned to the\ntablespace to be created with the USE LOGFILE GROUP clause.\nlogfile_group must be an existing log file group created with CREATE\nLOGFILE GROUP (see\nhttp://dev.mysql.com/doc/refman/5.1/en/create-logfile-group.html).\nMu… tablespaces may use the same log file group for UNDO logging.\n\nThe EXTENT_SIZE sets the size, in bytes, of the extents used by any\nfiles belonging to the tablespace. The default value is 1M. The minimum\nsize is 32K, and theoretical maximum is 2G, although the practical\nmaximum size depends on a number of factors. In most cases, changing\nthe extent size does not have any measurable effect on performance, and\nthe default value is recommended for all but the most unusual\nsituations.\n\nAn extent is a unit of disk space allocation. One extent is filled with\nas much data as that extent can contain before another extent is used.\nIn theory, up to 65,535 (64K) extents may used per data file; however,\nthe recommended maximum is 32,768 (32K). The recommended maximum size\nfor a single data file is 32G --- that is, 32K extents x 1 MB per\nextent. In addition, once an extent is allocated to a given partition,\nit cannot be used to store data from a different partition; an extent\ncannot store data from more than one partition. This means, for example\nthat a tablespace having a single datafile whose INITIAL_SIZE is 256 MB\nand whose EXTENT_SIZE is 128M has just two extents, and so can be used\nto store data from at most two different disk data table partitions.\n\nYou can see how many extents remain free in a given data file by\nquerying the INFORMATION_SCHEMA.FILES table, and so derive an estimate\nfor how much space remains free in the file. For further discussion and\nexamples, see http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nThe INITIAL_SIZE parameter sets the data file\'s total size in bytes.\nOnce the file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using ALTER TABLESPACE\n... ADD DATAFILE. See [HELP ALTER TABLESPACE].\n\nINITIAL_SIZE is optional; its default value is 128M.\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/29186))\n\nWhen setting EXTENT_SIZE or INITIAL_SIZE (either or both), you may\noptionally follow the number with a one-letter abbreviation for an\norder of magnitude, similar to those used in my.cnf. Generally, this is\none of the letters M (for megabytes) or G (for gigabytes).\n\nAUTOEXTEND_SIZE, MAX_SIZE, NODEGROUP, WAIT, and COMMENT are parsed but\nignored, and so have no effect in MySQL 5.1. These options are intended\nfor future expansion.\n\nThe ENGINE parameter determines the storage engine which uses this\ntablespace, with engine_name being the name of the storage engine. In\nMySQL 5.1, engine_name must be one of the values NDB or NDBCLUSTER.\n\nWhen CREATE TABLESPACE is used with ENGINE = NDB, a tablespace and\nassociated data file are created on each Cluster data node. You can\nverify that the data files were created and obtain information about\nthem by querying the INFORMATION_SCHEMA.FILES table. For example:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+-------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+-------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n+--------------------+-------------+----------------+\n2 rows in set (0.01 sec)\n\n(See http://dev.mysql.com/doc/refman/5.1/en/files-table.html.)\n\nCREATE TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (191,37,'CREATE TABLESPACE','Syntax:\nCREATE TABLESPACE tablespace_name\n ADD DATAFILE \'file_name\'\n USE LOGFILE GROUP logfile_group\n [EXTENT_SIZE [=] extent_size]\n [INITIAL_SIZE [=] initial_size]\n [AUTOEXTEND_SIZE [=] autoextend_size]\n [MAX_SIZE [=] max_size]\n [NODEGROUP [=] nodegroup_id]\n [WAIT]\n [COMMENT [=] comment_text]\n ENGINE [=] engine_name\n\nThis statement is used to create a tablespace, which can contain one or\nmore data files, providing storage space for tables. One data file is\ncreated and added to the tablespace using this statement. Additional\ndata files may be added to the tablespace by using the ALTER TABLESPACE\nstatement (see [HELP ALTER TABLESPACE]). For rules covering the naming\nof tablespaces, see\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and a log file group with the same name, or a\ntablespace and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/bug.php?id=31770))\n\nA log file group of one or more UNDO log files must be assigned to the\ntablespace to be created with the USE LOGFILE GROUP clause.\nlogfile_group must be an existing log file group created with CREATE\nLOGFILE GROUP (see\nhttp://dev.mysql.com/doc/refman/5.1/en/create-logfile-group.html).\nMu… tablespaces may use the same log file group for UNDO logging.\n\nThe EXTENT_SIZE sets the size, in bytes, of the extents used by any\nfiles belonging to the tablespace. The default value is 1M. The minimum\nsize is 32K, and theoretical maximum is 2G, although the practical\nmaximum size depends on a number of factors. In most cases, changing\nthe extent size does not have any measurable effect on performance, and\nthe default value is recommended for all but the most unusual\nsituations.\n\nAn extent is a unit of disk space allocation. One extent is filled with\nas much data as that extent can contain before another extent is used.\nIn theory, up to 65,535 (64K) extents may used per data file; however,\nthe recommended maximum is 32,768 (32K). The recommended maximum size\nfor a single data file is 32G --- that is, 32K extents x 1 MB per\nextent. In addition, once an extent is allocated to a given partition,\nit cannot be used to store data from a different partition; an extent\ncannot store data from more than one partition. This means, for example\nthat a tablespace having a single datafile whose INITIAL_SIZE is 256 MB\nand whose EXTENT_SIZE is 128M has just two extents, and so can be used\nto store data from at most two different disk data table partitions.\n\nYou can see how many extents remain free in a given data file by\nquerying the INFORMATION_SCHEMA.FILES table, and so derive an estimate\nfor how much space remains free in the file. For further discussion and\nexamples, see http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nThe INITIAL_SIZE parameter sets the data file\'s total size in bytes.\nOnce the file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using ALTER TABLESPACE\n... ADD DATAFILE. See [HELP ALTER TABLESPACE].\n\nINITIAL_SIZE is optional; its default value is 128M.\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/bug.php?id=29186))\n\nWhen setting EXTENT_SIZE or INITIAL_SIZE (either or both), you may\noptionally follow the number with a one-letter abbreviation for an\norder of magnitude, similar to those used in my.cnf. Generally, this is\none of the letters M (for megabytes) or G (for gigabytes).\n\nINITIAL_SIZE, EXTENT_SIZE, and UNDO_BUFFER_SIZE are subject to rounding\nas follows:\n\no EXTENT_SIZE and UNDO_BUFFER_SIZE are each rounded up to the nearest\n whole multiple of 32K.\n\no INITIAL_SIZE is rounded down to the nearest whole multiple of 32K.\n\n For data files, INITIAL_SIZE is subject to further rounding; the\n result just obtained is rounded up to the nearest whole multiple of\n EXTENT_SIZE (after any rounding).\n\nThe rounding just described has always (since Disk Data tablespaces\nwere introduced in MySQL 5.1.6) been performed implicitly, but\nbeginning with MySQL Cluster NDB 6.2.19, MySQL Cluster NDB 6.3.32,\nMySQL Cluster NDB 7.0.13, and MySQL Cluster NDB 7.1.2, this rounding is\ndone explicitly, and a warning is issued by the MySQL Server when any\nsuch rounding is performed. The rounded values are also used by the NDB\nkernel for calculating INFORMATION_SCHEMA.FILES column values and other\npurposes. However, to avoid an unexpected result, we suggest that you\nalways use whole multiples of 32K in specifying these options.\n\nAUTOEXTEND_SIZE, MAX_SIZE, NODEGROUP, WAIT, and COMMENT are parsed but\nignored, and so currently have no effect. These options are intended\nfor future expansion.\n\nThe ENGINE parameter determines the storage engine which uses this\ntablespace, with engine_name being the name of the storage engine. In\nMySQL 5.1, engine_name must be one of the values NDB or NDBCLUSTER.\n\nWhen CREATE TABLESPACE is used with ENGINE = NDB, a tablespace and\nassociated data file are created on each Cluster data node. You can\nverify that the data files were created and obtain information about\nthem by querying the INFORMATION_SCHEMA.FILES table. For example:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+-------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+-------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n+--------------------+-------------+----------------+\n2 rows in set (0.01 sec)\n\n(See http://dev.mysql.com/doc/refman/5.1/en/files-table.html.)\n\nCREATE TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (192,35,'INSERT FUNCTION','Syntax:\nINSERT(str,pos,len,newstr)\n\nReturns the string str, with the substring beginning at position pos\nand len characters long replaced by the string newstr. Returns the\noriginal string if pos is not within the length of the string. Replaces\nthe rest of the string from position pos if len is not within the\nlength of the rest of the string. Returns NULL if any argument is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT INSERT(\'Quadratic\', 3, 4, \'What\');\n -> \'QuWhattic\'\nmysql> SELECT INSERT(\'Quadratic\', -1, 4, \'What\');\n -> \'Quadratic\'\nmysql> SELECT INSERT(\'Quadratic\', 3, 100, \'What\');\n -> \'QuWhat\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (193,4,'CRC32','Syntax:\nCRC32(expr)\n\nComputes a cyclic redundancy check value and returns a 32-bit unsigned\nvalue. The result is NULL if the argument is NULL. The argument is\nexpected to be a string and (if possible) is treated as one if it is\nnot.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT CRC32(\'MySQL\');\n -> 3259397556\nmysql> SELECT CRC32(\'mysql\');\n -> 2501908538\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (194,13,'XOR','Syntax:\nXOR\n\nLogical XOR. Returns NULL if either operand is NULL. For non-NULL\noperands, evaluates to 1 if an odd number of operands is nonzero,\notherwise 0 is returned.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/logical-operators.html\n\n','mysql> SELECT 1 XOR 1;\n -> 0\nmysql> SELECT 1 XOR 0;\n -> 1\nmysql> SELECT 1 XOR NULL;\n -> NULL\nmysql> SELECT 1 XOR 1 XOR 1;\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/logical-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (195,12,'STARTPOINT','StartPoint(ls)\n\nReturns the Point that is the start point of the LineString value ls.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…','mysql> SET @ls = \'LineString(1 1,2 2,3 3)\';\nmysql> SELECT AsText(StartPoint(GeomFromText(@ls)));\n+---------------------------------------+\n| AsText(StartPoint(GeomFromText(@ls))) |\n+---------------------------------------+\n| POINT(1 1) |\n+---------------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (196,22,'DECLARE VARIABLE','Syntax:\nDECLARE var_name [, var_name] ... type [DEFAULT value]\n\nThis statement is used to declare local variables within stored\nprograms. To provide a default value for the variable, include a\nDEFAULT clause. The value can be specified as an expression; it need\nnot be a constant. If the DEFAULT clause is missing, the initial value\nis NULL.\n\nLocal variables are treated like stored routine parameters with respect\nto data type and overflow checking. See [HELP CREATE PROCEDURE].\n\nLocal variable names are not case sensitive.\n\nThe scope of a local variable is within the BEGIN ... END block where\nit is declared. The variable can be referred to in blocks nested within\nthe declaring block, except those blocks that declare a variable with\nthe same name.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/declare-local-variable.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-local-variable.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (197,9,'GRANT','Syntax:\nGRANT\n priv_type [(column_list)]\n [, priv_type [(column_list)]] ...\n ON [object_type] priv_level\n TO user [IDENTIFIED BY [PASSWORD] \'password\']\n [, user [IDENTIFIED BY [PASSWORD] \'password\']] ...\n [REQUIRE {NONE | ssl_option [[AND] ssl_option] ...}]\n [WITH with_option [with_option] ...]\n\nobject_type:\n TABLE\n | FUNCTION\n | PROCEDURE\n\npriv_level:\n *\n | *.*\n | db_name.*\n | db_name.tbl_name\n | tbl_name\n | db_name.routine_name\n\nwith_option:\n GRANT OPTION\n | MAX_QUERIES_PER_HOUR count\n | MAX_UPDATES_PER_HOUR count\n | MAX_CONNECTIONS_PER_HOUR count\n | MAX_USER_CONNECTIONS count\n\nssl_option:\n SSL\n | X509\n | CIPHER \'cipher\'\n | ISSUER \'issuer\'\n | SUBJECT \'subject\'\n\nThe GRANT statement enables system administrators to create MySQL user\naccounts and to grant rights to accounts. To use GRANT, you must have\nthe GRANT OPTION privilege, and you must have the privileges that you\nare granting. The REVOKE statement is related and enables\nadministrators to remove account privileges. To determine what\nprivileges an account has, use SHOW GRANTS. See [HELP REVOKE], and\n[HELP SHOW GRANTS].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/grant.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/grant.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (197,9,'GRANT','Syntax:\nGRANT\n priv_type [(column_list)]\n [, priv_type [(column_list)]] ...\n ON [object_type] priv_level\n TO user [IDENTIFIED BY [PASSWORD] \'password\']\n [, user [IDENTIFIED BY [PASSWORD] \'password\']] ...\n [REQUIRE {NONE | ssl_option [[AND] ssl_option] ...}]\n [WITH with_option ...]\n\nobject_type:\n TABLE\n | FUNCTION\n | PROCEDURE\n\npriv_level:\n *\n | *.*\n | db_name.*\n | db_name.tbl_name\n | tbl_name\n | db_name.routine_name\n\nssl_option:\n SSL\n | X509\n | CIPHER \'cipher\'\n | ISSUER \'issuer\'\n | SUBJECT \'subject\'\n\nwith_option:\n GRANT OPTION\n | MAX_QUERIES_PER_HOUR count\n | MAX_UPDATES_PER_HOUR count\n | MAX_CONNECTIONS_PER_HOUR count\n | MAX_USER_CONNECTIONS count\n\nThe GRANT statement enables system administrators to grant privileges\nto MySQL user accounts. GRANT also serves to specify other account\ncharacteristics such as use of secure connections and limits on access\nto server resources. To use GRANT, you must have the GRANT OPTION\nprivilege, and you must have the privileges that you are granting.\n\nNormally, CREATE USER is used to create an account and GRANT to define\nits privileges. However, if an account named in a GRANT statement does\nnot already exist, GRANT may create it under the conditions described\nlater in the discussion of the NO_AUTO_CREATE_USER SQL mode.\n\nThe REVOKE statement is related to GRANT and enables administrators to\nremove account privileges. To determine what privileges an account has,\nuse SHOW GRANTS. See [HELP REVOKE], and [HELP SHOW GRANTS].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/grant.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/grant.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (198,3,'MPOLYFROMTEXT','MPolyFromText(wkt[,srid]), MultiPolygonFromText(wkt[,srid])\n\nConstructs a MULTIPOLYGON value using its WKT representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (199,6,'MBRINTERSECTS','MBRIntersects(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 intersect.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (200,16,'BIT_OR','Syntax:\nBIT_OR(expr)\n\nReturns the bitwise OR of all bits in expr. The calculation is\nperformed with 64-bit (BIGINT) precision.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); @@ -286,15 +286,15 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (217,35,'ASCII','Syntax:\nASCII(str)\n\nReturns the numeric value of the leftmost character of the string str.\nReturns 0 if str is the empty string. Returns NULL if str is NULL.\nASCII() works for 8-bit characters.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT ASCII(\'2\');\n -> 50\nmysql> SELECT ASCII(2);\n -> 50\nmysql> SELECT ASCII(\'dx\');\n -> 100\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (218,4,'DIV','Syntax:\nDIV\n\nInteger division. Similar to FLOOR(), but is safe with BIGINT values.\nIncorrect results may occur for noninteger operands that exceed BIGINT\nrange.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html\n\n','mysql> SELECT 5 DIV 2;\n -> 2\n','http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (219,9,'RENAME USER','Syntax:\nRENAME USER old_user TO new_user\n [, old_user TO new_user] ...\n\nThe RENAME USER statement renames existing MySQL accounts. To use it,\nyou must have the global CREATE USER privilege or the UPDATE privilege\nfor the mysql database. An error occurs if any old account does not\nexist or any new account exists. Each account is named using the same\nformat as for the GRANT statement; for example, \'jeffrey\'@\'localhost\'.\nIf you specify only the user name part of the account name, a host name\npart of \'%\' is used. For additional information about specifying\naccount names, see [HELP GRANT].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/rename-user.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/rename-user.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (220,25,'SHOW SLAVE STATUS','Syntax:\nSHOW SLAVE STATUS\n\nThis statement provides status information on essential parameters of\nthe slave threads. It requires either the SUPER or REPLICATION CLIENT\nprivilege.\n\nIf you issue this statement using the mysql client, you can use a \\G\nstatement terminator rather than a semicolon to obtain a more readable\nvertical layout:\n\nmysql> SHOW SLAVE STATUS\\G\n*************************** 1. row ***************************\n Slave_IO_State: Waiting for master to send event\n Master_Host: localhost\n Master_User: root\n Master_Port: 3306\n Connect_Retry: 3\n Master_Log_File: gbichot-bin.005\n Read_Master_Log_Pos: 79\n Relay_Log_File: gbichot-relay-bin.005\n Relay_Log_Pos: 548\n Relay_Master_Log_File: gbichot-bin.005\n Slave_IO_Running: Yes\n Slave_SQL_Running: Yes\n Replicate_Do_DB:\n Replicate_Ignore_DB:\n Last_Errno: 0\n Last_Error:\n Skip_Counter: 0\n Exec_Master_Log_Pos: 79\n Relay_Log_Space: 552\n Until_Condition: None\n Until_Log_File:\n Until_Log_Pos: 0\n Master_SSL_Allowed: No\n Master_SSL_CA_File:\n Master_SSL_CA_Path:\n Master_SSL_Cert:\n Master_SSL_Cipher:\n Master_SSL_Key:\n Seconds_Behind_Master: 8\nMaster_SSL_Verify_Server_Cert: No\n Last_IO_Errno: 0\n Last_IO_Error:\n Last_SQL_Errno: 0\n Last_SQL_Error:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (220,25,'SHOW SLAVE STATUS','Syntax:\nSHOW SLAVE STATUS\n\nThis statement provides status information on essential parameters of\nthe slave threads. It requires either the SUPER or REPLICATION CLIENT\nprivilege.\n\nIf you issue this statement using the mysql client, you can use a \\G\nstatement terminator rather than a semicolon to obtain a more readable\nvertical layout:\n\nmysql> SHOW SLAVE STATUS\\G\n*************************** 1. row ***************************\n Slave_IO_State: Waiting for master to send event\n Master_Host: localhost\n Master_User: root\n Master_Port: 3306\n Connect_Retry: 3\n Master_Log_File: gbichot-bin.005\n Read_Master_Log_Pos: 79\n Relay_Log_File: gbichot-relay-bin.005\n Relay_Log_Pos: 548\n Relay_Master_Log_File: gbichot-bin.005\n Slave_IO_Running: Yes\n Slave_SQL_Running: Yes\n Replicate_Do_DB:\n Replicate_Ignore_DB:\n Replicate_Do_Table:\n Replicate_Ignore_Table:\n Replicate_Wild_Do_Table:\n Replicate_Wild_Ignore_Table:\n Last_Errno: 0\n Last_Error:\n Skip_Counter: 0\n Exec_Master_Log_Pos: 79\n Relay_Log_Space: 552\n Until_Condition: None\n Until_Log_File:\n Until_Log_Pos: 0\n Master_SSL_Allowed: No\n Master_SSL_CA_File:\n Master_SSL_CA_Path:\n Master_SSL_Cert:\n Master_SSL_Cipher:\n Master_SSL_Key:\n Seconds_Behind_Master: 8\nMaster_SSL_Verify_Server_Cert: No\n Last_IO_Errno: 0\n Last_IO_Error:\n Last_SQL_Errno: 0\n Last_SQL_Error:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (221,32,'GEOMETRY','MySQL provides a standard way of creating spatial columns for geometry\ntypes, for example, with CREATE TABLE or ALTER TABLE. Currently,\nspatial columns are supported for MyISAM, InnoDB, NDB, and ARCHIVE\ntables. See also the annotations about spatial indexes under [HELP\nSPATIAL].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-columns.html\n\n','CREATE TABLE geom (g GEOMETRY);\n','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-columns.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (222,12,'NUMPOINTS','NumPoints(ls)\n\nReturns the number of Point objects in the LineString value ls.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…','mysql> SET @ls = \'LineString(1 1,2 2,3 3)\';\nmysql> SELECT NumPoints(GeomFromText(@ls));\n+------------------------------+\n| NumPoints(GeomFromText(@ls)) |\n+------------------------------+\n| 3 |\n+------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (223,37,'ALTER LOGFILE GROUP','Syntax:\nALTER LOGFILE GROUP logfile_group\n ADD UNDOFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement adds an UNDO file named \'file_name\' to an existing log\nfile group logfile_group. An ALTER LOGFILE GROUP statement has one and\nonly one ADD UNDOFILE clause. No DROP UNDOFILE clause is currently\nsupported.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an undo log file with the same name, or an undo\nlog file and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for undo log files could not be longer than 128 characters.\n(Bug#31769 (http://bugs.mysql.com/31769))\n\nThe optional INITIAL_SIZE parameter sets the UNDO file\'s initial size\nin bytes; if not specified, the initial size default to 128M (128\nmegabytes). You may optionally follow size with a one-letter\nabbreviation for an order of magnitude, similar to those used in\nmy.cnf. Generally, this is one of the letters M (for megabytes) or G\n(for gigabytes).\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/29186))\n\nBeginning with MySQL Cluster NDB 2.1.18, 6.3.24, and 7.0.4, the minimum\nallowed value for INITIAL_SIZE is 1M. (Bug#29574\n(http://bugs.mysql.com/29574))\n\n*Note*: WAIT is parsed but otherwise ignored, and so has no effect in\nMySQL 5.1 and MySQL Cluster NDB 6.x. It is intended for future\nexpansion.\n\nThe ENGINE parameter (required) determines the storage engine which is\nused by this log file group, with engine_name being the name of the\nstorage engine. In MySQL 5.1 and MySQL Cluster NDB 6.x, the only\naccepted values for engine_name are "NDBCLUSTER" and "NDB". The two\nvalues are equivalent.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (223,37,'ALTER LOGFILE GROUP','Syntax:\nALTER LOGFILE GROUP logfile_group\n ADD UNDOFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement adds an UNDO file named \'file_name\' to an existing log\nfile group logfile_group. An ALTER LOGFILE GROUP statement has one and\nonly one ADD UNDOFILE clause. No DROP UNDOFILE clause is currently\nsupported.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an undo log file with the same name, or an undo\nlog file and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for undo log files could not be longer than 128 characters.\n(Bug#31769 (http://bugs.mysql.com/bug.php?id=31769))\n\nThe optional INITIAL_SIZE parameter sets the UNDO file\'s initial size\nin bytes; if not specified, the initial size default to 128M (128\nmegabytes). You may optionally follow size with a one-letter\nabbreviation for an order of magnitude, similar to those used in\nmy.cnf. Generally, this is one of the letters M (for megabytes) or G\n(for gigabytes).\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/bug.php?id=29186))\n\nBeginning with MySQL Cluster NDB 2.1.18, 6.3.24, and 7.0.4, the minimum\nallowed value for INITIAL_SIZE is 1M. (Bug#29574\n(http://bugs.mysql.com/bug.php?id=29574))\n\n*Note*: WAIT is parsed but otherwise ignored, and so has no effect in\nMySQL 5.1 and MySQL Cluster NDB 6.x. It is intended for future\nexpansion.\n\nThe ENGINE parameter (required) determines the storage engine which is\nused by this log file group, with engine_name being the name of the\nstorage engine. In MySQL 5.1 and MySQL Cluster NDB 6.x, the only\naccepted values for engine_name are "NDBCLUSTER" and "NDB". The two\nvalues are equivalent.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (224,18,'&','Syntax:\n&\n\nBitwise AND:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html\n\n','mysql> SELECT 29 & 15;\n -> 13\n','http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (225,30,'LOCALTIMESTAMP','Syntax:\nLOCALTIMESTAMP, LOCALTIMESTAMP()\n\nLOCALTIMESTAMP and LOCALTIMESTAMP() are synonyms for NOW().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (226,35,'CONVERT','Syntax:\nCONVERT(expr,type), CONVERT(expr USING transcoding_name)\n\nThe CONVERT() and CAST() functions take a value of one type and produce\na value of another type.\n\nThe type can be one of the following values:\n\no BINARY[(N)]\n\no CHAR[(N)]\n\no DATE\n\no DATETIME\n\no DECIMAL[(M[,D])]\n\no SIGNED [INTEGER]\n\no TIME\n\no UNSIGNED [INTEGER]\n\nBINARY produces a string with the BINARY data type. See\nhttp://dev.mysql.com/doc/refman/5.1/en/binary-varbinary.html for a\ndescription of how this affects comparisons. If the optional length N\nis given, BINARY(N) causes the cast to use no more than N bytes of the\nargument. Values shorter than N bytes are padded with 0x00 bytes to a\nlength of N.\n\nCHAR(N) causes the cast to use no more than N characters of the\nargument.\n\nCAST() and CONVERT(... USING ...) are standard SQL syntax. The\nnon-USING form of CONVERT() is ODBC syntax.\n\nCONVERT() with USING is used to convert data between different\ncharacter sets. In MySQL, transcoding names are the same as the\ncorresponding character set names. For example, this statement converts\nthe string \'abc\' in the default character set to the corresponding\nstring in the utf8 character set:\n\nSELECT CONVERT(\'abc\' USING utf8);\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html\n\n','SELECT enum_col FROM tbl_name ORDER BY CAST(enum_col AS CHAR);\n','http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (227,30,'ADDDATE','Syntax:\nADDDATE(date,INTERVAL expr unit), ADDDATE(expr,days)\n\nWhen invoked with the INTERVAL form of the second argument, ADDDATE()\nis a synonym for DATE_ADD(). The related function SUBDATE() is a\nsynonym for DATE_SUB(). For information on the INTERVAL unit argument,\nsee the discussion for DATE_ADD().\n\nmysql> SELECT DATE_ADD(\'2008-01-02\', INTERVAL 31 DAY);\n -> \'2008-02-02\'\nmysql> SELECT ADDDATE(\'2008-01-02\', INTERVAL 31 DAY);\n -> \'2008-02-02\'\n\nWhen invoked with the days form of the second argument, MySQL treats it\nas an integer number of days to be added to expr.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT ADDDATE(\'2008-01-02\', 31);\n -> \'2008-02-02\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (228,22,'REPEAT LOOP','Syntax:\n[begin_label:] REPEAT\n statement_list\nUNTIL search_condition\nEND REPEAT [end_label]\n\nThe statement list within a REPEAT statement is repeated until the\nsearch_condition is true. Thus, a REPEAT always enters the loop at\nleast once. statement_list consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter.\n\nA REPEAT statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html\n\n','mysql> delimiter //\n\nmysql> CREATE PROCEDURE dorepeat(p1 INT)\n -> BEGIN\n -> SET @x = 0;\n -> REPEAT SET @x = @x + 1; UNTIL @x > p1 END REPEAT;\n -> END\n -> //\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> CALL dorepeat(1000)//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @x//\n+------+\n| @x |\n+------+\n| 1001 |\n+------+\n1 row in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (228,22,'REPEAT LOOP','Syntax:\n[begin_label:] REPEAT\n statement_list\nUNTIL search_condition\nEND REPEAT [end_label]\n\nThe statement list within a REPEAT statement is repeated until the\nsearch_condition is true. Thus, a REPEAT always enters the loop at\nleast once. statement_list consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter.\n\nA REPEAT statement can be labeled. See [HELP BEGIN END] for the rules\nregarding label use.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html\n\n','mysql> delimiter //\n\nmysql> CREATE PROCEDURE dorepeat(p1 INT)\n -> BEGIN\n -> SET @x = 0;\n -> REPEAT SET @x = @x + 1; UNTIL @x > p1 END REPEAT;\n -> END\n -> //\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> CALL dorepeat(1000)//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @x//\n+------+\n| @x |\n+------+\n| 1001 |\n+------+\n1 row in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (229,37,'ALTER FUNCTION','Syntax:\nALTER FUNCTION func_name [characteristic ...]\n\ncharacteristic:\n { CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }\n | SQL SECURITY { DEFINER | INVOKER }\n | COMMENT \'string\'\n\nThis statement can be used to change the characteristics of a stored\nfunction. More than one change may be specified in an ALTER FUNCTION\nstatement. However, you cannot change the parameters or body of a\nstored function using this statement; to make such changes, you must\ndrop and re-create the function using DROP FUNCTION and CREATE\nFUNCTION.\n\nYou must have the ALTER ROUTINE privilege for the function. (That\nprivilege is granted automatically to the function creator.) If binary\nlogging is enabled, the ALTER FUNCTION statement might also require the\nSUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\n\…: http://dev.mysql.com/doc/refman/5.1/en/alter-function.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-function.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (230,20,'SMALLINT','SMALLINT[(M)] [UNSIGNED] [ZEROFILL]\n\nA small integer. The signed range is -32768 to 32767. The unsigned\nrange is 0 to 65535.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (231,20,'DOUBLE PRECISION','DOUBLE PRECISION[(M,D)] [UNSIGNED] [ZEROFILL], REAL[(M,D)] [UNSIGNED]\n[ZEROFILL]\n\nThese types are synonyms for DOUBLE. Exception: If the REAL_AS_FLOAT\nSQL mode is enabled, REAL is a synonym for FLOAT rather than DOUBLE.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); @@ -327,12 +327,12 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (258,4,'COS','Syntax:\nCOS(X)\n\nReturns the cosine of X, where X is given in radians.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT COS(PI());\n -> -1\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (259,30,'DATE FUNCTION','Syntax:\nDATE(expr)\n\nExtracts the date part of the date or datetime expression expr.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DATE(\'2003-12-31 01:02:03\');\n -> \'2003-12-31\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (260,37,'DROP TRIGGER','Syntax:\nDROP TRIGGER [IF EXISTS] [schema_name.]trigger_name\n\nThis statement drops a trigger. The schema (database) name is optional.\nIf the schema is omitted, the trigger is dropped from the default\nschema. DROP TRIGGER was added in MySQL 5.0.2. Its use requires the\nTRIGGER privilege for the table associated with the trigger. (This\nstatement requires the SUPER privilege prior to MySQL 5.1.6.)\n\nUse IF EXISTS to prevent an error from occurring for a trigger that\ndoes not exist. A NOTE is generated for a nonexistent trigger when\nusing IF EXISTS. See [HELP SHOW WARNINGS]. The IF EXISTS clause was\nadded in MySQL 5.1.14.\n\nTriggers for a table are also dropped if you drop the table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-trigger.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-trigger.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (261,25,'RESET MASTER','Syntax:\nRESET MASTER\n\nDeletes all binary logs listed in the index file, resets the binary log\nindex file to be empty, and creates a new binary log file. It is\nintended to be used only when the master is started for the first time.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-master.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (261,25,'RESET MASTER','Syntax:\nRESET MASTER\n\nDeletes all binary log files listed in the index file, resets the\nbinary log index file to be empty, and creates a new binary log file.\nThis statement is intended to be used only when the master is started\nfor the first time.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-master.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (262,4,'TAN','Syntax:\nTAN(X)\n\nReturns the tangent of X, where X is given in radians.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT TAN(PI());\n -> -1.2246063538224e-16\nmysql> SELECT TAN(PI()+1);\n -> 1.5574077246549\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (263,4,'PI','Syntax:\nPI()\n\nReturns the value of π (pi). The default number of decimal places\ndisplayed is seven, but MySQL uses the full double-precision value\ninternally.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT PI();\n -> 3.141593\nmysql> SELECT PI()+0.000000000000000000;\n -> 3.141592653589793116\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (264,30,'WEEKOFYEAR','Syntax:\nWEEKOFYEAR(date)\n\nReturns the calendar week of the date as a number in the range from 1\nto 53. WEEKOFYEAR() is a compatibility function that is equivalent to\nWEEK(date,3).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT WEEKOFYEAR(\'2008-02-20\');\n -> 8\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (265,4,'/','Syntax:\n/\n\nDivision:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html\n\n','mysql> SELECT 3/5;\n -> 0.60\n','http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (266,25,'PURGE BINARY LOGS','Syntax:\nPURGE { BINARY | MASTER } LOGS\n { TO \'log_name\' | BEFORE datetime_expr }\n\nThe binary log is a set of files that contain information about data\nmodifications made by the MySQL server. The log consists of a set of\nbinary log files, plus an index file.\n\nThe PURGE BINARY LOGS statement deletes all the binary log files listed\nin the log index file prior to the specified log file name or date. The\nlog files also are removed from the list recorded in the index file, so\nthat the given log file becomes the first.\n\nThis statement has no effect if the --log-bin option has not been\nenabled.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html\n\n','PURGE BINARY LOGS TO \'mysql-bin.010\';\nPURGE BINARY LOGS BEFORE \'2008-04-02 22:46:26\';\n','http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (266,25,'PURGE BINARY LOGS','Syntax:\nPURGE { BINARY | MASTER } LOGS\n { TO \'log_name\' | BEFORE datetime_expr }\n\nThe binary log is a set of files that contain information about data\nmodifications made by the MySQL server. The log consists of a set of\nbinary log files, plus an index file (see\nhttp://dev.mysql.com/doc/refman/5.1/en/binary-log.html).\n\nThe PURGE BINARY LOGS statement deletes all the binary log files listed\nin the log index file prior to the specified log file name or date.\nBINARY and MASTER are synonyms. Deleted log files also are removed from\nthe list recorded in the index file, so that the given log file becomes\nthe first in the list.\n\nThis statement has no effect if the server was not started with the\n--log-bin option to enable binary logging.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html\n\n','PURGE BINARY LOGS TO \'mysql-bin.010\';\nPURGE BINARY LOGS BEFORE \'2008-04-02 22:46:26\';\n','http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (267,16,'STDDEV_SAMP','Syntax:\nSTDDEV_SAMP(expr)\n\nReturns the sample standard deviation of expr (the square root of\nVAR_SAMP().\n\nSTDDEV_SAMP() returns NULL if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (268,15,'SCHEMA','Syntax:\nSCHEMA()\n\nThis function is a synonym for DATABASE().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (269,31,'MLINEFROMWKB','MLineFromWKB(wkb[,srid]), MultiLineStringFromWKB(wkb[,srid])\n\nConstructs a MULTILINESTRING value using its WKB representation and\nSRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); @@ -345,7 +345,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (276,26,'DUAL','You are allowed to specify DUAL as a dummy table name in situations\nwhere no tables are referenced:\n\nmysql> SELECT 1 + 1 FROM DUAL;\n -> 2\n\nDUAL is purely for the convenience of people who require that all\nSELECT statements should have FROM and possibly other clauses. MySQL\nmay ignore the clauses. MySQL does not require FROM DUAL if no tables\nare referenced.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/select.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/select.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (277,35,'INSTR','Syntax:\nINSTR(str,substr)\n\nReturns the position of the first occurrence of substring substr in\nstring str. This is the same as the two-argument form of LOCATE(),\nexcept that the order of the arguments is reversed.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT INSTR(\'foobarbar\', \'bar\');\n -> 4\nmysql> SELECT INSTR(\'xbar\', \'foobar\');\n -> 0\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (278,30,'NOW','Syntax:\nNOW()\n\nReturns the current date and time as a value in \'YYYY-MM-DD HH:MM:SS\'\nor YYYYMMDDHHMMSS.uuuuuu format, depending on whether the function is\nused in a string or numeric context. The value is expressed in the\ncurrent time zone.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT NOW();\n -> \'2007-12-15 23:50:26\'\nmysql> SELECT NOW() + 0;\n -> 20071215235026.000000\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (279,25,'SHOW ENGINES','Syntax:\nSHOW [STORAGE] ENGINES\n\nSHOW ENGINES displays status information about the server\'s storage\nengines. This is particularly useful for checking whether a storage\nengine is supported, or to see what the default engine is. SHOW TABLE\nTYPES is a deprecated synonym.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engines.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engines.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (279,25,'SHOW ENGINES','Syntax:\nSHOW [STORAGE] ENGINES\n\nSHOW ENGINES displays status information about the server\'s storage\nengines. This is particularly useful for checking whether a storage\nengine is supported, or to see what the default engine is. SHOW TABLE\nTYPES is a synonym, but is deprecated and is removed in MySQL 5.5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engines.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engines.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (280,17,'>=','Syntax:\n>=\n\nGreater than or equal:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 2 >= 2;\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (281,4,'EXP','Syntax:\nEXP(X)\n\nReturns the value of e (the base of natural logarithms) raised to the\npower of X. The inverse of this function is LOG() (using a single\nargument only) or LN().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT EXP(2);\n -> 7.3890560989307\nmysql> SELECT EXP(-2);\n -> 0.13533528323661\nmysql> SELECT EXP(0);\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (282,20,'LONGBLOB','LONGBLOB\n\nA BLOB column with a maximum length of 4,294,967,295 or 4GB (232 - 1)\nbytes. The effective maximum length of LONGBLOB columns depends on the\nconfigured maximum packet size in the client/server protocol and\navailable memory. Each LONGBLOB value is stored using a four-byte\nlength prefix that indicates the number of bytes in the value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); @@ -353,19 +353,19 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (284,20,'YEAR DATA TYPE','YEAR[(2|4)]\n\nA year in two-digit or four-digit format. The default is four-digit\nformat. In four-digit format, the allowable values are 1901 to 2155,\nand 0000. In two-digit format, the allowable values are 70 to 69,\nrepresenting years from 1970 to 2069. MySQL displays YEAR values in\nYYYY format, but allows you to assign values to YEAR columns using\neither strings or numbers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (285,16,'SUM','Syntax:\nSUM([DISTINCT] expr)\n\nReturns the sum of expr. If the return set has no rows, SUM() returns\nNULL. The DISTINCT keyword can be used in MySQL 5.1 to sum only the\ndistinct values of expr.\n\nSUM() returns NULL if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (286,4,'OCT','Syntax:\nOCT(N)\n\nReturns a string representation of the octal value of N, where N is a\nlonglong (BIGINT) number. This is equivalent to CONV(N,10,8). Returns\nNULL if N is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT OCT(12);\n -> \'14\'\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (287,30,'SYSDATE','Syntax:\nSYSDATE()\n\nReturns the current date and time as a value in \'YYYY-MM-DD HH:MM:SS\'\nor YYYYMMDDHHMMSS.uuuuuu format, depending on whether the function is\nused in a string or numeric context.\n\nSYSDATE() returns the time at which it executes. This differs from the\nbehavior for NOW(), which returns a constant time that indicates the\ntime at which the statement began to execute. (Within a stored function\nor trigger, NOW() returns the time at which the function or triggering\nstatement began to execute.)\n\nmysql> SELECT NOW(), SLEEP(2), NOW();\n+---------------------+----------+---------------------+\n| NOW() | SLEEP(2) | NOW() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:36 | 0 | 2006-04-12 13:47:36 |\n+---------------------+----------+---------------------+\n\nmysql> SELECT SYSDATE(), SLEEP(2), SYSDATE();\n+---------------------+----------+---------------------+\n| SYSDATE() | SLEEP(2) | SYSDATE() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:44 | 0 | 2006-04-12 13:47:46 |\n+---------------------+----------+---------------------+\n\nIn addition, the SET TIMESTAMP statement affects the value returned by\nNOW() but not by SYSDATE(). This means that timestamp settings in the\nbinary log have no effect on invocations of SYSDATE().\n\nBecause SYSDATE() can return different values even within the same\nstatement, and is not affected by SET TIMESTAMP, it is nondeterministic\nand therefore unsafe for replication if statement-based binary logging\nis used. If that is a problem, you can use row-based logging, or start\nthe server with the --sysdate-is-now option to cause SYSDATE() to be an\nalias for NOW(). The nondeterministic nature of SYSDATE() also means\nthat indexes cannot be used for evaluating expressions that refer to\nit.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (288,5,'UNINSTALL PLUGIN','Syntax:\nUNINSTALL PLUGIN plugin_name\n\nThis statement removes an installed plugin. You cannot uninstall a\nplugin if any table that uses it is open.\n\nplugin_name must be the name of some plugin that is listed in the\nmysql.plugin table. The server executes the plugin\'s deinitialization\nfunction and removes the row for the plugin from the mysql.plugin\ntable, so that subsequent server restarts will not load and initialize\nthe plugin. UNINSTALL PLUGIN does not remove the plugin\'s shared\nlibrary file.\n\nTo use UNINSTALL PLUGIN, you must have the DELETE privilege for the\nmysql.plugin table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (287,30,'SYSDATE','Syntax:\nSYSDATE()\n\nReturns the current date and time as a value in \'YYYY-MM-DD HH:MM:SS\'\nor YYYYMMDDHHMMSS.uuuuuu format, depending on whether the function is\nused in a string or numeric context.\n\nSYSDATE() returns the time at which it executes. This differs from the\nbehavior for NOW(), which returns a constant time that indicates the\ntime at which the statement began to execute. (Within a stored function\nor trigger, NOW() returns the time at which the function or triggering\nstatement began to execute.)\n\nmysql> SELECT NOW(), SLEEP(2), NOW();\n+---------------------+----------+---------------------+\n| NOW() | SLEEP(2) | NOW() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:36 | 0 | 2006-04-12 13:47:36 |\n+---------------------+----------+---------------------+\n\nmysql> SELECT SYSDATE(), SLEEP(2), SYSDATE();\n+---------------------+----------+---------------------+\n| SYSDATE() | SLEEP(2) | SYSDATE() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:44 | 0 | 2006-04-12 13:47:46 |\n+---------------------+----------+---------------------+\n\nIn addition, the SET TIMESTAMP statement affects the value returned by\nNOW() but not by SYSDATE(). This means that timestamp settings in the\nbinary log have no effect on invocations of SYSDATE().\n\nBecause SYSDATE() can return different values even within the same\nstatement, and is not affected by SET TIMESTAMP, it is nondeterministic\nand therefore unsafe for replication if statement-based binary logging\nis used. If that is a problem, you can use row-based logging.\n\nAlternatively, you can use the --sysdate-is-now option to cause\nSYSDATE() to be an alias for NOW(). This works if the option is used on\nboth the master and the slave.\n\nThe nondeterministic nature of SYSDATE() also means that indexes cannot\nbe used for evaluating expressions that refer to it.\n\nBeginning with MySQL 5.1.42, a warning is logged if you use this\nfunction when binlog_format is set to STATEMENT. (Bug#47995\n(http://bugs.mysql.com/bug.php?id=47995))\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (288,5,'UNINSTALL PLUGIN','Syntax:\nUNINSTALL PLUGIN plugin_name\n\nThis statement removes an installed server plugin. It requires the\nDELETE privilege for the mysql.plugin table.\n\nplugin_name must be the name of some plugin that is listed in the\nmysql.plugin table. The server executes the plugin\'s deinitialization\nfunction and removes the row for the plugin from the mysql.plugin\ntable, so that subsequent server restarts will not load and initialize\nthe plugin. UNINSTALL PLUGIN does not remove the plugin\'s shared\nlibrary file.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (289,31,'ASBINARY','AsBinary(g), AsWKB(g)\n\nConverts a value in internal geometry format to its WKB representation\nand returns the binary result.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…','SELECT AsBinary(g) FROM geom;\n','http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (290,35,'REPEAT FUNCTION','Syntax:\nREPEAT(str,count)\n\nReturns a string consisting of the string str repeated count times. If\ncount is less than 1, returns an empty string. Returns NULL if str or\ncount are NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT REPEAT(\'MySQL\', 3);\n -> \'MySQLMySQLMySQL\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (291,25,'SHOW TABLES','Syntax:\nSHOW [FULL] TABLES [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW TABLES lists the non-TEMPORARY tables in a given database. You can\nalso get this list using the mysqlshow db_name command. The LIKE\nclause, if present, indicates which table names to match. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nThis statement also lists any views in the database. The FULL modifier\nis supported such that SHOW FULL TABLES displays a second output\ncolumn. Values for the second column are BASE TABLE for a table and\nVIEW for a view.\n\nIf you have no privileges for a base table or view, it does not show up\nin the output from SHOW TABLES or mysqlshow db_name.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-tables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-tables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (292,30,'MAKEDATE','Syntax:\nMAKEDATE(year,dayofyear)\n\nReturns a date, given year and day-of-year values. dayofyear must be\ngreater than 0 or the result is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MAKEDATE(2011,31), MAKEDATE(2011,32);\n -> \'2011-01-31\', \'2011-02-01\'\nmysql> SELECT MAKEDATE(2011,365), MAKEDATE(2014,365);\n -> \'2011-12-31\', \'2014-12-31\'\nmysql> SELECT MAKEDATE(2011,0);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (293,35,'BINARY OPERATOR','Syntax:\nBINARY\n\nThe BINARY operator casts the string following it to a binary string.\nThis is an easy way to force a column comparison to be done byte by\nbyte rather than character by character. This causes the comparison to\nbe case sensitive even if the column isn\'t defined as BINARY or BLOB.\nBINARY also causes trailing spaces to be significant.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html\n\n','mysql> SELECT \'a\' = \'A\';\n -> 1\nmysql> SELECT BINARY \'a\' = \'A\';\n -> 0\nmysql> SELECT \'a\' = \'a \';\n -> 1\nmysql> SELECT BINARY \'a\' = \'a \';\n -> 0\n','http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (294,6,'MBROVERLAPS','MBROverlaps(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 overlap. The term spatially overlaps is\nused if two geometries intersect and their intersection results in a\ngeometry of the same dimension but not equal to either of the given\ngeometries.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (295,35,'SOUNDEX','Syntax:\nSOUNDEX(str)\n\nReturns a soundex string from str. Two strings that sound almost the\nsame should have identical soundex strings. A standard soundex string\nis four characters long, but the SOUNDEX() function returns an\narbitrarily long string. You can use SUBSTRING() on the result to get a\nstandard soundex string. All nonalphabetic characters in str are\nignored. All international alphabetic characters outside the A-Z range\nare treated as vowels.\n\n*Important*: When using SOUNDEX(), you should be aware of the following\nlimitations:\n\no This function, as currently implemented, is intended to work well\n with strings that are in the English language only. Strings in other\n languages may not produce reliable results.\n\no This function is not guaranteed to provide consistent results with\n strings that use multi-byte character sets, including utf-8.\n\n We hope to remove these limitations in a future release. See\n Bug#22638 (http://bugs.mysql.com/22638) for more information.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT SOUNDEX(\'Hello\');\n -> \'H400\'\nmysql> SELECT SOUNDEX(\'Quadratically\');\n -> \'Q36324\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (295,35,'SOUNDEX','Syntax:\nSOUNDEX(str)\n\nReturns a soundex string from str. Two strings that sound almost the\nsame should have identical soundex strings. A standard soundex string\nis four characters long, but the SOUNDEX() function returns an\narbitrarily long string. You can use SUBSTRING() on the result to get a\nstandard soundex string. All nonalphabetic characters in str are\nignored. All international alphabetic characters outside the A-Z range\nare treated as vowels.\n\n*Important*: When using SOUNDEX(), you should be aware of the following\nlimitations:\n\no This function, as currently implemented, is intended to work well\n with strings that are in the English language only. Strings in other\n languages may not produce reliable results.\n\no This function is not guaranteed to provide consistent results with\n strings that use multi-byte character sets, including utf-8.\n\n We hope to remove these limitations in a future release. See\n Bug#22638 (http://bugs.mysql.com/bug.php?id=22638) for more\n information.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT SOUNDEX(\'Hello\');\n -> \'H400\'\nmysql> SELECT SOUNDEX(\'Quadratically\');\n -> \'Q36324\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (296,6,'MBRTOUCHES','MBRTouches(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 touch. Two geometries spatially touch if\nthe interiors of the geometries do not intersect, but the boundary of\none of the geometries intersects either the boundary or the interior of\nthe other.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (297,37,'DROP EVENT','Syntax:\nDROP EVENT [IF EXISTS] event_name\n\nThis statement drops the event named event_name. The event immediately\nceases being active, and is deleted completely from the server.\n\nIf the event does not exist, the error ERROR 1517 (HY000): Unknown\nevent \'event_name\' results. You can override this and cause the\nstatement to generate a warning for nonexistent events instead using IF\nEXISTS.\n\nBeginning with MySQL 5.1.12, this statement requires the EVENT\nprivilege for the schema to which the event to be dropped belongs. (In\nMySQL 5.1.11 and earlier, an event could be dropped only by its\ndefiner, or by a user having the SUPER privilege.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-event.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-event.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (298,26,'INSERT SELECT','Syntax:\nINSERT [LOW_PRIORITY | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name [(col_name,...)]\n SELECT ...\n [ ON DUPLICATE KEY UPDATE col_name=expr, ... ]\n\nWith INSERT ... SELECT, you can quickly insert many rows into a table\nfrom one or many tables. For example:\n\nINSERT INTO tbl_temp2 (fld_id)\n SELECT tbl_temp1.fld_order_id\n FROM tbl_temp1 WHERE tbl_temp1.fld_order_id > 100;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/insert-select.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/insert-select.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (299,37,'CREATE PROCEDURE','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n PROCEDURE sp_name ([proc_parameter[,...]])\n [characteristic ...] routine_body\n\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n FUNCTION sp_name ([func_parameter[,...]])\n RETURNS type\n [characteristic ...] routine_body\n\nproc_parameter:\n [ IN | OUT | INOUT ] param_name type\n\nfunc_parameter:\n param_name type\n\ntype:\n Any valid MySQL data type\n\ncharacteristic:\n LANGUAGE SQL\n | [NOT] DETERMINISTIC\n | { CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }\n | SQL SECURITY { DEFINER | INVOKER }\n | COMMENT \'string\'\n\nroutine_body:\n Valid SQL procedure statement\n\nThese statements create stored routines. By default, a routine is\nassociated with the default database. To associate the routine\nexplicitly with a given database, specify the name as db_name.sp_name\nwhen you create it.\n\nThe CREATE FUNCTION statement is also used in MySQL to support UDFs\n(user-defined functions). See\nhttp://dev.mysql.com/doc/refman/5.1/en/adding-functions.html. A UDF can\nbe regarded as an external stored function. However, do note that\nstored functions share their namespace with UDFs. See\nhttp://dev.mysql.com/doc/refman/5.1/en/function-resolution.html, for\nthe rules describing how the server interprets references to different\nkinds of functions.\n\nTo invoke a stored procedure, use the CALL statement (see [HELP CALL]).\nTo invoke a stored function, refer to it in an expression. The function\nreturns a value during expression evaluation.\n\nTo execute the CREATE PROCEDURE or CREATE FUNCTION statement, it is\nnecessary to have the CREATE ROUTINE privilege. By default, MySQL\nautomatically grants the ALTER ROUTINE and EXECUTE privileges to the\nroutine creator. This behavior can be changed by disabling the\nautomatic_sp_privileges system variable. See\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-routines-privileges.html… binary logging is enabled, the CREATE FUNCTION statement might also\nrequire the SUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\n\… DEFINER and SQL SECURITY clauses specify the security context to be\nused when checking access privileges at routine execution time, as\ndescribed later.\n\nIf the routine name is the same as the name of a built-in SQL function,\na syntax error occurs unless you use a space between the name and the\nfollowing parenthesis when defining the routine or invoking it later.\nFor this reason, avoid using the names of existing SQL functions for\nyour own stored routines.\n\nThe IGNORE_SPACE SQL mode applies to built-in functions, not to stored\nroutines. It is always allowable to have spaces after a stored routine\nname, regardless of whether IGNORE_SPACE is enabled.\n\nThe parameter list enclosed within parentheses must always be present.\nIf there are no parameters, an empty parameter list of () should be\nused. Parameter names are not case sensitive.\n\nEach parameter is an IN parameter by default. To specify otherwise for\na parameter, use the keyword OUT or INOUT before the parameter name.\n\n*Note*: Specifying a parameter as IN, OUT, or INOUT is valid only for a\nPROCEDURE. (FUNCTION parameters are always regarded as IN parameters.)\n\nAn IN parameter passes a value into a procedure. The procedure might\nmodify the value, but the modification is not visible to the caller\nwhen the procedure returns. An OUT parameter passes a value from the\nprocedure back to the caller. Its initial value is NULL within the\nprocedure, and its value is visible to the caller when the procedure\nreturns. An INOUT parameter is initialized by the caller, can be\nmodified by the procedure, and any change made by the procedure is\nvisible to the caller when the procedure returns.\n\nFor each OUT or INOUT parameter, pass a user-defined variable in the\nCALL statement that invokes the procedure so that you can obtain its\nvalue when the procedure returns. If you are calling the procedure from\nwithin another stored procedure or function, you can also pass a\nroutine parameter or local routine variable as an IN or INOUT\nparameter.\n\nThe following example shows a simple stored procedure that uses an OUT\nparameter:\n\nmysql> delimiter //\n\nmysql> CREATE PROCEDURE simpleproc (OUT param1 INT)\n -> BEGIN\n -> SELECT COUNT(*) INTO param1 FROM t;\n -> END//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> delimiter ;\n\nmysql> CALL simpleproc(@a);\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @a;\n+------+\n| @a |\n+------+\n| 3 |\n+------+\n1 row in set (0.00 sec)\n\nThe example uses the mysql client delimiter command to change the\nstatement delimiter from ; to // while the procedure is being defined.\nThis allows the ; delimiter used in the procedure body to be passed\nthrough to the server rather than being interpreted by mysql itself.\nSee\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defini… RETURNS clause may be specified only for a FUNCTION, for which it\nis mandatory. It indicates the return type of the function, and the\nfunction body must contain a RETURN value statement. If the RETURN\nstatement returns a value of a different type, the value is coerced to\nthe proper type. For example, if a function specifies an ENUM or SET\nvalue in the RETURNS clause, but the RETURN statement returns an\ninteger, the value returned from the function is the string for the\ncorresponding ENUM member of set of SET members.\n\nThe following example function takes a parameter, performs an operation\nusing an SQL function, and returns the result. In this case, it is\nunnecessary to use delimiter because the function definition contains\nno internal ; statement delimiters:\n\nmysql> CREATE FUNCTION hello (s CHAR(20))\nmysql> RETURNS CHAR(50) DETERMINISTIC\n -> RETURN CONCAT(\'Hello, \',s,\'!\');\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT hello(\'world\');\n+----------------+\n| hello(\'world\') |\n+----------------+\n| Hello, world! |\n+----------------+\n1 row in set (0.00 sec)\n\nParameter types and function return types can be declared to use any\nvalid data type, except that the COLLATE attribute cannot be used.\n\nThe routine_body consists of a valid SQL procedure statement. This can\nbe a simple statement such as SELECT or INSERT, or it can be a compound\nstatement written using BEGIN and END. Compound statements can contain\ndeclarations, loops, and other control structure statements. The syntax\nfor these statements is described in\nhttp://dev.mysql.com/doc/refman/5.1/en/sql-syntax-compound-statements.h… allows routines to contain DDL statements, such as CREATE and\nDROP. MySQL also allows stored procedures (but not stored functions) to\ncontain SQL transaction statements such as COMMIT. Stored functions may\nnot contain statements that perform explicit or implicit commit or\nrollback. Support for these statements is not required by the SQL\nstandard, which states that each DBMS vendor may decide whether to\nallow them.\n\nStatements that return a result set can be used within a stored\nprocedcure but not within a stored function. This prohibition includes\nSELECT statements that do not have an INTO var_list clause and other\nstatements such as SHOW, EXPLAIN, and CHECK TABLE. For statements that\ncan be determined at function definition time to return a result set, a\nNot allowed to return a result set from a function error occurs\n(ER_SP_NO_RETSET). For statements that can be determined only at\nruntime to return a result set, a PROCEDURE %s can\'t return a result\nset in the given context error occurs (ER_SP_BADSELECT).\n\nUSE statements within stored routines are disallowed. When a routine is\ninvoked, an implicit USE db_name is performed (and undone when the\nroutine terminates). The causes the routine to have the given default\ndatabase while it executes. References to objects in databases other\nthan the routine default database should be qualified with the\nappropriate database name.\n\nFor additional information about statements that are not allowed in\nstored routines, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-program-restrictions.htm… information about invoking stored procedures from within programs\nwritten in a language that has a MySQL interface, see [HELP CALL].\n\nMySQL stores the sql_mode system variable setting that is in effect at\nthe time a routine is created, and always executes the routine with\nthis setting in force, regardless of the server SQL mode in effect when\nthe routine is invoked.\n\nThe switch from the SQL mode of the invoker to that of the routine\noccurs after evaluation of arguments and assignment of the resulting\nvalues to routine parameters. If you define a routine in strict SQL\nmode but invoke it in nonstrict mode, assignment of arguments to\nroutine parameters does not take place in strict mode. If you require\nthat expressions passed to a routine be assigned in strict SQL mode,\nyou should invoke the routine with strict mode in effect.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (299,37,'CREATE PROCEDURE','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n PROCEDURE sp_name ([proc_parameter[,...]])\n [characteristic ...] routine_body\n\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n FUNCTION sp_name ([func_parameter[,...]])\n RETURNS type\n [characteristic ...] routine_body\n\nproc_parameter:\n [ IN | OUT | INOUT ] param_name type\n\nfunc_parameter:\n param_name type\n\ntype:\n Any valid MySQL data type\n\ncharacteristic:\n LANGUAGE SQL\n | [NOT] DETERMINISTIC\n | { CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }\n | SQL SECURITY { DEFINER | INVOKER }\n | COMMENT \'string\'\n\nroutine_body:\n Valid SQL procedure statement\n\nThese statements create stored routines. By default, a routine is\nassociated with the default database. To associate the routine\nexplicitly with a given database, specify the name as db_name.sp_name\nwhen you create it.\n\nThe CREATE FUNCTION statement is also used in MySQL to support UDFs\n(user-defined functions). See\nhttp://dev.mysql.com/doc/refman/5.1/en/adding-functions.html. A UDF can\nbe regarded as an external stored function. However, do note that\nstored functions share their namespace with UDFs. See\nhttp://dev.mysql.com/doc/refman/5.1/en/function-resolution.html, for\nthe rules describing how the server interprets references to different\nkinds of functions.\n\nTo invoke a stored procedure, use the CALL statement (see [HELP CALL]).\nTo invoke a stored function, refer to it in an expression. The function\nreturns a value during expression evaluation.\n\nTo execute the CREATE PROCEDURE or CREATE FUNCTION statement, it is\nnecessary to have the CREATE ROUTINE privilege. By default, MySQL\nautomatically grants the ALTER ROUTINE and EXECUTE privileges to the\nroutine creator. This behavior can be changed by disabling the\nautomatic_sp_privileges system variable. See\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-routines-privileges.html… binary logging is enabled, the CREATE FUNCTION statement might also\nrequire the SUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\nS… may also be required depending on the DEFINER value, as described\nlater.\n\nThe DEFINER and SQL SECURITY clauses specify the security context to be\nused when checking access privileges at routine execution time, as\ndescribed later.\n\nIf the routine name is the same as the name of a built-in SQL function,\na syntax error occurs unless you use a space between the name and the\nfollowing parenthesis when defining the routine or invoking it later.\nFor this reason, avoid using the names of existing SQL functions for\nyour own stored routines.\n\nThe IGNORE_SPACE SQL mode applies to built-in functions, not to stored\nroutines. It is always allowable to have spaces after a stored routine\nname, regardless of whether IGNORE_SPACE is enabled.\n\nThe parameter list enclosed within parentheses must always be present.\nIf there are no parameters, an empty parameter list of () should be\nused. Parameter names are not case sensitive.\n\nEach parameter is an IN parameter by default. To specify otherwise for\na parameter, use the keyword OUT or INOUT before the parameter name.\n\n*Note*: Specifying a parameter as IN, OUT, or INOUT is valid only for a\nPROCEDURE. (FUNCTION parameters are always regarded as IN parameters.)\n\nAn IN parameter passes a value into a procedure. The procedure might\nmodify the value, but the modification is not visible to the caller\nwhen the procedure returns. An OUT parameter passes a value from the\nprocedure back to the caller. Its initial value is NULL within the\nprocedure, and its value is visible to the caller when the procedure\nreturns. An INOUT parameter is initialized by the caller, can be\nmodified by the procedure, and any change made by the procedure is\nvisible to the caller when the procedure returns.\n\nFor each OUT or INOUT parameter, pass a user-defined variable in the\nCALL statement that invokes the procedure so that you can obtain its\nvalue when the procedure returns. If you are calling the procedure from\nwithin another stored procedure or function, you can also pass a\nroutine parameter or local routine variable as an IN or INOUT\nparameter.\n\nThe following example shows a simple stored procedure that uses an OUT\nparameter:\n\nmysql> delimiter //\n\nmysql> CREATE PROCEDURE simpleproc (OUT param1 INT)\n -> BEGIN\n -> SELECT COUNT(*) INTO param1 FROM t;\n -> END//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> delimiter ;\n\nmysql> CALL simpleproc(@a);\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @a;\n+------+\n| @a |\n+------+\n| 3 |\n+------+\n1 row in set (0.00 sec)\n\nThe example uses the mysql client delimiter command to change the\nstatement delimiter from ; to // while the procedure is being defined.\nThis allows the ; delimiter used in the procedure body to be passed\nthrough to the server rather than being interpreted by mysql itself.\nSee\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defini… RETURNS clause may be specified only for a FUNCTION, for which it\nis mandatory. It indicates the return type of the function, and the\nfunction body must contain a RETURN value statement. If the RETURN\nstatement returns a value of a different type, the value is coerced to\nthe proper type. For example, if a function specifies an ENUM or SET\nvalue in the RETURNS clause, but the RETURN statement returns an\ninteger, the value returned from the function is the string for the\ncorresponding ENUM member of set of SET members.\n\nThe following example function takes a parameter, performs an operation\nusing an SQL function, and returns the result. In this case, it is\nunnecessary to use delimiter because the function definition contains\nno internal ; statement delimiters:\n\nmysql> CREATE FUNCTION hello (s CHAR(20))\nmysql> RETURNS CHAR(50) DETERMINISTIC\n -> RETURN CONCAT(\'Hello, \',s,\'!\');\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT hello(\'world\');\n+----------------+\n| hello(\'world\') |\n+----------------+\n| Hello, world! |\n+----------------+\n1 row in set (0.00 sec)\n\nParameter types and function return types can be declared to use any\nvalid data type, except that the COLLATE attribute cannot be used.\n\nThe routine_body consists of a valid SQL procedure statement. This can\nbe a simple statement such as SELECT or INSERT, or it can be a compound\nstatement written using BEGIN and END. Compound statements can contain\ndeclarations, loops, and other control structure statements. The syntax\nfor these statements is described in\nhttp://dev.mysql.com/doc/refman/5.1/en/sql-syntax-compound-statements.h… allows routines to contain DDL statements, such as CREATE and\nDROP. MySQL also allows stored procedures (but not stored functions) to\ncontain SQL transaction statements such as COMMIT. Stored functions may\nnot contain statements that perform explicit or implicit commit or\nrollback. Support for these statements is not required by the SQL\nstandard, which states that each DBMS vendor may decide whether to\nallow them.\n\nStatements that return a result set can be used within a stored\nprocedcure but not within a stored function. This prohibition includes\nSELECT statements that do not have an INTO var_list clause and other\nstatements such as SHOW, EXPLAIN, and CHECK TABLE. For statements that\ncan be determined at function definition time to return a result set, a\nNot allowed to return a result set from a function error occurs\n(ER_SP_NO_RETSET). For statements that can be determined only at\nruntime to return a result set, a PROCEDURE %s can\'t return a result\nset in the given context error occurs (ER_SP_BADSELECT).\n\nUSE statements within stored routines are disallowed. When a routine is\ninvoked, an implicit USE db_name is performed (and undone when the\nroutine terminates). This causes the routine to have the given default\ndatabase while it executes. References to objects in databases other\nthan the routine default database should be qualified with the\nappropriate database name.\n\nFor additional information about statements that are not allowed in\nstored routines, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-program-restrictions.htm… information about invoking stored procedures from within programs\nwritten in a language that has a MySQL interface, see [HELP CALL].\n\nMySQL stores the sql_mode system variable setting that is in effect at\nthe time a routine is created, and always executes the routine with\nthis setting in force, regardless of the server SQL mode in effect when\nthe routine is invoked.\n\nThe switch from the SQL mode of the invoker to that of the routine\noccurs after evaluation of arguments and assignment of the resulting\nvalues to routine parameters. If you define a routine in strict SQL\nmode but invoke it in nonstrict mode, assignment of arguments to\nroutine parameters does not take place in strict mode. If you require\nthat expressions passed to a routine be assigned in strict SQL mode,\nyou should invoke the routine with strict mode in effect.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (300,20,'VARBINARY','VARBINARY(M)\n\nThe VARBINARY type is similar to the VARCHAR type, but stores binary\nbyte strings rather than nonbinary character strings. M represents the\nmaximum column length in bytes.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (301,25,'LOAD INDEX','Syntax:\nLOAD INDEX INTO CACHE\n tbl_index_list [, tbl_index_list] ...\n\ntbl_index_list:\n tbl_name\n [[INDEX|KEY] (index_name[, index_name] ...)]\n [IGNORE LEAVES]\n\nThe LOAD INDEX INTO CACHE statement preloads a table index into the key\ncache to which it has been assigned by an explicit CACHE INDEX\nstatement, or into the default key cache otherwise. LOAD INDEX INTO\nCACHE is used only for MyISAM tables. It is not supported for tables\nhaving user-defined partitioning (see\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-limitations.html.)… IGNORE LEAVES modifier causes only blocks for the nonleaf nodes of\nthe index to be preloaded.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-index.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-index.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (302,26,'UNION','Syntax:\nSELECT ...\nUNION [ALL | DISTINCT] SELECT ...\n[UNION [ALL | DISTINCT] SELECT ...]\n\nUNION is used to combine the result from multiple SELECT statements\ninto a single result set.\n\nThe column names from the first SELECT statement are used as the column\nnames for the results returned. Selected columns listed in\ncorresponding positions of each SELECT statement should have the same\ndata type. (For example, the first column selected by the first\nstatement should have the same type as the first column selected by the\nother statements.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/union.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/union.html'); @@ -384,16 +384,16 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (315,15,'FOUND_ROWS','Syntax:\nFOUND_ROWS()\n\nA SELECT statement may include a LIMIT clause to restrict the number of\nrows the server returns to the client. In some cases, it is desirable\nto know how many rows the statement would have returned without the\nLIMIT, but without running the statement again. To obtain this row\ncount, include a SQL_CALC_FOUND_ROWS option in the SELECT statement,\nand then invoke FOUND_ROWS() afterward:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name\n -> WHERE id > 100 LIMIT 10;\nmysql> SELECT FOUND_ROWS();\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (316,15,'SYSTEM_USER','Syntax:\nSYSTEM_USER()\n\nSYSTEM_USER() is a synonym for USER().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (317,29,'CROSSES','Crosses(g1,g2)\n\nReturns 1 if g1 spatially crosses g2. Returns NULL if g1 is a Polygon\nor a MultiPolygon, or if g2 is a Point or a MultiPoint. Otherwise,\nreturns 0.\n\nThe term spatially crosses denotes a spatial relation between two given\ngeometries that has the following properties:\n\no The two geometries intersect\n\no Their intersection results in a geometry that has a dimension that is\n one less than the maximum dimension of the two given geometries\n\no Their intersection is not equal to either of the two given geometries\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (318,26,'TRUNCATE TABLE','Syntax:\nTRUNCATE [TABLE] tbl_name\n\nTRUNCATE TABLE empties a table completely. It requires the DROP\nprivilege as of MySQL 5.1.16. (Before 5.1.16, it requires the DELETE\nprivilege.\n\nLogically, TRUNCATE TABLE is equivalent to a DELETE statement that\ndeletes all rows, but there are practical differences under some\ncircumstances.\n\nFor an InnoDB table, InnoDB processes TRUNCATE TABLE by deleting rows\none by one if there are any FOREIGN KEY constraints that reference the\ntable. If there are no FOREIGN KEY constraints, InnoDB performs fast\ntruncation by dropping the original table and creating an empty one\nwith the same definition, which is much faster than deleting rows one\nby one. The AUTO_INCREMENT counter is reset by TRUNCATE TABLE,\nregardless of whether there is a FOREIGN KEY constraint.\n\nIn the case that FOREIGN KEY constraints reference the table, InnoDB\ndeletes rows one by one and processes the constraints on each one. If\nthe FOREIGN KEY constraint specifies DELETE CASCADE, rows from the\nchild (referenced) table are deleted, and the truncated table becomes\nempty. If the FOREIGN KEY constraint does not specify CASCADE, the\nTRUNCATE statement deletes rows one by one and stops if it encounters a\nparent row that is referenced by the child, returning this error:\n\nERROR 1451 (23000): Cannot delete or update a parent row: a foreign\nkey constraint fails (`test`.`child`, CONSTRAINT `child_ibfk_1`\nFOREIGN KEY (`parent_id`) REFERENCES `parent` (`id`))\n\nThis is the same as a DELETE statement with no WHERE clause.\n\nThe count of rows affected by TRUNCATE TABLE is accurate only when it\nis mapped to a DELETE statement.\n\nFor other storage engines, TRUNCATE TABLE differs from DELETE in the\nfollowing ways in MySQL 5.1:\n\no Truncate operations drop and re-create the table, which is much\n faster than deleting rows one by one, particularly for large tables.\n\no Truncate operations cause an implicit commit.\n\no Truncation operations cannot be performed if the session holds an\n active table lock.\n\no Truncation operations do not return a meaningful value for the number\n of deleted rows. The usual result is "0 rows affected," which should\n be interpreted as "no information."\n\no As long as the table format file tbl_name.frm is valid, the table can\n be re-created as an empty table with TRUNCATE TABLE, even if the data\n or index files have become corrupted.\n\no The table handler does not remember the last used AUTO_INCREMENT\n value, but starts counting from the beginning. This is true even for\n MyISAM and InnoDB, which normally do not reuse sequence values.\n\no When used with partitioned tables, TRUNCATE TABLE preserves the\n partitioning; that is, the data and index files are dropped and\n re-created, while the partition definitions (.par) file is\n unaffected.\n\no Since truncation of a table does not make any use of DELETE, the\n TRUNCATE statement does not invoke ON DELETE triggers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/truncate.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/truncate.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (318,26,'TRUNCATE TABLE','Syntax:\nTRUNCATE [TABLE] tbl_name\n\nTRUNCATE TABLE empties a table completely. It requires the DROP\nprivilege as of MySQL 5.1.16. (Before 5.1.16, it requires the DELETE\nprivilege).\n\nLogically, TRUNCATE TABLE is equivalent to a DELETE statement that\ndeletes all rows, but there are practical differences under some\ncircumstances.\n\nFor an InnoDB table, InnoDB processes TRUNCATE TABLE by deleting rows\none by one if there are any FOREIGN KEY constraints that reference the\ntable. If there are no FOREIGN KEY constraints, InnoDB performs fast\ntruncation by dropping the original table and creating an empty one\nwith the same definition, which is much faster than deleting rows one\nby one. The AUTO_INCREMENT counter is reset to zero by TRUNCATE TABLE,\nregardless of whether there is a FOREIGN KEY constraint.\n\nIn the case that FOREIGN KEY constraints reference the table, InnoDB\ndeletes rows one by one and processes the constraints on each one. If\nthe FOREIGN KEY constraint specifies DELETE CASCADE, rows from the\nchild (referenced) table are deleted, and the truncated table becomes\nempty. If the FOREIGN KEY constraint does not specify CASCADE, the\nTRUNCATE TABLE statement deletes rows one by one and stops if it\nencounters a parent row that is referenced by the child, returning this\nerror:\n\nERROR 1451 (23000): Cannot delete or update a parent row: a foreign\nkey constraint fails (`test`.`child`, CONSTRAINT `child_ibfk_1`\nFOREIGN KEY (`parent_id`) REFERENCES `parent` (`id`))\n\nThis is the same as a DELETE statement with no WHERE clause.\n\nThe count of rows affected by TRUNCATE TABLE is accurate only when it\nis mapped to a DELETE statement.\n\nFor other storage engines, TRUNCATE TABLE differs from DELETE in the\nfollowing ways in MySQL 5.1:\n\no Truncate operations drop and re-create the table, which is much\n faster than deleting rows one by one, particularly for large tables.\n\no Truncate operations cause an implicit commit.\n\no Truncation operations cannot be performed if the session holds an\n active table lock.\n\no Truncation operations do not return a meaningful value for the number\n of deleted rows. The usual result is "0 rows affected," which should\n be interpreted as "no information."\n\no As long as the table format file tbl_name.frm is valid, the table can\n be re-created as an empty table with TRUNCATE TABLE, even if the data\n or index files have become corrupted.\n\no The table handler does not remember the last used AUTO_INCREMENT\n value, but starts counting from the beginning. This is true even for\n MyISAM and InnoDB, which normally do not reuse sequence values.\n\no When used with partitioned tables, TRUNCATE TABLE preserves the\n partitioning; that is, the data and index files are dropped and\n re-created, while the partition definitions (.par) file is\n unaffected.\n\no Since truncation of a table does not make any use of DELETE, the\n TRUNCATE TABLE statement does not invoke ON DELETE triggers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/truncate-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/truncate-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (319,16,'BIT_XOR','Syntax:\nBIT_XOR(expr)\n\nReturns the bitwise XOR of all bits in expr. The calculation is\nperformed with 64-bit (BIGINT) precision.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (320,30,'CURRENT_DATE','Syntax:\nCURRENT_DATE, CURRENT_DATE()\n\nCURRENT_DATE and CURRENT_DATE() are synonyms for CURDATE().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (321,25,'START SLAVE','Syntax:\nSTART SLAVE [thread_type [, thread_type] ... ]\nSTART SLAVE [SQL_THREAD] UNTIL\n MASTER_LOG_FILE = \'log_name\', MASTER_LOG_POS = log_pos\nSTART SLAVE [SQL_THREAD] UNTIL\n RELAY_LOG_FILE = \'log_name\', RELAY_LOG_POS = log_pos\n\nthread_type: IO_THREAD | SQL_THREAD\n\nSTART SLAVE with no thread_type options starts both of the slave\nthreads. The I/O thread reads queries from the master server and stores\nthem in the relay log. The SQL thread reads the relay log and executes\nthe queries. START SLAVE requires the SUPER privilege.\n\nIf START SLAVE succeeds in starting the slave threads, it returns\nwithout any error. However, even in that case, it might be that the\nslave threads start and then later stop (for example, because they do\nnot manage to connect to the master or read its binary logs, or some\nother problem). START SLAVE does not warn you about this. You must\ncheck the slave\'s error log for error messages generated by the slave\nthreads, or check that they are running satisfactorily with SHOW SLAVE\nSTATUS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/start-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/start-slave.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (321,25,'START SLAVE','Syntax:\nSTART SLAVE [thread_type [, thread_type] ... ]\nSTART SLAVE [SQL_THREAD] UNTIL\n MASTER_LOG_FILE = \'log_name\', MASTER_LOG_POS = log_pos\nSTART SLAVE [SQL_THREAD] UNTIL\n RELAY_LOG_FILE = \'log_name\', RELAY_LOG_POS = log_pos\n\nthread_type: IO_THREAD | SQL_THREAD\n\nSTART SLAVE with no thread_type options starts both of the slave\nthreads. The I/O thread reads events from the master server and stores\nthem in the relay log. The SQL thread reads events from the relay log\nand executes them. START SLAVE requires the SUPER privilege.\n\nIf START SLAVE succeeds in starting the slave threads, it returns\nwithout any error. However, even in that case, it might be that the\nslave threads start and then later stop (for example, because they do\nnot manage to connect to the master or read its binary log, or some\nother problem). START SLAVE does not warn you about this. You must\ncheck the slave\'s error log for error messages generated by the slave\nthreads, or check that they are running satisfactorily with SHOW SLAVE\nSTATUS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/start-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/start-slave.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (322,2,'AREA','Area(poly)\n\nReturns as a double-precision number the area of the Polygon value\npoly, as measured in its spatial reference system.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…','mysql> SET @poly = \'Polygon((0 0,0 3,3 0,0 0),(1 1,1 2,2 1,1 1))\';\nmysql> SELECT Area(GeomFromText(@poly));\n+---------------------------+\n| Area(GeomFromText(@poly)) |\n+---------------------------+\n| 4 |\n+---------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (323,22,'BEGIN END','Syntax:\n[begin_label:] BEGIN\n [statement_list]\nEND [end_label]\n\nBEGIN ... END syntax is used for writing compound statements, which can\nappear within stored programs. A compound statement can contain\nmultiple statements, enclosed by the BEGIN and END keywords.\nstatement_list represents a list of one or more statements, each\nterminated by a semicolon (;) statement delimiter. statement_list is\noptional, which means that the empty compound statement (BEGIN END) is\nlegal.\n\nUse of multiple statements requires that a client is able to send\nstatement strings containing the ; statement delimiter. This is handled\nin the mysql command-line client with the delimiter command. Changing\nthe ; end-of-statement delimiter (for example, to //) allows ; to be\nused in a program body. For an example, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defining.html.\… compound statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/begin-end.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/begin-end.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (324,25,'FLUSH','Syntax:\nFLUSH [NO_WRITE_TO_BINLOG | LOCAL]\n flush_option [, flush_option] ...\n\nThe FLUSH statement clears or reloads various internal caches used by\nMySQL. To execute FLUSH, you must have the RELOAD privilege.\n\nThe RESET statement is similar to FLUSH. See [HELP RESET].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/flush.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/flush.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (323,22,'BEGIN END','Syntax:\n[begin_label:] BEGIN\n [statement_list]\nEND [end_label]\n\nBEGIN ... END syntax is used for writing compound statements, which can\nappear within stored programs. A compound statement can contain\nmultiple statements, enclosed by the BEGIN and END keywords.\nstatement_list represents a list of one or more statements, each\nterminated by a semicolon (;) statement delimiter. statement_list is\noptional, which means that the empty compound statement (BEGIN END) is\nlegal.\n\nUse of multiple statements requires that a client is able to send\nstatement strings containing the ; statement delimiter. This is handled\nin the mysql command-line client with the delimiter command. Changing\nthe ; end-of-statement delimiter (for example, to //) allows ; to be\nused in a program body. For an example, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defining.html.\… BEGIN ... END block can be labeled. Labels follow these rules:\n\no end_label cannot be given unless begin_label is also present.\n\no If both begin_label and end_label are present, they must be the same.\n\no Labels can be up to 16 characters long.\n\nLabels are also allowed for the LOOP, REPEAT, and WHILE statements.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/begin-end.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/begin-end.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (324,25,'FLUSH','Syntax:\nFLUSH [NO_WRITE_TO_BINLOG | LOCAL]\n flush_option [, flush_option] ...\n\nThe FLUSH statement clears or reloads various internal caches used by\nMySQL. One variant acquires a lock. To execute FLUSH, you must have the\nRELOAD privilege.\n\nBy default, FLUSH statements are written to the binary log so that they\nwill be replicated to replication slaves. Logging can be suppressed\nwith the optional NO_WRITE_TO_BINLOG keyword or its alias LOCAL.\n\n*Note*: FLUSH LOGS, FLUSH MASTER, FLUSH SLAVE, and FLUSH TABLES WITH\nREAD LOCK are not written to the binary log in any case because they\nwould cause problems if replicated to a slave.\n\nThe RESET statement is similar to FLUSH. See [HELP RESET], for\ninformation about using the RESET statement with replication.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/flush.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/flush.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (325,25,'SHOW PROCEDURE STATUS','Syntax:\nSHOW PROCEDURE STATUS\n [LIKE \'pattern\' | WHERE expr]\n\nThis statement is a MySQL extension. It returns characteristics of a\nstored procedure, such as the database, name, type, creator, creation\nand modification dates, and character set information. A similar\nstatement, SHOW FUNCTION STATUS, displays information about stored\nfunctions (see [HELP SHOW FUNCTION STATUS]).\n\nThe LIKE clause, if present, indicates which procedure or function\nnames to match. The WHERE clause can be given to select rows using more\ngeneral conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-procedure-status.html\n\n','mysql> SHOW PROCEDURE STATUS LIKE \'sp1\'\\G\n*************************** 1. row ***************************\n Db: test\n Name: sp1\n Type: PROCEDURE\n Definer: testuser@localhost\n Modified: 2004-08-03 15:29:37\n Created: 2004-08-03 15:29:37\n Security_type: DEFINER\n Comment:\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n','http://dev.mysql.com/doc/refman/5.1/en/show-procedure-status.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (326,25,'SHOW WARNINGS','Syntax:\nSHOW WARNINGS [LIMIT [offset,] row_count]\nSHOW COUNT(*) WARNINGS\n\nSHOW WARNINGS shows the error, warning, and note messages that resulted\nfrom the last statement that generated messages in the current session.\nIt shows nothing if the last statement used a table and generated no\nmessages. (That is, a statement that uses a table but generates no\nmessages clears the message list.) Statements that do not use tables\nand do not generate messages have no effect on the message list.\n\nWarnings are generated for DML statements such as INSERT, UPDATE, and\nLOAD DATA INFILE as well as DDL statements such as CREATE TABLE and\nALTER TABLE.\n\nA related statement, SHOW ERRORS, shows only the errors. See [HELP SHOW\nERRORS].\n\nThe SHOW COUNT(*) WARNINGS statement displays the total number of\nerrors, warnings, and notes. You can also retrieve this number from the\nwarning_count variable:\n\nSHOW COUNT(*) WARNINGS;\nSELECT @@warning_count;\n\nThe value of warning_count might be greater than the number of messages\ndisplayed by SHOW WARNINGS if the max_error_count system variable is\nset so low that not all messages are stored. An example shown later in\nthis section demonstrates how this can happen.\n\nThe LIMIT clause has the same syntax as for the SELECT statement. See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (327,27,'DESCRIBE','Syntax:\n{DESCRIBE | DESC} tbl_name [col_name | wild]\n\nDESCRIBE provides information about the columns in a table. It is a\nshortcut for SHOW COLUMNS FROM. These statements also display\ninformation for views. (See [HELP SHOW COLUMNS].)\n\ncol_name can be a column name, or a string containing the SQL "%" and\n"_" wildcard characters to obtain output only for the columns with\nnames matching the string. There is no need to enclose the string\nwithin quotes unless it contains spaces or other special characters.\n\nmysql> DESCRIBE City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nThe description for SHOW COLUMNS provides more information about the\noutput columns (see [HELP SHOW COLUMNS]).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/describe.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/describe.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (326,27,'DESCRIBE','Syntax:\n{DESCRIBE | DESC} tbl_name [col_name | wild]\n\nDESCRIBE provides information about the columns in a table. It is a\nshortcut for SHOW COLUMNS FROM. These statements also display\ninformation for views. (See [HELP SHOW COLUMNS].)\n\ncol_name can be a column name, or a string containing the SQL "%" and\n"_" wildcard characters to obtain output only for the columns with\nnames matching the string. There is no need to enclose the string\nwithin quotes unless it contains spaces or other special characters.\n\nmysql> DESCRIBE City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nThe description for SHOW COLUMNS provides more information about the\noutput columns (see [HELP SHOW COLUMNS]).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/describe.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/describe.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (327,25,'SHOW WARNINGS','Syntax:\nSHOW WARNINGS [LIMIT [offset,] row_count]\nSHOW COUNT(*) WARNINGS\n\nSHOW WARNINGS shows the error, warning, and note messages that resulted\nfrom the last statement that generated messages in the current session.\nIt shows nothing if the last statement used a table and generated no\nmessages. (That is, a statement that uses a table but generates no\nmessages clears the message list.) Statements that do not use tables\nand do not generate messages have no effect on the message list.\n\nWarnings are generated for DML statements such as INSERT, UPDATE, and\nLOAD DATA INFILE as well as DDL statements such as CREATE TABLE and\nALTER TABLE.\n\nA related statement, SHOW ERRORS, shows only the errors. See [HELP SHOW\nERRORS].\n\nThe SHOW COUNT(*) WARNINGS statement displays the total number of\nerrors, warnings, and notes. You can also retrieve this number from the\nwarning_count variable:\n\nSHOW COUNT(*) WARNINGS;\nSELECT @@warning_count;\n\nThe value of warning_count might be greater than the number of messages\ndisplayed by SHOW WARNINGS if the max_error_count system variable is\nset so low that not all messages are stored. An example shown later in\nthis section demonstrates how this can happen.\n\nThe LIMIT clause has the same syntax as for the SELECT statement. See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (328,9,'DROP USER','Syntax:\nDROP USER user [, user] ...\n\nThe DROP USER statement removes one or more MySQL accounts. It removes\nprivilege rows for the account from all grant tables. To use this\nstatement, you must have the global CREATE USER privilege or the DELETE\nprivilege for the mysql database. Each account is named using the same\nformat as for the GRANT statement; for example, \'jeffrey\'@\'localhost\'.\nIf you specify only the user name part of the account name, a host name\npart of \'%\' is used. For additional information about specifying\naccount names, see [HELP GRANT].\n\nWith DROP USER, you can remove an account and its privileges as\nfollows:\n\nDROP USER user;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-user.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-user.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (329,16,'STDDEV_POP','Syntax:\nSTDDEV_POP(expr)\n\nReturns the population standard deviation of expr (the square root of\nVAR_POP()). You can also use STD() or STDDEV(), which are equivalent\nbut not standard SQL.\n\nSTDDEV_POP() returns NULL if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (330,25,'SHOW CHARACTER SET','Syntax:\nSHOW CHARACTER SET\n [LIKE \'pattern\' | WHERE expr]\n\nThe SHOW CHARACTER SET statement shows all available character sets.\nThe LIKE clause, if present, indicates which character set names to\nmatch. The WHERE clause can be given to select rows using more general\nconditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html. For example:\n\nmysql> SHOW CHARACTER SET LIKE \'latin%\';\n+---------+-----------------------------+-------------------+--------+\n| Charset | Description | Default collation | Maxlen |\n+---------+-----------------------------+-------------------+--------+\n| latin1 | cp1252 West European | latin1_swedish_ci | 1 |\n| latin2 | ISO 8859-2 Central European | latin2_general_ci | 1 |\n| latin5 | ISO 8859-9 Turkish | latin5_turkish_ci | 1 |\n| latin7 | ISO 8859-13 Baltic | latin7_general_ci | 1 |\n+---------+-----------------------------+-------------------+--------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-character-set.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-character-set.html'); @@ -407,10 +407,10 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (338,14,'VALUES','Syntax:\nVALUES(col_name)\n\nIn an INSERT ... ON DUPLICATE KEY UPDATE statement, you can use the\nVALUES(col_name) function in the UPDATE clause to refer to column\nvalues from the INSERT portion of the statement. In other words,\nVALUES(col_name) in the UPDATE clause refers to the value of col_name\nthat would be inserted, had no duplicate-key conflict occurred. This\nfunction is especially useful in multiple-row inserts. The VALUES()\nfunction is meaningful only in INSERT ... ON DUPLICATE KEY UPDATE\nstatements and returns NULL otherwise.\nhttp://dev.mysql.com/doc/refman/5.1/en/insert-on-duplicate.html…: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','mysql> INSERT INTO table (a,b,c) VALUES (1,2,3),(4,5,6)\n -> ON DUPLICATE KEY UPDATE c=VALUES(a)+VALUES(b);\n','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (339,35,'SUBSTRING_INDEX','Syntax:\nSUBSTRING_INDEX(str,delim,count)\n\nReturns the substring from string str before count occurrences of the\ndelimiter delim. If count is positive, everything to the left of the\nfinal delimiter (counting from the left) is returned. If count is\nnegative, everything to the right of the final delimiter (counting from\nthe right) is returned. SUBSTRING_INDEX() performs a case-sensitive\nmatch when searching for delim.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT SUBSTRING_INDEX(\'www.mysql.com\', \'.\', 2);\n -> \'www.mysql\'\nmysql> SELECT SUBSTRING_INDEX(\'www.mysql.com\', \'.\', -2);\n -> \'mysql.com\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (340,11,'ENCODE','Syntax:\nENCODE(str,pass_str)\n\nEncrypt str using pass_str as the password. To decrypt the result, use\nDECODE().\n\nThe result is a binary string of the same length as str.\n\nThe strength of the encryption is based on how good the random\ngenerator is. It should suffice for short strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (341,22,'LOOP','Syntax:\n[begin_label:] LOOP\n statement_list\nEND LOOP [end_label]\n\nLOOP implements a simple loop construct, enabling repeated execution of\nthe statement list, which consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter. The statements\nwithin the loop are repeated until the loop is exited; usually this is\naccomplished with a LEAVE statement.\n\nA LOOP statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (341,22,'LOOP','Syntax:\n[begin_label:] LOOP\n statement_list\nEND LOOP [end_label]\n\nLOOP implements a simple loop construct, enabling repeated execution of\nthe statement list, which consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter. The statements\nwithin the loop are repeated until the loop is exited; usually this is\naccomplished with a LEAVE statement.\n\nA LOOP statement can be labeled. See [HELP BEGIN END] for the rules\nregarding label use.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (342,4,'TRUNCATE','Syntax:\nTRUNCATE(X,D)\n\nReturns the number X, truncated to D decimal places. If D is 0, the\nresult has no decimal point or fractional part. D can be negative to\ncause D digits left of the decimal point of the value X to become zero.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT TRUNCATE(1.223,1);\n -> 1.2\nmysql> SELECT TRUNCATE(1.999,1);\n -> 1.9\nmysql> SELECT TRUNCATE(1.999,0);\n -> 1\nmysql> SELECT TRUNCATE(-1.999,1);\n -> -1.9\nmysql> SELECT TRUNCATE(122,-2);\n -> 100\nmysql> SELECT TRUNCATE(10.28*100,0);\n -> 1028\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (343,30,'TIMESTAMPADD','Syntax:\nTIMESTAMPADD(unit,interval,datetime_expr)\n\nAdds the integer expression interval to the date or datetime expression\ndatetime_expr. The unit for interval is given by the unit argument,\nwhich should be one of the following values: FRAC_SECOND\n(microseconds), SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER, or\nYEAR.\n\nBeginning with MySQL 5.1.24, it is possible to use MICROSECOND in place\nof FRAC_SECOND with this function, and FRAC_SECOND is deprecated.\n\nThe unit value may be specified using one of keywords as shown, or with\na prefix of SQL_TSI_. For example, DAY and SQL_TSI_DAY both are legal.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMESTAMPADD(MINUTE,1,\'2003-01-02\');\n -> \'2003-01-02 00:01:00\'\nmysql> SELECT TIMESTAMPADD(WEEK,1,\'2003-01-02\');\n -> \'2003-01-09\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (344,25,'SHOW','SHOW has many forms that provide information about databases, tables,\ncolumns, or status information about the server. This section describes\nthose following:\n\nSHOW AUTHORS\nSHOW CHARACTER SET [like_or_where]\nSHOW COLLATION [like_or_where]\nSHOW [FULL] COLUMNS FROM tbl_name [FROM db_name] [like_or_where]\nSHOW CONTRIBUTORS\nSHOW CREATE DATABASE db_name\nSHOW CREATE EVENT event_name\nSHOW CREATE FUNCTION func_name\nSHOW CREATE PROCEDURE proc_name\nSHOW CREATE TABLE tbl_name\nSHOW CREATE TRIGGER trigger_name\nSHOW CREATE VIEW view_name\nSHOW DATABASES [like_or_where]\nSHOW ENGINE engine_name {STATUS | MUTEX}\nSHOW [STORAGE] ENGINES\nSHOW ERRORS [LIMIT [offset,] row_count]\nSHOW EVENTS\nSHOW FUNCTION CODE func_name\nSHOW FUNCTION STATUS [like_or_where]\nSHOW GRANTS FOR user\nSHOW INDEX FROM tbl_name [FROM db_name]\nSHOW INNODB STATUS\nSHOW OPEN TABLES [FROM db_name] [like_or_where]\nSHOW PLUGINS\nSHOW PROCEDURE CODE proc_name\nSHOW PROCEDURE STATUS [like_or_where]\nSHOW PRIVILEGES\nSHOW [FULL] PROCESSLIST\nSHOW PROFILE [types] [FOR QUERY n] [OFFSET n] [LIMIT n]\nSHOW PROFILES\nSHOW SCHEDULER STATUS\nSHOW [GLOBAL | SESSION] STATUS [like_or_where]\nSHOW TABLE STATUS [FROM db_name] [like_or_where]\nSHOW TABLES [FROM db_name] [like_or_where]\nSHOW TRIGGERS [FROM db_name] [like_or_where]\nSHOW [GLOBAL | SESSION] VARIABLES [like_or_where]\nSHOW WARNINGS [LIMIT [offset,] row_count]\n\nlike_or_where:\n LIKE \'pattern\'\n | WHERE expr\n\nIf the syntax for a given SHOW statement includes a LIKE \'pattern\'\npart, \'pattern\' is a string that can contain the SQL "%" and "_"\nwildcard characters. The pattern is useful for restricting statement\noutput to matching values.\n\nSeveral SHOW statements also accept a WHERE clause that provides more\nflexibility in specifying which rows to display. See\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (343,30,'TIMESTAMPADD','Syntax:\nTIMESTAMPADD(unit,interval,datetime_expr)\n\nAdds the integer expression interval to the date or datetime expression\ndatetime_expr. The unit for interval is given by the unit argument,\nwhich should be one of the following values: FRAC_SECOND\n(microseconds), SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER, or\nYEAR.\n\nBeginning with MySQL 5.1.24, it is possible to use MICROSECOND in place\nof FRAC_SECOND with this function, and FRAC_SECOND is deprecated.\nFRAC_SECOND is removed in MySQL 5.5.\n\nThe unit value may be specified using one of keywords as shown, or with\na prefix of SQL_TSI_. For example, DAY and SQL_TSI_DAY both are legal.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMESTAMPADD(MINUTE,1,\'2003-01-02\');\n -> \'2003-01-02 00:01:00\'\nmysql> SELECT TIMESTAMPADD(WEEK,1,\'2003-01-02\');\n -> \'2003-01-09\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (344,25,'SHOW','SHOW has many forms that provide information about databases, tables,\ncolumns, or status information about the server. This section describes\nthose following:\n\nSHOW AUTHORS\nSHOW CHARACTER SET [like_or_where]\nSHOW COLLATION [like_or_where]\nSHOW [FULL] COLUMNS FROM tbl_name [FROM db_name] [like_or_where]\nSHOW CONTRIBUTORS\nSHOW CREATE DATABASE db_name\nSHOW CREATE EVENT event_name\nSHOW CREATE FUNCTION func_name\nSHOW CREATE PROCEDURE proc_name\nSHOW CREATE TABLE tbl_name\nSHOW CREATE TRIGGER trigger_name\nSHOW CREATE VIEW view_name\nSHOW DATABASES [like_or_where]\nSHOW ENGINE engine_name {STATUS | MUTEX}\nSHOW [STORAGE] ENGINES\nSHOW ERRORS [LIMIT [offset,] row_count]\nSHOW EVENTS\nSHOW FUNCTION CODE func_name\nSHOW FUNCTION STATUS [like_or_where]\nSHOW GRANTS FOR user\nSHOW INDEX FROM tbl_name [FROM db_name]\nSHOW INNODB STATUS\nSHOW OPEN TABLES [FROM db_name] [like_or_where]\nSHOW PLUGINS\nSHOW PROCEDURE CODE proc_name\nSHOW PROCEDURE STATUS [like_or_where]\nSHOW PRIVILEGES\nSHOW [FULL] PROCESSLIST\nSHOW PROFILE [types] [FOR QUERY n] [OFFSET n] [LIMIT n]\nSHOW PROFILES\nSHOW SCHEDULER STATUS\nSHOW [GLOBAL | SESSION] STATUS [like_or_where]\nSHOW TABLE STATUS [FROM db_name] [like_or_where]\nSHOW [FULL] TABLES [FROM db_name] [like_or_where]\nSHOW TRIGGERS [FROM db_name] [like_or_where]\nSHOW [GLOBAL | SESSION] VARIABLES [like_or_where]\nSHOW WARNINGS [LIMIT [offset,] row_count]\n\nlike_or_where:\n LIKE \'pattern\'\n | WHERE expr\n\nIf the syntax for a given SHOW statement includes a LIKE \'pattern\'\npart, \'pattern\' is a string that can contain the SQL "%" and "_"\nwildcard characters. The pattern is useful for restricting statement\noutput to matching values.\n\nSeveral SHOW statements also accept a WHERE clause that provides more\nflexibility in specifying which rows to display. See\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (345,17,'GREATEST','Syntax:\nGREATEST(value1,value2,...)\n\nWith two or more arguments, returns the largest (maximum-valued)\nargument. The arguments are compared using the same rules as for\nLEAST().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT GREATEST(2,0);\n -> 2\nmysql> SELECT GREATEST(34.0,3.0,5.0,767.0);\n -> 767.0\nmysql> SELECT GREATEST(\'B\',\'A\',\'C\');\n -> \'C\'\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (346,25,'SHOW VARIABLES','Syntax:\nSHOW [GLOBAL | SESSION] VARIABLES\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW VARIABLES shows the values of MySQL system variables. This\ninformation also can be obtained using the mysqladmin variables\ncommand. The LIKE clause, if present, indicates which variable names to\nmatch. The WHERE clause can be given to select rows using more general\nconditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html. This\nstatement does not require any privilege. It requires only the ability\nto connect to the server.\n\nWith the GLOBAL modifier, SHOW VARIABLES displays the values that are\nused for new connections to MySQL. With SESSION, it displays the values\nthat are in effect for the current connection. If no modifier is\npresent, the default is SESSION. LOCAL is a synonym for SESSION.\nWith a LIKE clause, the statement displays only rows for those\nvariables with names that match the pattern. To obtain the row for a\nspecific variable, use a LIKE clause as shown:\n\nSHOW VARIABLES LIKE \'max_join_size\';\nSHOW SESSION VARIABLES LIKE \'max_join_size\';\n\nTo get a list of variables whose name match a pattern, use the "%"\nwildcard character in a LIKE clause:\n\nSHOW VARIABLES LIKE \'%size%\';\nSHOW GLOBAL VARIABLES LIKE \'%size%\';\n\nWildcard characters can be used in any position within the pattern to\nbe matched. Strictly speaking, because "_" is a wildcard that matches\nany single character, you should escape it as "\\_" to match it\nliterally. In practice, this is rarely necessary.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-variables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-variables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (347,25,'BINLOG','Syntax:\nBINLOG \'str\'\n\nBINLOG is an internal-use statement. It is generated by the mysqlbinlog\nprogram as the printable representation of certain events in binary log\nfiles. (See http://dev.mysql.com/doc/refman/5.1/en/mysqlbinlog.html.)\nThe \'str\' value is a base 64-encoded string the that server decodes to\ndetermine the data change indicated by the corresponding event. This\nstatement requires the SUPER privilege. It was added in MySQL 5.1.5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/binlog.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/binlog.html'); @@ -422,15 +422,15 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (353,26,'SELECT','Syntax:\nSELECT\n [ALL | DISTINCT | DISTINCTROW ]\n [HIGH_PRIORITY]\n [STRAIGHT_JOIN]\n [SQL_SMALL_RESULT] [SQL_BIG_RESULT] [SQL_BUFFER_RESULT]\n [SQL_CACHE | SQL_NO_CACHE] [SQL_CALC_FOUND_ROWS]\n select_expr [, select_expr ...]\n [FROM table_references\n [WHERE where_condition]\n [GROUP BY {col_name | expr | position}\n [ASC | DESC], ... [WITH ROLLUP]]\n [HAVING where_condition]\n [ORDER BY {col_name | expr | position}\n [ASC | DESC], ...]\n [LIMIT {[offset,] row_count | row_count OFFSET offset}]\n [PROCEDURE procedure_name(argument_list)]\n [INTO OUTFILE \'file_name\'\n [CHARACTER SET charset_name]\n export_options\n | INTO DUMPFILE \'file_name\'\n | INTO var_name [, var_name]]\n [FOR UPDATE | LOCK IN SHARE MODE]]\n\nSELECT is used to retrieve rows selected from one or more tables, and\ncan include UNION statements and subqueries. See [HELP UNION], and\nhttp://dev.mysql.com/doc/refman/5.1/en/subqueries.html.\n\nThe most commonly used clauses of SELECT statements are these:\n\no Each select_expr indicates a column that you want to retrieve. There\n must be at least one select_expr.\n\no table_references indicates the table or tables from which to retrieve\n rows. Its syntax is described in [HELP JOIN].\n\no The WHERE clause, if given, indicates the condition or conditions\n that rows must satisfy to be selected. where_condition is an\n expression that evaluates to true for each row to be selected. The\n statement selects all rows if there is no WHERE clause.\n\n In the WHERE clause, you can use any of the functions and operators\n that MySQL supports, except for aggregate (summary) functions. See\n http://dev.mysql.com/doc/refman/5.1/en/functions.html.\n\nSELECT can also be used to retrieve rows computed without reference to\nany table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/select.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/select.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (354,4,'COT','Syntax:\nCOT(X)\n\nReturns the cotangent of X.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT COT(12);\n -> -1.5726734063977\nmysql> SELECT COT(0);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (355,25,'SHOW CREATE EVENT','Syntax:\nSHOW CREATE EVENT event_name\n\nThis statement displays the CREATE EVENT statement needed to re-create\na given event. For example (using the same event e_daily defined and\nthen altered in [HELP SHOW EVENTS]):\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-create-event.html\n\n','mysql> SHOW CREATE EVENT test.e_daily\\G\n*************************** 1. row ***************************\n Event: e_daily\n sql_mode:\n time_zone: SYSTEM\n Create Event: CREATE EVENT `e_daily`\n ON SCHEDULE EVERY 1 DAY\n STARTS CURRENT_TIMESTAMP + INTERVAL 6 HOUR\n ON COMPLETION NOT PRESERVE\n ENABLE\n COMMENT \'Saves total number of sessions then\n clears the table each day\'\n DO BEGIN\n INSERT INTO site_activity.totals (time, total)\n SELECT CURRENT_TIMESTAMP, COUNT(*)\n FROM site_activity.sessions;\n DELETE FROM site_activity.sessions;\n END\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n','http://dev.mysql.com/doc/refman/5.1/en/show-create-event.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (356,19,'BACKUP TABLE','Syntax:\nBACKUP TABLE tbl_name [, tbl_name] ... TO \'/path/to/backup/directory\'\n\n*Note*: This statement is deprecated. We are working on a better\nreplacement for it that will provide online backup capabilities. In the\nmeantime, the mysqlhotcopy script can be used instead.\n\nBACKUP TABLE copies to the backup directory the minimum number of table\nfiles needed to restore the table, after flushing any buffered changes\nto disk. The statement works only for MyISAM tables. It copies the .frm\ndefinition and .MYD data files. The .MYI index file can be rebuilt from\nthose two files. The directory should be specified as a full path name.\nTo restore the table, use RESTORE TABLE.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/backup-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/backup-table.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (356,19,'BACKUP TABLE','Syntax:\nBACKUP TABLE tbl_name [, tbl_name] ... TO \'/path/to/backup/directory\'\n\n*Note*: This statement is deprecated and is removed in MySQL 5.5. As an\nalternative, mysqldump or mysqlhotcopy can be used instead.\n\nBACKUP TABLE copies to the backup directory the minimum number of table\nfiles needed to restore the table, after flushing any buffered changes\nto disk. The statement works only for MyISAM tables. It copies the .frm\ndefinition and .MYD data files. The .MYI index file can be rebuilt from\nthose two files. The directory should be specified as a full path name.\nTo restore the table, use RESTORE TABLE.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/backup-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/backup-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (357,35,'LOAD_FILE','Syntax:\nLOAD_FILE(file_name)\n\nReads the file and returns the file contents as a string. To use this\nfunction, the file must be located on the server host, you must specify\nthe full path name to the file, and you must have the FILE privilege.\nThe file must be readable by all and its size less than\nmax_allowed_packet bytes. If the secure_file_priv system variable is\nset to a nonempty directory name, the file to be loaded must be located\nin that directory.\n\nIf the file does not exist or cannot be read because one of the\npreceding conditions is not satisfied, the function returns NULL.\n\nAs of MySQL 5.1.6, the character_set_filesystem system variable\ncontrols interpretation of file names that are given as literal\nstrings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> UPDATE t\n SET blob_col=LOAD_FILE(\'/tmp/picture\')\n WHERE id=1;\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (358,25,'LOAD TABLE FROM MASTER','Syntax:\nLOAD TABLE tbl_name FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated in\nversions 4.1 of MySQL and above. We will introduce a more advanced\ntechnique (called "online backup") in a future version. That technique\nwill have the additional advantage of working with more storage\nengines.\n\nFor MySQL 5.1 and earlier, the recommended alternative solution to\nusing LOAD DATA FROM MASTER or LOAD TABLE FROM MASTER is using\nmysqldump or mysqlhotcopy. The latter requires Perl and two Perl\nmodules (DBI and DBD:mysql) and works for MyISAM and ARCHIVE tables\nonly. With mysqldump, you can create SQL dumps on the master and pipe\n(or copy) these to a mysql client on the slave. This has the advantage\nof working for all storage engines, but can be quite slow, since it\nworks using SELECT.\n\nTransfers a copy of the table from the master to the slave. This\nstatement is implemented mainly debugging LOAD DATA FROM MASTER\noperations. To use LOAD TABLE, the account used for connecting to the\nmaster server must have the RELOAD and SUPER privileges on the master\nand the SELECT privilege for the master table to load. On the slave\nside, the user that issues LOAD TABLE FROM MASTER must have privileges\nfor dropping and creating the table.\n\nThe conditions for LOAD DATA FROM MASTER apply here as well. For\nexample, LOAD TABLE FROM MASTER works only for MyISAM tables. The\ntimeout notes for LOAD DATA FROM MASTER apply as well.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (358,25,'LOAD TABLE FROM MASTER','Syntax:\nLOAD TABLE tbl_name FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated as\nof MySQL 4.1 and removed in MySQL 5.5.\n\nThe recommended alternative solution to using LOAD DATA FROM MASTER or\nLOAD TABLE FROM MASTER is using mysqldump or mysqlhotcopy. The latter\nrequires Perl and two Perl modules (DBI and DBD:mysql) and works for\nMyISAM and ARCHIVE tables only. With mysqldump, you can create SQL\ndumps on the master and pipe (or copy) these to a mysql client on the\nslave. This has the advantage of working for all storage engines, but\ncan be quite slow, since it works using SELECT.\n\nTransfers a copy of the table from the master to the slave. This\nstatement is implemented mainly debugging LOAD DATA FROM MASTER\noperations. To use LOAD TABLE, the account used for connecting to the\nmaster server must have the RELOAD and SUPER privileges on the master\nand the SELECT privilege for the master table to load. On the slave\nside, the user that issues LOAD TABLE FROM MASTER must have privileges\nfor dropping and creating the table.\n\nThe conditions for LOAD DATA FROM MASTER apply here as well. For\nexample, LOAD TABLE FROM MASTER works only for MyISAM tables. The\ntimeout notes for LOAD DATA FROM MASTER apply as well.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (359,3,'POINTFROMTEXT','PointFromText(wkt[,srid])\n\nConstructs a POINT value using its WKT representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (360,16,'GROUP_CONCAT','Syntax:\nGROUP_CONCAT(expr)\n\nThis function returns a string result with the concatenated non-NULL\nvalues from a group. It returns NULL if there are no non-NULL values.\nThe full syntax is as follows:\n\nGROUP_CONCAT([DISTINCT] expr [,expr ...]\n [ORDER BY {unsigned_integer | col_name | expr}\n [ASC | DESC] [,col_name ...]]\n [SEPARATOR str_val])\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','mysql> SELECT student_name,\n -> GROUP_CONCAT(test_score)\n -> FROM student\n -> GROUP BY student_name;\n','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (361,30,'DATE_FORMAT','Syntax:\nDATE_FORMAT(date,format)\n\nFormats the date value according to the format string.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DATE_FORMAT(\'2009-10-04 22:23:00\', \'%W %M %Y\');\n -> \'Sunday October 2009\'\nmysql> SELECT DATE_FORMAT(\'2007-10-04 22:23:00\', \'%H:%i:%s\');\n -> \'22:23:00\'\nmysql> SELECT DATE_FORMAT(\'1900-10-04 22:23:00\',\n -> \'%D %y %a %d %m %b %j\');\n -> \'4th 00 Thu 04 10 Oct 277\'\nmysql> SELECT DATE_FORMAT(\'1997-10-04 22:23:00\',\n -> \'%H %k %I %r %T %S %w\');\n -> \'22 22 10 10:23:00 PM 22:23:00 00 6\'\nmysql> SELECT DATE_FORMAT(\'1999-01-01\', \'%X %V\');\n -> \'1998 52\'\nmysql> SELECT DATE_FORMAT(\'2006-06-00\', \'%d\');\n -> \'00\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (362,15,'BENCHMARK','Syntax:\nBENCHMARK(count,expr)\n\nThe BENCHMARK() function executes the expression expr repeatedly count\ntimes. It may be used to time how quickly MySQL processes the\nexpression. The result value is always 0. The intended use is from\nwithin the mysql client, which reports query execution times:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT BENCHMARK(1000000,ENCODE(\'hello\',\'goodbye\'));\n+----------------------------------------------+\n| BENCHMARK(1000000,ENCODE(\'hello\',\'goodbye\')) |\n+----------------------------------------------+\n| 0 |\n+----------------------------------------------+\n1 row in set (4.74 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (363,30,'YEAR','Syntax:\nYEAR(date)\n\nReturns the year for date, in the range 1000 to 9999, or 0 for the\n"zero" date.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT YEAR(\'1987-01-01\');\n -> 1987\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (364,25,'SHOW ENGINE','Syntax:\nSHOW ENGINE engine_name {STATUS | MUTEX}\n\nSHOW ENGINE displays operational information about a storage engine.\nThe following statements currently are supported:\n\nSHOW ENGINE INNODB STATUS\nSHOW ENGINE INNODB MUTEX\nSHOW ENGINE {NDB | NDBCLUSTER} STATUS\n\nOlder (and now deprecated) synonyms are SHOW INNODB STATUS for SHOW\nENGINE INNODB STATUS and SHOW MUTEX STATUS for SHOW ENGINE INNODB\nMUTEX.\n\nIn MySQL 5.0, SHOW ENGINE INNODB MUTEX is invoked as SHOW MUTEX STATUS.\nThe latter statement displays similar information but in a somewhat\ndifferent output format.\n\nSHOW ENGINE BDB LOGS formerly displayed status information about BDB\nlog files. As of MySQL 5.1.12, the BDB storage engine is not supported,\nand this statement produces a warning.\n\nSHOW ENGINE INNODB STATUS displays extensive information from the\nstandard InnoDB Monitor about the state of the InnoDB storage engine.\nFor information about the standard monitor and other InnoDB Monitors\nthat provide information about InnoDB processing, see\nhttp://dev.mysql.com/doc/refman/5.1/en/innodb-monitors.html.\n\nSHOW ENGINE INNODB MUTEX displays InnoDB mutex statistics. From MySQL\n5.1.2 to 5.1.14, the statement displays the following output fields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The mutex name and the source file where it is implemented. Example:\n &pool->mutex:mem0pool.c\n\n The mutex name indicates its purpose. For example, the log_sys mutex\n is used by the InnoDB logging subsystem and indicates how intensive\n logging activity is. The buf_pool mutex protects the InnoDB buffer\n pool.\n\no Status\n\n The mutex status. The fields contains several values:\n\n o count indicates how many times the mutex was requested.\n\n o spin_waits indicates how many times the spinlock had to run.\n\n o spin_rounds indicates the number of spinlock rounds. (spin_rounds\n divided by spin_waits provides the average round count.)\n\n o os_waits indicates the number of operating system waits. This\n occurs when the spinlock did not work (the mutex was not locked\n during the spinlock and it was necessary to yield to the operating\n system and wait).\n\n o os_yields indicates the number of times a the thread trying to lock\n a mutex gave up its timeslice and yielded to the operating system\n (on the presumption that allowing other threads to run will free\n the mutex so that it can be locked).\n\n o os_wait_times indicates the amount of time (in ms) spent in\n operating system waits, if the timed_mutexes system variable is 1\n (ON). If timed_mutexes is 0 (OFF), timing is disabled, so\n os_wait_times is 0. timed_mutexes is off by default.\n\nFrom MySQL 5.1.15 on, the statement displays the following output\nfields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The source file where the mutex is implemented, and the line number\n in the file where the mutex is created. The line number may change\n depending on your version of MySQL.\n\no Status\n\n This field displays the same values as previously described (count,\n spin_waits, spin_rounds, os_waits, os_yields, os_wait_times), but\n only if UNIV_DEBUG was defined at MySQL compilation time (for\n example, in include/univ.h in the InnoDB part of the MySQL source\n tree). If UNIV_DEBUG was not defined, the statement displays only the\n os_waits value. In the latter case (without UNIV_DEBUG), the\n information on which the output is based is insufficient to\n distinguish regular mutexes and mutexes that protect rw-locks (which\n allow multiple readers or a single writer). Consequently, the output\n may appear to contain multiple rows for the same mutex.\n\nInformation from this statement can be used to diagnose system\nproblems. For example, large values of spin_waits and spin_rounds may\nindicate scalability problems.\n\nIf the server has the NDBCLUSTER storage engine enabled, SHOW ENGINE\nNDB STATUS displays cluster status information such as the number of\nconnected data nodes, the cluster connectstring, and cluster binlog\nepochs, as well as counts of various Cluster API objects created by the\nMySQL Server when connected to the cluster. Sample output from this\nstatement is shown here:\n\nmysql> SHOW ENGINE NDB STATUS;\n+------------+-----------------------+--------------------------------------------------+\n| Type | Name | Status |\n+------------+-----------------------+--------------------------------------------------+\n| ndbcluster | connection | cluster_node_id=7,\n connected_host=192.168.0.103, connected_port=1186, number_of_data_nodes=4,\n number_of_ready_data_nodes=3, connect_count=0 |\n| ndbcluster | NdbTransaction | created=6, free=0, sizeof=212 |\n| ndbcluster | NdbOperation | created=8, free=8, sizeof=660 |\n| ndbcluster | NdbIndexScanOperation | created=1, free=1, sizeof=744 |\n| ndbcluster | NdbIndexOperation | created=0, free=0, sizeof=664 |\n| ndbcluster | NdbRecAttr | created=1285, free=1285, sizeof=60 |\n| ndbcluster | NdbApiSignal | created=16, free=16, sizeof=136 |\n| ndbcluster | NdbLabel | created=0, free=0, sizeof=196 |\n| ndbcluster | NdbBranch | created=0, free=0, sizeof=24 |\n| ndbcluster | NdbSubroutine | created=0, free=0, sizeof=68 |\n| ndbcluster | NdbCall | created=0, free=0, sizeof=16 |\n| ndbcluster | NdbBlob | created=1, free=1, sizeof=264 |\n| ndbcluster | NdbReceiver | created=4, free=0, sizeof=68 |\n| ndbcluster | binlog | latest_epoch=155467, latest_trans_epoch=148126,\n latest_received_binlog_epoch=0, latest_handled_binlog_epoch=0,\n latest_applied_binlog_epoch=0 |\n+------------+-----------------------+--------------------------------------------------+\n\nThe rows with connection and binlog in the Name column were added to\nthe output of this statement in MySQL 5.1. The Status column in each of\nthese rows provides information about the MySQL server\'s connection to\nthe cluster and about the cluster binary log\'s status, respectively.\nThe Status information is in the form of comma-delimited set of\nname/value pairs.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engine.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engine.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (364,25,'SHOW ENGINE','Syntax:\nSHOW ENGINE engine_name {STATUS | MUTEX}\n\nSHOW ENGINE displays operational information about a storage engine.\nThe following statements currently are supported:\n\nSHOW ENGINE INNODB STATUS\nSHOW ENGINE INNODB MUTEX\nSHOW ENGINE {NDB | NDBCLUSTER} STATUS\n\nOlder (and now deprecated) synonyms are SHOW INNODB STATUS for SHOW\nENGINE INNODB STATUS and SHOW MUTEX STATUS for SHOW ENGINE INNODB\nMUTEX. SHOW INNODB STATUS and SHOW MUTEX STATUS are removed in MySQL\n5.5.\n\nIn MySQL 5.0, SHOW ENGINE INNODB MUTEX is invoked as SHOW MUTEX STATUS.\nThe latter statement displays similar information but in a somewhat\ndifferent output format.\n\nSHOW ENGINE BDB LOGS formerly displayed status information about BDB\nlog files. As of MySQL 5.1.12, the BDB storage engine is not supported,\nand this statement produces a warning.\n\nSHOW ENGINE INNODB STATUS displays extensive information from the\nstandard InnoDB Monitor about the state of the InnoDB storage engine.\nFor information about the standard monitor and other InnoDB Monitors\nthat provide information about InnoDB processing, see\nhttp://dev.mysql.com/doc/refman/5.1/en/innodb-monitors.html.\n\nSHOW ENGINE INNODB MUTEX displays InnoDB mutex statistics. From MySQL\n5.1.2 to 5.1.14, the statement displays the following output fields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The mutex name and the source file where it is implemented. Example:\n &pool->mutex:mem0pool.c\n\n The mutex name indicates its purpose. For example, the log_sys mutex\n is used by the InnoDB logging subsystem and indicates how intensive\n logging activity is. The buf_pool mutex protects the InnoDB buffer\n pool.\n\no Status\n\n The mutex status. The fields contains several values:\n\n o count indicates how many times the mutex was requested.\n\n o spin_waits indicates how many times the spinlock had to run.\n\n o spin_rounds indicates the number of spinlock rounds. (spin_rounds\n divided by spin_waits provides the average round count.)\n\n o os_waits indicates the number of operating system waits. This\n occurs when the spinlock did not work (the mutex was not locked\n during the spinlock and it was necessary to yield to the operating\n system and wait).\n\n o os_yields indicates the number of times a the thread trying to lock\n a mutex gave up its timeslice and yielded to the operating system\n (on the presumption that allowing other threads to run will free\n the mutex so that it can be locked).\n\n o os_wait_times indicates the amount of time (in ms) spent in\n operating system waits, if the timed_mutexes system variable is 1\n (ON). If timed_mutexes is 0 (OFF), timing is disabled, so\n os_wait_times is 0. timed_mutexes is off by default.\n\nFrom MySQL 5.1.15 on, the statement displays the following output\nfields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The source file where the mutex is implemented, and the line number\n in the file where the mutex is created. The line number may change\n depending on your version of MySQL.\n\no Status\n\n This field displays the same values as previously described (count,\n spin_waits, spin_rounds, os_waits, os_yields, os_wait_times), but\n only if UNIV_DEBUG was defined at MySQL compilation time (for\n example, in include/univ.h in the InnoDB part of the MySQL source\n tree). If UNIV_DEBUG was not defined, the statement displays only the\n os_waits value. In the latter case (without UNIV_DEBUG), the\n information on which the output is based is insufficient to\n distinguish regular mutexes and mutexes that protect rw-locks (which\n allow multiple readers or a single writer). Consequently, the output\n may appear to contain multiple rows for the same mutex.\n\nInformation from this statement can be used to diagnose system\nproblems. For example, large values of spin_waits and spin_rounds may\nindicate scalability problems.\n\nIf the server has the NDBCLUSTER storage engine enabled, SHOW ENGINE\nNDB STATUS displays cluster status information such as the number of\nconnected data nodes, the cluster connectstring, and cluster binlog\nepochs, as well as counts of various Cluster API objects created by the\nMySQL Server when connected to the cluster. Sample output from this\nstatement is shown here:\n\nmysql> SHOW ENGINE NDB STATUS;\n+------------+-----------------------+--------------------------------------------------+\n| Type | Name | Status |\n+------------+-----------------------+--------------------------------------------------+\n| ndbcluster | connection | cluster_node_id=7,\n connected_host=192.168.0.103, connected_port=1186, number_of_data_nodes=4,\n number_of_ready_data_nodes=3, connect_count=0 |\n| ndbcluster | NdbTransaction | created=6, free=0, sizeof=212 |\n| ndbcluster | NdbOperation | created=8, free=8, sizeof=660 |\n| ndbcluster | NdbIndexScanOperation | created=1, free=1, sizeof=744 |\n| ndbcluster | NdbIndexOperation | created=0, free=0, sizeof=664 |\n| ndbcluster | NdbRecAttr | created=1285, free=1285, sizeof=60 |\n| ndbcluster | NdbApiSignal | created=16, free=16, sizeof=136 |\n| ndbcluster | NdbLabel | created=0, free=0, sizeof=196 |\n| ndbcluster | NdbBranch | created=0, free=0, sizeof=24 |\n| ndbcluster | NdbSubroutine | created=0, free=0, sizeof=68 |\n| ndbcluster | NdbCall | created=0, free=0, sizeof=16 |\n| ndbcluster | NdbBlob | created=1, free=1, sizeof=264 |\n| ndbcluster | NdbReceiver | created=4, free=0, sizeof=68 |\n| ndbcluster | binlog | latest_epoch=155467, latest_trans_epoch=148126,\n latest_received_binlog_epoch=0, latest_handled_binlog_epoch=0,\n latest_applied_binlog_epoch=0 |\n+------------+-----------------------+--------------------------------------------------+\n\nThe rows with connection and binlog in the Name column were added to\nthe output of this statement in MySQL 5.1. The Status column in each of\nthese rows provides information about the MySQL server\'s connection to\nthe cluster and about the cluster binary log\'s status, respectively.\nThe Status information is in the form of comma-delimited set of\nname/value pairs.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engine.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engine.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (365,14,'NAME_CONST','Syntax:\nNAME_CONST(name,value)\n\nReturns the given value. When used to produce a result set column,\nNAME_CONST() causes the column to have the given name. The arguments\nshould be constants.\n\nmysql> SELECT NAME_CONST(\'myname\', 14);\n+--------+\n| myname |\n+--------+\n| 14 |\n+--------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (366,14,'RELEASE_LOCK','Syntax:\nRELEASE_LOCK(str)\n\nReleases the lock named by the string str that was obtained with\nGET_LOCK(). Returns 1 if the lock was released, 0 if the lock was not\nestablished by this thread (in which case the lock is not released),\nand NULL if the named lock did not exist. The lock does not exist if it\nwas never obtained by a call to GET_LOCK() or if it has previously been\nreleased.\n\nThe DO statement is convenient to use with RELEASE_LOCK(). See [HELP\nDO].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (367,17,'IS NULL','Syntax:\nIS NULL\n\nTests whether a value is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 1 IS NULL, 0 IS NULL, NULL IS NULL;\n -> 0, 0, 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); @@ -465,7 +465,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (396,35,'LENGTH','Syntax:\nLENGTH(str)\n\nReturns the length of the string str, measured in bytes. A multi-byte\ncharacter counts as multiple bytes. This means that for a string\ncontaining five two-byte characters, LENGTH() returns 10, whereas\nCHAR_LENGTH() returns 5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT LENGTH(\'text\');\n -> 4\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (397,30,'STR_TO_DATE','Syntax:\nSTR_TO_DATE(str,format)\n\nThis is the inverse of the DATE_FORMAT() function. It takes a string\nstr and a format string format. STR_TO_DATE() returns a DATETIME value\nif the format string contains both date and time parts, or a DATE or\nTIME value if the string contains only date or time parts. If the date,\ntime, or datetime value extracted from str is illegal, STR_TO_DATE()\nreturns NULL and produces a warning.\n\nThe server scans str attempting to match format to it. The format\nstring can contain literal characters and format specifiers beginning\nwith %. Literal characters in format must match literally in str.\nFormat specifiers in format must match a date or time part in str. For\nthe specifiers that can be used in format, see the DATE_FORMAT()\nfunction description.\n\nmysql> SELECT STR_TO_DATE(\'01,5,2013\',\'%d,%m,%Y\');\n -> \'2013-05-01\'\nmysql> SELECT STR_TO_DATE(\'May 1, 2013\',\'%M %d,%Y\');\n -> \'2013-05-01\'\n\nScanning starts at the beginning of str and fails if format is found\nnot to match. Extra characters at the end of str are ignored.\n\nmysql> SELECT STR_TO_DATE(\'a09:30:17\',\'a%h:%i:%s\');\n -> \'09:30:17\'\nmysql> SELECT STR_TO_DATE(\'a09:30:17\',\'%h:%i:%s\');\n -> NULL\nmysql> SELECT STR_TO_DATE(\'09:30:17a\',\'%h:%i:%s\');\n -> \'09:30:17\'\n\nUnspecified date or time parts have a value of 0, so incompletely\nspecified values in str produce a result with some or all parts set to\n0:\n\nmysql> SELECT STR_TO_DATE(\'abc\',\'abc\');\n -> \'0000-00-00\'\nmysql> SELECT STR_TO_DATE(\'9\',\'%m\');\n -> \'0000-09-00\'\nmysql> SELECT STR_TO_DATE(\'9\',\'%s\');\n -> \'00:00:09\'\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (398,10,'Y','Y(p)\n\nReturns the Y-coordinate value for the point p as a double-precision\nnumber.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#poi…','mysql> SET @pt = \'Point(56.7 53.34)\';\nmysql> SELECT Y(GeomFromText(@pt));\n+----------------------+\n| Y(GeomFromText(@pt)) |\n+----------------------+\n| 53.34 |\n+----------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#poi…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (399,25,'SHOW INNODB STATUS','Syntax:\nSHOW INNODB STATUS\n\nIn MySQL 5.1, this is a deprecated synonym for SHOW ENGINE INNODB\nSTATUS. See [HELP SHOW ENGINE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (399,25,'SHOW INNODB STATUS','Syntax:\nSHOW INNODB STATUS\n\nIn MySQL 5.1, this is a deprecated synonym for SHOW ENGINE INNODB\nSTATUS. See [HELP SHOW ENGINE]. SHOW INNODB STATUS is removed in MySQL\n5.5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (400,19,'CHECKSUM TABLE','Syntax:\nCHECKSUM TABLE tbl_name [, tbl_name] ... [ QUICK | EXTENDED ]\n\nCHECKSUM TABLE reports a table checksum.\n\nWith QUICK, the live table checksum is reported if it is available, or\nNULL otherwise. This is very fast. A live checksum is enabled by\nspecifying the CHECKSUM=1 table option when you create the table;\ncurrently, this is supported only for MyISAM tables. See [HELP CREATE\nTABLE].\n\nWith EXTENDED, the entire table is read row by row and the checksum is\ncalculated. This can be very slow for large tables.\n\nIf neither QUICK nor EXTENDED is specified, MySQL returns a live\nchecksum if the table storage engine supports it and scans the table\notherwise.\n\nFor a nonexistent table, CHECKSUM TABLE returns NULL and generates a\nwarning.\n\nThe checksum value depends on the table row format. If the row format\nchanges, the checksum also changes. For example, the storage format for\nVARCHAR changed between MySQL 4.1 and 5.0, so if a 4.1 table is\nupgraded to MySQL 5.0, the checksum value may change.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/checksum-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/checksum-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (401,2,'NUMINTERIORRINGS','NumInteriorRings(poly)\n\nReturns the number of interior rings in the Polygon value poly.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…','mysql> SET @poly =\n -> \'Polygon((0 0,0 3,3 3,3 0,0 0),(1 1,1 2,2 2,2 1,1 1))\';\nmysql> SELECT NumInteriorRings(GeomFromText(@poly));\n+---------------------------------------+\n| NumInteriorRings(GeomFromText(@poly)) |\n+---------------------------------------+\n| 1 |\n+---------------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (402,2,'INTERIORRINGN','InteriorRingN(poly,N)\n\nReturns the N-th interior ring for the Polygon value poly as a\nLineString. Rings are numbered beginning with 1.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…','mysql> SET @poly =\n -> \'Polygon((0 0,0 3,3 3,3 0,0 0),(1 1,1 2,2 2,2 1,1 1))\';\nmysql> SELECT AsText(InteriorRingN(GeomFromText(@poly),1));\n+----------------------------------------------+\n| AsText(InteriorRingN(GeomFromText(@poly),1)) |\n+----------------------------------------------+\n| LINESTRING(1 1,1 2,2 2,2 1,1 1) |\n+----------------------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…'); @@ -481,12 +481,12 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (412,35,'RIGHT','Syntax:\nRIGHT(str,len)\n\nReturns the rightmost len characters from the string str, or NULL if\nany argument is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT RIGHT(\'foobarbar\', 4);\n -> \'rbar\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (413,30,'DATEDIFF','Syntax:\nDATEDIFF(expr1,expr2)\n\nDATEDIFF() returns expr1 - expr2 expressed as a value in days from one\ndate to the other. expr1 and expr2 are date or date-and-time\nexpressions. Only the date parts of the values are used in the\ncalculation.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DATEDIFF(\'2007-12-31 23:59:59\',\'2007-12-30\');\n -> 1\nmysql> SELECT DATEDIFF(\'2010-11-30 23:59:59\',\'2010-12-31\');\n -> -31\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (414,37,'DROP TABLESPACE','Syntax:\nDROP TABLESPACE tablespace_name\n ENGINE [=] engine_name\n\nThis statement drops a tablespace that was previously created using\nCREATE TABLESPACE (see [HELP CREATE TABLESPACE]).\n\n*Important*: The tablespace to be dropped must not contain any data\nfiles; in other words, before you can drop a tablespace, you must first\ndrop each of its data files using ALTER TABLESPACE ... DROP DATAFILE\n(see [HELP ALTER TABLESPACE]).\n\nThe ENGINE clause (required) specifies the storage engine used by the\ntablespace. In MySQL 5.1, the only accepted values for engine_name are\nNDB and NDBCLUSTER.\n\nDROP TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/drop-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-tablespace.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (415,37,'DROP PROCEDURE','Syntax:\nDROP {PROCEDURE | FUNCTION} [IF EXISTS] sp_name\n\nThis statement is used to drop a stored procedure or function. That is,\nthe specified routine is removed from the server. You must have the\nALTER ROUTINE privilege for the routine. (That privilege is granted\nautomatically to the routine creator.)\n\nThe IF EXISTS clause is a MySQL extension. It prevents an error from\noccurring if the procedure or function does not exist. A warning is\nproduced that can be viewed with SHOW WARNINGS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (415,37,'DROP PROCEDURE','Syntax:\nDROP {PROCEDURE | FUNCTION} [IF EXISTS] sp_name\n\nThis statement is used to drop a stored procedure or function. That is,\nthe specified routine is removed from the server. You must have the\nALTER ROUTINE privilege for the routine. (If the\nautomatic_sp_privileges system variable is enabled, that privilege and\nEXECUTE are granted automatically to the routine creator when the\nroutine is created and dropped from the creator when the routine is\ndropped. See\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-routines-privileges.html… IF EXISTS clause is a MySQL extension. It prevents an error from\noccurring if the procedure or function does not exist. A warning is\nproduced that can be viewed with SHOW WARNINGS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (416,19,'CHECK TABLE','Syntax:\nCHECK TABLE tbl_name [, tbl_name] ... [option] ...\n\noption = {FOR UPGRADE | QUICK | FAST | MEDIUM | EXTENDED | CHANGED}\n\nCHECK TABLE checks a table or tables for errors. CHECK TABLE works for\nMyISAM, InnoDB, and ARCHIVE tables. Starting with MySQL 5.1.9, CHECK\nTABLE is also valid for CSV tables, see\nhttp://dev.mysql.com/doc/refman/5.1/en/csv-storage-engine.html. For\nMyISAM tables, the key statistics are updated as well.\n\nCHECK TABLE can also check views for problems, such as tables that are\nreferenced in the view definition that no longer exist.\n\nBeginning with MySQL 5.1.27, CHECK TABLE is also supported for\npartitioned tables. Also beginning with MySQL 5.1.27, you can use ALTER\nTABLE ... CHECK PARTITION to check one or more partitions; for more\ninformation, see [HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/check-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/check-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (417,35,'BIN','Syntax:\nBIN(N)\n\nReturns a string representation of the binary value of N, where N is a\nlonglong (BIGINT) number. This is equivalent to CONV(N,10,2). Returns\nNULL if N is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT BIN(12);\n -> \'1100\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (418,5,'INSTALL PLUGIN','Syntax:\nINSTALL PLUGIN plugin_name SONAME \'plugin_library\'\n\nThis statement installs a plugin.\n\nplugin_name is the name of the plugin as defined in the plugin\ndeclaration structure contained in the library file. Plugin names are\nnot case sensitive. For maximal compatibility, plugin names should be\nlimited to ASCII letters, digits, and underscore, because they are used\nin C source files, shell command lines, M4 and Bourne shell scripts,\nand SQL environments.\n\nplugin_library is the name of the shared library that contains the\nplugin code. The name includes the file name extension (for example,\nlibmyplugin.so or libmyplugin.dylib).\n\nThe shared library must be located in the plugin directory (that is,\nthe directory named by the plugin_dir system variable). The library\nmust be in the plugin directory itself, not in a subdirectory. By\ndefault, plugin_dir is plugin directory under the directory named by\nthe pkglibdir configuration variable, but it can be changed by setting\nthe value of plugin_dir at server startup. For example, set its value\nin a my.cnf file:\n\n[mysqld]\nplugin_dir=/path/to/plugin/directory\n\nIf the value of plugin_dir is a relative path name, it is taken to be\nrelative to the MySQL base directory (the value of the basedir system\nvariable).\n\nINSTALL PLUGIN adds a line to the mysql.plugin table that describes the\nplugin. This table contains the plugin name and library file name.\n\nAs of MySQL 5.1.33, INSTALL PLUGIN causes the server to read option\n(my.cnf) files just as during server startup. This enables the plugin\nto pick up any relevant options from those files. It is possible to add\nplugin options to an option file even before loading a plugin (if the\nloose prefix is used). It is also possible to uninstall a plugin, edit\nmy.cnf, and install the plugin again. Restarting the plugin this way\nenables it to the new option values without a server restart.\n\nBefore MySQL 5.1.33, a plugin is started with each option set to its\ndefault value.\n\nINSTALL PLUGIN also loads and initializes the plugin code to make the\nplugin available for use. A plugin is initialized by executing its\ninitialization function, which handles any setup that the plugin must\nperform before it can be used.\n\nTo use INSTALL PLUGIN, you must have the INSERT privilege for the\nmysql.plugin table.\n\nAt server startup, the server loads and initializes any plugin that is\nlisted in the mysql.plugin table. This means that a plugin is installed\nwith INSTALL PLUGIN only once, not every time the server starts. Plugin\nloading at startup does not occur if the server is started with the\n--skip-grant-tables option.\n\nWhen the server shuts down, it executes the deinitialization function\nfor each plugin that is loaded so that the plugin has a change to\nperform any final cleanup.\n\nFor options that control individual plugin loading at server startup,\nsee http://dev.mysql.com/doc/refman/5.1/en/server-plugin-options.html.\nIf you need to load plugins for a single server startup when the\n--skip-grant-tables option is given (which tells the server not to read\nsystem tables), use the --plugin-load option. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-options.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (419,22,'DECLARE CURSOR','Syntax:\nDECLARE cursor_name CURSOR FOR select_statement\n\nThis statement declares a cursor. Multiple cursors may be declared in a\nstored program, but each cursor in a given block must have a unique\nname.\n\nThe SELECT statement cannot have an INTO clause.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (420,26,'LOAD DATA','Syntax:\nLOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE \'file_name\'\n [REPLACE | IGNORE]\n INTO TABLE tbl_name\n [CHARACTER SET charset_name]\n [{FIELDS | COLUMNS}\n [TERMINATED BY \'string\']\n [[OPTIONALLY] ENCLOSED BY \'char\']\n [ESCAPED BY \'char\']\n ]\n [LINES\n [STARTING BY \'string\']\n [TERMINATED BY \'string\']\n ]\n [IGNORE number LINES]\n [(col_name_or_user_var,...)]\n [SET col_name = expr,...]\n\nThe LOAD DATA INFILE statement reads rows from a text file into a table\nat a very high speed. The file name must be given as a literal string.\n\nLOAD DATA INFILE is the complement of SELECT ... INTO OUTFILE. (See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.) To write data from\na table to a file, use SELECT ... INTO OUTFILE. To read the file back\ninto a table, use LOAD DATA INFILE. The syntax of the FIELDS and LINES\nclauses is the same for both statements. Both clauses are optional, but\nFIELDS must precede LINES if both are specified.\n\nFor more information about the efficiency of INSERT versus LOAD DATA\nINFILE and speeding up LOAD DATA INFILE, see\nhttp://dev.mysql.com/doc/refman/5.1/en/insert-speed.html.\n\nThe character set indicated by the character_set_database system\nvariable is used to interpret the information in the file. SET NAMES\nand the setting of character_set_client do not affect interpretation of\ninput. If the contents of the input file use a character set that\ndiffers from the default, it is usually preferable to specify the\ncharacter set of the file by using the CHARACTER SET clause, which is\navailable as of MySQL 5.1.17. A character set of binary specifies "no\nconversion."\n\nLOAD DATA INFILE interprets all fields in the file as having the same\ncharacter set, regardless of the data types of the columns into which\nfield values are loaded. For proper interpretation of file contents,\nyou must ensure that it was written with the correct character set. For\nexample, if you write a data file with mysqldump -T or by issuing a\nSELECT ... INTO OUTFILE statement in mysql, be sure to use a\n--default-character-set option with mysqldump or mysql so that output\nis written in the character set to be used when the file is loaded with\nLOAD DATA INFILE.\n\nNote that it is currently not possible to load data files that use the\nucs2, utf16, or utf32 character set.\n\nAs of MySQL 5.1.6, the character_set_filesystem system variable\ncontrols the interpretation of the file name.\n\nYou can also load data files by using the mysqlimport utility; it\noperates by sending a LOAD DATA INFILE statement to the server. The\n--local option causes mysqlimport to read data files from the client\nhost. You can specify the --compress option to get better performance\nover slow networks if the client and server support the compressed\nprotocol. See http://dev.mysql.com/doc/refman/5.1/en/mysqlimport.html.\n\nIf you use LOW_PRIORITY, execution of the LOAD DATA statement is\ndelayed until no other clients are reading from the table. This affects\nonly storage engines that use only table-level locking (MyISAM, MEMORY,\nMERGE).\n\nIf you specify CONCURRENT with a MyISAM table that satisfies the\ncondition for concurrent inserts (that is, it contains no free blocks\nin the middle), other threads can retrieve data from the table while\nLOAD DATA is executing. Using this option affects the performance of\nLOAD DATA a bit, even if no other thread is using the table at the same\ntime.\n\nCONCURRENT is not replicated when using statement-based replication;\nhowever, it is replicated when using row-based replication. See\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-features-load-data.…, for more information.\n\n*Note*: Prior to MySQL 5.1.23, LOAD DATA performed very poorly when\nimporting into partitioned tables. The statement now uses buffering to\nimprove performance; however, the buffer uses 130 KB memory per\npartition to achieve this. (Bug#26527 (http://bugs.mysql.com/26527))\n\nThe LOCAL keyword, if specified, is interpreted with respect to the\nclient end of the connection:\n\no If LOCAL is specified, the file is read by the client program on the\n client host and sent to the server. The file can be given as a full\n path name to specify its exact location. If given as a relative path\n name, the name is interpreted relative to the directory in which the\n client program was started.\n\no If LOCAL is not specified, the file must be located on the server\n host and is read directly by the server. The server uses the\n following rules to locate the file:\n\n o If the file name is an absolute path name, the server uses it as\n given.\n\n o If the file name is a relative path name with one or more leading\n components, the server searches for the file relative to the\n server\'s data directory.\n\n o If a file name with no leading components is given, the server\n looks for the file in the database directory of the default\n database.\n\nNote that, in the non-LOCAL case, these rules mean that a file named as\n./myfile.txt is read from the server\'s data directory, whereas the file\nnamed as myfile.txt is read from the database directory of the default\ndatabase. For example, if db1 is the default database, the following\nLOAD DATA statement reads the file data.txt from the database directory\nfor db1, even though the statement explicitly loads the file into a\ntable in the db2 database:\n\nLOAD DATA INFILE \'data.txt\' INTO TABLE db2.my_table;\n\nWindows path names are specified using forward slashes rather than\nbackslashes. If you do use backslashes, you must double them.\n\n*Note*: A regression in MySQL 5.1.40 caused the database referenced in\na fully qualified table name to be ignored by LOAD DATA when using\nreplication with either STATEMENT or MIXED as the binary logging\nformat; this could lead to problems if the table was not in the current\ndatabase. As a workaround, you can specify the correct database with\nthe USE statement prior to executing LOAD DATA. If necessary, you can\nreset the current database with a second USE statement following the\nLOAD DATA statement. This issue was fixed in MySQL 5.1.41. (Bug#48297\n(http://bugs.mysql.com/48297))\n\nFor security reasons, when reading text files located on the server,\nthe files must either reside in the database directory or be readable\nby all. Also, to use LOAD DATA INFILE on server files, you must have\nthe FILE privilege. See\nhttp://dev.mysql.com/doc/refman/5.1/en/privileges-provided.html. For\nnon-LOCAL load operations, if the secure_file_priv system variable is\nset to a nonempty directory name, the file to be loaded must be located\nin that directory.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (418,5,'INSTALL PLUGIN','Syntax:\nINSTALL PLUGIN plugin_name SONAME \'shared_library_name\'\n\nThis statement installs a server plugin. It requires the INSERT\nprivilege for the mysql.plugin table.\n\nplugin_name is the name of the plugin as defined in the plugin\ndescriptor structure contained in the library file (see\nhttp://dev.mysql.com/doc/refman/5.1/en/plugin-data-structures.html).\n… names are not case sensitive. For maximal compatibility, plugin\nnames should be limited to ASCII letters, digits, and underscore\nbecause they are used in C source files, shell command lines, M4 and\nBourne shell scripts, and SQL environments.\n\nshared_library_name is the name of the shared library that contains the\nplugin code. The name includes the file name extension (for example,\nlibmyplugin.so, libmyplugin.dll, or libmyplugin.dylib).\n\nThe shared library must be located in the plugin directory (the\ndirectory named by the plugin_dir system variable). The library must be\nin the plugin directory itself, not in a subdirectory. By default,\nplugin_dir is the plugin directory under the directory named by the\npkglibdir configuration variable, but it can be changed by setting the\nvalue of plugin_dir at server startup. For example, set its value in a\nmy.cnf file:\n\n[mysqld]\nplugin_dir=/path/to/plugin/directory\n\nIf the value of plugin_dir is a relative path name, it is taken to be\nrelative to the MySQL base directory (the value of the basedir system\nvariable).\n\nINSTALL PLUGIN loads and initializes the plugin code to make the plugin\navailable for use. A plugin is initialized by executing its\ninitialization function, which handles any setup that the plugin must\nperform before it can be used. When the server shuts down, it executes\nthe deinitialization function for each plugin that is loaded so that\nthe plugin has a change to perform any final cleanup.\n\nINSTALL PLUGIN also registers the plugin by adding a line that\nindicates the plugin name and library file name to the mysql.plugin\ntable. At server startup, the server loads and initializes any plugin\nthat is listed in the mysql.plugin table. This means that a plugin is\ninstalled with INSTALL PLUGIN only once, not every time the server\nstarts. Plugin loading at startup does not occur if the server is\nstarted with the --skip-grant-tables option.\n\nA plugin library can contain multiple plugins. For each of them to be\ninstalled, use a separate INSTALL PLUGIN statement. Each statement\nnames a different plugin, but all of them specify the same library\nname.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (419,22,'DECLARE CURSOR','Syntax:\nDECLARE cursor_name CURSOR FOR select_statement\n\nThis statement declares a cursor. Multiple cursors may be declared in a\nstored program, but each cursor in a given block must have a unique\nname.\n\nThe SELECT statement cannot have an INTO clause.\n\nFor information available through SHOW statements, it is possible in\nmany cases to obtain equivalent information by using a cursor with an\nINFORMATION_SCHEMA table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (420,26,'LOAD DATA','Syntax:\nLOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE \'file_name\'\n [REPLACE | IGNORE]\n INTO TABLE tbl_name\n [CHARACTER SET charset_name]\n [{FIELDS | COLUMNS}\n [TERMINATED BY \'string\']\n [[OPTIONALLY] ENCLOSED BY \'char\']\n [ESCAPED BY \'char\']\n ]\n [LINES\n [STARTING BY \'string\']\n [TERMINATED BY \'string\']\n ]\n [IGNORE number LINES]\n [(col_name_or_user_var,...)]\n [SET col_name = expr,...]\n\nThe LOAD DATA INFILE statement reads rows from a text file into a table\nat a very high speed. The file name must be given as a literal string.\n\nLOAD DATA INFILE is the complement of SELECT ... INTO OUTFILE. (See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.) To write data from\na table to a file, use SELECT ... INTO OUTFILE. To read the file back\ninto a table, use LOAD DATA INFILE. The syntax of the FIELDS and LINES\nclauses is the same for both statements. Both clauses are optional, but\nFIELDS must precede LINES if both are specified.\n\nFor more information about the efficiency of INSERT versus LOAD DATA\nINFILE and speeding up LOAD DATA INFILE, see\nhttp://dev.mysql.com/doc/refman/5.1/en/insert-speed.html.\n\nThe character set indicated by the character_set_database system\nvariable is used to interpret the information in the file. SET NAMES\nand the setting of character_set_client do not affect interpretation of\ninput. If the contents of the input file use a character set that\ndiffers from the default, it is usually preferable to specify the\ncharacter set of the file by using the CHARACTER SET clause, which is\navailable as of MySQL 5.1.17. A character set of binary specifies "no\nconversion."\n\nLOAD DATA INFILE interprets all fields in the file as having the same\ncharacter set, regardless of the data types of the columns into which\nfield values are loaded. For proper interpretation of file contents,\nyou must ensure that it was written with the correct character set. For\nexample, if you write a data file with mysqldump -T or by issuing a\nSELECT ... INTO OUTFILE statement in mysql, be sure to use a\n--default-character-set option with mysqldump or mysql so that output\nis written in the character set to be used when the file is loaded with\nLOAD DATA INFILE.\n\nNote that it is currently not possible to load data files that use the\nucs2 character set.\n\nAs of MySQL 5.1.6, the character_set_filesystem system variable\ncontrols the interpretation of the file name.\n\nYou can also load data files by using the mysqlimport utility; it\noperates by sending a LOAD DATA INFILE statement to the server. The\n--local option causes mysqlimport to read data files from the client\nhost. You can specify the --compress option to get better performance\nover slow networks if the client and server support the compressed\nprotocol. See http://dev.mysql.com/doc/refman/5.1/en/mysqlimport.html.\n\nIf you use LOW_PRIORITY, execution of the LOAD DATA statement is\ndelayed until no other clients are reading from the table. This affects\nonly storage engines that use only table-level locking (such as MyISAM,\nMEMORY, and MERGE).\n\nIf you specify CONCURRENT with a MyISAM table that satisfies the\ncondition for concurrent inserts (that is, it contains no free blocks\nin the middle), other threads can retrieve data from the table while\nLOAD DATA is executing. Using this option affects the performance of\nLOAD DATA a bit, even if no other thread is using the table at the same\ntime.\n\nPrior to MySQL 5.1.43, CONCURRENT was not replicated when using\nstatement-based replication (see Bug#34628\n(http://bugs.mysql.com/bug.php?id=34628)) However, it is replicated\nwhen using row-based replication, regardless of the version. See\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-features-load-data.…, for more information.\n\n*Note*: Prior to MySQL 5.1.23, LOAD DATA performed very poorly when\nimporting into partitioned tables. The statement now uses buffering to\nimprove performance; however, the buffer uses 130KB memory per\npartition to achieve this. (Bug#26527\n(http://bugs.mysql.com/bug.php?id=26527))\n\nThe LOCAL keyword, if specified, is interpreted with respect to the\nclient end of the connection:\n\no If LOCAL is specified, the file is read by the client program on the\n client host and sent to the server. The file can be given as a full\n path name to specify its exact location. If given as a relative path\n name, the name is interpreted relative to the directory in which the\n client program was started.\n\no If LOCAL is not specified, the file must be located on the server\n host and is read directly by the server. The server uses the\n following rules to locate the file:\n\n o If the file name is an absolute path name, the server uses it as\n given.\n\n o If the file name is a relative path name with one or more leading\n components, the server searches for the file relative to the\n server\'s data directory.\n\n o If a file name with no leading components is given, the server\n looks for the file in the database directory of the default\n database.\n\nNote that, in the non-LOCAL case, these rules mean that a file named as\n./myfile.txt is read from the server\'s data directory, whereas the file\nnamed as myfile.txt is read from the database directory of the default\ndatabase. For example, if db1 is the default database, the following\nLOAD DATA statement reads the file data.txt from the database directory\nfor db1, even though the statement explicitly loads the file into a\ntable in the db2 database:\n\nLOAD DATA INFILE \'data.txt\' INTO TABLE db2.my_table;\n\nWindows path names are specified using forward slashes rather than\nbackslashes. If you do use backslashes, you must double them.\n\n*Note*: A regression in MySQL 5.1.40 caused the database referenced in\na fully qualified table name to be ignored by LOAD DATA when using\nreplication with either STATEMENT or MIXED as the binary logging\nformat; this could lead to problems if the table was not in the current\ndatabase. As a workaround, you can specify the correct database with\nthe USE statement prior to executing LOAD DATA. If necessary, you can\nreset the default database with a second USE statement following the\nLOAD DATA statement. This issue was fixed in MySQL 5.1.41. (Bug#48297\n(http://bugs.mysql.com/bug.php?id=48297))\n\nFor security reasons, when reading text files located on the server,\nthe files must either reside in the database directory or be readable\nby all. Also, to use LOAD DATA INFILE on server files, you must have\nthe FILE privilege. See\nhttp://dev.mysql.com/doc/refman/5.1/en/privileges-provided.html. For\nnon-LOCAL load operations, if the secure_file_priv system variable is\nset to a nonempty directory name, the file to be loaded must be located\nin that directory.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (421,23,'MULTILINESTRING','MultiLineString(ls1,ls2,...)\n\nConstructs a MultiLineString value using LineString or WKB LineString\narguments.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (422,30,'LOCALTIME','Syntax:\nLOCALTIME, LOCALTIME()\n\nLOCALTIME and LOCALTIME() are synonyms for NOW().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (423,3,'MPOINTFROMTEXT','MPointFromText(wkt[,srid]), MultiPointFromText(wkt[,srid])\n\nConstructs a MULTIPOINT value using its WKT representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); @@ -525,7 +525,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (456,20,'TINYBLOB','TINYBLOB\n\nA BLOB column with a maximum length of 255 (28 - 1) bytes. Each\nTINYBLOB value is stored using a one-byte length prefix that indicates\nthe number of bytes in the value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (457,8,'SAVEPOINT','Syntax:\nSAVEPOINT identifier\nROLLBACK [WORK] TO [SAVEPOINT] identifier\nRELEASE SAVEPOINT identifier\n\nInnoDB supports the SQL statements SAVEPOINT, ROLLBACK TO SAVEPOINT,\nRELEASE SAVEPOINT and the optional WORK keyword for ROLLBACK.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/savepoint.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/savepoint.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (458,15,'USER','Syntax:\nUSER()\n\nReturns the current MySQL user name and host name as a string in the\nutf8 character set.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT USER();\n -> \'davida@localhost\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (459,37,'ALTER TABLE','Syntax:\nALTER [ONLINE | OFFLINE] [IGNORE] TABLE tbl_name\n alter_specification [, alter_specification] ...\n\nalter_specification:\n table_options\n | ADD [COLUMN] col_name column_definition\n [FIRST | AFTER col_name ]\n | ADD [COLUMN] (col_name column_definition,...)\n | ADD {INDEX|KEY} [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]] PRIMARY KEY\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n UNIQUE [INDEX|KEY] [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD FULLTEXT [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD SPATIAL [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n FOREIGN KEY [index_name] (index_col_name,...)\n reference_definition\n | ALTER [COLUMN] col_name {SET DEFAULT literal | DROP DEFAULT}\n | CHANGE [COLUMN] old_col_name new_col_name column_definition\n [FIRST|AFTER col_name]\n | MODIFY [COLUMN] col_name column_definition\n [FIRST | AFTER col_name]\n | DROP [COLUMN] col_name\n | DROP PRIMARY KEY\n | DROP {INDEX|KEY} index_name\n | DROP FOREIGN KEY fk_symbol\n | DISABLE KEYS\n | ENABLE KEYS\n | RENAME [TO] new_tbl_name\n | ORDER BY col_name [, col_name] ...\n | CONVERT TO CHARACTER SET charset_name [COLLATE collation_name]\n | [DEFAULT] CHARACTER SET [=] charset_name [COLLATE [=] collation_name]\n | DISCARD TABLESPACE\n | IMPORT TABLESPACE\n | partition_options\n | ADD PARTITION (partition_definition)\n | DROP PARTITION partition_names\n | COALESCE PARTITION number\n | REORGANIZE PARTITION [partition_names INTO (partition_definitions)]\n | ANALYZE PARTITION partition_names\n | CHECK PARTITION partition_names\n | OPTIMIZE PARTITION partition_names\n | REBUILD PARTITION partition_names\n | REPAIR PARTITION partition_names\n | REMOVE PARTITIONING\n\nindex_col_name:\n col_name [(length)] [ASC | DESC]\n\nindex_type:\n USING {BTREE | HASH | RTREE}\n\nindex_option:\n KEY_BLOCK_SIZE [=] value\n | index_type\n | WITH PARSER parser_name\n | COMMENT \'string\'\n\ntable_options:\n table_option [[,] table_option] ...\n\nALTER TABLE enables you to change the structure of an existing table.\nFor example, you can add or delete columns, create or destroy indexes,\nchange the type of existing columns, or rename columns or the table\nitself. You can also change the comment for the table and type of the\ntable.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-table.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (459,37,'ALTER TABLE','Syntax:\nALTER [ONLINE | OFFLINE] [IGNORE] TABLE tbl_name\n {table_options | partitioning_specification}\n\ntable_options:\n table_option [, table_option] ...\n\ntable_option:\n ADD [COLUMN] col_name column_definition\n [FIRST | AFTER col_name ]\n | ADD [COLUMN] (col_name column_definition,...)\n | ADD {INDEX|KEY} [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]] PRIMARY KEY\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n UNIQUE [INDEX|KEY] [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD FULLTEXT [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD SPATIAL [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n FOREIGN KEY [index_name] (index_col_name,...)\n reference_definition\n | ALTER [COLUMN] col_name {SET DEFAULT literal | DROP DEFAULT}\n | CHANGE [COLUMN] old_col_name new_col_name column_definition\n [FIRST|AFTER col_name]\n | MODIFY [COLUMN] col_name column_definition\n [FIRST | AFTER col_name]\n | DROP [COLUMN] col_name\n | DROP PRIMARY KEY\n | DROP {INDEX|KEY} index_name\n | DROP FOREIGN KEY fk_symbol\n | DISABLE KEYS\n | ENABLE KEYS\n | RENAME [TO] new_tbl_name\n | ORDER BY col_name [, col_name] ...\n | CONVERT TO CHARACTER SET charset_name [COLLATE collation_name]\n | [DEFAULT] CHARACTER SET [=] charset_name [COLLATE [=] collation_name]\n | DISCARD TABLESPACE\n | IMPORT TABLESPACE\n\npartitioning_specification:\n ADD PARTITION (partition_definition)\n | DROP PARTITION partition_names\n | COALESCE PARTITION number\n | REORGANIZE PARTITION [partition_names INTO (partition_definitions)]\n | ANALYZE PARTITION {partition_names | ALL }\n | CHECK PARTITION {partition_names | ALL }\n | OPTIMIZE PARTITION {partition_names | ALL }\n | REBUILD PARTITION {partition_names | ALL }\n | REPAIR PARTITION {partition_names | ALL }\n | PARTITION BY partitioning_expression\n | REMOVE PARTITIONING\n\nindex_col_name:\n col_name [(length)] [ASC | DESC]\n\nindex_type:\n USING {BTREE | HASH | RTREE}\n\nindex_option:\n KEY_BLOCK_SIZE [=] value\n | index_type\n | WITH PARSER parser_name\n\ntable_options:\n table_option [[,] table_option] ... (see CREATE TABLE options)\n\nALTER TABLE enables you to change the structure of an existing table.\nFor example, you can add or delete columns, create or destroy indexes,\nchange the type of existing columns, or rename columns or the table\nitself. You can also change the comment for the table and type of the\ntable.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (460,31,'MPOINTFROMWKB','MPointFromWKB(wkb[,srid]), MultiPointFromWKB(wkb[,srid])\n\nConstructs a MULTIPOINT value using its WKB representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (461,20,'CHAR BYTE','The CHAR BYTE data type is an alias for the BINARY data type. This is a\ncompatibility feature.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (462,19,'REPAIR TABLE','Syntax:\nREPAIR [NO_WRITE_TO_BINLOG | LOCAL] TABLE\n tbl_name [, tbl_name] ...\n [QUICK] [EXTENDED] [USE_FRM]\n\nREPAIR TABLE repairs a possibly corrupted table. By default, it has the\nsame effect as myisamchk --recover tbl_name. REPAIR TABLE works for\nMyISAM and for ARCHIVE tables. Starting with MySQL 5.1.9, REPAIR is\nalso valid for CSV tables. See\nhttp://dev.mysql.com/doc/refman/5.1/en/myisam-storage-engine.html, and\nhttp://dev.mysql.com/doc/refman/5.1/en/archive-storage-engine.html, and\nhttp://dev.mysql.com/doc/refman/5.1/en/csv-storage-engine.html\n\nThis statement requires SELECT and INSERT privileges for the table.\n\nBeginning with MySQL 5.1.27, REPAIR TABLE is also supported for\npartitioned tables. However, the USE_FRM option cannot be used with\nthis statement on a partitioned table.\n\nAlso beginning with MySQL 5.1.27, you can use ALTER TABLE ... REPAIR\nPARTITION to repair one or more partitions; for more information, see\n[HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/repair-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/repair-table.html'); @@ -535,12 +535,12 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (466,19,'ANALYZE TABLE','Syntax:\nANALYZE [NO_WRITE_TO_BINLOG | LOCAL] TABLE\n tbl_name [, tbl_name] ...\n\nANALYZE TABLE analyzes and stores the key distribution for a table.\nDuring the analysis, the table is locked with a read lock for MyISAM.\nFor InnoDB the table is locked with a write lock. This statement works\nwith MyISAM and InnoDB tables. For MyISAM tables, this statement is\nequivalent to using myisamchk --analyze.\n\nFor more information on how the analysis works within InnoDB, see\nhttp://dev.mysql.com/doc/refman/5.1/en/innodb-restrictions.html.\n\nMy… uses the stored key distribution to decide the order in which\ntables should be joined when you perform a join on something other than\na constant. In addition, key distributions can be used when deciding\nwhich indexes to use for a specific table within a query.\n\nThis statement requires SELECT and INSERT privileges for the table.\n\nBeginning with MySQL 5.1.27, ANALYZE TABLE is also supported for\npartitioned tables. Also beginning with MySQL 5.1.27, you can use ALTER\nTABLE ... ANALYZE PARTITION to analyze one or more partitions; for more\ninformation, see [HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/analyze-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/analyze-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (467,30,'MICROSECOND','Syntax:\nMICROSECOND(expr)\n\nReturns the microseconds from the time or datetime expression expr as a\nnumber in the range from 0 to 999999.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MICROSECOND(\'12:00:00.123456\');\n -> 123456\nmysql> SELECT MICROSECOND(\'2009-12-31 23:59:59.000010\');\n -> 10\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (468,37,'CONSTRAINT','InnoDB supports foreign key constraints. The syntax for a foreign key\nconstraint definition in InnoDB looks like this:\n\n[CONSTRAINT [symbol]] FOREIGN KEY\n [index_name] (index_col_name, ...)\n REFERENCES tbl_name (index_col_name,...)\n [ON DELETE reference_option]\n [ON UPDATE reference_option]\n\nreference_option:\n RESTRICT | CASCADE | SET NULL | NO ACTION\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/innodb-foreign-key-constraints.html\…','CREATE TABLE product (category INT NOT NULL, id INT NOT NULL,\n price DECIMAL,\n PRIMARY KEY(category, id)) ENGINE=INNODB;\nCREATE TABLE customer (id INT NOT NULL,\n PRIMARY KEY (id)) ENGINE=INNODB;\nCREATE TABLE product_order (no INT NOT NULL AUTO_INCREMENT,\n product_category INT NOT NULL,\n product_id INT NOT NULL,\n customer_id INT NOT NULL,\n PRIMARY KEY(no),\n INDEX (product_category, product_id),\n FOREIGN KEY (product_category, product_id)\n REFERENCES product(category, id)\n ON UPDATE CASCADE ON DELETE RESTRICT,\n INDEX (customer_id),\n FOREIGN KEY (customer_id)\n REFERENCES customer(id)) ENGINE=INNODB;\n','http://dev.mysql.com/doc/refman/5.1/en/innodb-foreign-key-constraints.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (469,37,'CREATE SERVER','Syntax:\nCREATE SERVER server_name\n FOREIGN DATA WRAPPER wrapper_name\n OPTIONS (option [, option] ...)\n\noption:\n { HOST character-literal\n | DATABASE character-literal\n | USER character-literal\n | PASSWORD character-literal\n | SOCKET character-literal\n | OWNER character-literal\n | PORT numeric-literal }\n\nThis statement creates the definition of a server for use with the\nFEDERATED storage engine. The CREATE SERVER statement creates a new row\nwithin the servers table within the mysql database. This statement\nrequires the SUPER privilege.\n\nThe server_name should be a unique reference to the server. Server\ndefinitions are global within the scope of the server, it is not\npossible to qualify the server definition to a specific database.\nserver_name has a maximum length of 64 characters (names longer than 64\ncharacters are silently truncated), and is case insensitive. You may\nspecify the name as a quoted string.\n\nThe wrapper_name should be mysql, and may be quoted with single quotes.\nOther values for wrapper_name are not currently supported.\n\nFor each option you must specify either a character literal or numeric\nliteral. Character literals are UTF-8, support a maximum length of 64\ncharacters and default to a blank (empty) string. String literals are\nsilently truncated to 64 characters. Numeric literals must be a number\nbetween 0 and 9999, default value is 0.\n\n*Note*: Note that the OWNER option is currently not applied, and has no\neffect on the ownership or operation of the server connection that is\ncreated.\n\nThe CREATE SERVER statement creates an entry in the mysql.server table\nthat can later be used with the CREATE TABLE statement when creating a\nFEDERATED table. The options that you specify will be used to populate\nthe columns in the mysql.server table. The table columns are\nServer_name, Host, Db, Username, Password, Port and Socket.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-server.html\n\n','CREATE SERVER s\nFOREIGN DATA WRAPPER mysql\nOPTIONS (USER \'Remote\', HOST \'192.168.1.106\', DATABASE \'test\');\n','http://dev.mysql.com/doc/refman/5.1/en/create-server.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (469,37,'CREATE SERVER','Syntax:\nCREATE SERVER server_name\n FOREIGN DATA WRAPPER wrapper_name\n OPTIONS (option [, option] ...)\n\noption:\n { HOST character-literal\n | DATABASE character-literal\n | USER character-literal\n | PASSWORD character-literal\n | SOCKET character-literal\n | OWNER character-literal\n | PORT numeric-literal }\n\nThis statement creates the definition of a server for use with the\nFEDERATED storage engine. The CREATE SERVER statement creates a new row\nwithin the servers table within the mysql database. This statement\nrequires the SUPER privilege.\n\nThe server_name should be a unique reference to the server. Server\ndefinitions are global within the scope of the server, it is not\npossible to qualify the server definition to a specific database.\nserver_name has a maximum length of 64 characters (names longer than 64\ncharacters are silently truncated), and is case insensitive. You may\nspecify the name as a quoted string.\n\nThe wrapper_name should be mysql, and may be quoted with single quotes.\nOther values for wrapper_name are not currently supported.\n\nFor each option you must specify either a character literal or numeric\nliteral. Character literals are UTF-8, support a maximum length of 64\ncharacters and default to a blank (empty) string. String literals are\nsilently truncated to 64 characters. Numeric literals must be a number\nbetween 0 and 9999, default value is 0.\n\n*Note*: Note that the OWNER option is currently not applied, and has no\neffect on the ownership or operation of the server connection that is\ncreated.\n\nThe CREATE SERVER statement creates an entry in the mysql.servers table\nthat can later be used with the CREATE TABLE statement when creating a\nFEDERATED table. The options that you specify will be used to populate\nthe columns in the mysql.servers table. The table columns are\nServer_name, Host, Db, Username, Password, Port and Socket.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-server.html\n\n','CREATE SERVER s\nFOREIGN DATA WRAPPER mysql\nOPTIONS (USER \'Remote\', HOST \'192.168.1.106\', DATABASE \'test\');\n','http://dev.mysql.com/doc/refman/5.1/en/create-server.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (470,35,'FIELD','Syntax:\nFIELD(str,str1,str2,str3,...)\n\nReturns the index (position) of str in the str1, str2, str3, ... list.\nReturns 0 if str is not found.\n\nIf all arguments to FIELD() are strings, all arguments are compared as\nstrings. If all arguments are numbers, they are compared as numbers.\nOtherwise, the arguments are compared as double.\n\nIf str is NULL, the return value is 0 because NULL fails equality\ncomparison with any value. FIELD() is the complement of ELT().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT FIELD(\'ej\', \'Hej\', \'ej\', \'Heja\', \'hej\', \'foo\');\n -> 2\nmysql> SELECT FIELD(\'fo\', \'Hej\', \'ej\', \'Heja\', \'hej\', \'foo\');\n -> 0\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (471,30,'MAKETIME','Syntax:\nMAKETIME(hour,minute,second)\n\nReturns a time value calculated from the hour, minute, and second\narguments.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MAKETIME(12,15,30);\n -> \'12:15:30\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (472,30,'CURDATE','Syntax:\nCURDATE()\n\nReturns the current date as a value in \'YYYY-MM-DD\' or YYYYMMDD format,\ndepending on whether the function is used in a string or numeric\ncontext.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT CURDATE();\n -> \'2008-06-13\'\nmysql> SELECT CURDATE() + 0;\n -> 20080613\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (473,9,'SET PASSWORD','Syntax:\nSET PASSWORD [FOR user] =\n {\n PASSWORD(\'some password\')\n | OLD_PASSWORD(\'some password\')\n | \'encrypted password\'\n }\n\nThe SET PASSWORD statement assigns a password to an existing MySQL user\naccount.\n\nIf the password is specified using the PASSWORD() or OLD_PASSWORD()\nfunction, the literal text of the password should be given. If the\npassword is specified without using either function, the password\nshould be the already-encrypted password value as returned by\nPASSWORD().\n\nWith no FOR clause, this statement sets the password for the current\nuser. Any client that has connected to the server using a nonanonymous\naccount can change the password for that account.\n\nWith a FOR clause, this statement sets the password for a specific\naccount on the current server host. Only clients that have the UPDATE\nprivilege for the mysql database can do this. The user value should be\ngiven in user_name@host_name format, where user_name and host_name are\nexactly as they are listed in the User and Host columns of the\nmysql.user table entry. For example, if you had an entry with User and\nHost column values of \'bob\' and \'%.loc.gov\', you would write the\nstatement like this:\n\nSET PASSWORD FOR \'bob\'@\'%.loc.gov\' = PASSWORD(\'newpass\');\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-password.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-password.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (474,37,'ALTER TABLESPACE','Syntax:\nALTER TABLESPACE tablespace_name\n {ADD|DROP} DATAFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement can be used either to add a new data file, or to drop a\ndata file from a tablespace.\n\nThe ADD DATAFILE variant allows you to specify an initial size using an\nINITIAL_SIZE clause, where size is measured in bytes; the default value\nis 128M (128 megabytes). You may optionally follow this integer value\nwith a one-letter abbreviation for an order of magnitude, similar to\nthose used in my.cnf. Generally, this is one of the letters M (for\nmegabytes) or G (for gigabytes).\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an data file with the same name, or an undo log\nfile and a with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/31770))\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/29186))\n\nOnce a data file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using additional ALTER\nTABLESPACE ... ADD DATAFILE statements.\n\nUsing DROP DATAFILE with ALTER TABLESPACE drops the data file\n\'file_name\' from the tablespace. This file must already have been added\nto the tablespace using CREATE TABLESPACE or ALTER TABLESPACE;\notherwise an error will result.\n\nBoth ALTER TABLESPACE ... ADD DATAFILE and ALTER TABLESPACE ... DROP\nDATAFILE require an ENGINE clause which specifies the storage engine\nused by the tablespace. In MySQL 5.1, the only accepted values for\nengine_name are NDB and NDBCLUSTER.\n\nWAIT is parsed but otherwise ignored, and so has no effect in MySQL\n5.1. It is intended for future expansion.\n\nWhen ALTER TABLESPACE ... ADD DATAFILE is used with ENGINE = NDB, a\ndata file is created on each Cluster data node. You can verify that the\ndata files were created and obtain information about them by querying\nthe INFORMATION_SCHEMA.FILES table. For example, the following query\nshows all data files belonging to the tablespace named newts:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+--------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+--------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=4 |\n+--------------------+--------------+----------------+\n2 rows in set (0.03 sec)\n\nSee http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nALTER TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (474,37,'ALTER TABLESPACE','Syntax:\nALTER TABLESPACE tablespace_name\n {ADD|DROP} DATAFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement can be used either to add a new data file, or to drop a\ndata file from a tablespace.\n\nThe ADD DATAFILE variant allows you to specify an initial size using an\nINITIAL_SIZE clause, where size is measured in bytes; the default value\nis 128M (128 megabytes). You may optionally follow this integer value\nwith a one-letter abbreviation for an order of magnitude, similar to\nthose used in my.cnf. Generally, this is one of the letters M (for\nmegabytes) or G (for gigabytes).\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an data file with the same name, or an undo log\nfile and a with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/bug.php?id=31770))\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/bug.php?id=29186))\n\nINITIAL_SIZE is rounded as for CREATE TABLESPACE. Beginning with MySQL\nCluster NDB 6.2.19, MySQL Cluster NDB 6.3.32, MySQL Cluster NDB 7.0.13,\nand MySQL Cluster NDB 7.1.2, this rounding is done explicitly (also as\nwith CREATE TABLESPACE).\n\nOnce a data file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using additional ALTER\nTABLESPACE ... ADD DATAFILE statements.\n\nUsing DROP DATAFILE with ALTER TABLESPACE drops the data file\n\'file_name\' from the tablespace. This file must already have been added\nto the tablespace using CREATE TABLESPACE or ALTER TABLESPACE;\notherwise an error will result.\n\nBoth ALTER TABLESPACE ... ADD DATAFILE and ALTER TABLESPACE ... DROP\nDATAFILE require an ENGINE clause which specifies the storage engine\nused by the tablespace. In MySQL 5.1, the only accepted values for\nengine_name are NDB and NDBCLUSTER.\n\nWAIT is parsed but otherwise ignored, and so has no effect in MySQL\n5.1. It is intended for future expansion.\n\nWhen ALTER TABLESPACE ... ADD DATAFILE is used with ENGINE = NDB, a\ndata file is created on each Cluster data node. You can verify that the\ndata files were created and obtain information about them by querying\nthe INFORMATION_SCHEMA.FILES table. For example, the following query\nshows all data files belonging to the tablespace named newts:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+--------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+--------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=4 |\n+--------------------+--------------+----------------+\n2 rows in set (0.03 sec)\n\nSee http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nALTER TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (475,20,'ENUM','ENUM(\'value1\',\'value2\',...) [CHARACTER SET charset_name] [COLLATE\ncollation_name]\n\nAn enumeration. A string object that can have only one value, chosen\nfrom the list of values \'value1\', \'value2\', ..., NULL or the special \'\'\nerror value. An ENUM column can have a maximum of 65,535 distinct\nvalues. ENUM values are represented internally as integers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (476,7,'IF FUNCTION','Syntax:\nIF(expr1,expr2,expr3)\n\nIf expr1 is TRUE (expr1 <> 0 and expr1 <> NULL) then IF() returns\nexpr2; otherwise it returns expr3. IF() returns a numeric or string\nvalue, depending on the context in which it is used.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html\n\n','mysql> SELECT IF(1>2,2,3);\n -> 3\nmysql> SELECT IF(1<2,\'yes\',\'no\');\n -> \'yes\'\nmysql> SELECT IF(STRCMP(\'test\',\'test1\'),\'no\',\'yes\');\n -> \'no\'\n','http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (477,15,'DATABASE','Syntax:\nDATABASE()\n\nReturns the default (current) database name as a string in the utf8\ncharacter set. If there is no default database, DATABASE() returns\nNULL. Within a stored routine, the default database is the database\nthat the routine is associated with, which is not necessarily the same\nas the database that is the default in the calling context.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT DATABASE();\n -> \'test\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); @@ -556,9 +556,9 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (487,22,'RETURN','Syntax:\nRETURN expr\n\nThe RETURN statement terminates execution of a stored function and\nreturns the value expr to the function caller. There must be at least\none RETURN statement in a stored function. There may be more than one\nif the function has multiple exit points.\n\nThis statement is not used in stored procedures, triggers, or events.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/return.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/return.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (488,25,'SHOW COLLATION','Syntax:\nSHOW COLLATION\n [LIKE \'pattern\' | WHERE expr]\n\nThis statement lists collations supported by the server. By default,\nthe output from SHOW COLLATION includes all available collations. The\nLIKE clause, if present, indicates which collation names to match. The\nWHERE clause can be given to select rows using more general conditions,\nas discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html. For example:\n\nmysql> SHOW COLLATION LIKE \'latin1%\';\n+-------------------+---------+----+---------+----------+---------+\n| Collation | Charset | Id | Default | Compiled | Sortlen |\n+-------------------+---------+----+---------+----------+---------+\n| latin1_german1_ci | latin1 | 5 | | | 0 |\n| latin1_swedish_ci | latin1 | 8 | Yes | Yes | 0 |\n| latin1_danish_ci | latin1 | 15 | | | 0 |\n| latin1_german2_ci | latin1 | 31 | | Yes | 2 |\n| latin1_bin | latin1 | 47 | | Yes | 0 |\n| latin1_general_ci | latin1 | 48 | | | 0 |\n| latin1_general_cs | latin1 | 49 | | | 0 |\n| latin1_spanish_ci | latin1 | 94 | | | 0 |\n+-------------------+---------+----+---------+----------+---------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-collation.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-collation.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (489,4,'LOG','Syntax:\nLOG(X), LOG(B,X)\n\nIf called with one parameter, this function returns the natural\nlogarithm of X. If X is less than or equal to 0, then NULL is returned.\n\nThe inverse of this function (when called with a single argument) is\nthe EXP() function.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT LOG(2);\n -> 0.69314718055995\nmysql> SELECT LOG(-2);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (490,25,'SET SQL_LOG_BIN','Syntax:\nSET sql_log_bin = {0|1}\n\nDisables or enables binary logging for the current connection\n(sql_log_bin is a session variable) if the client has the SUPER\nprivilege. The statement is refused with an error if the client does\nnot have that privilege.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (490,25,'SET SQL_LOG_BIN','Syntax:\nSET sql_log_bin = {0|1}\n\nDisables or enables binary logging for the current session (sql_log_bin\nis a session variable) if the client has the SUPER privilege. The\nstatement fails with an error if the client does not have that\nprivilege.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (491,17,'!=','Syntax:\n<>, !=\n\nNot equal:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT \'.01\' <> \'0.01\';\n -> 1\nmysql> SELECT .01 <> \'0.01\';\n -> 0\nmysql> SELECT \'zapp\' <> \'zappp\';\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (492,22,'WHILE','Syntax:\n[begin_label:] WHILE search_condition DO\n statement_list\nEND WHILE [end_label]\n\nThe statement list within a WHILE statement is repeated as long as the\nsearch_condition is true. statement_list consists of one or more\nstatements.\n\nA WHILE statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/while-statement.html\n\n','CREATE PROCEDURE dowhile()\nBEGIN\n DECLARE v1 INT DEFAULT 5;\n\n WHILE v1 > 0 DO\n ...\n SET v1 = v1 - 1;\n END WHILE;\nEND\n','http://dev.mysql.com/doc/refman/5.1/en/while-statement.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (492,22,'WHILE','Syntax:\n[begin_label:] WHILE search_condition DO\n statement_list\nEND WHILE [end_label]\n\nThe statement list within a WHILE statement is repeated as long as the\nsearch_condition is true. statement_list consists of one or more\nstatements.\n\nA WHILE statement can be labeled. See [HELP BEGIN END] for the rules\nregarding label use.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/while-statement.html\n\n','CREATE PROCEDURE dowhile()\nBEGIN\n DECLARE v1 INT DEFAULT 5;\n\n WHILE v1 > 0 DO\n ...\n SET v1 = v1 - 1;\n END WHILE;\nEND\n','http://dev.mysql.com/doc/refman/5.1/en/while-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (493,11,'AES_DECRYPT','Syntax:\nAES_DECRYPT(crypt_str,key_str)\n\nThis function allows decryption of data using the official AES\n(Advanced Encryption Standard) algorithm. For more information, see the\ndescription of AES_ENCRYPT().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (494,30,'DAYNAME','Syntax:\nDAYNAME(date)\n\nReturns the name of the weekday for date. As of MySQL 5.1.12, the\nlanguage used for the name is controlled by the value of the\nlc_time_names system variable\n(http://dev.mysql.com/doc/refman/5.1/en/locale-support.html).\n\n…: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DAYNAME(\'2007-02-03\');\n -> \'Saturday\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (495,15,'COERCIBILITY','Syntax:\nCOERCIBILITY(str)\n\nReturns the collation coercibility value of the string argument.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT COERCIBILITY(\'abc\' COLLATE latin1_swedish_ci);\n -> 0\nmysql> SELECT COERCIBILITY(USER());\n -> 3\nmysql> SELECT COERCIBILITY(\'abc\');\n -> 4\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); @@ -567,7 +567,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (498,4,'RADIANS','Syntax:\nRADIANS(X)\n\nReturns the argument X, converted from degrees to radians. (Note that\nπ radians equals 180 degrees.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT RADIANS(90);\n -> 1.5707963267949\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (499,15,'COLLATION','Syntax:\nCOLLATION(str)\n\nReturns the collation of the string argument.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT COLLATION(\'abc\');\n -> \'latin1_swedish_ci\'\nmysql> SELECT COLLATION(_utf8\'abc\');\n -> \'utf8_general_ci\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (500,17,'COALESCE','Syntax:\nCOALESCE(value,...)\n\nReturns the first non-NULL value in the list, or NULL if there are no\nnon-NULL values.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT COALESCE(NULL,1);\n -> 1\nmysql> SELECT COALESCE(NULL,NULL,NULL);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (501,15,'VERSION','Syntax:\nVERSION()\n\nReturns a string that indicates the MySQL server version. The string\nuses the utf8 character set.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT VERSION();\n -> \'5.1.41-standard\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (501,15,'VERSION','Syntax:\nVERSION()\n\nReturns a string that indicates the MySQL server version. The string\nuses the utf8 character set.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT VERSION();\n -> \'5.1.46-standard\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (502,35,'MAKE_SET','Syntax:\nMAKE_SET(bits,str1,str2,...)\n\nReturns a set value (a string containing substrings separated by ","\ncharacters) consisting of the strings that have the corresponding bit\nin bits set. str1 corresponds to bit 0, str2 to bit 1, and so on. NULL\nvalues in str1, str2, ... are not appended to the result.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT MAKE_SET(1,\'a\',\'b\',\'c\');\n -> \'a\'\nmysql> SELECT MAKE_SET(1 | 4,\'hello\',\'nice\',\'world\');\n -> \'hello,world\'\nmysql> SELECT MAKE_SET(1 | 4,\'hello\',\'nice\',NULL,\'world\');\n -> \'hello\'\nmysql> SELECT MAKE_SET(0,\'a\',\'b\',\'c\');\n -> \'\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (503,35,'FIND_IN_SET','Syntax:\nFIND_IN_SET(str,strlist)\n\nReturns a value in the range of 1 to N if the string str is in the\nstring list strlist consisting of N substrings. A string list is a\nstring composed of substrings separated by "," characters. If the first\nargument is a constant string and the second is a column of type SET,\nthe FIND_IN_SET() function is optimized to use bit arithmetic. Returns\n0 if str is not in strlist or if strlist is the empty string. Returns\nNULL if either argument is NULL. This function does not work properly\nif the first argument contains a comma (",") character.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT FIND_IN_SET(\'b\',\'a,b,c,d\');\n -> 2\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); @@ -1115,7 +1115,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (464,43); insert into help_relation (help_topic_id,help_keyword_id) values (459,43); insert into help_relation (help_topic_id,help_keyword_id) values (469,44); -insert into help_relation (help_topic_id,help_keyword_id) values (115,44); +insert into help_relation (help_topic_id,help_keyword_id) values (116,44); insert into help_relation (help_topic_id,help_keyword_id) values (464,44); insert into help_relation (help_topic_id,help_keyword_id) values (209,44); insert into help_relation (help_topic_id,help_keyword_id) values (420,44); @@ -1222,7 +1222,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (464,92); insert into help_relation (help_topic_id,help_keyword_id) values (67,93); insert into help_relation (help_topic_id,help_keyword_id) values (431,93); -insert into help_relation (help_topic_id,help_keyword_id) values (326,93); +insert into help_relation (help_topic_id,help_keyword_id) values (327,93); insert into help_relation (help_topic_id,help_keyword_id) values (282,94); insert into help_relation (help_topic_id,help_keyword_id) values (83,95); insert into help_relation (help_topic_id,help_keyword_id) values (57,95); @@ -1268,7 +1268,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (112,109); insert into help_relation (help_topic_id,help_keyword_id) values (459,109); insert into help_relation (help_topic_id,help_keyword_id) values (38,110); -insert into help_relation (help_topic_id,help_keyword_id) values (116,110); +insert into help_relation (help_topic_id,help_keyword_id) values (117,110); insert into help_relation (help_topic_id,help_keyword_id) values (261,110); insert into help_relation (help_topic_id,help_keyword_id) values (148,110); insert into help_relation (help_topic_id,help_keyword_id) values (122,111); @@ -1340,7 +1340,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (197,137); insert into help_relation (help_topic_id,help_keyword_id) values (310,138); insert into help_relation (help_topic_id,help_keyword_id) values (344,139); -insert into help_relation (help_topic_id,help_keyword_id) values (115,139); +insert into help_relation (help_topic_id,help_keyword_id) values (116,139); insert into help_relation (help_topic_id,help_keyword_id) values (453,139); insert into help_relation (help_topic_id,help_keyword_id) values (48,139); insert into help_relation (help_topic_id,help_keyword_id) values (120,139); @@ -1361,7 +1361,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (353,147); insert into help_relation (help_topic_id,help_keyword_id) values (419,147); insert into help_relation (help_topic_id,help_keyword_id) values (344,148); -insert into help_relation (help_topic_id,help_keyword_id) values (326,148); +insert into help_relation (help_topic_id,help_keyword_id) values (327,148); insert into help_relation (help_topic_id,help_keyword_id) values (464,149); insert into help_relation (help_topic_id,help_keyword_id) values (178,150); insert into help_relation (help_topic_id,help_keyword_id) values (95,151); @@ -1441,7 +1441,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (419,181); insert into help_relation (help_topic_id,help_keyword_id) values (196,181); insert into help_relation (help_topic_id,help_keyword_id) values (301,182); -insert into help_relation (help_topic_id,help_keyword_id) values (115,182); +insert into help_relation (help_topic_id,help_keyword_id) values (116,182); insert into help_relation (help_topic_id,help_keyword_id) values (420,182); insert into help_relation (help_topic_id,help_keyword_id) values (358,182); insert into help_relation (help_topic_id,help_keyword_id) values (353,183); @@ -1597,7 +1597,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (86,251); insert into help_relation (help_topic_id,help_keyword_id) values (184,252); insert into help_relation (help_topic_id,help_keyword_id) values (353,253); -insert into help_relation (help_topic_id,help_keyword_id) values (327,254); +insert into help_relation (help_topic_id,help_keyword_id) values (326,254); insert into help_relation (help_topic_id,help_keyword_id) values (353,254); insert into help_relation (help_topic_id,help_keyword_id) values (360,254); insert into help_relation (help_topic_id,help_keyword_id) values (312,255); @@ -1633,7 +1633,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (87,264); insert into help_relation (help_topic_id,help_keyword_id) values (60,264); insert into help_relation (help_topic_id,help_keyword_id) values (355,264); -insert into help_relation (help_topic_id,help_keyword_id) values (327,265); +insert into help_relation (help_topic_id,help_keyword_id) values (326,265); insert into help_relation (help_topic_id,help_keyword_id) values (191,266); insert into help_relation (help_topic_id,help_keyword_id) values (197,267); insert into help_relation (help_topic_id,help_keyword_id) values (230,268); @@ -1695,7 +1695,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (58,286); insert into help_relation (help_topic_id,help_keyword_id) values (391,286); insert into help_relation (help_topic_id,help_keyword_id) values (325,286); -insert into help_relation (help_topic_id,help_keyword_id) values (326,286); +insert into help_relation (help_topic_id,help_keyword_id) values (327,286); insert into help_relation (help_topic_id,help_keyword_id) values (120,286); insert into help_relation (help_topic_id,help_keyword_id) values (189,286); insert into help_relation (help_topic_id,help_keyword_id) values (449,286); @@ -1742,7 +1742,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (184,308); insert into help_relation (help_topic_id,help_keyword_id) values (420,309); insert into help_relation (help_topic_id,help_keyword_id) values (388,310); -insert into help_relation (help_topic_id,help_keyword_id) values (117,310); +insert into help_relation (help_topic_id,help_keyword_id) values (115,310); insert into help_relation (help_topic_id,help_keyword_id) values (459,311); insert into help_relation (help_topic_id,help_keyword_id) values (25,312); insert into help_relation (help_topic_id,help_keyword_id) values (344,312); @@ -2002,7 +2002,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (474,444); insert into help_relation (help_topic_id,help_keyword_id) values (344,445); insert into help_relation (help_topic_id,help_keyword_id) values (39,446); -insert into help_relation (help_topic_id,help_keyword_id) values (115,446); +insert into help_relation (help_topic_id,help_keyword_id) values (116,446); insert into help_relation (help_topic_id,help_keyword_id) values (266,446); insert into help_relation (help_topic_id,help_keyword_id) values (58,446); insert into help_relation (help_topic_id,help_keyword_id) values (184,446);

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2709)
by knielsen＠knielsen-hq.org 28 Apr '10

28 Apr '10

#At lp:maria 2709 knielsen(a)knielsen-hq.org 2010-04-28 Imported MySQL documentation files from ../mysql-5.1.46 modified: Docs/INSTALL-BINARY INSTALL-SOURCE INSTALL-WIN-SOURCE man/comp_err.1 man/innochecksum.1 man/make_win_bin_dist.1 man/msql2mysql.1 man/my_print_defaults.1 man/myisam_ftdump.1 man/myisamchk.1 man/myisamlog.1 man/myisampack.1 man/mysql-stress-test.pl.1 man/mysql-test-run.pl.1 man/mysql.1 man/mysql.server.1 man/mysql_client_test.1 man/mysql_config.1 man/mysql_convert_table_format.1 man/mysql_find_rows.1 man/mysql_fix_extensions.1 man/mysql_fix_privilege_tables.1 man/mysql_install_db.1 man/mysql_secure_installation.1 man/mysql_setpermission.1 man/mysql_tzinfo_to_sql.1 man/mysql_upgrade.1 man/mysql_waitpid.1 man/mysql_zap.1 man/mysqlaccess.1 man/mysqladmin.1 man/mysqlbinlog.1 man/mysqlbug.1 man/mysqlcheck.1 man/mysqld.8 man/mysqld_multi.1 man/mysqld_safe.1 man/mysqldump.1 man/mysqldumpslow.1 man/mysqlhotcopy.1 man/mysqlimport.1 man/mysqlmanager.8 man/mysqlshow.1 man/mysqlslap.1 man/mysqltest.1 man/ndbd.8 man/ndbd_redo_log_reader.1 man/ndbmtd.8 man/perror.1 man/replace.1 man/resolve_stack_dump.1 man/resolveip.1 scripts/fill_help_tables.sql === modified file 'Docs/INSTALL-BINARY' --- a/Docs/INSTALL-BINARY 2009-12-01 07:24:05 +0000 +++ b/Docs/INSTALL-BINARY 2010-04-28 13:06:11 +0000 @@ -3,9 +3,7 @@ This section covers the installation of MySQL binary distributions that are provided for various platforms in the form of compressed - tar files (files with a .tar.gz extension). See Section 2.2, - "Installing MySQL from Generic Binaries on Unix/Linux," for a - detailed list. + tar files (files with a .tar.gz extension). To obtain MySQL, see Section 2.1.3, "How to Get MySQL." @@ -23,7 +21,7 @@ MySQL tar file binary distributions have names of the form mysql-VERSION-OS.tar.gz, where VERSION is a number (for example, - 5.1.41), and OS indicates the type of operating system for which + 5.1.46), and OS indicates the type of operating system for which the distribution is intended (for example, pc-linux-i686). In addition to these generic packages, we also offer binaries in @@ -48,7 +46,7 @@ first. If you run into problems and need to file a bug report, please use - the instructions in Section 1.6, "How to Report Bugs or Problems." + the instructions in Section 1.7, "How to Report Bugs or Problems." The basic commands that you must execute to install and use a MySQL binary distribution are: === modified file 'INSTALL-SOURCE' --- a/INSTALL-SOURCE 2009-12-01 07:24:05 +0000 +++ b/INSTALL-SOURCE 2010-04-28 13:06:11 +0000 @@ -17,8 +17,7 @@ Chapter 2. Installing and Upgrading MySQ platform. Please note that not all platforms are equally suitable for running MySQL, and that not all platforms on which MySQL is - known to run are officially supported by Sun Microsystems, - Inc.: + known to run are officially supported by Oracle Corporation: 2. Choose which distribution to install. Several versions of MySQL are available, and most are @@ -77,12 +76,11 @@ Chapter 2. Installing and Upgrading MySQ Important - Sun Microsystems, Inc. does not necessarily provide official - support for all the platforms listed in this section. For - information about those platforms that are officially supported, - see MySQL Server Supported Platforms - (http://www.mysql.com/support/supportedplatforms.html) on the - MySQL Web site. + Oracle Corporation does not necessarily provide official support + for all the platforms listed in this section. For information + about those platforms that are officially supported, see + http://www.mysql.com/support/supportedplatforms.html on the MySQL + Web site. We use GNU Autoconf, so it is possible to port MySQL to all modern systems that have a C++ compiler and a working implementation of @@ -148,7 +146,7 @@ Important by the ability of the file system to deal with large files and dealing with them efficiently. - * Our level of expertise here at Sun Microsystems, Inc. with the + * Our level of expertise here at Oracle Corporation with the platform. If we know a platform well, we enable platform-specific optimizations and fixes at compile time. We can also provide advice on configuring your system optimally @@ -184,17 +182,16 @@ Important new features are being added that could affect stability. * MySQL 5.0 is the previous stable (production-quality) release - series. + series. MySQL 5.0 is now at the end of the product lifecycle. + Active development and support for this version has ended. + Extended support for MySQL 5.0 remains available. According to + the http://www.mysql.com/about/legal/lifecycle/, only Security + and Severity Level 1 issues are still being fixed for MySQL + 5.0. * MySQL 4.1, 4.0, and 3.23 are old stable (production-quality) - release series. MySQL 4.1 is now at the end of the product - lifecycle. Active development and support for these versions - has ended. - Extended support for MySQL 4.1 remains available. According to - the MySQL Lifecycle Policy - (http://www.mysql.com/about/legal/lifecycle/) only Security - and Severity Level 1 issues are still being fixed for MySQL - 4.1. + release series. Active development and support for these + versions has ended. We do not believe in a complete code freeze because this prevents us from making bugfixes and other fixes that must be done. By @@ -228,7 +225,7 @@ Important the code on which future releases are to be based. The MySQL naming scheme uses release names that consist of three - numbers and a suffix; for example, mysql-5.0.12-beta. The numbers + numbers and a suffix; for example, mysql-5.0.14-rc. The numbers within the release name are interpreted as follows: * The first number (5) is the major version and describes the @@ -238,7 +235,7 @@ Important the major version and release level constitute the release series number. - * The third number (12) is the version number within the release + * The third number (14) is the version number within the release series. This is incremented for each new release. Usually you want the latest version for the series you have chosen. @@ -307,11 +304,6 @@ Important actually made the code faster. See Section 7.1.3, "The MySQL Benchmark Suite." - * The crash-me test - This test tries to determine what features the database - supports and what its capabilities and limitations are. See - Section 7.1.3, "The MySQL Benchmark Suite." - We also test the newest MySQL version in our internal production environment, on at least one machine. We have more than 100GB of data to work with. @@ -475,8 +467,8 @@ Important shell> md5sum package_name Example: -shell> md5sum mysql-standard-5.1.41-linux-i686.tar.gz -aaab65abbec64d5e907dcd41b8699945 mysql-standard-5.1.41-linux-i686.ta +shell> md5sum mysql-standard-5.1.46-linux-i686.tar.gz +aaab65abbec64d5e907dcd41b8699945 mysql-standard-5.1.46-linux-i686.ta r.gz You should verify that the resulting checksum (the string of @@ -520,8 +512,7 @@ Note named build(a)mysql.com. Alternatively, you can cut and paste the key directly from the following text: -----BEGIN PGP PUBLIC KEY BLOCK----- -Version: GnuPG v1.0.6 (GNU/Linux) -Comment: For info see http://www.gnupg.org +Version: GnuPG v1.4.5 (GNU/Linux) mQGiBD4+owwRBAC14GIfUfCyEDSIePvEW3SAFUdJBtoQHH/nJKZyQT7h9bPlUWC3 RODjQReyCITRrdwyrKUGku2FmeVGwn2u2WmDMNABLnpprWPkBdCk96+OmSLN9brZ @@ -533,81 +524,26 @@ kYpXBACmWpP8NJTkamEnPCia2ZoOHODANwpUkP43 QJEXM6FSbi0LLtZciNlYsafwAPEOMDKpMqAK6IyisNtPvaLd8lH0bPAnWqcyefep rv0sxxqUEMcM3o7wwgfN83POkDasDbs3pjwPhxvhz6//62zQJ7Q7TXlTUUwgUGFj a2FnZSBzaWduaW5nIGtleSAod3d3Lm15c3FsLmNvbSkgPGJ1aWxkQG15c3FsLmNv -bT6IXQQTEQIAHQUCR6yUtAUJDTBYqAULBwoDBAMVAwIDFgIBAheAAAoJEIxxjTtQ -cuH1rpIAn38+BlBI815Dou9VXMIAsQEk4G3tAJ9+Cz69Y/Xwm611lzteJrCAA32+ -aYhMBBMRAgAMBQI+PqPRBYMJZgC7AAoJEElQ4SqycpHyJOEAn1mxHijft00bKXvu +bT6IXQQTEQIAHQULBwoDBAMVAwIDFgIBAheABQJLcC5lBQkQ8/JZAAoJEIxxjTtQ +cuH1oD4AoIcOQ4EoGsZvy06D0Ei5vcsWEy8dAJ4g46i3WEcdSWxMhcBSsPz65sh5 +lohMBBMRAgAMBQI+PqPRBYMJZgC7AAoJEElQ4SqycpHyJOEAn1mxHijft00bKXvu cSo/pECUmppiAJ41M9MRVj5VcdH/KN/KjRtW6tHFPYhMBBMRAgAMBQI+QoIDBYMJ YiKJAAoJELb1zU3GuiQ/lpEAoIhpp6BozKI8p6eaabzF5MlJH58pAKCu/ROofK8J -Eg2aLos+5zEYrB/LsohGBBARAgAGBQI/rOOvAAoJEK/FI0h4g3QP9pYAoNtSISDD -AAU2HafyAYlLD/yUC4hKAJ0czMsBLbo0M/xPaJ6Ox9Q5Hmw2uIhGBBARAgAGBQI/ -tEN3AAoJEIWWr6swc05mxsMAnRag9X61Ygu1kbfBiqDku4czTd9pAJ4q5W8KZ0+2 -ujTrEPN55NdWtnXj4YhGBBARAgAGBQJDW7PqAAoJEIvYLm8wuUtcf3QAnRCyqF0C -pMCTdIGc7bDO5I7CIMhTAJ0UTGx0O1d/VwvdDiKWj45N2tNbYIhGBBMRAgAGBQJE -8TMmAAoJEPZJxPRgk1MMCnEAoIm2pP0sIcVh9Yo0YYGAqORrTOL3AJwIbcy+e8HM -NSoNV5u51RnrVKie34hMBBARAgAMBQJBgcsBBYMGItmLAAoJEBhZ0B9ne6HsQo0A -nA/LCTQ3P5kvJvDhg1DsfVTFnJxpAJ49WFjg/kIcaN5iP1JfaBAITZI3H4hMBBAR -AgAMBQJBgcs0BYMGItlYAAoJEIHC9+viE7aSIiMAnRVTVVAfMXvJhV6D5uHfWeeD -046TAJ4kjwP2bHyd6DjCymq+BdEDz63axohMBBARAgAMBQJBgctiBYMGItkqAAoJ -EGtw7Nldw/RzCaoAmwWM6+Rj1zl4D/PIys5nW48Hql3hAJ0bLOBthv96g+7oUy9U -j09Uh41lF4hMBBARAgAMBQJB0JMkBYMF1BFoAAoJEH0lygrBKafCYlUAoIb1r5D6 -qMLMPMO1krHk3MNbX5b5AJ4vryx5fw6iJctC5GWJ+Y8ytXab34hMBBARAgAMBQJC -K1u6BYMFeUjSAAoJEOYbpIkV67mr8xMAoJMy+UJC0sqXMPSxh3BUsdcmtFS+AJ9+ -Z15LpoOnAidTT/K9iODXGViK6ohMBBIRAgAMBQJAKlk6BYMHektSAAoJEDyhHzSU -+vhhJlwAnA/gOdwOThjO8O+dFtdbpKuImfXJAJ0TL53QKp92EzscZSz49lD2YkoE -qohMBBIRAgAMBQJAPfq6BYMHZqnSAAoJEPLXXGPjnGWcst8AoLQ3MJWqttMNHDbl -xSyzXhFGhRU8AJ4ukRzfNJqElQHQ00ZM2WnCVNzOUIhMBBIRAgAMBQJBDgqEBYMG -lpoIAAoJEDnKK/Q9aopf/N0AniE2fcCKO1wDIwusuGVlC+JvnnWbAKDDoUSEYuNn -5qzRbrzWW5zBno/Nb4hMBBIRAgAMBQJCgKU0BYMFI/9YAAoJEAQNwIV8g5+o4yQA -nA9QOFLV5POCddyUMqB/fnctuO9eAJ4sJbLKP/Z3SAiTpKrNo+XZRxauqIhMBBMR -AgAMBQI+TU2EBYMJV1cIAAoJEC27dr+t1MkzBQwAoJU+RuTVSn+TI+uWxUpT82/d -s5NkAJ9bnNodffyMMK7GyMiv/TzifiTD+4hMBBMRAgAMBQJB14B2BYMFzSQWAAoJ -EGbv28jNgv0+P7wAn13uu8YkhwfNMJJhWdpK2/qM/4AQAJ40drnKW2qJ5EEIJwtx -pwapgrzWiYhMBBMRAgAMBQJCGIEOBYMFjCN+AAoJEHbBAxyiMW6hoO4An0Ith3Kx -5/sixbjZR9aEjoePGTNKAJ94SldLiESaYaJx2lGIlD9bbVoHQYhdBBMRAgAdBQJH -rJTPBQkNMFioBQsHCgMEAxUDAgMWAgECF4AACgkQjHGNO1By4fV0KgCgsLpG2wP0 -rc3s07Fync9g7MfairMAoIUefSNKrGTsTxvLeyH4DLzJW/QFiHsEMBECADsFAkJ3 -NfU0HQBPb3BzLi4uIHNob3VsZCBoYXZlIGJlZW4gbG9jYWwhIEknbSAqc28qIHN0 -dXBpZC4uLgAKCRA5yiv0PWqKX+9HAJ0WjTx/rqgouK4QCrOV/2IOU+jMQQCfYSC8 -JgsIIeN8aiyuStTdYrk0VWCIjwQwEQIATwUCRW8Av0gdAFNob3VsZCBoYXZlIGJl -ZW4gYSBsb2NhbCBzaWduYXR1cmUsIG9yIHNvbWV0aGluZyAtIFdURiB3YXMgSSB0 -aGlua2luZz8ACgkQOcor9D1qil+g+wCfcFWoo5qUl4XTE9K8tH3Q+xGWeYYAnjii -KxjtOXc0ls+BlqXxbfZ9uqBsiQIiBBABAgAMBQJBgcuFBYMGItkHAAoJEKrj5s5m -oURoqC8QAIISudocbJRhrTAROOPoMsReyp46Jdp3iL1oFDGcPfkZSBwWh8L+cJjh -dycIwwSeZ1D2h9S5Tc4EnoE0khsS6wBpuAuih5s//coRqIIiLKEdhTmNqulkCH5m -imCzc5zXWZDW0hpLr2InGsZMuh2QCwAkB4RTBM+r18cUXMLV4YHKyjIVaDhsiPP/ -MKUj6rJNsUDmDq1GiJdOjySjtCFjYADlQYSD7zcd1vpqQLThnZBESvEoCqumEfOP -xemNU6xAB0CL+pUpB40pE6Un6Krr5h6yZxYZ/N5vzt0Y3B5UUMkgYDSpjbulNvaU -TFiOxEU3gJvXc1+h0BsxM7FwBZnuMA8LEA+UdQb76YcyuFBcROhmcEUTiducLu84 -E2BZ2NSBdymRQKSinhvXsEWlH6Txm1gtJLynYsvPi4B4JxKbb+awnFPusL8W+gfz -jbygeKdyqzYgKj3M79R3geaY7Q75Kxl1UogiOKcbI5VZvg47OQCWeeERnejqEAdx -EQiwGA/ARhVOP/1l0LQA7jg2P1xTtrBqqC2ufDB+v+jhXaCXxstKSW1lTbv/b0d6 -454UaOUV7RisN39pE2zFvJvY7bwfiwbUJVmYLm4rWJAEOJLIDtDRtt2h8JahDObm -3CWkpadjw57S5v1c/mn+xV9yTgVx5YUfC/788L1HNKXfeVDq8zbAiQIiBBMBAgAM -BQJCnwocBYMFBZpwAAoJENjCCglaJFfPIT4P/25zvPp8ixqV85igs3rRqMBtBsj+ -5EoEW6DJnlGhoi26yf1nasC2frVasWG7i4JIm0U3WfLZERGDjR/nqlOCEqsP5gS3 -43N7r4UpDkBsYh0WxH/ZtST5llFK3zd7XgtxvqKL98l/OSgijH2W2SJ9DGpjtO+T -iegq7igtJzw7Vax9z/LQH2xhRQKZR9yernwMSYaJ72i9SyWbK3k0+e95fGnlR5pF -zlGq320rYHgD7v9yoQ2t1klsAxK6e3b7Z+RiJG6cAU8o8F0kGxjWzF4v8D1op7S+ -IoRdB0Bap01ko0KLyt3+g4/33/2UxsW50BtfqcvYNJvU4bZns1YSqAgDOOanBhg8 -Ip5XPlDxH6J/3997n5JNj/nk5ojfd8nYfe/5TjflWNiput6tZ7frEki1wl6pTNbv -V9C1eLUJMSXfDZyHtUXmiP9DKNpsucCUeBKWRKLqnsHLkLYydsIeUJ8+ciKc+EWh -FxEY+Ml72cXAaz5BuW9L8KHNzZZfez/ZJabiARQpFfjOwAnmhzJ9r++TEKRLEr96 -taUI9/8nVPvT6LnBpcM38Td6dJ639YvuH3ilAqmPPw50YvglIEe4BUYD5r52Seqc -8XQowouGOuBX4vs7zgWFuYA/s9ebfGaIw+uJd/56Xl9ll6q5CghqB/yt1EceFEnF -CAjQc2SeRo6qzx22uQINBD4+ox0QCADv4Yl/Fsx1jjCyU+eMf2sXg3ap9awQ3+XF -pmglhzdrozTZYKceXpqFPb+0ErbDVAjhgW15HjuAK+2Bvo7Ukd986jYd8uZENGJG -N3UNMIep7JfsIeFyCGP901GVbZnSXlAURyZX1TRWGndoV9YLhSN+zctT6GQBbMTv -NoPlwf0nvK//rG5lXDjXXHSHhSqxNxYy7SIzUHMQupfUNjsvCg8Rv871GRt/h+Yt -7XUTMhoJrg+oBFdBlzh2FKKcy3ordfgGtGwpN+jMG7vgXjsPwiVt/m9Jgdu4Tmn/ -WggPOeSD+nyRb7cXG5avJxyKoVNw3PbXnLJff0tcWeUvMpRv8XkbAAMFB/4vCqpr -wIatF+w4AnGKbrcId+3LmZRzmtRKdOyUZgQg4JHUF5Bq7I9ls8OwMP0xnVlpJp9q -cW/AUbouXH3GRTu3Or68ouhaSbi7nF/e+fnlWOdJ3VpD15CdRxeIvhycEahNs5Yj -f0RzLOCyXMF0L74w+NxBNwDunolRWw/qgAHcVBaDni25SjQRzxuwzxvcS/jYua5B -Pk10ocbAexdM+2XSSWThtCTg5qMeyLLUExqGlPbuNaMmUyIlz4hYnSaCGQoe33bq -z/KZ91/keR1DVzK+zPm2vJUjcXHvxd5Jh9C+67CqnYfXf2lcYSSDSfop1Q5611la -F7vRgY0/DXKNYlPUiEwEGBECAAwFAkeslPwFCQ0wWN8ACgkQjHGNO1By4fWlzgCf -Qj3rkfcljYZOuLOn50J7PFuF7FoAnjwWGhwVi9+Fm2B5RZvpo++BBkdP -=Xquv +Eg2aLos+5zEYrB/LsrkCDQQ+PqMdEAgA7+GJfxbMdY4wslPnjH9rF4N2qfWsEN/l +xaZoJYc3a6M02WCnHl6ahT2/tBK2w1QI4YFteR47gCvtgb6O1JHffOo2HfLmRDRi +Rjd1DTCHqeyX7CHhcghj/dNRlW2Z0l5QFEcmV9U0Vhp3aFfWC4Ujfs3LU+hkAWzE +7zaD5cH9J7yv/6xuZVw411x0h4UqsTcWMu0iM1BzELqX1DY7LwoPEb/O9Rkbf4fm +Le11EzIaCa4PqARXQZc4dhSinMt6K3X4BrRsKTfozBu74F47D8Ilbf5vSYHbuE5p +/1oIDznkg/p8kW+3FxuWrycciqFTcNz215yyX39LXFnlLzKUb/F5GwADBQf+Lwqq +a8CGrRfsOAJxim63CHfty5mUc5rUSnTslGYEIOCR1BeQauyPZbPDsDD9MZ1ZaSaf +anFvwFG6Llx9xkU7tzq+vKLoWkm4u5xf3vn55VjnSd1aQ9eQnUcXiL4cnBGoTbOW +I39EcyzgslzBdC++MPjcQTcA7p6JUVsP6oAB3FQWg54tuUo0Ec8bsM8b3Ev42Lmu +QT5NdKHGwHsXTPtl0klk4bQk4OajHsiy1BMahpT27jWjJlMiJc+IWJ0mghkKHt92 +6s/ymfdf5HkdQ1cyvsz5tryVI3Fx78XeSYfQvuuwqp2H139pXGEkg0n6KdUOetdZ +Whe70YGNPw1yjWJT1IhMBBgRAgAMBQI+PqMdBQkJZgGAAAoJEIxxjTtQcuH17p4A +n3r1QpVC9yhnW2cSAjq+kr72GX0eAJ4295kl6NxYEuFApmr1+0uUq/SlsQ== +=Mski + -----END PGP PUBLIC KEY BLOCK----- To import the build key into your personal public GPG keyring, use @@ -650,8 +586,8 @@ pg-signature.html signature, which also is available from the download page. The signature file has the same name as the distribution file with an .asc extension, as shown by the examples in the following table. - Distribution file mysql-standard-5.1.41-linux-i686.tar.gz - Signature file mysql-standard-5.1.41-linux-i686.tar.gz.asc + Distribution file mysql-standard-5.1.46-linux-i686.tar.gz + Signature file mysql-standard-5.1.46-linux-i686.tar.gz.asc Make sure that both files are stored in the same directory and then run the following command to verify the signature for the @@ -659,7 +595,7 @@ pg-signature.html shell> gpg --verify package_name.asc Example: -shell> gpg --verify mysql-standard-5.1.41-linux-i686.tar.gz.asc +shell> gpg --verify mysql-standard-5.1.46-linux-i686.tar.gz.asc gpg: Signature made Tue 12 Jul 2005 23:35:41 EST using DSA key ID 507 2E1F5 gpg: Good signature from "MySQL Package signing key (www.mysql.com) < @@ -679,8 +615,8 @@ build(a)mysql.com>" shell> rpm --checksig package_name.rpm Example: -shell> rpm --checksig MySQL-server-5.1.41-0.glibc23.i386.rpm -MySQL-server-5.1.41-0.glibc23.i386.rpm: md5 gpg OK +shell> rpm --checksig MySQL-server-5.1.46-0.glibc23.i386.rpm +MySQL-server-5.1.46-0.glibc23.i386.rpm: md5 gpg OK Note @@ -705,7 +641,7 @@ shell> rpm --import mysql_pubkey.asc This section describes the default layout of the directories created by installing binary or source distributions provided by - Sun Microsystems, Inc. A distribution provided by another vendor + Oracle Corporation. A distribution provided by another vendor might use a layout different from those shown here. Installations created from our Linux RPM distributions result in @@ -773,9 +709,7 @@ shell> rpm --import mysql_pubkey.asc This section covers the installation of MySQL binary distributions that are provided for various platforms in the form of compressed - tar files (files with a .tar.gz extension). See Section 2.2, - "Installing MySQL from Generic Binaries on Unix/Linux," for a - detailed list. + tar files (files with a .tar.gz extension). To obtain MySQL, see Section 2.1.3, "How to Get MySQL." @@ -793,7 +727,7 @@ shell> rpm --import mysql_pubkey.asc MySQL tar file binary distributions have names of the form mysql-VERSION-OS.tar.gz, where VERSION is a number (for example, - 5.1.41), and OS indicates the type of operating system for which + 5.1.46), and OS indicates the type of operating system for which the distribution is intended (for example, pc-linux-i686). In addition to these generic packages, we also offer binaries in @@ -818,7 +752,7 @@ shell> rpm --import mysql_pubkey.asc first. If you run into problems and need to file a bug report, please use - the instructions in Section 1.6, "How to Report Bugs or Problems." + the instructions in Section 1.7, "How to Report Bugs or Problems." The basic commands that you must execute to install and use a MySQL binary distribution are: @@ -987,7 +921,7 @@ Note MySQL source distributions are provided as compressed tar archives and have names of the form mysql-VERSION.tar.gz, where VERSION is - a number like 5.1.41. + a number like 5.1.46. You need the following tools to build and install MySQL from source: @@ -1005,15 +939,9 @@ Note systems with a deficient tar, you should install GNU tar first. - * A working ANSI C++ compiler. gcc 2.95.2 or later, SGI C++, and - SunPro C++ are some of the compilers that are known to work. - libg++ is not needed when using gcc. gcc 2.7.x has a bug that - makes it impossible to compile some perfectly legal C++ files, - such as sql/sql_base.cc. If you have only gcc 2.7.x, you must - upgrade your gcc to be able to compile MySQL. gcc 2.8.1 is - also known to have problems on some platforms, so it should be - avoided if a newer compiler exists for the platform. gcc - 2.95.2 or later is recommended. + * A working ANSI C++ compiler. GCC 3.2 or later, Sun Studio 10 + or later, Visual Studio 2005 or later, and many current + vendor-supplied compilers are known to work. * A good make program. GNU make is always recommended and is sometimes required. (BSD make fails, and vendor-provided make @@ -1035,7 +963,7 @@ CFLAGS="-O3" CXX=gcc CXXFLAGS="-O3 -feli On most systems, this gives you a fast and stable binary. If you run into problems and need to file a bug report, please use - the instructions in Section 1.6, "How to Report Bugs or Problems." + the instructions in Section 1.7, "How to Report Bugs or Problems." 2.3.1. Source Installation Overview @@ -1121,7 +1049,7 @@ shell> make from config.log that you think can help solve the problem. Also include the last couple of lines of output from configure. To file a bug report, please use the instructions - in Section 1.6, "How to Report Bugs or Problems." + in Section 1.7, "How to Report Bugs or Problems." If the compile fails, see Section 2.3.4, "Dealing with Problems Compiling MySQL," for help. @@ -1497,10 +1425,9 @@ shell> ./configure --with-charset=CHARSE cp1251, cp1256, cp1257, cp850, cp852, cp866, cp932, dec8, eucjpms, euckr, gb2312, gbk, geostd8, greek, hebrew, hp8, keybcs2, koi8r, koi8u, latin1, latin2, latin5, latin7, macce, - macroman, sjis, swe7, tis620, ucs2, ujis, utf8. See Section - 9.2, "The Character Set Used for Data and Sorting." - (Additional character sets might be available. Check the - output from ./configure --help for the current list.) + macroman, sjis, swe7, tis620, ucs2, ujis, utf8. (Additional + character sets might be available. Check the output from + ./configure --help for the current list.) The default collation may also be specified. MySQL uses the latin1_swedish_ci collation by default. To change this, use the --with-collation option: @@ -1602,7 +1529,7 @@ shell> ./configure --with-debug * When given with --enable-community-features, the --enable-profiling option enables the statement profiling capability exposed by the SHOW PROFILE and SHOW PROFILES - statements. (See Section 12.5.5.33, "SHOW PROFILES Syntax.") + statements. (See Section 12.4.5.33, "SHOW PROFILES Syntax.") This option was added in MySQL 5.1.24. It is enabled by default as of MySQL 5.1.28; to disable it, use --disable-profiling. @@ -1610,7 +1537,7 @@ shell> ./configure --with-debug * See Section 2.1, "General Installation Guidance," for options that pertain to particular operating systems. - * See Section 5.5.7.2, "Using SSL Connections," for options that + * See Section 5.5.6.2, "Using SSL Connections," for options that pertain to configuring MySQL to support secure (encrypted) connections. @@ -1664,12 +1591,12 @@ Caution (either a binary or source distribution). To obtain the most recent development source tree, you must have - Bazaar installed. You can obtain Bazaar from the Bazaar VCS - Website (http://bazaar-vcs.org) Bazaar is supported by any - platform that supports Python, and is therefore compatible with - any Linux, Unix, Windows or Mac OS X host. Instructions for - downloading and installing Bazaar on the different platforms are - available on the Bazaar website. + Bazaar installed. You can obtain Bazaar from the Bazaar VCS Web + site (http://bazaar-vcs.org) Bazaar is supported by any platform + that supports Python, and is therefore compatible with any Linux, + Unix, Windows or Mac OS X host. Instructions for downloading and + installing Bazaar on the different platforms are available on the + Bazaar Web site. All MySQL projects are hosted on Launchpad (http://launchpad.net/) MySQL projects, including MySQL server, @@ -1752,7 +1679,7 @@ shell> bzr log page. If you see diffs (changes) or code that you have a question about, do not hesitate to send email to the MySQL internals - mailing list. See Section 1.5.1, "MySQL Mailing Lists." Also, + mailing list. See Section 1.6.1, "MySQL Mailing Lists." Also, if you think you have a better idea on how to do something, send an email message to the list with a patch. @@ -1816,7 +1743,7 @@ shell> make 6. If you have gotten to the make stage, but the distribution does not compile, please enter the problem into our bugs - database using the instructions given in Section 1.6, "How to + database using the instructions given in Section 1.7, "How to Report Bugs or Problems." If you have installed the latest versions of the required GNU tools, and they crash trying to process our configuration files, please report that also. @@ -2092,7 +2019,7 @@ implicit declaration of function `int st * Before any upgrade, back up your databases, including the mysql database that contains the grant tables. See Section - 6.1, "Database Backup Methods." + 6.2, "Database Backup Methods." * Read all the notes in Section 2.4.1.1, "Upgrading from MySQL 5.0 to 5.1." These notes enable you to identify upgrade issues @@ -2117,7 +2044,7 @@ implicit declaration of function `int st * If you are running MySQL Server on Windows, see Section 2.5.7, "Upgrading MySQL on Windows." - * If you are using replication, see Section 16.3.3, "Upgrading a + * If you are using replication, see Section 16.4.3, "Upgrading a Replication Setup," for information on upgrading your replication setup. @@ -2251,7 +2178,7 @@ Note done before upgrading. Use of this statement with a version of MySQL different from the one used to create the table (that is, using it after upgrading) may damage the table. See - Section 12.5.2.6, "REPAIR TABLE Syntax." + Section 12.4.2.6, "REPAIR TABLE Syntax." * After you upgrade to a new version of MySQL, run mysql_upgrade (see Section 4.4.8, "mysql_upgrade --- Check Tables for MySQL @@ -2274,7 +2201,7 @@ Note * If you are running MySQL Server on Windows, see Section 2.5.7, "Upgrading MySQL on Windows." - * If you are using replication, see Section 16.3.3, "Upgrading a + * If you are using replication, see Section 16.4.3, "Upgrading a Replication Setup," for information on upgrading your replication setup. @@ -2322,13 +2249,13 @@ Note upgrading, and reload them into MySQL 5.1 after upgrading. * Known issue: The fix for - Bug#23491: http://bugs.mysql.com/23491 introduced a problem - with SHOW CREATE VIEW, which is used by mysqldump. This causes - an incompatibility when upgrading from versions affected by - that bug fix (MySQL 5.0.40 through 5.0.43, MySQL 5.1.18 - through 5.1.19): If you use mysqldump before upgrading from an - affected version and reload the data after upgrading to a - higher version, you must drop and recreate your views. + Bug#23491: http://bugs.mysql.com/bug.php?id=23491 introduced a + problem with SHOW CREATE VIEW, which is used by mysqldump. + This causes an incompatibility when upgrading from versions + affected by that bug fix (MySQL 5.0.40 through 5.0.43, MySQL + 5.1.18 through 5.1.19): If you use mysqldump before upgrading + from an affected version and reload the data after upgrading + to a higher version, you must drop and recreate your views. * Known issue: Dumps performed by using mysqldump to generate a dump file before the upgrade and reloading the file after @@ -2456,11 +2383,11 @@ RENAME TABLE table_b TO `table b`; * Incompatible change: MySQL 5.1 implements support for a plugin API that allows the loading and unloading of components at runtime, without restarting the server. Section 22.2, "The - MySQL Plugin Interface." The plugin API requires the - mysql.plugin table. After upgrading from an older version of - MySQL, you should run the mysql_upgrade command to create this - table. See Section 4.4.8, "mysql_upgrade --- Check Tables for - MySQL Upgrade." + MySQL Plugin API." The plugin API requires the mysql.plugin + table. After upgrading from an older version of MySQL, you + should run the mysql_upgrade command to create this table. See + Section 4.4.8, "mysql_upgrade --- Check Tables for MySQL + Upgrade." Plugins are installed in the directory named by the plugin_dir system variable. This variable also controls the location from which the server loads user-defined functions (UDFs), which is @@ -2718,7 +2645,7 @@ REPAIR TABLE tbl_name QUICK; specifies the locale that controls the language used to display day and month names and abbreviations. This variable affects the output from the DATE_FORMAT(), DAYNAME() and - MONTHNAME() functions. See Section 9.8, "MySQL Server Locale + MONTHNAME() functions. See Section 9.7, "MySQL Server Locale Support." * As of MySQL 5.1.9, mysqld_safe no longer implicitly invokes @@ -2749,7 +2676,7 @@ REPAIR TABLE tbl_name QUICK; to reload them into an upgraded server. Handlers that contain illegal label references will be rejected. For more information about condition handlers and writing them - to avoid invalid jumps, see Section 12.8.4.2, "DECLARE for + to avoid invalid jumps, see Section 12.7.4.2, "DECLARE for Handlers." * Incompatible change: The parser accepted statements that @@ -2758,13 +2685,13 @@ REPAIR TABLE tbl_name QUICK; contain unclosed /*-comments now are rejected with a syntax error. This fix has the potential to cause incompatibilities. Because - of Bug#26302: http://bugs.mysql.com/26302, which caused the - trailing */ to be truncated from comments in views, stored - routines, triggers, and events, it is possible that objects of - those types may have been stored with definitions that now - will be rejected as syntactically invalid. Such objects should - be dropped and re-created so that their definitions do not - contain truncated comments. + of Bug#26302: http://bugs.mysql.com/bug.php?id=26302, which + caused the trailing */ to be truncated from comments in views, + stored routines, triggers, and events, it is possible that + objects of those types may have been stored with definitions + that now will be rejected as syntactically invalid. Such + objects should be dropped and re-created so that their + definitions do not contain truncated comments. * Incompatible change: Multiple-table DELETE statements containing ambiguous aliases could have unintended side @@ -2851,21 +2778,20 @@ mysql> source /tmp/triggers.sql // mysqldump or mysqlhotcopy can be used as alternatives. * The LOAD DATA FROM MASTER and LOAD TABLE FROM MASTER - statements are deprecated. See Section 12.6.2.2, "LOAD DATA + statements are deprecated. See Section 12.5.2.2, "LOAD DATA FROM MASTER Syntax," for recommended alternatives. * The INSTALL PLUGIN and UNINSTALL PLUGIN statements that are used for the plugin API are new. So is the WITH PARSER clause for FULLTEXT index creation that associates a parser plugin - with a full-text index. Section 22.2, "The MySQL Plugin - Interface." + with a full-text index. Section 22.2, "The MySQL Plugin API." C API Changes: * Incompatible change: As of MySQL 5.1.7, the mysql_stmt_attr_get() C API function returns a boolean rather than an unsigned int for STMT_ATTR_UPDATE_MAX_LENGTH. - (Bug#16144: http://bugs.mysql.com/16144) + (Bug#16144: http://bugs.mysql.com/bug.php?id=16144) 2.4.2. Downgrading MySQL @@ -2930,10 +2856,10 @@ mysql> source /tmp/triggers.sql // 5. Reload the dump file into the older server. Your tables should be accessible. - It might also be the case that the structure of the system tables - in the mysql database has changed and that downgrading introduces - some loss of functionality or requires some adjustments. Here are - some examples: + It might also be the case that system tables in the mysql database + have changed and that downgrading introduces some loss of + functionality or requires some adjustments. Here are some + examples: * Trigger creation requires the TRIGGER privilege as of MySQL 5.1. In MySQL 5.0, there is no TRIGGER privilege and SUPER is @@ -2944,6 +2870,12 @@ mysql> source /tmp/triggers.sql // * Triggers were added in MySQL 5.0, so if you downgrade from 5.0 to 4.1, you cannot use triggers at all. + * The mysql.proc.comment column definition changed between MySQL + 5.1 and 5.5. After a downgrade from 5.5 to 5.1, this table is + seen as corrupt and in need of repair. To workaround this + problem, execute mysql_upgrade from the version of MySQL to + which you downgraded. + 2.4.2.1. Downgrading to MySQL 5.0 When downgrading to MySQL 5.0 from MySQL 5.1, you should keep in @@ -2979,9 +2911,10 @@ mysql> source /tmp/triggers.sql // --all-databases option). Instead, you should run mysqldump --routines prior to performing the downgrade and run the stored routines DDL statements following the downgrade. - See Bug#11986: http://bugs.mysql.com/11986, - Bug#30029: http://bugs.mysql.com/30029, and - Bug#30660: http://bugs.mysql.com/30660, for more information. + See Bug#11986: http://bugs.mysql.com/bug.php?id=11986, + Bug#30029: http://bugs.mysql.com/bug.php?id=30029, and + Bug#30660: http://bugs.mysql.com/bug.php?id=30660, for more + information. * Triggers. Trigger creation requires the TRIGGER privilege as of MySQL 5.1. In MySQL 5.0, there is no TRIGGER privilege and @@ -3060,10 +2993,10 @@ mysql> source /tmp/triggers.sql // report, the bug number is given. The list applies both for binary upgrades and downgrades. For - example, Bug#27877: http://bugs.mysql.com/27877 was fixed in MySQL - 5.1.24 and 5.4.0, so it applies to upgrades from versions older - than 5.1.24 to 5.1.24 or newer, and to downgrades from 5.1.24 or - newer to versions older than 5.1.24. + example, Bug#27877: http://bugs.mysql.com/bug.php?id=27877 was + fixed in MySQL 5.1.24 and 5.4.0, so it applies to upgrades from + versions older than 5.1.24 to 5.1.24 or newer, and to downgrades + from 5.1.24 or newer to versions older than 5.1.24. In many cases, you can use CHECK TABLE ... FOR UPGRADE to identify tables for which index rebuilding is required. (It will report: @@ -3073,33 +3006,36 @@ mysql> source /tmp/triggers.sql // TABLE. However, the use of CHECK TABLE applies only after upgrades, not downgrades. Also, CHECK TABLE is not applicable to all storage engines. For details about which storage engines CHECK - TABLE supports, see Section 12.5.2.3, "CHECK TABLE Syntax." + TABLE supports, see Section 12.4.2.3, "CHECK TABLE Syntax." Changes that cause index rebuilding to be necessary: - * MySQL 5.0.48, 5.1.21 (Bug#29461: http://bugs.mysql.com/29461) + * MySQL 5.0.48, 5.1.21 + (Bug#29461: http://bugs.mysql.com/bug.php?id=29461) Affects indexes for columns that use any of these character sets: eucjpms, euc_kr, gb2312, latin7, macce, ujis Affected tables can be detected by CHECK TABLE ... FOR UPGRADE as of MySQL 5.1.29, 5.4.0 (see - Bug#39585: http://bugs.mysql.com/39585) + Bug#39585: http://bugs.mysql.com/bug.php?id=39585) - * MySQL 5.0.48, 5.1.23 (Bug#27562: http://bugs.mysql.com/27562) + * MySQL 5.0.48, 5.1.23 + (Bug#27562: http://bugs.mysql.com/bug.php?id=27562) Affects indexes that use the ascii_general_ci collation for columns that contain any of these characters: '`' GRAVE ACCENT, '[' LEFT SQUARE BRACKET, '\' REVERSE SOLIDUS, ']' RIGHT SQUARE BRACKET, '~' TILDE Affected tables can be detected by CHECK TABLE ... FOR UPGRADE as of MySQL 5.1.29, 5.4.0 (see - Bug#39585: http://bugs.mysql.com/39585) + Bug#39585: http://bugs.mysql.com/bug.php?id=39585) - * MySQL 5.1.24, 5.4.0 (Bug#27877: http://bugs.mysql.com/27877) + * MySQL 5.1.24, 5.4.0 + (Bug#27877: http://bugs.mysql.com/bug.php?id=27877) Affects indexes that use the utf8_general_ci or ucs2_general_ci collation for columns that contain 'ß' LATIN SMALL LETTER SHARP S (German). Affected tables can be detected by CHECK TABLE ... FOR UPGRADE as of MySQL 5.1.30, 5.4.0 (see - Bug#40053: http://bugs.mysql.com/40053) + Bug#40053: http://bugs.mysql.com/bug.php?id=40053) 2.4.4. Rebuilding or Repairing Tables or Indexes @@ -3107,10 +3043,12 @@ mysql> source /tmp/triggers.sql // necessitated by changes to MySQL such as how data types are handled or changes to character set handling. For example, an error in a collation might have been corrected, necessitating a - table rebuild to rebuild the indexes for character columns that - use the collation. It might also be that a table repair or upgrade - should be done as indicated by a table check operation such as - that performed by CHECK TABLE, mysqlcheck, or mysql_upgrade. + table rebuild to update the indexes for character columns that use + the collation. (For examples, see Section 2.4.3, "Checking Whether + Tables or Indexes Must Be Rebuilt.") It might also be that a table + repair or upgrade should be done as indicated by a table check + operation such as that performed by CHECK TABLE, mysqlcheck, or + mysql_upgrade. Methods for rebuilding a table include dumping and reloading it, or using ALTER TABLE or REPAIR TABLE. @@ -3120,26 +3058,25 @@ Note If you are rebuilding tables because a different version of MySQL will not handle them after a binary (in-place) upgrade or downgrade, you must use the dump-and-reload method. Dump the - tables before upgrading or downgrading (using your original - version of MySQL), and reload the tables after upgrading or - downgrading (after installing the new version). + tables before upgrading or downgrading using your original version + of MySQL. Then reload the tables after upgrading or downgrading. If you use the dump-and-reload method of rebuilding tables only for the purpose of rebuilding indexes, you can perform the dump either before or after upgrading or downgrading. Reloading still must be done afterward. - To re-create a table by dumping and reloading it, use mysqldump to + To rebuild a table by dumping and reloading it, use mysqldump to create a dump file and mysql to reload the file: shell> mysqldump db_name t1 > dump.sql shell> mysql db_name < dump.sql - To recreate all the tables in a single database, specify the + To rebuild all the tables in a single database, specify the database name without any following table name: shell> mysqldump db_name > dump.sql shell> mysql db_name < dump.sql - To recreate all tables in all databases, use the --all-databases + To rebuild all tables in all databases, use the --all-databases option: shell> mysqldump --all-databases > dump.sql shell> mysql < dump.sql @@ -3165,7 +3102,7 @@ mysql> REPAIR TABLE t1; the file, as described earlier. For specifics about which storage engines REPAIR TABLE supports, - see Section 12.5.2.6, "REPAIR TABLE Syntax." + see Section 12.4.2.6, "REPAIR TABLE Syntax." mysqlcheck --repair provides command-line access to the REPAIR TABLE statement. This can be a more convenient means of repairing @@ -3447,7 +3384,7 @@ Note below for reference: * Windows Essentials --- this package has a file name similar to - mysql-essential-5.1.41-win32.msi and is supplied as a + mysql-essential-5.1.46-win32.msi and is supplied as a Microsoft Installer (MSI) package. The package includes the minimum set of files needed to install MySQL on Windows, including the MySQL Server Instance Config Wizard. This @@ -3458,7 +3395,7 @@ Note MySQL with the MSI Package." * Windows MSI Installer (Complete) --- this package has a file - name similar to mysql-5.1.41-win32.zip and contains all files + name similar to mysql-5.1.46-win32.zip and contains all files needed for a complete Windows installation, including the MySQL Server Instance Config Wizard. This package includes optional components such as the embedded server and benchmark @@ -3467,7 +3404,7 @@ Note MySQL with the MSI Package." * Without installer --- this package has a file name similar to - mysql-noinstall-5.1.41-win32.zip and contains all the files + mysql-noinstall-5.1.46-win32.zip and contains all the files found in the Complete install package, with the exception of the MySQL Server Instance Config Wizard. This package does not include an automated installer, and must be manually installed @@ -3618,7 +3555,7 @@ Note feedback of users like you. If you find that the MySQL Installation Wizard is lacking some feature important to you, or if you discover a bug, please report it in our bugs database using - the instructions given in Section 1.6, "How to Report Bugs or + the instructions given in Section 1.7, "How to Report Bugs or Problems." 2.5.3.1.1. Downloading and Starting the MySQL Installation Wizard @@ -3720,7 +3657,7 @@ Note directory. In a default installation it contains C:\Program Files\MySQL\MySQL Server 5.1\. The Version string contains the release number. For example, for an installation of MySQL Server - 5.1.41, the key contains a value of 5.1.41. + 5.1.46, the key contains a value of 5.1.46. These registry keys are used to help external tools identify the installed location of the MySQL server, preventing a complete scan @@ -3963,8 +3900,8 @@ shell> msiexec /x /quiet mysql-5.1.39.ms Apart from making changes to the my.ini file by running the MySQL Server Instance Config Wizard again, you can modify it by opening it with a text editor and making any necessary changes. You can - also modify the server configuration with the MySQL Administrator - (http://www.mysql.com/products/administrator/) utility. For more + also modify the server configuration with the + http://www.mysql.com/products/administrator/ utility. For more information about server configuration, see Section 5.1.2, "Server Command Options." @@ -4262,17 +4199,31 @@ Warning 2.5.4.11. The Security Options Dialog - It is strongly recommended that you set a root password for your - MySQL server, and the MySQL Server Instance Config Wizard requires - by default that you do so. If you do not wish to set a root - password, uncheck the box next to the Modify Security Settings - option. - MySQL Server Instance Config Wizard: Security - - To set the root password, enter the desired password into both the - New root password and Confirm boxes. If you are reconfiguring an - existing server, you need to enter the existing root password into - the Current root password box. + The content of the security options portion of the MySQL Server + Instance Configuration Wizard will depend on whether this is a new + installation, or modifying an existing installation. + + * Setting the root password for a new installation + It is strongly recommended that you set a root password for + your MySQL server, and the MySQL Server Instance Config Wizard + requires by default that you do so. If you do not wish to set + a root password, uncheck the box next to the Modify Security + Settings option. + MySQL Server Instance Config Wizard: Security + + * To set the root password, enter the desired password into both + the New root password and Confirm boxes. + Setting the root password for an existing installation + If you are modifying the configuration of an existing + configuration, or you are installing an upgrade and the MySQL + Server Instance Configuration Wizard has detected an existing + MySQL system, then you must enter the existing password for + root before changing the configuration information. + MySQL Server Instance Config Wizard: Security (Existing + Installation) + If you want to change the current root password, enter the + desired new password into both the New root password and + Confirm boxes. To allow root logins from across the network, check the box next to the Enable root access from remote machines option. This @@ -4718,7 +4669,7 @@ InnoDB: foreign key constraint system ta something like this, which indicates that the server is ready to service client connections: mysqld: ready for connections -Version: '5.1.41' socket: '' port: 3306 +Version: '5.1.46' socket: '' port: 3306 The server continues to write to the console any further diagnostic output it produces. You can open a new console window @@ -5104,7 +5055,7 @@ C:\> sc delete mysql Windows. 2. You should always back up your current MySQL installation - before performing an upgrade. See Section 6.1, "Database + before performing an upgrade. See Section 6.2, "Database Backup Methods." 3. Download the latest Windows distribution of MySQL from @@ -5389,7 +5340,7 @@ ROM db" mysql names that are compatible with the current ANSI code pages. For example, the following Japanese directory name will not work in the Western locale (code page 1252): -datadir="C:/维基百科关于中文维基百科" +datadir="C:/私たちのプロジェクトのデータ" The same limitation applies to directory and file names referred to in SQL statements, such as the data file path name in LOAD DATA INFILE. @@ -5451,10 +5402,9 @@ Note from the Bazaar tree. For production use, we do not advise using a MySQL server built by yourself from source. Normally, it is best to use precompiled binary distributions of MySQL that are built - specifically for optimal performance on Windows by Sun - Microsystems, Inc. Instructions for installing binary - distributions are available in Section 2.5, "Installing MySQL on - Windows." + specifically for optimal performance on Windows by Oracle + Corporation. Instructions for installing binary distributions are + available in Section 2.5, "Installing MySQL on Windows." To build MySQL on Windows from source, you must satisfy the following system, compiler, and resource requirements: @@ -5514,8 +5464,8 @@ Note You also need a MySQL source distribution for Windows, which can be obtained two ways: - * Obtain a source distribution packaged by Sun Microsystems, - Inc. These are available from http://dev.mysql.com/downloads/. + * Obtain a source distribution packaged by Oracle Corporation. + These are available from http://dev.mysql.com/downloads/. * Package a source distribution yourself from the latest Bazaar developer source tree. For instructions on pulling the latest @@ -5525,19 +5475,20 @@ Note If you find something not working as expected, or you have suggestions about ways to improve the current build process on Windows, please send a message to the win32 mailing list. See - Section 1.5.1, "MySQL Mailing Lists." + Section 1.6.1, "MySQL Mailing Lists." 2.5.10.1. Building MySQL from Source Using CMake and Visual Studio You can build MySQL on Windows by using a combination of cmake and Microsoft Visual Studio .NET 2003 (7.1), Microsoft Visual Studio - 2005 (8.0) or Microsoft Visual C++ 2005 Express Edition. You must - have the appropriate Microsoft Platform SDK installed. + 2005 (8.0), Microsoft Visual Studio 2008 (9.0) or Microsoft Visual + C++ 2005 Express Edition. You must have the appropriate Microsoft + Platform SDK installed. Note To compile from the source code on Windows you must use the - standard source distribution (for example, mysql-5.1.41.tar.gz). + standard source distribution (for example, mysql-5.1.46.tar.gz). You build from the same distribution as used to build MySQL on Unix, Linux and other platforms. Do not use the Windows Source distributions as they do not contain the necessary configuration @@ -5551,8 +5502,19 @@ Note tool that can read .zip files. This directory is the work directory in the following instructions. - 2. Using a command shell, navigate to the work directory and run - the following command: +Note + You must run the commands in the win directory from the + top-level source directory. Do not change into the win + directory, as the commands will not be executed correctly. + + 2. Start a command shell. If you have not configured the PATH and + other environment variables for all command shells, you may be + able to start a command shell from the Start Menu within the + Windows Visual Studio menu that contains the necessary + environment changes. + + 3. Within the command shell, navigate to the work directory and + run the following command: C:\workdir>win\configure.js options If you have associated the .js file extension with an application such as a text editor, then you may need to use @@ -5603,16 +5565,19 @@ C:\workdir>cscript win\configure.js opti C:\workdir>win\configure.js WITH_INNOBASE_STORAGE_ENGINE WITH_PARTITION_STORAGE_ENGINE MYSQL_SERVER_SUFFIX=-pro - 3. From the work directory, execute the win\build-vs8.bat or - win\build-vs71.bat file, depending on the version of Visual + 4. From the work directory, execute the win\build-vs9.bat + (Windows Visual Studio 2008), win\build-vs8.bat (Windows + Visual Studio 2005), or win\build-vs71.bat (Windows Visual + Stidion 2003) script, depending on the version of Visual Studio you have installed. The script invokes CMake, which generates the mysql.sln solution file. - You can also use win\build-vs8_x64.bat to build the 64-bit - version of MySQL. However, you cannot build the 64-bit version - with Visual Studio Express Edition. You must use Visual Studio - 2005 (8.0) or higher. + You can also use the corresponding 64-bit file (for example + win\build-vs8_x64.bat or win\build-vs9_x64.bat) to build the + 64-bit version of MySQL. However, you cannot build the 64-bit + version with Visual Studio Express Edition. You must use + Visual Studio 2005 (8.0) or higher. - 4. From the work directory, open the generated mysql.sln file + 5. From the work directory, open the generated mysql.sln file with Visual Studio and select the proper configuration using the Configuration menu. The menu provides Debug, Release, RelwithDebInfo, MinRelInfo options. Then select Solution > @@ -5621,7 +5586,7 @@ C:\workdir>win\configure.js WITH_INNOBAS important later when you run the test script because that script needs to know which configuration you used. - 5. Test the server. The server built using the preceding + 6. Test the server. The server built using the preceding instructions expects that the MySQL base directory and data directory are C:\mysql and C:\mysql\data by default. If you want to test your server using the source tree root directory @@ -5681,29 +5646,36 @@ C:\> mkdir C:\mysql\sql-bench Installation Notes." 2. From the work directory, copy into the C:\mysql directory the - following directories: + following files and directories: C:\> cd \workdir -C:\workdir> copy client_release\*.exe C:\mysql\bin -C:\workdir> copy client_debug\mysqld.exe C:\mysql\bin\mysqld-debug.ex -e +C:\workdir> mkdir C:\mysql +C:\workdir> mkdir C:\mysql\bin +C:\workdir> copy client\Release\*.exe C:\mysql\bin +C:\workdir> copy sql\Release\mysqld.exe C:\mysql\bin\mysqld.exe C:\workdir> xcopy scripts\*.* C:\mysql\scripts /E C:\workdir> xcopy share\*.* C:\mysql\share /E If you want to compile other clients and link them to MySQL, you should also copy several libraries and header files: -C:\workdir> copy lib_debug\mysqlclient.lib C:\mysql\lib\debug -C:\workdir> copy lib_debug\libmysql.* C:\mysql\lib\debug -C:\workdir> copy lib_debug\zlib.* C:\mysql\lib\debug -C:\workdir> copy lib_release\mysqlclient.lib C:\mysql\lib\opt -C:\workdir> copy lib_release\libmysql.* C:\mysql\lib\opt -C:\workdir> copy lib_release\zlib.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\debug +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\opt +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\opt C:\workdir> copy include\*.h C:\mysql\include C:\workdir> copy libmysql\libmysql.def C:\mysql\include + +Note + If you have compiled a Debug, rather than Release solution, + you can replace Release with Debug in the source file names + shown above. If you want to benchmark MySQL, you should also do this: C:\workdir> xcopy sql-bench\*.* C:\mysql\bench /E After installation, set up and start the server in the same way as - for binary Windows distributions. See Section 2.5, "Installing - MySQL on Windows." + for binary Windows distributions. This includes creating the + system tables by running mysql_install_db. For more information, + see Section 2.5, "Installing MySQL on Windows." 2.5.11. Compiling MySQL Clients on Windows @@ -6626,14 +6598,23 @@ DLTLIB LIB(MYSQLINST) Upgrading an existing MySQL instance - You need to execute the upgrade command, MYSQLINST/UPGMYSQL. You - must specify 6 parameters to perform an upgrade: + You need to execute the upgrade command, MYSQLINST/UPGMYSQL. + +Note + + You cannot use MYSQLINST/UPGMYSQL to upgrade between major + versions of MySQL (for example from 5.0 to 5.1). For information + and advice on migrating between major versions you can use the + advice provided in Section 2.4.1.1, "Upgrading from MySQL 5.0 to + 5.1." + + You must specify 6 parameters to perform an upgrade: * DIR('/QOpenSys/usr/local/') --- sets the installation location for the MySQL files. The directory will be created if it does not already exist. This is the directory that the MySQL server will be installed into, inside a directory with a name - matching the version and release. For example if installing + matching the version and release. For example, if installing MySQL 5.1.39 with the DIR set to /QOpenSys/usr/local/ would result in /QOpenSys/usr/local/mysql-5.1.39-i5os-power64 and a symbolic link to this directory will be created in @@ -7167,7 +7148,7 @@ shell> bin/mysqld_safe --user=mysql & logged in to the system as mysql, in which case you can omit the --user option from the command. Further instructions for running MySQL as an unprivileged user - are given in Section 5.3.5, "How to Run MySQL as a Normal + are given in Section 5.3.6, "How to Run MySQL as a Normal User." If you neglected to create the grant tables before proceeding to this step, the following message appears in the error log @@ -7185,10 +7166,10 @@ shell> bin/mysqladmin variables on your platform and version of MySQL, but should be similar to that shown here: shell> bin/mysqladmin version -mysqladmin Ver 14.12 Distrib 5.1.41, for pc-linux-gnu on i686 +mysqladmin Ver 14.12 Distrib 5.1.46, for pc-linux-gnu on i686 ... -Server version 5.1.41 +Server version 5.1.46 Protocol version 10 Connection Localhost via UNIX socket UNIX socket /var/lib/mysql/mysql.sock @@ -7292,7 +7273,7 @@ shell> mysql -vvf test < ./tests/auto_in The MySQL 5.1 installation procedure creates time zone tables in the mysql database. However, you must populate the tables manually - using the instructions in Section 9.7, "MySQL Server Time Zone + using the instructions in Section 9.6, "MySQL Server Time Zone Support." 2.13.1.1. Problems Running mysql_install_db @@ -7323,7 +7304,7 @@ mysqld ended carefully. The log should be located in the directory XXXXXX named by the error message and should indicate why mysqld didn't start. If you do not understand what happened, include - the log when you post a bug report. See Section 1.6, "How to + the log when you post a bug report. See Section 1.7, "How to Report Bugs or Problems." * There is a mysqld process running @@ -7972,7 +7953,7 @@ Note MYSQL_PS1 The command prompt to use in the mysql command-line client. MYSQL_PWD The default password when connecting to mysqld. Note - that using this is insecure. See Section 5.5.6.2, "End-User + that using this is insecure. See Section 5.3.2.2, "End-User Guidelines for Password Security." MYSQL_TCP_PORT The default TCP/IP port number. MYSQL_UNIX_PORT The default Unix socket file name; used for === modified file 'INSTALL-WIN-SOURCE' --- a/INSTALL-WIN-SOURCE 2009-12-01 07:24:05 +0000 +++ b/INSTALL-WIN-SOURCE 2010-04-28 13:06:11 +0000 @@ -13,10 +13,9 @@ Note from the Bazaar tree. For production use, we do not advise using a MySQL server built by yourself from source. Normally, it is best to use precompiled binary distributions of MySQL that are built - specifically for optimal performance on Windows by Sun - Microsystems, Inc. Instructions for installing binary - distributions are available in Section 2.5, "Installing MySQL on - Windows." + specifically for optimal performance on Windows by Oracle + Corporation. Instructions for installing binary distributions are + available in Section 2.5, "Installing MySQL on Windows." To build MySQL on Windows from source, you must satisfy the following system, compiler, and resource requirements: @@ -76,8 +75,8 @@ Note You also need a MySQL source distribution for Windows, which can be obtained two ways: - * Obtain a source distribution packaged by Sun Microsystems, - Inc. These are available from http://dev.mysql.com/downloads/. + * Obtain a source distribution packaged by Oracle Corporation. + These are available from http://dev.mysql.com/downloads/. * Package a source distribution yourself from the latest Bazaar developer source tree. For instructions on pulling the latest @@ -87,19 +86,20 @@ Note If you find something not working as expected, or you have suggestions about ways to improve the current build process on Windows, please send a message to the win32 mailing list. See - Section 1.5.1, "MySQL Mailing Lists." + Section 1.6.1, "MySQL Mailing Lists." 2.5.10.1. Building MySQL from Source Using CMake and Visual Studio You can build MySQL on Windows by using a combination of cmake and Microsoft Visual Studio .NET 2003 (7.1), Microsoft Visual Studio - 2005 (8.0) or Microsoft Visual C++ 2005 Express Edition. You must - have the appropriate Microsoft Platform SDK installed. + 2005 (8.0), Microsoft Visual Studio 2008 (9.0) or Microsoft Visual + C++ 2005 Express Edition. You must have the appropriate Microsoft + Platform SDK installed. Note To compile from the source code on Windows you must use the - standard source distribution (for example, mysql-5.1.41.tar.gz). + standard source distribution (for example, mysql-5.1.46.tar.gz). You build from the same distribution as used to build MySQL on Unix, Linux and other platforms. Do not use the Windows Source distributions as they do not contain the necessary configuration @@ -113,8 +113,19 @@ Note tool that can read .zip files. This directory is the work directory in the following instructions. - 2. Using a command shell, navigate to the work directory and run - the following command: +Note + You must run the commands in the win directory from the + top-level source directory. Do not change into the win + directory, as the commands will not be executed correctly. + + 2. Start a command shell. If you have not configured the PATH and + other environment variables for all command shells, you may be + able to start a command shell from the Start Menu within the + Windows Visual Studio menu that contains the necessary + environment changes. + + 3. Within the command shell, navigate to the work directory and + run the following command: C:\workdir>win\configure.js options If you have associated the .js file extension with an application such as a text editor, then you may need to use @@ -165,16 +176,19 @@ C:\workdir>cscript win\configure.js opti C:\workdir>win\configure.js WITH_INNOBASE_STORAGE_ENGINE WITH_PARTITION_STORAGE_ENGINE MYSQL_SERVER_SUFFIX=-pro - 3. From the work directory, execute the win\build-vs8.bat or - win\build-vs71.bat file, depending on the version of Visual + 4. From the work directory, execute the win\build-vs9.bat + (Windows Visual Studio 2008), win\build-vs8.bat (Windows + Visual Studio 2005), or win\build-vs71.bat (Windows Visual + Stidion 2003) script, depending on the version of Visual Studio you have installed. The script invokes CMake, which generates the mysql.sln solution file. - You can also use win\build-vs8_x64.bat to build the 64-bit - version of MySQL. However, you cannot build the 64-bit version - with Visual Studio Express Edition. You must use Visual Studio - 2005 (8.0) or higher. + You can also use the corresponding 64-bit file (for example + win\build-vs8_x64.bat or win\build-vs9_x64.bat) to build the + 64-bit version of MySQL. However, you cannot build the 64-bit + version with Visual Studio Express Edition. You must use + Visual Studio 2005 (8.0) or higher. - 4. From the work directory, open the generated mysql.sln file + 5. From the work directory, open the generated mysql.sln file with Visual Studio and select the proper configuration using the Configuration menu. The menu provides Debug, Release, RelwithDebInfo, MinRelInfo options. Then select Solution > @@ -183,7 +197,7 @@ C:\workdir>win\configure.js WITH_INNOBAS important later when you run the test script because that script needs to know which configuration you used. - 5. Test the server. The server built using the preceding + 6. Test the server. The server built using the preceding instructions expects that the MySQL base directory and data directory are C:\mysql and C:\mysql\data by default. If you want to test your server using the source tree root directory @@ -243,26 +257,33 @@ C:\> mkdir C:\mysql\sql-bench Installation Notes." 2. From the work directory, copy into the C:\mysql directory the - following directories: + following files and directories: C:\> cd \workdir -C:\workdir> copy client_release\*.exe C:\mysql\bin -C:\workdir> copy client_debug\mysqld.exe C:\mysql\bin\mysqld-debug.ex -e +C:\workdir> mkdir C:\mysql +C:\workdir> mkdir C:\mysql\bin +C:\workdir> copy client\Release\*.exe C:\mysql\bin +C:\workdir> copy sql\Release\mysqld.exe C:\mysql\bin\mysqld.exe C:\workdir> xcopy scripts\*.* C:\mysql\scripts /E C:\workdir> xcopy share\*.* C:\mysql\share /E If you want to compile other clients and link them to MySQL, you should also copy several libraries and header files: -C:\workdir> copy lib_debug\mysqlclient.lib C:\mysql\lib\debug -C:\workdir> copy lib_debug\libmysql.* C:\mysql\lib\debug -C:\workdir> copy lib_debug\zlib.* C:\mysql\lib\debug -C:\workdir> copy lib_release\mysqlclient.lib C:\mysql\lib\opt -C:\workdir> copy lib_release\libmysql.* C:\mysql\lib\opt -C:\workdir> copy lib_release\zlib.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\debug +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\debug +C:\workdir> copy lib\Release\mysqlclient.lib C:\mysql\lib\opt +C:\workdir> copy lib\Release\libmysql.* C:\mysql\lib\opt +C:\workdir> copy lib\Release\zlib.* C:\mysql\lib\opt C:\workdir> copy include\*.h C:\mysql\include C:\workdir> copy libmysql\libmysql.def C:\mysql\include + +Note + If you have compiled a Debug, rather than Release solution, + you can replace Release with Debug in the source file names + shown above. If you want to benchmark MySQL, you should also do this: C:\workdir> xcopy sql-bench\*.* C:\mysql\bench /E After installation, set up and start the server in the same way as - for binary Windows distributions. See Section 2.5, "Installing - MySQL on Windows." + for binary Windows distributions. This includes creating the + system tables by running mysql_install_db. For more information, + see Section 2.5, "Installing MySQL on Windows." === modified file 'man/comp_err.1' --- a/man/comp_err.1 2009-12-01 07:24:05 +0000 +++ b/man/comp_err.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBcomp_err\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBCOMP_ERR\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBCOMP_ERR\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -63,7 +63,7 @@ shell> \fBcomp_err [\fR\fB\fIoptions\fR\ .\} .PP \fBcomp_err\fR -supports the options described in the following list\&. +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -254,7 +254,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/innochecksum.1' --- a/man/innochecksum.1 2009-12-01 07:24:05 +0000 +++ b/man/innochecksum.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBinnochecksum\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBINNOCHECKSUM\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBINNOCHECKSUM\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -59,7 +59,7 @@ shell> \fBinnochecksum [\fR\fB\fIoptions .\} .PP \fBinnochecksum\fR -supports the options described in the following list\&. For options that refer to page numbers, the numbers are zero\-based\&. +supports the following options\&. For options that refer to page numbers, the numbers are zero\-based\&. .sp .RS 4 .ie n \{\ @@ -141,7 +141,7 @@ Verbose mode; print a progress indicator .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/make_win_bin_dist.1' --- a/man/make_win_bin_dist.1 2009-12-01 07:24:05 +0000 +++ b/man/make_win_bin_dist.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmake_win_bin_dist\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMAKE_WIN_BIN_DIST" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMAKE_WIN_BIN_DIST" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -67,7 +67,7 @@ bin/mysqld\-max\&.exe=\&.\&./my\-max\-bu If you specify a directory, the entire directory will be copied\&. .PP \fBmake_win_bin_dist\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -169,7 +169,7 @@ directories)\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/msql2mysql.1' --- a/man/msql2mysql.1 2009-12-01 07:24:05 +0000 +++ b/man/msql2mysql.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmsql2mysql\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMSQL2MYSQL\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMSQL2MYSQL\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -60,7 +60,7 @@ utility to make the function name substi .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/my_print_defaults.1' --- a/man/my_print_defaults.1 2009-12-01 07:24:05 +0000 +++ b/man/my_print_defaults.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmy_print_defaults\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMY_PRINT_DEFAULTS" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMY_PRINT_DEFAULTS" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -53,7 +53,7 @@ shell> \fBmy_print_defaults mysqlcheck c The output consists of options, one per line, in the form that they would be specified on the command line\&. .PP \fBmy_print_defaults\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -195,7 +195,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisam_ftdump.1' --- a/man/myisam_ftdump.1 2009-12-01 07:24:05 +0000 +++ b/man/myisam_ftdump.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisam_ftdump\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAM_FTDUMP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAM_FTDUMP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -33,7 +33,13 @@ indexes in MyISAM tables\&. It reads the MyISAM -index file directly, so it must be run on the server host where the table is located +index file directly, so it must be run on the server host where the table is located\&. Before using +\fBmyisam_ftdump\fR, be sure to issue a +FLUSH TABLES +statement first if the server is running\&. +.PP +\fBmyisam_ftdump\fR +scans and dumps the entire index, which is not particularly fast\&. On the other hand, the distribution of words changes infrequently, so it need not be run often\&. .PP Invoke \fBmyisam_ftdump\fR @@ -120,6 +126,20 @@ shell> \fBmyisam_ftdump /usr/local/mysql .RE .\} .PP +You can use +\fBmyisam_ftdump\fR +to generate a list of index entries in order of frequency of occurrence like this: +.sp +.if n \{\ +.RS 4 +.\} +.nf +shell> \fBmyisam_ftdump \-c mytexttable 1 | sort \-r\fR +.fi +.if n \{\ +.RE +.\} +.PP \fBmyisam_ftdump\fR supports the following options: .sp @@ -222,7 +242,7 @@ Verbose mode\&. Print more output about .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisamchk.1' --- a/man/myisamchk.1 2009-12-01 07:24:05 +0000 +++ b/man/myisamchk.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisamchk\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAMCHK\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAMCHK\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -219,12 +219,16 @@ CHECK TABLE instead of \fBmyisamchk\fR to check tables\&. See -Section\ \&12.5.2.3, \(lqCHECK TABLE Syntax\(rq\&. +Section\ \&12.4.2.3, \(lqCHECK TABLE Syntax\(rq\&. .sp .5v .RE .PP \fBmyisamchk\fR -supports the options in the following table\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[myisamchk] +option file group\&. +\fBmyisamchk\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .SH "MYISAMCHK GENERAL OPTIONS" .\" options: myisamchk @@ -521,18 +525,7 @@ system variable\&. For more information, myisam_stats_method in Section\ \&5.1.4, \(lqServer System Variables\(rq, and -Section\ \&7.4.7, \(lqMyISAM Index Statistics Collection\(rq\&. For MySQL 5\&.1, -stats_method -was added in MySQL 5\&.0\&.14\&. For older versions, the statistics collection method is equivalent to -nulls_equal\&. -.PP -The -ft_min_word_len -and -ft_max_word_len -variables are available as of MySQL 4\&.0\&.0\&. -ft_stopword_file -is available as of MySQL 4\&.0\&.19\&. +Section\ \&7.4.7, \(lqMyISAM Index Statistics Collection\(rq\&. .PP ft_min_word_len and @@ -824,7 +817,7 @@ file as \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -981,7 +974,7 @@ and \fB\-\-quick\fR, \fB\-q\fR .sp -Achieve a faster repair by not modifying the data file\&. You can specify this option twice to force +Achieve a faster repair by modifying only the index file, not the data file\&. You can specify this option twice to force \fBmyisamchk\fR to modify the original data file in case of duplicate keys\&. .RE @@ -1469,7 +1462,11 @@ The format used to store table rows\&. T Fixed length\&. Other possible values are Compressed and -Packed\&. +Packed\&. (Packed +corresponds to what +SHOW TABLE STATUS +reports as +Dynamic\&.) .RE .sp .RS 4 @@ -1585,7 +1582,7 @@ The number of rows in the table\&. Deleted blocks .sp How many deleted blocks still have reserved space\&. You can optimize your table to minimize this space\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -1613,7 +1610,7 @@ Data records\&. Deleted data .sp How many bytes of unreclaimed deleted data there are\&. You can optimize your table to minimize this space\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -2100,7 +2097,7 @@ The number of rows in the table\&. Deleted blocks .sp How many deleted blocks still have reserved space\&. You can optimize your table to minimize this space\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -2238,7 +2235,7 @@ What percentage of the data file is unus Blocks/Record .sp Average number of blocks per row (that is, how many links a fragmented row is composed of)\&. This is always 1\&.0 for fixed\-format tables\&. This value should stay as close to 1\&.0 as possible\&. If it gets too large, you can reorganize the table\&. See -Section\ \&6.4.4, \(lqTable Optimization\(rq\&. +Section\ \&6.6.4, \(lqMyISAM Table Optimization\(rq\&. .RE .sp .RS 4 @@ -2447,7 +2444,7 @@ instead of .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisamlog.1' --- a/man/myisamlog.1 2009-12-01 07:24:05 +0000 +++ b/man/myisamlog.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisamlog\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAMLOG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAMLOG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -225,7 +225,7 @@ Display version information\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/myisampack.1' --- a/man/myisampack.1 2009-12-01 07:24:05 +0000 +++ b/man/myisampack.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmyisampack\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYISAMPACK\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYISAMPACK\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -100,7 +100,7 @@ to rebuild its indexes\&. \fBmyisamchk\fR(1)\&. .PP \fBmyisampack\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options\&. It also reads option files and supports the options for processing them described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -149,7 +149,7 @@ Make a backup of each table\'s data file \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -832,7 +832,7 @@ option to .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql-stress-test.pl.1' --- a/man/mysql-stress-test.pl.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql-stress-test.pl.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql-stress-test.pl\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQL\-STRESS\-TE" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQL\-STRESS\-TE" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -75,9 +75,9 @@ Display a help message and exit\&. .\} .\" mysql-stress-test.pl: abort-on-error option .\" abort-on-error option: mysql-stress-test.pl -\fB\-\-abort\-on\-error\fR +\fB\-\-abort\-on\-error=\fR\fB\fIN\fR\fR .sp -Unknown\&. +Causes the program to abort if an error with severity less than or equal to N was encountered\&. Set to 1 to abort on any error\&. .RE .sp .RS 4 @@ -169,7 +169,8 @@ program\&. .\" server-database option: mysql-stress-test.pl \fB\-\-server\-database=\fR\fB\fIdb_name\fR\fR .sp -The database to use for the tests\&. +The database to use for the tests\&. The default is +test\&. .RE .sp .RS 4 @@ -333,7 +334,7 @@ option\&. \fB\-\-stress\-init\-file[=\fR\fB\fIpath\fR\fR\fB]\fR .sp \fIfile_name\fR -is the location of the file that contains the list of tests\&. If missing, the default file is +is the location of the file that contains the list of tests to be run once to initialize the database for the testing\&. If missing, the default file is stress_init\&.txt in the test suite directory\&. .RE @@ -464,21 +465,6 @@ The duration of stress testing in second .sp -1 .IP \(bu 2.3 .\} -.\" mysql-stress-test.pl: test-suffix option -.\" test-suffix option: mysql-stress-test.pl -\fB\-\-test\-suffix=\fR\fB\fIstr\fR\fR -.sp -Unknown\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-stress-test.pl: threads option .\" threads option: mysql-stress-test.pl \fB\-\-threads=\fR\fB\fIN\fR\fR @@ -503,7 +489,7 @@ Verbose mode\&. Print more information a .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql-test-run.pl.1' --- a/man/mysql-test-run.pl.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql-test-run.pl.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql-test-run.pl\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQL\-TEST\-RUN\" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQL\-TEST\-RUN\" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -30,8 +30,7 @@ The \fBmysql\-test\-run\&.pl\fR Perl script is the main application used to run the MySQL test suite\&. It invokes \fBmysqltest\fR -to run individual test cases\&. (Prior to MySQL 4\&.1, a similar shell script, -\fBmysql\-test\-run\fR, can be used instead\&.) +to run individual test cases\&. .PP Invoke \fBmysql\-test\-run\&.pl\fR @@ -84,7 +83,7 @@ shell> \fBmysql\-test\-run\&.pl t/mytest .RE .\} .PP -As of MySQL 5\&.1\&.23, a suite name can be given as part of the test name\&. That is, the syntax for naming a test is: +A suite name can be given as part of the test name\&. That is, the syntax for naming a test is: .sp .if n \{\ .RS 4 @@ -98,7 +97,14 @@ As of MySQL 5\&.1\&.23, a suite name can .PP If a suite name is given, \fBmysql\-test\-run\&.pl\fR -looks in that suite for the test\&. With no suite name, +looks in that suite for the test\&. The test file corresponding to a test named +\fIsuite_name\&.test_name\fR +is found in +suite/\fIsuite_name\fR/t/\fItest_name\fR\&.test\&. There is also an implicit suite name +main +for the tests in the top +t +directory\&. With no suite name, \fBmysql\-test\-run\&.pl\fR looks in the default list of suites for a match and runs the test in any suites where it finds the test\&. Suppose that the default suite list is main, @@ -131,11 +137,11 @@ rpl)\&. \fB\-\-skip\-test\fR has the opposite effect of skipping test cases for which the names share a common prefix\&. .PP -As of MySQL 5\&.0\&.54/5\&.1\&.23/6\&.0\&.5, the argument for the +The argument for the \fB\-\-do\-test\fR and \fB\-\-skip\-test\fR -options allows more flexible specification of which tests to perform or skip\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, +options also allows more flexible specification of which tests to perform or skip\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, \fB\-\-do\-test=testa\fR matches tests that begin with testa, @@ -150,7 +156,7 @@ main followed by testa with anything in between\&. In the latter case, the pattern match is not anchored to the beginning of the test name, so it also matches names such as -xmainytestz\&. +xmainytesta\&. .PP To perform setup prior to running tests, \fBmysql\-test\-run\&.pl\fR @@ -160,8 +166,7 @@ with the \fB\-\-bootstrap\fR and \fB\-\-skip\-grant\-tables\fR -options (see -\m[blue]\fBTypical \fBconfigure\fR Options\fR\m[]\&\s-2\u[1]\d\s+2)\&. If MySQL was configured with the +options\&. If MySQL was configured with the \fB\-\-disable\-grant\-options\fR option, \fB\-\-bootstrap\fR, @@ -217,28 +222,11 @@ shell> \fB\&./mysqltest\-run\&.pl \-\-fo .RE .\} .PP -If you have a copy of -\fBmysqld\fR -running on the machine where you want to run the test suite, you do not have to stop it, as long as it is not using ports -9306 -or -9307\&. If either of those ports is taken, you should set the -MTR_BUILD_THREAD -environment variable to an appropriate value, and the test suite will use a different set of ports for master, slave, NDB, and Instance Manager)\&. For example: -.sp -.if n \{\ -.RS 4 -.\} -.nf -shell> \fBexport MTR_BUILD_THREAD=31\fR -shell> \fB\&./mysql\-test\-run\&.pl [\fR\fB\fIoptions\fR\fR\fB] [\fR\fB\fItest_name\fR\fR\fB]\fR -.fi -.if n \{\ -.RE -.\} -.PP \fBmysql\-test\-run\&.pl\fR -defines several environment variables\&. Some of them are listed in the following table\&. +uses several environment variables\&. Some of them are listed in the following table\&. Some of these are set from the outside and used by +\fBmysql\-test\-run\&.pl\fR, others are set by +\fBmysql\-test\-run\&.pl\fR +instead, and may be referred to in tests\&. .TS allbox tab(:); l l @@ -246,6 +234,12 @@ l l l l l l l l +l l +l l +l l +l l +l l +l l l l. T{ \fBVariable\fR @@ -253,15 +247,52 @@ T}:T{ \fBMeaning\fR T} T{ -MYSQL_TEST +MTR_VERSION T}:T{ -Path name to \fBmysqltest\fR binary +If set to 1, will run the older version 1 of + \fBmysql\-test\-run\&.pl\fR\&. This will affect + what functionailty is available and what command line + options are supported\&. T} T{ -MYSQLTEST_VARDIR +MTR_MEM T}:T{ -Path name to the var directory that is used for - logs, temporary files, and so forth +If set to anything, will run tests with files in "memory" using tmpfs or + ramdisk\&. Not available on Windows\&. Same as + \fB\-\-mem\fR option +T} +T{ +MTR_PARALLEL +T}:T{ +If set, defines number of parallel threads executing tests\&. Same as + \fB\-\-parallel\fR option +T} +T{ +MTR_BUILD_THREAD +T}:T{ +If set, defines which port number range is used for the server +T} +T{ +MTR_PORT_BASE +T}:T{ +If set, defines which port number range is used for the server +T} +T{ +MTR_\fINAME\fR_TIMEOUT +T}:T{ +Setting of a timeout in minutes or seconds, corresponding to command + line option + \fB\-\-\fR\fB\fIname\fR\fR\fB\-timeout\fR\&. + Avaliable timeout names are TESTCASE, + SUITE (both in minutes) and + START, SHUTDOWN + (both in seconds)\&. These variables are supported from + MySQL 5\&.1\&.44\&. +T} +T{ +MYSQL_TEST +T}:T{ +Path name to \fBmysqltest\fR binary T} T{ MYSQLD_BOOTSTRAP @@ -269,18 +300,31 @@ T}:T{ Full path name to \fBmysqld\fR that has all options enabled T} T{ -MASTER_MYPORT +MYSQLTEST_VARDIR +T}:T{ +Path name to the var directory that is used for + logs, temporary files, and so forth +T} +T{ +MYSQL_TEST_DIR T}:T{ -??? +Full path to the mysql\-test directory where tests + are being run from T} T{ -MASTER_MYSOCK +MYSQL_TMP_DIR T}:T{ -??? +Path to temp directory used for temporary files during tests T} .TE .sp 1 .PP +The variable +MTR_PORT_BASE +was added in MySQL 5\&.1\&.45 as a more logical replacement for +MTR_BUILD_THREAD\&. It gives the actual port number directly (will be rounded down to a multiple of 10)\&. If you use +MTR_BUILD_THREAD, the port number is found by multiplying this by 10 and adding 10000\&. +.PP Tests sometimes rely on certain environment variables being defined\&. For example, certain tests assume that MYSQL_TEST is defined so that @@ -288,16 +332,23 @@ is defined so that can invoke itself with exec $MYSQL_TEST\&. .PP +Other tests may refer to the last three variables listed in the preceeding table, to locate files to read or write\&. For example, tests that need to create files will typically put them in +$MYSQL_TMP_DIR/\fIfile_name\fR\&. +.PP +If you are running +\fBmysql\-test\-run\&.pl\fR +version 1 by setting +MTR_VERSION, note that this only affects the test driver, not the test client (and its language) or the tests themselves\&. +.PP +A few tests might not run with version 1 because they depend on some feature of version 2\&. You may have those tests skipped by adding the test name to the file +lib/v1/incompatible\&.tests\&. This feature is available from MySQL 5\&.1\&.40\&. +.PP \fBmysql\-test\-run\&.pl\fR supports the options in the following list\&. An argument of \fB\-\-\fR tells \fBmysql\-test\-run\&.pl\fR -not to process any following arguments as options\&. (A description of differences between the options supported by -\fBmysql\-test\-run\&.pl\fR -and -\fBmysql\-test\-run\fR -appears following the list\&.) +not to process any following arguments as options\&. .sp .RS 4 .ie n \{\ @@ -323,11 +374,16 @@ Display a help message and exit\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: bench option -.\" bench option: mysql-test-run.pl -\fB\-\-bench\fR +.\" mysql-test-run.pl: big-test option +.\" big-test option: mysql-test-run.pl +\fB\-\-big\-test\fR .sp -Run the benchmark suite\&. +Allow tests marked as "big" to run\&. Tests can be thus marked by including the line +\fB\-\-source include/big_test\&.inc\fR, and they will only be run if this option is given, or if the environment variable +BIG_TEST +is set to 1\&. +.sp +This is typically done for tests that take very long to run, or that use very much resources, so that they are not suitable for running as part of a normal test suite run\&. .RE .sp .RS 4 @@ -338,12 +394,25 @@ Run the benchmark suite\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: benchdir option -.\" benchdir option: mysql-test-run.pl -\fB\-\-benchdir=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: build-thread option +.\" build-thread option: mysql-test-run.pl +\fB\-\-build\-thread=\fR\fB\fInumber\fR\fR +.sp +Specify a number to calculate port numbers from\&. The formula is 10 * +\fIbuild_thread\fR ++ 10000\&. Instead of a number, it can be set to +auto, which is also the default value, in which case +\fBmysql\-test\-run\&.pl\fR +will allocate a number unique to this host\&. +.sp +The value (number or +auto) can also be set with the +MTR_BUILD_THREAD +environment variable\&. .sp -The directory where the benchmark suite is located\&. The default path is -\&.\&./\&.\&./mysql\-bench\&. +From MySQL 5\&.1\&.45, the more logical +\fB\-\-port\-base\fR +is supported as an alternative\&. .RE .sp .RS 4 @@ -354,14 +423,14 @@ The directory where the benchmark suite .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: big option -.\" big option: mysql-test-run.pl -\fB\-\-big\-test\fR +.\" mysql-test-run.pl: callgrind option +.\" callgrind option: mysql-test-run.pl +\fB\-\-callgrind\fR .sp -Pass the -\fB\-\-big\-test\fR -option to -\fBmysqltest\fR\&. +Instructs +\fBvalgrind\fR +to use +\fBcallgrind\fR\&. .RE .sp .RS 4 @@ -376,7 +445,7 @@ option to .\" check-testcases option: mysql-test-run.pl \fB\-\-check\-testcases\fR .sp -Check test cases for side effects\&. +Check test cases for side effects\&. This is done by checking system state before and after each test case; if there is any difference, a warning to that effect will be written, but the test case will not be marked as failed because of it\&. This check is enabled by default\&. .RE .sp .RS 4 @@ -389,9 +458,9 @@ Check test cases for side effects\&. .\} .\" mysql-test-run.pl: client-bindir option .\" client-bindir option: mysql-test-run.pl -\fB\-\-client\-bindir\fR +\fB\-\-client\-bindir=\fR\fB\fIpath\fR\fR .sp -The path to the directory where client binaries are located\&. This option was added in MySQL 5\&.0\&.66/5\&.1\&.27\&. +The path to the directory where client binaries are located\&. .RE .sp .RS 4 @@ -423,7 +492,7 @@ debugger\&. .\} .\" mysql-test-run.pl: client-debugger option .\" client-debugger option: mysql-test-run.pl -\fB\-\-client\-debugger\fR +\fB\-\-client\-debugger=\fR\fB\fIdebugger\fR\fR .sp Start \fBmysqltest\fR @@ -459,9 +528,9 @@ debugger\&. .\} .\" mysql-test-run.pl: client-libdir option .\" client-libdir option: mysql-test-run.pl -\fB\-\-client\-libdir\fR +\fB\-\-client\-libdir=\fR\fB\fIpath\fR\fR .sp -The path to the directory where client libraries are located\&. This option was added in MySQL 5\&.0\&.66/5\&.1\&.27\&. +The path to the directory where client libraries are located\&. .RE .sp .RS 4 @@ -493,8 +562,6 @@ is to create a combinations file in the suite directory\&. The file should contain a section of options for each test run\&. See Section\ \&4.9, \(lqPassing Options from mysql-test-run.pl to mysqld or mysqltest\(rq\&. -.sp -This option was added in MySQL 5\&.1\&.23/6\&.0\&.4\&. .RE .sp .RS 4 @@ -511,7 +578,8 @@ This option was added in MySQL 5\&.1\&.2 .sp Write \fIstr\fR -to the output\&. +to the output within lines filled with +#, as a form of banner\&. .RE .sp .RS 4 @@ -593,7 +661,7 @@ Dump trace output for all clients and se .\} .\" mysql-test-run.pl: debugger option .\" debugger option: mysql-test-run.pl -\fB\-\-debugger\fR +\fB\-\-debugger=\fR\fB\fIdebugger\fR\fR .sp Start \fBmysqld\fR @@ -627,7 +695,37 @@ does not fail if Debug Sync is not compi For information about using the Debug Sync facility for testing, see Section\ \&4.14, \(lqThread Synchronization in Test Cases\(rq\&. .sp -This option was added in MySQL 5\&.4\&.4/6\&.0\&.6\&. +This option was added in MySQL 5\&.1\&.41/5\&.5\&.0/6\&.0\&.6\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: defaults-file option +.\" default-file option: mysql-test-run.pl +\fB\-\-defaults\-file=\fR\fB\fIfile_name\fR\fR +.sp +Use the named file as fixed config file template for all tests\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: defaults_extra_file option +.\" default_extra_file option: mysql-test-run.pl +\fB\-\-defaults_extra_file=\fR\fB\fIfile_name\fR\fR +.sp +Add setting from the named file to all generated configs\&. .RE .sp .RS 4 @@ -646,9 +744,9 @@ Run all test cases having a name that be \fIprefix\fR value\&. This option provides a convenient way to run a family of similarly named tests\&. .sp -As of MySQL 5\&.0\&.54/5\&.1\&.23/6\&.0\&.5, the argument for the +The argument for the \fB\-\-do\-test\fR -option allows more flexible specification of which tests to perform\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, +option also allows more flexible specification of which tests to perform\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value\&. For example, \fB\-\-do\-test=testa\fR matches tests that begin with testa, @@ -691,6 +789,23 @@ built with the embedded server\&. .sp -1 .IP \(bu 2.3 .\} +.\" mysql-test-run.pl: enable-disabled option +.\" enable-disabled option: mysql-test-run.pl +\fB\-\-enable\-disabled\fR +.sp +Ignore any +disabled\&.def +file, and run also tests marked as disbaled\&. Success or failure of those tests will be reported the same way as other tests\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} .\" mysql-test-run.pl: experimental option .\" experimental option: mysql-test-run.pl \fB\-\-experimental=\fR\fB\fIfile_name\fR\fR @@ -699,7 +814,7 @@ Specify a file that contains a list of t [ exp\-fail ] code rather than [ fail ] -if they fail\&. This option was added in MySQL 5\&.1\&.33/6\&.0\&.11\&. +if they fail\&. This option was added in MySQL 5\&.1\&.33\&. .sp For an example of a file that might be specified via this option, see mysql\-test/collections/default\&.experimental\&. @@ -716,8 +831,25 @@ mysql\-test/collections/default\&.experi .\" mysql-test-run.pl: extern option .\" extern option: mysql-test-run.pl \fB\-\-extern\fR +\fIoption\fR=\fIvalue\fR .sp -Use an already running server\&. +Use an already running server\&. The option/value pair is what is needed by the +\fBmysql\fR +client to connect to the server\&. Each +\fB\-\-extern\fR +can only take one option/value pair as argument, so it you need more you need to repeat +\fB\-\-extern\fR +for each of them\&. Example: +.sp +.if n \{\ +.RS 4 +.\} +.nf + \&./mysql\-test\-run\&.pl \-\-extern socket=var/tmp/mysqld\&.1\&.sock alias +.fi +.if n \{\ +.RE +.\} .sp Note: If a test case has an \&.opt @@ -736,7 +868,8 @@ file that requires the server to be rest .\" fast option: mysql-test-run.pl \fB\-\-fast\fR .sp -Do not clean up from earlier test runs\&. +Do not perform controlled shutdown when servers need to be restarted or at the end of the test run\&. This is equivalent to using +\-\-shutdown\-timeout=0\&. .RE .sp .RS 4 @@ -809,6 +942,8 @@ debugger\&. Run tests with the \fBgprof\fR profiling tool\&. +\fB\-\-gprof\fR +was added in 5\&.1\&.45\&. .RE .sp .RS 4 @@ -819,12 +954,13 @@ profiling tool\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: im-mysqld1-port option -.\" im-mysqld1-port option: mysql-test-run.pl -\fB\-\-im\-mysqld1\-port\fR +.\" mysql-test-run.pl: manual-ddd option +.\" manual-ddd option: mysql-test-run.pl +\fB\-\-manual\-ddd\fR .sp -TCP/IP port number to use for the first -\fBmysqld\fR, controlled by Instance Manager\&. +Use a server that has already been started by the user in the +\fBddd\fR +debugger\&. .RE .sp .RS 4 @@ -835,12 +971,11 @@ TCP/IP port number to use for the first .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: im-mysqld2-port option -.\" im-mysqld2-port option: mysql-test-run.pl -\fB\-\-im\-mysqld2\-port\fR +.\" mysql-test-run.pl: manual-debug option +.\" manual-debug option: mysql-test-run.pl +\fB\-\-manual\-debug\fR .sp -TCP/IP port number to use for the second -\fBmysqld\fR, controlled by Instance Manager\&. +Use a server that has already been started by the user in a debugger\&. .RE .sp .RS 4 @@ -851,12 +986,13 @@ TCP/IP port number to use for the second .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: im-port option -.\" im-port option: mysql-test-run.pl -\fB\-\-im\-port\fR +.\" mysql-test-run.pl: manual-gdb option +.\" manual-gdb option: mysql-test-run.pl +\fB\-\-manual\-gdb\fR .sp -TCP/IP port number to use for -\fBmysqld\fR, controlled by Instance Manager\&. +Use a server that has already been started by the user in the +\fBgdb\fR +debugger\&. .RE .sp .RS 4 @@ -867,14 +1003,12 @@ TCP/IP port number to use for .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: log-warnings option -.\" log-warnings option: mysql-test-run.pl -\fB\-\-log\-warnings\fR +.\" mysql-test-run.pl: mark-progress option +.\" mark-progress option: mysql-test-run.pl +\fB\-\-mark\-progress\fR .sp -Pass the -\fB\-\-log\-warnings\fR -option to -\fBmysqld\fR\&. +Marks progress with timing (in milliseconds) and line number in +var/log/\fItestname\fR\&.progress\&. .RE .sp .RS 4 @@ -885,11 +1019,14 @@ option to .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: manual-debug option -.\" manual-debug option: mysql-test-run.pl -\fB\-\-manual\-debug\fR +.\" mysql-test-run.pl: max-connections option +.\" max-connections option: mysql-test-run.pl +\fB\-\-max\-connections=\fR\fB\fInum\fR\fR .sp -Use a server that has already been started by the user in a debugger\&. +The maximum number of simultaneous server connections that may be used per test\&. If not set, the maximum is 128\&. Minimum allowed limit is 8, maximum is 5120\&. Corresponds to the same option for +\fBmysqltest\fR\&. +.sp +This option is available from MySQL 5\&.1\&.45\&. .RE .sp .RS 4 @@ -900,13 +1037,12 @@ Use a server that has already been start .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: manual-gdb option -.\" manual-gdb option: mysql-test-run.pl -\fB\-\-manual\-gdb\fR +.\" mysql-test-run.pl: max-save-core option +.\" max-save-core option: mysql-test-run.pl +\fB\-\-max\-save\-core=\fR\fB\fIN\fR\fR .sp -Use a server that has already been started by the user in the -\fBgdb\fR -debugger\&. +Limit the number of core files saved, to avoid filling up disks in case of a frequently crashing server\&. Defaults to 5, set to 0 for no limit\&. May also be set with the environment variable +MTR_MAX_SAVE_CORE .RE .sp .RS 4 @@ -917,13 +1053,12 @@ debugger\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: master-binary option -.\" master-binary option: mysql-test-run.pl -\fB\-\-master\-binary=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: max-save-datadir option +.\" max-save-datadir option: mysql-test-run.pl +\fB\-\-max\-save\-datadir=\fR\fB\fIN\fR\fR .sp -Specify the path of the -\fBmysqld\fR -binary to use for master servers\&. +Limit the number of data directories saved after failed tests, to avoid filling up disks in case of frequent failures\&. Defaults to 20, set to 0 for no limit\&. May also be set with the environment variable +MTR_MAX_SAVE_DATADIR .RE .sp .RS 4 @@ -934,11 +1069,12 @@ binary to use for master servers\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: master_port option -.\" master_port option: mysql-test-run.pl -\fB\-\-master_port=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: max-test-fail option +.\" max-test-fail option: mysql-test-run.pl +\fB\-\-max\-test\-fail=\fR\fB\fIN\fR\fR .sp -Specify the TCP/IP port number for the first master server to use\&. Observe that the option name has an underscore and not a dash\&. +Stop execution after the specified number of tests have failed, to avoid using up resources (and time) in case of massive failures\&. retries are noe counted, nor are failures of tests marked experimental\&. Defaults to 10, set to 0 for no limit\&. May also be set with the environment variable +MTR_MAX_TEST_FAIL .RE .sp .RS 4 @@ -953,7 +1089,9 @@ Specify the TCP/IP port number for the f .\" mem option: mysql-test-run.pl \fB\-\-mem\fR .sp -Run the test suite in memory, using tmpfs or ramdisk\&. This can decrease test times significantly\&. +This option is not supported on Windows\&. +.sp +Run the test suite in memory, using tmpfs or ramdisk\&. This can decrease test times significantly, in particular if you would otherwise be running over a remote file system\&. \fBmysql\-test\-run\&.pl\fR attempts to find a suitable location using a built\-in list of standard locations for tmpfs and puts the var @@ -966,7 +1104,14 @@ MTR_MEM[=\fIdir_name\fR]\&. If \fIdir_name\fR is given, it is added to the beginning of the list of locations to search, so it takes precedence over any built\-in locations\&. .sp -This option was added in MySQL 4\&.1\&.22, 5\&.0\&.30, and 5\&.1\&.13\&. +Once you have run tests with +\fB\-\-mem\fR +within a +mysql\-testdirectory, a soflink +var +will have been set up to the temporary directory, and this will be re\-used the next time, until the soflink is deleted\&. Thus, you do not have to repeat the +\fB\-\-mem\fR +option next time\&. .RE .sp .RS 4 @@ -996,25 +1141,6 @@ Section\ \&4.9, \(lqPassing Options from .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: mysqltest option -.\" mysqltest option: mysql-test-run.pl -\fB\-\-mysqltest=\fR\fB\fIvalue\fR\fR -.sp -Extra options to pass to -\fBmysqltest\fR\&. The value should consist of one or more comma\-separated -\fBmysqltest\fR -options\&. See -Section\ \&4.9, \(lqPassing Options from mysql-test-run.pl to mysqld or mysqltest\(rq\&. This option was added in MySQL 6\&.0\&.6\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: ndb-connectstring option .\" ndb-connectstring option: mysql-test-run.pl \fB\-\-ndb\-connectstring=\fR\fB\fIstr\fR\fR @@ -1034,15 +1160,13 @@ from starting a cluster\&. It is assumed .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndb-connectstring-slave option -.\" ndb-connectstring-slave option: mysql-test-run.pl -\fB\-\-ndb\-connectstring\-slave=\fR\fB\fIstr\fR\fR +.\" mysql-test-run.pl: nocheck-testcases option +.\" nocheck-testcases option: mysql-test-run.pl +\fB\-\-nocheck\-testcases\fR .sp -Pass -\fB\-\-ndb\-connectstring=\fR\fB\fIstr\fR\fR -to slave MySQL servers\&. This option also prevents -\fBmysql\-test\-run\&.pl\fR -from starting a cluster\&. It is assumed that there is already a cluster running to which the server can connect with the given connectstring\&. +Disable the check for test case side effects; see +\fB\-\-check\-testcases\fR +for a description\&. .RE .sp .RS 4 @@ -1053,11 +1177,11 @@ from starting a cluster\&. It is assumed .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndb-extra-test option -.\" ndb-extra-test option: mysql-test-run.pl -\fB\-\-ndb\-extra\-test\fR +.\" mysql-test-run.pl: noreorder option +.\" noreorder option: mysql-test-run.pl +\fB\-\-noreorder\fR .sp -Unknown\&. +Do not reorder tests to reduce number of restarts, but run them in exactly the order given\&. If a whole suite is to be run, the tests are run in alphabetical order, though similiar combinations will be grouped together\&. If more than one suite is listed, the tests are run one suite at a time, in the order listed\&. .RE .sp .RS 4 @@ -1068,14 +1192,13 @@ Unknown\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndbcluster-port option -.\" ndbcluster-port option: mysql-test-run.pl -.\" mysql-test-run.pl: ndbcluster_port option -.\" ndbcluster_port option: mysql-test-run.pl -\fB\-\-ndbcluster\-port=\fR\fB\fIport_num\fR\fR, -\fB\-\-ndbcluster_port=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: notimer option +.\" notimer option: mysql-test-run.pl +\fB\-\-notimer\fR .sp -Specify the TCP/IP port number that NDB Cluster should use\&. +Cause +\fBmysqltest\fR +not to generate a timing file\&. The effect of this is that the report from each test case does not include the timing in milliseconds as it normally does\&. .RE .sp .RS 4 @@ -1086,11 +1209,11 @@ Specify the TCP/IP port number that NDB .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: ndbcluster-port-slave option -.\" ndbcluster-port-slave option: mysql-test-run.pl -\fB\-\-ndbcluster\-port\-slave=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: nowarnings option +.\" nowarnings option: mysql-test-run.pl +\fB\-\-nowarnings\fR .sp -Specify the TCP/IP port number that the slave NDB Cluster should use\&. +Do not look for and report errors and warning in the server logs\&. .RE .sp .RS 4 @@ -1101,13 +1224,16 @@ Specify the TCP/IP port number that the .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: netware option -.\" netware option: mysql-test-run.pl -\fB\-\-netware\fR +.\" mysql-test-run.pl: parallel option +.\" parallel option: mysql-test-run.pl +\fB\-\-parallel={\fR\fB\fIN\fR\fR\fB|auto}\fR .sp -Run -\fBmysqld\fR -with options needed on NetWare\&. +Run tests using +\fIN\fR +parallel threads\&. By default, 1 thread is used\&. Use +\fB\-\-parallel=auto\fR +for auto\-setting of +\fIN\fR\&. The auto value was added in MySQL 5\&.1\&.36\&. .RE .sp .RS 4 @@ -1118,13 +1244,24 @@ with options needed on NetWare\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: notimer option -.\" notimer option: mysql-test-run.pl -\fB\-\-notimer\fR +.\" mysql-test-run.pl: port-base option +.\" port-base option: mysql-test-run.pl +\fB\-\-port\-base=\fR\fB\fIP\fR\fR .sp -Cause -\fBmysqltest\fR -not to generate a timing file\&. +Specify base of port numbers to be used; a block of 10 will be allocated\&. +\fIP\fR +should be divisible by 10; if it is not, it will be rounded down\&. If running with more than one parallel test thread, thread 2 will use the next block of 10 and so on\&. +.sp +If the port number is given as +auto, which is also the default, +\fBmysql\-test\-run\&.pl\fRwill allocate a number unique to this host\&. The value may also be given with the environment variable +MTR_PORT_BASE\&. +.sp +\fB\-\-port\-base\fR +was added in MySQL 5\&.1\&.45 as a more logical alternative to +\fB\-\-build\-thread\fR\&. If both are used, +\fB\-\-port\-base\fR +takes presedence\&. .RE .sp .RS 4 @@ -1135,16 +1272,11 @@ not to generate a timing file\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: parallel option -.\" parallel option: mysql-test-run.pl -\fB\-\-parallel={\fR\fB\fIN\fR\fR\fB|auto}\fR +.\" mysql-test-run.pl: print-testcases option +.\" print-testcases option: mysql-test-run.pl +\fB\-\-print\-testcases\fR .sp -Run tests using -\fIN\fR -parallel threads\&. By default, 1 thread is used\&. Use -\fB\-\-parallel=auto\fR -for auto\-setting of -\fIN\fR\&. This option was added in MySQL 5\&.1\&.36\&. +Do not run any tests, but print details about all tests, in the order they would have been run\&. .RE .sp .RS 4 @@ -1195,7 +1327,24 @@ option to .\" reorder option: mysql-test-run.pl \fB\-\-reorder\fR .sp -Reorder tests to minimize the number of server restarts needed\&. +Reorder tests to minimize the number of server restarts needed\&. This is the default behavior\&. There is no guarantee that a particular set of tests will always end up in the same order\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: repeat option +.\" repeat option: mysql-test-run.pl +\fB\-\-repeat=\fR\fB\fIN\fR\fR +.sp +Run each test +\fIN\fR +number of times\&. .RE .sp .RS 4 @@ -1214,8 +1363,33 @@ Display the output of SHOW ENGINES and SHOW VARIABLES\&. This can be used to verify that binaries are built with all required features\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysql-test-run.pl: retry option +.\" retry option: mysql-test-run.pl +\fB\-\-retry=\fR\fB\fIN\fR\fR .sp -This option was added in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.14\&. +If a test fails, it is retried up to a maximum of +\fIN\fR +runs, but will terminate after 2 failures\&. Default is 3, set to 1 or 0 for no retries\&. This option has no effect unless +\fB\-\-force\fR +is also used; without it, test execution will terminate after the first failure\&. +.sp +The +\fB\-\-retry\fR +and +\fB\-\-retry\-failure\fR +options do not affect how many times a test repeated with +\fB\-\-repeat\fR +may fail in total, as each repetition is considered a new test case, which may in turn be retried if it fails\&. .RE .sp .RS 4 @@ -1226,13 +1400,11 @@ This option was added in MySQL 4\&.1\&.2 .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: script-debug option -.\" script-debug option: mysql-test-run.pl -\fB\-\-script\-debug\fR +.\" mysql-test-run.pl: retry-failure option +.\" retry-failure option: mysql-test-run.pl +\fB\-\-retry\-failure=\fR\fB\fIN\fR\fR .sp -Enable debug output for -\fBmysql\-test\-run\&.pl\fR -itself\&. +Allow a failed and retried test to fail more than the default 2 times before giving it up\&. Setting it to 0 or 1 effectively turns off retries .RE .sp .RS 4 @@ -1243,11 +1415,11 @@ itself\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: skip-im option -.\" skip-im option: mysql-test-run.pl -\fB\-\-skip\-im\fR +.\" mysql-test-run.pl: shutdown-timeout option +.\" shutdown-timeout option: mysql-test-run.pl +\fB\-\-shutdown\-timeout=\fR\fB\fISECONDS\fR\fR .sp -Do not start Instance Manager; skip Instance Manager test cases\&. +Max number of seconds to wait for servers to do controlled shutdown before killing them\&. Default is 10\&. .RE .sp .RS 4 @@ -1258,11 +1430,11 @@ Do not start Instance Manager; skip Inst .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: skip-master-binlog option -.\" skip-master-binlog option: mysql-test-run.pl -\fB\-\-skip\-master\-binlog\fR +.\" mysql-test-run.pl: skip-combinations option +.\" skip-combinations option: mysql-test-run.pl +\fB\-\-skip\-combinations\fR .sp -Do not enable master server binary logging\&. +Do not apply combinations; ignore combinations file or option\&. .RE .sp .RS 4 @@ -1324,21 +1496,6 @@ Skip replication test cases\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: skip-slave-binlog option -.\" skip-slave-binlog option: mysql-test-run.pl -\fB\-\-skip\-slave\-binlog\fR -.sp -Do not enable master server binary logging\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: skip-ssl option .\" skip-ssl option: mysql-test-run.pl \fB\-\-skip\-ssl\fR @@ -1362,7 +1519,7 @@ with support for SSL connections\&. .sp Specify a regular expression to be applied to test case names\&. Cases with names that match the expression are skipped\&. tests to skip\&. .sp -As of MySQL 5\&.0\&.54/5\&.1\&.23/6\&.0\&.5, the argument for the +The argument for the \fB\-\-skip\-test\fR option allows more flexible specification of which tests to skip\&. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern\&. See the description of the \fB\-\-do\-test\fR @@ -1393,13 +1550,14 @@ are passed to the master server\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: slave-binary option -.\" slave-binary option: mysql-test-run.pl -\fB\-\-slave\-binary=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: sleep option +.\" sleep option: mysql-test-run.pl +\fB\-\-sleep=\fR\fB\fIN\fR\fR .sp -Specify the path of the -\fBmysqld\fR -binary to use for slave servers\&. +Pass +\fB\-\-sleep=\fR\fB\fIN\fR\fR +to +\fBmysqltest\fR\&. .RE .sp .RS 4 @@ -1410,82 +1568,14 @@ binary to use for slave servers\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: slave_port option -.\" slave_port option: mysql-test-run.pl -\fB\-\-slave_port=\fR\fB\fIport_num\fR\fR +.\" mysql-test-run.pl: sp-protocol option +.\" sp-protocol option: mysql-test-run.pl +\fB\-\-sp\-protocol\fR .sp -Specify the TCP/IP port number for the first master server to use\&. Observe that the option name has an underscore and not a dash\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: sleep option -.\" sleep option: mysql-test-run.pl -\fB\-\-sleep=\fR\fB\fIN\fR\fR -.sp -Pass -\fB\-\-sleep=\fR\fB\fIN\fR\fR -to -\fBmysqltest\fR\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: small-bench option -.\" small-bench option: mysql-test-run.pl -\fB\-\-small\-bench\fR -.sp -Run the benchmarks with the -\fB\-\-small\-tests\fR -and -\fB\-\-small\-tables\fR -options\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: socket option -.\" socket option: mysql-test-run.pl -\fB\-\-socket=\fR\fB\fIfile_name\fR\fR -.sp -For connections to -localhost, the Unix socket file to use, or, on Windows, the name of the named pipe to use\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: sp-protocol option -.\" sp-protocol option: mysql-test-run.pl -\fB\-\-sp\-protocol\fR -.sp -Pass the -\fB\-\-sp\-protocol\fR -option to -\fBmysqltest\fR\&. +Pass the +\fB\-\-sp\-protocol\fR +option to +\fBmysqltest\fR\&. .RE .sp .RS 4 @@ -1520,39 +1610,11 @@ Couldn\'t find support for SSL .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: start option -.\" start option: mysql-test-run.pl -\fB\-\-start\fR -.sp -Initialize and start servers with the startup settings for the first specified test case\&. For example: -.sp -.if n \{\ -.RS 4 -.\} -.nf -shell> \fBcd mysql\-test\fR -shell> \fB\&./mysql\-test\-run\&.pl \-\-start alias &\fR -.fi -.if n \{\ -.RE -.\} -.sp -This option was added in MySQL 5\&.1\&.32/6\&.0\&.11\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: start-and-exit option .\" start-and-exit option: mysql-test-run.pl -\fB\-\-start\-and\-exit\fR +\fB\-\-start\fR .sp -Initialize and start servers with the startup settings for the specified test case or cases, if any, and then exit\&. You can use this option to start a server to which you can connect later\&. For example, after building a source distribution you can start a server and connect to it with the +Initialize and start servers with the startup settings for the specified test case\&. You can use this option to start a server to which you can connect later\&. For example, after building a source distribution you can start a server and connect to it with the \fBmysql\fR client like this: .sp @@ -1561,12 +1623,19 @@ client like this: .\} .nf shell> \fBcd mysql\-test\fR -shell> \fB\&./mysql\-test\-run\&.pl \-\-start\-and\-exit\fR +shell> \fB\&./mysql\-test\-run\&.pl \-\-start alias &\fR shell> \fB\&.\&./mysql \-S \&./var/tmp/master\&.sock \-h localhost \-u root\fR .fi .if n \{\ .RE .\} +.sp +If no tests are named on the command line, the server(s) will be started with settings for the first test that would have been run without the +\fB\-\-start\fR +option\&. +.sp +\fBmysql\-test\-run\&.pl\fR +will stop once the server has been started, but will terminate if the server dies\&. If killed, it will also shut down the server\&. .RE .sp .RS 4 @@ -1581,7 +1650,8 @@ shell> \fB\&.\&./mysql \-S \&./var/tmp/m .\" start-dirty option: mysql-test-run.pl \fB\-\-start\-dirty\fR .sp -Start servers (without initialization) for the specified test case or cases, if any, and then exit\&. You can then manually run the test cases\&. +This is similar to +\fB\-\-start\fR, but will skip the database initialization phase and assume that database files are already available\&. Usually this means you must have run another test first\&. .RE .sp .RS 4 @@ -1627,161 +1697,6 @@ output for .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: stress option -.\" stress option: mysql-test-run.pl -\fB\-\-stress\fR -.sp -Run the stress test\&. The other -\fB\-\-stress\-\fR\fB\fIxxx\fR\fR -options apply in this case\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-init-file option -.\" stress-init-file option: mysql-test-run.pl -\fB\-\-stress\-init\-file=\fR\fB\fIfile_name\fR\fR -.sp -\fIfile_name\fR -is the location of the file that contains the list of tests\&. The default file is -stress_init\&.txt -in the test suite directory\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-loop-count option -.\" stress-loop-count option: mysql-test-run.pl -\fB\-\-stress\-loop\-count=\fR\fB\fIN\fR\fR -.sp -In sequential stress\-test mode, the number of loops to execute before exiting\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-mode option -.\" stress-mode option: mysql-test-run.pl -\fB\-\-stress\-mode=\fR\fB\fImode\fR\fR -.sp -This option indicates the test order in stress\-test mode\&. The -\fImode\fR -value is either -random -to select tests in random order or -seq -to run tests in each thread in the order specified in the test list file\&. The default mode is -random\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-suite option -.\" stress-suite option: mysql-test-run.pl -\fB\-\-stress\-suite=\fR\fB\fIsuite_name\fR\fR -.sp -The name of the test suite to use for stress testing\&. The default suite name is -main -(the regular test suite located in the -mysql\-test -directory)\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-test-count option -.\" stress-test-count option: mysql-test-run.pl -\fB\-\-stress\-test\-count=\fR\fB\fIN\fR\fR -.sp -For stress testing, the number of tests to execute before exiting\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-test-duration option -.\" stress-test-duration option: mysql-test-run.pl -\fB\-\-stress\-test\-duration=\fR\fB\fIN\fR\fR -.sp -For stress testing, the duration of stress testing in seconds\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-test-file option -.\" stress-test-file option: mysql-test-run.pl -\fB\-\-stress\-test\-file=\fR\fB\fIfile_name\fR\fR -.sp -The file that contains the list of tests to use in stress testing\&. The tests should be named without the -\&.test -extension\&. The default file is -stress_tests\&.txt -in the test suite directory\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: stress-threads option -.\" stress-threads option: mysql-test-run.pl -\fB\-\-stress\-threads=\fR\fB\fIN\fR\fR -.sp -The number of threads to use in stress testing\&. The default is 5\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: suite option .\" suite option: mysql-test-run.pl \fB\-\-suite=\fR\fB\fIsuite_name\fR\fR @@ -1831,30 +1746,12 @@ Specify the maximum test case runtime\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: timer option -.\" timer option: mysql-test-run.pl -\fB\-\-timer\fR -.sp -Cause -\fBmysqltest\fR -to generate a timing file\&. The default file is named -\&./var/log/timer\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: tmpdir option -.\" tmpdir option: mysql-test-run.pl -\fB\-\-tmpdir=\fR\fB\fIpath\fR\fR +.\" mysql-test-run.pl: timediff option +.\" timediff option: mysql-test-run.pl +\fB\-\-timediff\fR .sp -The directory where temporary file are stored\&. The default location is -\&./var/tmp\&. +Adds to each test report for a test case, the total time in sconds and milliseconds passed since the preceding test ended\&. This option can only be used together with +\fB\-\-timestamp\fR, and has no effect without it\&. .RE .sp .RS 4 @@ -1865,12 +1762,14 @@ The directory where temporary file are s .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: unified-diff option -.\" unified-diff option: mysql-test-run.pl -\fB\-\-unified\-diff\fR, -\fB\-\-udiff\fR +.\" mysql-test-run.pl: timer option +.\" timer option: mysql-test-run.pl +\fB\-\-timer\fR .sp -Use unified diff format when presenting differences between expected and actual test case results\&. +Cause +\fBmysqltest\fR +to generate a timing file\&. The default file is named +\&./var/log/timer\&. .RE .sp .RS 4 @@ -1881,11 +1780,11 @@ Use unified diff format when presenting .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: use-old-data option -.\" use-old-data option: mysql-test-run.pl -\fB\-\-use\-old\-data\fR +.\" mysql-test-run.pl: timestamp option +.\" timestamp option: mysql-test-run.pl +\fB\-\-timestamp\fR .sp -Do not install the test databases\&. (Use existing ones\&.) +Prints a timestamp before the test case name in each test report line, showing when the test ended\&. .RE .sp .RS 4 @@ -1896,11 +1795,14 @@ Do not install the test databases\&. (Us .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: user-test option -.\" user-test option: mysql-test-run.pl -\fB\-\-user\-test=\fR\fB\fIval\fR\fR +.\" mysql-test-run.pl: tmpdir option +.\" tmpdir option: mysql-test-run.pl +\fB\-\-tmpdir=\fR\fB\fIpath\fR\fR .sp -Unused\&. +The directory where temporary file are stored\&. The default location is +\&./var/tmp\&. The environment variable +MYSQL_TMP_DIR +will be set to the path for this directory, whether it has the default value or has been set explicitly\&. This may be referred to in tests\&. .RE .sp .RS 4 @@ -1935,7 +1837,11 @@ Run and \fBmysqld\fR with -\fBvalgrind\fR\&. +\fBvalgrind\fR\&. Thiks and the following +\fB\-\-valgrind\fR +options require that the executables have been build with +\fBvalgrind\fR +support\&. .RE .sp .RS 4 @@ -1946,16 +1852,13 @@ with .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: valgrind-all option -.\" valgrind-all option: mysql-test-run.pl -\fB\-\-valgrind\-all\fR +.\" mysql-test-run.pl: valgrind-mysqld option +.\" valgrind-mysqld option: mysql-test-run.pl +\fB\-\-valgrind\-mysqld\fR .sp -Like -\fB\-\-valgrind\fR, but passes the -\fB\-\-verbose\fR -and -\fB\-\-show\-reachable\fR -options to +Run the +\fBmysqld\fR +server with \fBvalgrind\fR\&. .RE .sp @@ -1985,30 +1888,9 @@ with .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: valgrind-mysqltest-all option -.\" valgrind-mysqltest-all option: mysql-test-run.pl -\fB\-\-valgrind\-mysqltest\-all\fR -.sp -Like -\fB\-\-valgrind\-mysqltest\fR, but passes the -\fB\-\-verbose\fR -and -\fB\-\-show\-reachable\fR -options to -\fBvalgrind\fR\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysql-test-run.pl: valgrind-options option .\" valgrind-options option: mysql-test-run.pl -\fB\-\-valgrind\-options=\fR\fB\fIstr\fR\fR +\fB\-\-valgrind\-option=\fR\fB\fIstr\fR\fR .sp Extra options to pass to \fBvalgrind\fR\&. @@ -2044,7 +1926,9 @@ executable\&. \fB\-\-vardir=\fR\fB\fIpath\fR\fR .sp Specify the path where files generated during the test run are stored\&. The default location is -\&./var\&. +\&./var\&. The environment variable +MYSQLTEST_VARDIR +will be set to the path for this directory, whether it has the default value or has been set explicitly\&. This may be referred to in tests\&. .RE .sp .RS 4 @@ -2055,14 +1939,11 @@ Specify the path where files generated d .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: view-protocol option -.\" view-protocol option: mysql-test-run.pl -\fB\-\-view\-protocol\fR +.\" mysql-test-run.pl: verbose option +.\" verbose option: mysql-test-run.pl +\fB\-\-verbose\fR .sp -Pass the -\fB\-\-view\-protocol\fR -option to -\fBmysqltest\fR\&. +Give more verbose output regarding test execution\&. Use the option twice to get even more output\&. Note that the output generated within each test case is not affected\&. .RE .sp .RS 4 @@ -2073,13 +1954,11 @@ option to .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: vs-config option -.\" vs-config option: mysql-test-run.pl -\fB\-\-vs\-config=\fR\fB\fIconfig_val\fR\fR +.\" mysql-test-run.pl: verbose-restart option +.\" verbose-restart option: mysql-test-run.pl +\fB\-\-verbose\-restart\fR .sp -Specify the configuration used to build MySQL (for example, -\fB\-\-vs\-config=debug\fR -\fB\-\-vs\-config=release\fR)\&. This option is for Windows only\&. It is available as of MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.14\&. +Write when and why servers are restarted between test cases\&. .RE .sp .RS 4 @@ -2090,11 +1969,14 @@ Specify the configuration used to build .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: wait-timeout option -.\" wait-timeout option: mysql-test-run.pl -\fB\-\-wait\-timeout=\fR\fB\fIN\fR\fR +.\" mysql-test-run.pl: view-protocol option +.\" view-protocol option: mysql-test-run.pl +\fB\-\-view\-protocol\fR .sp -Unused? +Pass the +\fB\-\-view\-protocol\fR +option to +\fBmysqltest\fR\&. .RE .sp .RS 4 @@ -2105,12 +1987,13 @@ Unused? .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: warnings option -.\" warnings option: mysql-test-run.pl -\fB\-\-warnings\fR +.\" mysql-test-run.pl: vs-config option +.\" vs-config option: mysql-test-run.pl +\fB\-\-vs\-config=\fR\fB\fIconfig_val\fR\fR .sp -This option is a synonym for -\fB\-\-log\-warnings\fR\&. +Specify the configuration used to build MySQL (for example, +\fB\-\-vs\-config=debug\fR +\fB\-\-vs\-config=release\fR)\&. This option is for Windows only\&. .RE .sp .RS 4 @@ -2121,11 +2004,17 @@ This option is a synonym for .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: with-ndbcluster option -.\" with-ndbcluster option: mysql-test-run.pl -\fB\-\-with\-ndbcluster\fR +.\" mysql-test-run.pl: wait-all option +.\" wait-all option: mysql-test-run.pl +\fB\-\-wait\-all\fR .sp -Use NDB Cluster and enable test cases that require it\&. +If +\fB\-\-start\fR +or +\fB\-\-start\-dirty\fR +is used, wait for all servers to exit before termination\&. Otherise, it will terminate if one (of several) servers is restarted\&. +.sp +This option was added in MySQL 5\&.1\&.36\&. .RE .sp .RS 4 @@ -2136,11 +2025,12 @@ Use NDB Cluster and enable test cases th .sp -1 .IP \(bu 2.3 .\} -.\" mysql-test-run.pl: with-ndbcluster-all option -.\" with-ndbcluster-all option: mysql-test-run.pl -\fB\-\-with\-ndbcluster\-all\fR +.\" mysql-test-run.pl: warnings option +.\" warnings option: mysql-test-run.pl +\fB\-\-warnings\fR .sp -Use NDB Cluster in all tests\&. +Search the server log for errors or warning after each test and report any suspicious ones; if any are found, the test will be marked as failed\&. This is the default behavior, it may be turned off with +\fB\-\-nowarnings\fR\&. .RE .sp .RS 4 @@ -2159,100 +2049,10 @@ Run only test cases that have ndb in their name\&. .RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: with-ndbcluster-slave option -.\" with-ndbcluster-slave option: mysql-test-run.pl -\fB\-\-with\-ndbcluster\-slave\fR -.sp -Unknown\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysql-test-run.pl: with-openssl option -.\" with-openssl option: mysql-test-run.pl -\fB\-\-with\-openssl\fR -.sp -This option is a synonym for -\fB\-\-ssl\fR\&. -.RE -.if n \{\ -.sp -.\} -.RS 4 -.it 1 an-trap -.nr an-no-space-flag 1 -.nr an-break-flag 1 -.br -.ps +1 -\fBNote\fR -.ps -1 -.br -.PP -\fBmysql\-test\-run\fR -supports the following options not supported by -\fBmysql\-test\-run\&.pl\fR: -\fB\-\-local\fR, -\fB\-\-local\-master\fR, -\fB\-\-ndb\-verbose\fR, -\fB\-\-ndb_mgm\-extra\-opts\fR, -\fB\-\-ndb_mgmd\-extra\-opts\fR, -\fB\-\-ndbd\-extra\-opts\fR, -\fB\-\-old\-master\fR, -\fB\-\-purify\fR, -\fB\-\-use\-old\-data\fR, -\fB\-\-valgrind\-mysqltest\-all\fR\&. -.PP -Conversely, -\fBmysql\-test\-run\&.pl\fR -supports the following options not supported by -\fBmysql\-test\-run\fR: -\fB\-\-benchdir\fR, -\fB\-\-check\-testcases\fR, -\fB\-\-client\-ddd\fR, -\fB\-\-client\-debugger\fR, -\fB\-\-cursor\-protocol\fR, -\fB\-\-debugger\fR, -\fB\-\-im\-mysqld1\-port\fR, -\fB\-\-im\-mysqld2\-port\fR, -\fB\-\-im\-port\fR, -\fB\-\-manual\-debug\fR, -\fB\-\-netware\fR, -\fB\-\-notimer\fR, -\fB\-\-reorder\fR, -\fB\-\-script\-debug\fR, -\fB\-\-skip\-im\fR, -\fB\-\-skip\-ssl\fR, -\fB\-\-sp\-protocol\fR, -\fB\-\-start\-dirty\fR, -\fB\-\-suite\fR, -\fB\-\-suite\-timeout\fR, -\fB\-\-testcase\-timeout\fR, -\fB\-\-udiff\fR, -\fB\-\-unified\-diff\fR,, -\fB\-\-valgrind\-path\fR, -\fB\-\-vardir\fR, -\fB\-\-view\-protocol\fR\&. -.sp .5v -.RE .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -2260,12 +2060,6 @@ This documentation is distributed in the .PP You should have received a copy of the GNU General Public License along with the program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA or see http://www.gnu.org/licenses/. .sp -.SH "NOTES" -.IP " 1." 4 -Typical \fBconfigure\fR Options -.RS 4 -\%http://dev.mysql.com/doc/refman/5.1/en/configure-options.html -.RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, which may already be installed locally and which is also available === modified file 'man/mysql.1' --- a/man/mysql.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -103,7 +103,13 @@ shell> \fBmysql \fR\fB\fIdb_name\fR\fR\f .\" startup parameters: mysql .PP \fBmysql\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysql] +and +[client] +option file groups\&. +\fBmysql\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -182,7 +188,7 @@ option\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -345,7 +351,7 @@ latin1 character set by default\&. You can usually fix such issues by using this option to force the client to use the system character set instead\&. .sp See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq, for more information\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq, for more information\&. .RE .sp .RS 4 @@ -550,10 +556,8 @@ the section called \(lqMYSQL COMMANDS\(r \fB\-\-no\-auto\-rehash\fR, \fB\-A\fR .sp -Deprecated form of -\fB\-skip\-auto\-rehash\fR\&. Use -\fB\-\-disable\-auto\-rehash\fR -instead\&. See the description for +This has the same effect as +\fB\-skip\-auto\-rehash\fR\&. See the description for \fB\-\-auto\-rehash\fR\&. .RE .sp @@ -589,6 +593,8 @@ Do not beep when errors occur\&. Deprecated, use \fB\-\-disable\-named\-commands\fR instead\&. +\fB\-\-no\-named\-commands\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -607,6 +613,8 @@ Deprecated form of \fB\-\-skip\-pager\fR\&. See the \fB\-\-pager\fR option\&. +\fB\-\-no\-pager\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -621,8 +629,12 @@ option\&. .\" no-tee option: mysql \fB\-\-no\-tee\fR .sp -Do not copy output to a file\&. -the section called \(lqMYSQL COMMANDS\(rq, discusses tee files further\&. +Deprecated form of +\fB\-\-skip\-tee\fR\&. See the +\fB\-\-tee\fR +option\&. +\fB\-\-no\-tee\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -684,10 +696,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysql\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -703,7 +717,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -950,9 +964,7 @@ option\&. \fB\-\-skip\-column\-names\fR, \fB\-N\fR .sp -Do not write column names in results\&. The short format, -\fB\-N\fR -is deprecated, use the long format instead\&. +Do not write column names in results\&. .RE .sp .RS 4 @@ -968,9 +980,7 @@ is deprecated, use the long format inste \fB\-\-skip\-line\-numbers\fR, \fB\-L\fR .sp -Do not write line numbers for errors\&. Useful when you want to compare result files that include error messages\&. The short format, -\fB\-L\fR -is deprecated, use the long format instead\&. +Do not write line numbers for errors\&. Useful when you want to compare result files that include error messages\&. .RE .sp .RS 4 @@ -1005,7 +1015,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -1229,7 +1239,7 @@ shell> \fBmysql \-\-xml \-uroot \-e "SHO You can also set the following variables by using \fB\-\-\fR\fB\fIvar_name\fR\fR\fB=\fR\fB\fIvalue\fR\fR\&. The \fB\-\-set\-variable\fR -format is deprecated\&. +format is deprecated and is removed in MySQL 5\&.5\&. .sp .RS 4 .ie n \{\ @@ -1325,7 +1335,7 @@ environment variable\&. The \&.mysql_history should be protected with a restrictive access mode because sensitive information might be written to it, such as the text of SQL statements that contain passwords\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. .PP If you do not want to maintain a history file, first remove \&.mysql_history @@ -2805,7 +2815,7 @@ Section\ \&21.9.11, \(lqControlling Auto .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -2817,7 +2827,7 @@ You should have received a copy of the G .IP " 1." 4 Bug#25946 .RS 4 -\%http://bugs.mysql.com/25946 +\%http://bugs.mysql.com/bug.php?id=25946 .RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, === modified file 'man/mysql.server.1' --- a/man/mysql.server.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql.server.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql.server\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL\&.SERVER\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL\&.SERVER\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -62,7 +62,7 @@ sections, although you should rename suc when using MySQL 5\&.1\&. .PP \fBmysql\&.server\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -176,7 +176,7 @@ The login user name to use for running .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_client_test.1' --- a/man/mysql_client_test.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_client_test.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_client_test\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQL_CLIENT_TEST" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQL_CLIENT_TEST" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -40,10 +40,15 @@ and its test language\&. is similar but used for testing the embedded server\&. Both programs are run as part of the test suite\&. .PP The source code for the programs can be found in in -test/mysql_client_test\&.c +tests/mysql_client_test\&.c in a source distribution\&. The program serves as a good source of examples illustrating how to use various features of the client API\&. .PP \fBmysql_client_test\fR +is used in a test by the same name in the main tests suite of +\fBmysql\-test\-run\&.pl\fR +but may also be run directly\&. Unlike the other programs listed here, it does not read an external description of what tests to run\&. Instead, all tests are coded into the program, which is written to cover all aspects of the C language API\&. +.PP +\fBmysql_client_test\fR supports the following options: .sp .RS 4 @@ -70,10 +75,10 @@ Display a help message and exit\&. .sp -1 .IP \(bu 2.3 .\} -\fB\-b \fR\fB\fIdir_name\fR\fR, +\fB\-\-basedir=\fR\fB\fIdir_name\fR\fR, .\" mysql_client_test: basedir option .\" basedir option: mysql_client_test -\fB\-\-basedir=\fR\fB\fIdir_name\fR\fR +\fB\-b \fR\fB\fIdir_name\fR\fR .sp The base directory for the tests\&. .RE @@ -86,10 +91,10 @@ The base directory for the tests\&. .sp -1 .IP \(bu 2.3 .\} -\fB\-t \fR\fB\fIcount\fR\fR, +\fB\-\-count=\fR\fB\fIcount\fR\fR, .\" mysql_client_test: count option .\" count option: mysql_client_test -\fB\-\-count=\fR\fB\fIcount\fR\fR +\fB\-t \fR\fB\fIcount\fR\fR .sp The number of times to execute the tests\&. .RE @@ -137,10 +142,10 @@ value is .sp -1 .IP \(bu 2.3 .\} -\fB\-g \fR\fB\fIoption\fR\fR, +\fB\-\-getopt\-ll\-test=\fR\fB\fIoption\fR\fR, .\" mysql_client_test: getopt-ll-test option .\" getopt-ll-test option: mysql_client_test -\fB\-\-getopt\-ll\-test=\fR\fB\fIoption\fR\fR +\fB\-g \fR\fB\fIoption\fR\fR .sp Option to use for testing bugs in the getopt @@ -213,10 +218,10 @@ The TCP/IP port number to use for the co .sp -1 .IP \(bu 2.3 .\} -\fB\-A \fR\fB\fIarg\fR\fR, +\fB\-\-server\-arg=\fR\fB\fIarg\fR\fR, .\" mysql_client_test: server-arg option .\" server-arg option: mysql_client_test -\fB\-\-server\-arg=\fR\fB\fIarg\fR\fR +\fB\-A \fR\fB\fIarg\fR\fR .sp Argument to send to the embedded server\&. .RE @@ -229,8 +234,8 @@ Argument to send to the embedded server\ .sp -1 .IP \(bu 2.3 .\} -\fB\-T\fR, -\fB\-\-show\-tests\fR +\fB\-\-show\-tests\fR, +\fB\-T\fR .sp Show all test names\&. .RE @@ -277,12 +282,13 @@ localhost .sp -1 .IP \(bu 2.3 .\} -\fB\-c\fR, -\fB\-\-testcase\fR +\fB\-\-testcase\fR, +\fB\-c\fR .sp -The option may disable some code when run as a -\fBmysql\-test\-run\&.pl\fR -test case\&. +The option is used when called from +\fBmysql\-test\-run\&.pl\fR, so that +\fBmysql_client_test\fR +may optionally behave in a different way than if called manually, for example by skipping some tests\&. Currently, there is no difference in behavior but the option is included in order to make this possible\&. .RE .sp .RS 4 @@ -320,7 +326,7 @@ mysql\-test/var\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_config.1' --- a/man/mysql_config.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_config.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_config\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_CONFIG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_CONFIG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -30,7 +30,7 @@ mysql_config \- get compile options for provides you with useful information for compiling your MySQL client and connecting it to MySQL\&. .PP \fBmysql_config\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -224,7 +224,7 @@ this way, be sure to invoke it within ba .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_convert_table_format.1' --- a/man/mysql_convert_table_format.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_convert_table_format.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_convert_table_format\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_CONVERT_TAB" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_CONVERT_TAB" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -115,10 +115,10 @@ Connect to the MySQL server on the given .\" password option: mysql_convert_table_format \fB\-\-password=\fR\fB\fIpassword\fR\fR .sp -The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. You can use an option file to avoid giving the password on the command line\&. +The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -216,7 +216,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_find_rows.1' --- a/man/mysql_find_rows.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_find_rows.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_find_rows\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_FIND_ROWS\F" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_FIND_ROWS\F" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -151,7 +151,7 @@ Start output from this row\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_fix_extensions.1' --- a/man/mysql_fix_extensions.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_fix_extensions.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_fix_extensions\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_FIX_EXTENSI" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_FIX_EXTENSI" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -61,7 +61,7 @@ shell> \fBmysql_fix_extensions \fR\fB\fI .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_fix_privilege_tables.1' --- a/man/mysql_fix_privilege_tables.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_fix_privilege_tables.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_fix_privilege_tables\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_FIX_PRIVILE" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_FIX_PRIVILE" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -131,7 +131,9 @@ mysql> \fBSOURCE share/mysql_fix_privile .ps -1 .br .PP -Prior to version 5\&.1\&.17, this script is found in the +Prior to version 5\&.1\&.17, the +mysql_fix_privilege_tables\&.sql +script is found in the scripts directory\&. .sp .5v @@ -157,7 +159,7 @@ After running the script, stop the serve .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_install_db.1' --- a/man/mysql_install_db.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_install_db.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_install_db\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_INSTALL_DB\" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_INSTALL_DB\" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -94,8 +94,12 @@ environment variable to the full path na will use that server\&. .PP \fBmysql_install_db\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at -Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. +supports the following options, which can be specified on the command line or in the +[mysql_install_db] +and (if they are common to +\fBmysqld\fR) +[mysqld] +option file groups\&. .sp .RS 4 .ie n \{\ @@ -248,7 +252,7 @@ For internal use\&. This option is used .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_secure_installation.1' --- a/man/mysql_secure_installation.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_secure_installation.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_secure_installation\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_SECURE_INST" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_SECURE_INST" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -96,7 +96,7 @@ The script will prompt you to determine .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_setpermission.1' --- a/man/mysql_setpermission.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_setpermission.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_setpermission\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_SETPERMISSI" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_SETPERMISSI" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -109,10 +109,10 @@ Connect to the MySQL server on the given .\" password option: mysql_setpermission \fB\-\-password=\fR\fB\fIpassword\fR\fR .sp -The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. You can use an option file to avoid giving the password on the command line\&. +The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -163,7 +163,7 @@ The MySQL user name to use when connecti .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_tzinfo_to_sql.1' --- a/man/mysql_tzinfo_to_sql.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_tzinfo_to_sql.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_tzinfo_to_sql\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_TZINFO_TO_S" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_TZINFO_TO_S" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -33,11 +33,11 @@ program loads the time zone tables in th mysql database\&. It is used on systems that have a zoneinfo -database (the set of files describing time zones)\&. Examples of such systems are Linux, FreeBSD, Sun Solaris, and Mac OS X\&. One likely location for these files is the +database (the set of files describing time zones)\&. Examples of such systems are Linux, FreeBSD, Solaris, and Mac OS X\&. One likely location for these files is the /usr/share/zoneinfo directory (/usr/share/lib/zoneinfo on Solaris)\&. If your system does not have a zoneinfo database, you can use the downloadable package described in -Section\ \&9.7, \(lqMySQL Server Time Zone Support\(rq\&. +Section\ \&9.6, \(lqMySQL Server Time Zone Support\(rq\&. .PP \fBmysql_tzinfo_to_sql\fR can be invoked several ways: @@ -113,7 +113,7 @@ After running .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_upgrade.1' --- a/man/mysql_upgrade.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_upgrade.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_upgrade\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_UPGRADE\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_UPGRADE\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -52,6 +52,24 @@ for manual table repair strategies\&. .nr an-break-flag 1 .br .ps +1 +\fBNote\fR +.ps -1 +.br +.PP +On Windows Server 2008 and Windows Vista, you must run +\fBmysql_upgrade\fR +with administrator privileges\&. You can do this by running a Command Prompt as Administrator and running the command\&. Failure to do so may result in the upgrade failing to execute correctly\&. +.sp .5v +.RE +.if n \{\ +.sp +.\} +.RS 4 +.it 1 an-trap +.nr an-no-space-flag 1 +.nr an-break-flag 1 +.br +.ps +1 \fBCaution\fR .ps -1 .br @@ -59,7 +77,7 @@ for manual table repair strategies\&. You should always back up your current MySQL installation \fIbefore\fR performing an upgrade\&. See -Section\ \&6.1, \(lqDatabase Backup Methods\(rq\&. +Section\ \&6.2, \(lqDatabase Backup Methods\(rq\&. .PP Some upgrade incompatibilities may require special handling before you upgrade your MySQL installation and run \fBmysql_upgrade\fR\&. See @@ -132,7 +150,7 @@ FOR UPGRADE option of the CHECK TABLE statement (see -Section\ \&12.5.2.3, \(lqCHECK TABLE Syntax\(rq)\&. +Section\ \&12.4.2.3, \(lqCHECK TABLE Syntax\(rq)\&. .RE .sp .RS 4 @@ -144,7 +162,7 @@ Section\ \&12.5.2.3, \(lqCHECK TABLE Syn .IP \(bu 2.3 .\} \fIfix_priv_tables\fR -represents a script generated interally by +represents a script generated internally by \fBmysql_upgrade\fR that contains SQL statements to upgrade the tables in the mysql @@ -198,15 +216,17 @@ was added as a shell script and worked o is an executable binary and is available on all systems\&. .PP \fBmysql_upgrade\fR -supports the options in the following list\&. It also reads option files (the +supports the following options, which can be specified on the command line or in the [mysql_upgrade] and [client] -groups) and supports the options for processing them described at -Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. Other options are passed to +option file groups\&. Other options are passed to \fBmysqlcheck\fR\&. For example, it might be necessary to specify the \fB\-\-password[=\fR\fB\fIpassword\fR\fR\fB]\fR option\&. +\fBmysql_upgrade\fR +also supports the options for processing option files described at +Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 .ie n \{\ @@ -375,7 +395,7 @@ This option was introduced in MySQL 5\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_waitpid.1' --- a/man/mysql_waitpid.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_waitpid.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_waitpid\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_WAITPID\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_WAITPID\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -120,7 +120,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysql_zap.1' --- a/man/mysql_zap.1 2009-12-01 07:24:05 +0000 +++ b/man/mysql_zap.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysql_zap\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQL_ZAP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQL_ZAP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -116,7 +116,7 @@ Test mode\&. Display information about e .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlaccess.1' --- a/man/mysqlaccess.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlaccess.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlaccess\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLACCESS\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLACCESS\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -54,7 +54,7 @@ shell> \fBmysqlaccess [\fR\fB\fIhost_nam .\} .PP \fBmysqlaccess\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -221,10 +221,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlaccess\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. .RE .sp .RS 4 @@ -319,13 +321,15 @@ Undo the most recent changes to the temp The password to use when connecting to the server as the superuser\&. If you omit the \fIpassword\fR value following the -\fB\-\-password\fR +\fB\-\-spassword\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlaccess\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. .RE .sp .RS 4 @@ -419,7 +423,7 @@ error will occur when you run .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqladmin.1' --- a/man/mysqladmin.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqladmin.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqladmin\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLADMIN\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLADMIN\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -46,7 +46,7 @@ shell> \fBmysqladmin [\fR\fB\fIoptions\f .\} .PP \fBmysqladmin\fR -supports the commands described in the following list\&. Some of the commands take an argument following the command name\&. +supports the following commands\&. Some of the commands take an argument following the command name\&. .sp .RS 4 .ie n \{\ @@ -211,7 +211,7 @@ old\-password \fInew\-password\fR This is like the password command but stores the password using the old (pre\-4\&.1) password\-hashing format\&. (See -Section\ \&5.5.6.3, \(lqPassword Hashing in MySQL\(rq\&.) +Section\ \&5.3.2.3, \(lqPassword Hashing in MySQL\(rq\&.) .RE .sp .RS 4 @@ -304,7 +304,7 @@ statement\&. If the \fB\-\-verbose\fR option is given, the output is like that of SHOW FULL PROCESSLIST\&. (See -Section\ \&12.5.5.31, \(lqSHOW PROCESSLIST Syntax\(rq\&.) +Section\ \&12.4.5.31, \(lqSHOW PROCESSLIST Syntax\(rq\&.) .RE .sp .RS 4 @@ -586,7 +586,13 @@ waits until the server\'s process ID fil .\" startup parameters: mysqladmin .PP \fBmysqladmin\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqladmin] +and +[client] +option file groups\&. +\fBmysqladmin\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -618,7 +624,7 @@ Display a help message and exit\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -720,7 +726,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -794,10 +800,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqladmin\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -813,7 +821,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -940,7 +948,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -1030,7 +1038,7 @@ You can also set the following variables \fB\-\-\fR\fB\fIvar_name\fR\fR\fB=\fR\fB\fIvalue\fR\fR The \fB\-\-set\-variable\fR -format is deprecated\&. syntax: +format is deprecated and is removed in MySQL 5\&.5\&. syntax: .sp .RS 4 .ie n \{\ @@ -1064,7 +1072,7 @@ The maximum number of seconds to wait fo .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlbinlog.1' --- a/man/mysqlbinlog.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlbinlog.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlbinlog\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLBINLOG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLBINLOG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -34,7 +34,7 @@ utility\&. You can also use \fBmysqlbinlog\fR to display the contents of relay log files written by a slave server in a replication setup because relay logs have the same format as binary logs\&. The binary log and relay log are discussed further in Section\ \&5.2.4, \(lqThe Binary Log\(rq, and -Section\ \&16.4.2, \(lqReplication Relay and Status Files\(rq\&. +Section\ \&16.2.2, \(lqReplication Relay and Status Files\(rq\&. .PP Invoke \fBmysqlbinlog\fR @@ -64,18 +64,52 @@ shell> \fBmysqlbinlog binlog\&.0000003\f .\} .PP The output includes events contained in -binlog\&.000003\&. Event information includes the statement, the ID of the server on which it was executed, the timestamp when the statement was executed, how much time it took, and so forth\&. +binlog\&.000003\&. For statement\-based logging, event information includes the SQL statement, the ID of the server on which it was executed, the timestamp when the statement was executed, how much time it took, and so forth\&. For row\-based logging, the event indicates a row change rather than an SQL statement\&. See +Section\ \&16.1.2, \(lqReplication Formats\(rq, for information about logging modes\&. +.PP +Events are preceded by header comments that provide additional information\&. For example: +.sp +.if n \{\ +.RS 4 +.\} +.nf +# at 141 +#100309 9:28:36 server id 123 end_log_pos 245 + Query thread_id=3350 exec_time=11 error_code=0 +.fi +.if n \{\ +.RE +.\} +.PP +In the first line, the number following +at +indicates the starting position of the event in the binary log file\&. +.PP +The second line starts with a date and time indicating when the statement started on the server where the event originated\&. For replication, this timestamp is propagated to slave servers\&. +server id +is the +server_id +value of the server where the event originated\&. +end_log_pos +indicates where the next event starts (that is, it is the end position of the current event + 1)\&. +thread_id +indicates which thread executed the event\&. +exec_time +is the time spent executing the event, on a master server\&. On a slave, it is the difference of the end execution time on the slave minus the beginning execution time on the master\&. The difference serves as an indicator of how much replication lags behind the master\&. +error_code +indicates the result from executing the event\&. Zero means that no error occurred\&. .PP The output from \fBmysqlbinlog\fR can be re\-executed (for example, by using it as input to -\fBmysql\fR) to reapply the statements in the log\&. This is useful for recovery operations after a server crash\&. For other usage examples, see the discussion later in this section\&. +\fBmysql\fR) to redo the statements in the log\&. This is useful for recovery operations after a server crash\&. For other usage examples, see the discussion later in this section and +Section\ \&6.5, \(lqPoint-in-Time (Incremental) Recovery Using the Binary Log\(rq\&. .PP Normally, you use \fBmysqlbinlog\fR to read binary log files directly and apply them to the local MySQL server\&. It is also possible to read binary logs from a remote server by using the \fB\-\-read\-from\-remote\-server\fR -option\&. When you read remote binary logs, the connection parameter options can be given to indicate how to connect to the server\&. These options are +option\&. To read remote binary logs, the connection parameter options can be given to indicate how to connect to the server\&. These options are \fB\-\-host\fR, \fB\-\-password\fR, \fB\-\-port\fR, @@ -86,7 +120,13 @@ option\&. When you read remote binary lo option\&. .PP \fBmysqlbinlog\fR -supports the following options\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlbinlog] +and +[client] +option file groups\&. +\fBmysqlbinlog\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -248,7 +288,7 @@ the section called \(lqMYSQLBINLOG ROW E \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -264,15 +304,140 @@ Section\ \&9.2, \(lqThe Character Set Us \fB\-\-database=\fR\fB\fIdb_name\fR\fR, \fB\-d \fR\fB\fIdb_name\fR\fR .sp -List entries for just this database (local log only)\&. You can only specify one database with this option \- if you specify multiple +This option causes +\fBmysqlbinlog\fR +to output entries from the binary log (local log only) that occur while +\fIdb_name\fR +is been selected as the default database by +USE\&. +.sp +The +\fB\-\-database\fR +option for +\fBmysqlbinlog\fR +is similar to the +\fB\-\-binlog\-do\-db\fR +option for +\fBmysqld\fR, but can be used to specify only one database\&. If +\fB\-\-database\fR +is given multiple times, only the last instance is used\&. +.sp +The effects of this option depend on whether the statement\-based or row\-based logging format is in use, in the same way that the effects of +\fB\-\-binlog\-do\-db\fR +depend on whether statement\-based or row\-based logging is in use\&. +.PP +\fBStatement-based logging\fR. The \fB\-\-database\fR -options, only the last one is used\&. This option forces +option works as follows: +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +While +\fIdb_name\fR +is the default database, statements are output whether they modify tables in +\fIdb_name\fR +or a different database\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +Unless +\fIdb_name\fR +is selected as the default database, statements are not output, even if they modify tables in +\fIdb_name\fR\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +There is an exception for +CREATE DATABASE, +ALTER DATABASE, and +DROP DATABASE\&. The database being +\fIcreated, altered, or dropped\fR +is considered to be the default database when determining whether to output the statement\&. +.RE +.RS 4 +Suppose that the binary log was created by executing these statements using statement\-based\-logging: +.sp +.if n \{\ +.RS 4 +.\} +.nf +INSERT INTO test\&.t1 (i) VALUES(100); +INSERT INTO db2\&.t2 (j) VALUES(200); +USE test; +INSERT INTO test\&.t1 (i) VALUES(101); +INSERT INTO t1 (i) VALUES(102); +INSERT INTO db2\&.t2 (j) VALUES(201); +USE db2; +INSERT INTO test\&.t1 (i) VALUES(103); +INSERT INTO db2\&.t2 (j) VALUES(202); +INSERT INTO t2 (j) VALUES(203); +.fi +.if n \{\ +.RE +.\} +.sp +\fBmysqlbinlog \-\-database=test\fR +does not output the first two +INSERT +statements because there is no default database\&. It outputs the three +INSERT +statements following +USE test, but not the three +INSERT +statements following +USE db2\&. +.sp +\fBmysqlbinlog \-\-database=db2\fR +does not output the first two +INSERT +statements because there is no default database\&. It does not output the three +INSERT +statements following +USE test, but does output the three +INSERT +statements following +USE db2\&. +.PP +\fBRow-based logging\fR. +\fBmysqlbinlog\fR +outputs only entries that change tables belonging to +\fIdb_name\fR\&. The default database has no effect on this\&. Suppose that the binary log just described was created using row\-based logging rather than statement\-based logging\&. +\fBmysqlbinlog \-\-database=test\fR +outputs only those entries that modify +t1 +in the test database, regardless of whether +USE +was issued or what the default database is\&. +If a server is running with +binlog_format +set to +MIXED +and you want it to be possible to use \fBmysqlbinlog\fR -to output entries from the binary log where the default database (that is, the one selected by -USE) is -\fIdb_name\fR\&. Note that this does not replicate cross\-database statements such as -UPDATE \fIsome_db\&.some_table\fR SET foo=\'bar\' -while having selected a different database or no database\&. +with the +\fB\-\-database\fR +option, you must ensure that tables that are modified are in the database selected by +USE\&. (In particular, no cross\-database updates should be used\&.) .if n \{\ .sp .\} @@ -406,7 +571,7 @@ stops if it reads such an event\&. \fB\-H\fR .sp Display a hex dump of the log in comments, as described in -the section called \(lqMYSQLBINLOG HEX DUMP FORMAT\(rq\&. This output can be helpful for replication debugging\&. This option was added in MySQL 5\&.1\&.2\&. +the section called \(lqMYSQLBINLOG HEX DUMP FORMAT\(rq\&. The hex output can be helpful for replication debugging\&. This option was added in MySQL 5\&.1\&.2\&. .RE .sp .RS 4 @@ -482,10 +647,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlbinlog\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -514,12 +681,13 @@ The TCP/IP port number to use for connec .\} .\" mysqlbinlog: position option .\" position option: mysqlbinlog -\fB\-\-position=\fR\fB\fIN\fR\fR, -\fB\-j \fR\fB\fIN\fR\fR +\fB\-\-position=\fR\fB\fIN\fR\fR .sp Deprecated\&. Use \fB\-\-start\-position\fR instead\&. +\fB\-\-position\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -590,7 +758,7 @@ Direct output to the given file\&. .\" server-id option: mysqlbinlog \fB\-\-server\-id=\fR\fB\fIid\fR\fR .sp -Extract only those events created by the server having the given server ID\&. This option is available as of MySQL 5\&.1\&.4\&. +Display only those events created by the server having the given server ID\&. This option is available as of MySQL 5\&.1\&.4\&. .RE .sp .RS 4 @@ -677,7 +845,7 @@ shell> \fBmysqlbinlog \-\-start\-datetim .\} .sp This option is useful for point\-in\-time recovery\&. See -Section\ \&6.2, \(lqExample Backup and Recovery Strategy\(rq\&. +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -690,10 +858,14 @@ Section\ \&6.2, \(lqExample Backup and R .\} .\" mysqlbinlog: start-position option .\" start-position option: mysqlbinlog -\fB\-\-start\-position=\fR\fB\fIN\fR\fR +\fB\-\-start\-position=\fR\fB\fIN\fR\fR, +\fB\-j \fR\fB\fIN\fR\fR .sp Start reading the binary log at the first event having a position equal to or greater than \fIN\fR\&. This option applies to the first log file named on the command line\&. +.sp +This option is useful for point\-in\-time recovery\&. See +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -715,6 +887,9 @@ argument\&. This option is useful for po option for information about the \fIdatetime\fR value\&. +.sp +This option is useful for point\-in\-time recovery\&. See +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -731,6 +906,9 @@ value\&. .sp Stop reading the binary log at the first event having a position equal to or greater than \fIN\fR\&. This option applies to the last log file named on the command line\&. +.sp +This option is useful for point\-in\-time recovery\&. See +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .RE .sp .RS 4 @@ -779,7 +957,7 @@ The MySQL user name to use when connecti \fB\-\-verbose\fR, \fB\-v\fR .sp -Reconstruct row events and display them as commented SQL statements\&. If given twice, the output includes comments to indicate column data types and some metadata\&. This option was added in MySQL 5\&.1\&.28\&. +Reconstruct row events and display them as commented SQL statements\&. If this option is given twice, the output includes comments to indicate column data types and some metadata\&. This option was added in MySQL 5\&.1\&.28\&. .sp For examples that show the effect of \fB\-\-base64\-output\fR @@ -804,33 +982,6 @@ the section called \(lqMYSQLBINLOG ROW E .sp Display version information and exit\&. .RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} -.\" mysqlbinlog: write-binlog option -.\" write-binlog option: mysqlbinlog -\fB\-\-write\-binlog\fR -.sp -This option is enabled by default, so that -ANALYZE TABLE, -OPTIMIZE TABLE, and -REPAIR TABLE -statements generated by -\fBmysqlcheck\fR -are written to the binary log\&. Use -\fB\-\-skip\-write\-binlog\fR -to cause -NO_WRITE_TO_BINLOG -to be added to the statements so that they are not logged\&. Use the -\fB\-\-skip\-write\-binlog\fR -when these statements should not be sent to replication slaves or run when using the binary logs for recovery from backup\&. This option was added in MySQL 5\&.1\&.18\&. -.RE .PP You can also set the following variable by using \fB\-\-\fR\fB\fIvar_name\fR\fR\fB=\fR\fB\fIvalue\fR\fR @@ -854,14 +1005,14 @@ You can pipe the output of \fBmysqlbinlog\fR into the \fBmysql\fR -client to execute the statements contained in the binary log\&. This is used to recover from a crash when you have an old backup (see -Section\ \&6.1, \(lqDatabase Backup Methods\(rq)\&. For example: +client to execute the events contained in the binary log\&. This technique is used to recover from a crash when you have an old backup (see +Section\ \&6.5, \(lqPoint-in-Time (Incremental) Recovery Using the Binary Log\(rq)\&. For example: .sp .if n \{\ .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.000001 | mysql\fR +shell> \fBmysqlbinlog binlog\&.000001 | mysql \-u root \-p\fR .fi .if n \{\ .RE @@ -873,7 +1024,7 @@ Or: .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.[0\-9]* | mysql\fR +shell> \fBmysqlbinlog binlog\&.[0\-9]* | mysql \-u root \-p\fR .fi .if n \{\ .RE @@ -883,12 +1034,25 @@ You can also redirect the output of \fBmysqlbinlog\fR to a text file instead, if you need to modify the statement log first (for example, to remove statements that you do not want to execute for some reason)\&. After editing the file, execute the statements that it contains by using it as input to the \fBmysql\fR -program\&. +program: +.sp +.if n \{\ +.RS 4 +.\} +.nf +shell> \fBmysqlbinlog binlog\&.000001 > tmpfile\fR +shell> \&.\&.\&. \fIedit tmpfile\fR \&.\&.\&. +shell> \fBmysql \-u root \-p < tmpfile\fR +.fi +.if n \{\ +.RE +.\} .PP +When \fBmysqlbinlog\fR -has the +is invoked with the \fB\-\-start\-position\fR -option, which prints only those statements with an offset in the binary log greater than or equal to a given position (the given position must match the start of one event)\&. It also has options to stop and start when it sees an event with a given date and time\&. This enables you to perform point\-in\-time recovery using the +option, it displays only those events with an offset in the binary log greater than or equal to a given position (the given position must match the start of one event)\&. It also has options to stop and start when it sees an event with a given date and time\&. This enables you to perform point\-in\-time recovery using the \fB\-\-stop\-datetime\fR option (to be able to say, for example, \(lqroll forward my databases to how they were today at 10:30 a\&.m\&.\(rq)\&. @@ -900,8 +1064,8 @@ If you have more than one binary log to .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.000001 | mysql # DANGER!!\fR -shell> \fBmysqlbinlog binlog\&.000002 | mysql # DANGER!!\fR +shell> \fBmysqlbinlog binlog\&.000001 | mysql \-u root \-p # DANGER!!\fR +shell> \fBmysqlbinlog binlog\&.000002 | mysql \-u root \-p # DANGER!!\fR .fi .if n \{\ .RE @@ -918,13 +1082,14 @@ process attempts to use the table, the s .PP To avoid problems like this, use a \fIsingle\fR -connection to execute the contents of all binary logs that you want to process\&. Here is one way to do so: +\fBmysql\fR +process to execute the contents of all binary logs that you want to process\&. Here is one way to do so: .sp .if n \{\ .RS 4 .\} .nf -shell> \fBmysqlbinlog binlog\&.000001 binlog\&.000002 | mysql\fR +shell> \fBmysqlbinlog binlog\&.000001 binlog\&.000002 | mysql \-u root \-p\fR .fi .if n \{\ .RE @@ -938,7 +1103,7 @@ Another approach is to write all the log .nf shell> \fBmysqlbinlog binlog\&.000001 > /tmp/statements\&.sql\fR shell> \fBmysqlbinlog binlog\&.000002 >> /tmp/statements\&.sql\fR -shell> \fBmysql \-e "source /tmp/statements\&.sql"\fR +shell> \fBmysql \-u root \-p \-e "source /tmp/statements\&.sql"\fR .fi .if n \{\ .RE @@ -962,10 +1127,10 @@ LOAD DATA INFILE statements to LOAD DATA LOCAL INFILE statements (that is, it adds -LOCAL), both the client and the server that you use to process the statements must be configured to allow +LOCAL), both the client and the server that you use to process the statements must be configured with the LOCAL -capability\&. See -Section\ \&5.3.4, \(lqSecurity Issues with LOAD DATA LOCAL\(rq\&. +capability enabled\&. See +Section\ \&5.3.5, \(lqSecurity Issues with LOAD DATA LOCAL\(rq\&. .if n \{\ .sp .\} @@ -991,7 +1156,9 @@ automatically deleted because they are n .PP The \fB\-\-hexdump\fR -option produces a hex dump of the log contents: +option causes +\fBmysqlbinlog\fR +to produce a hex dump of the binary log contents: .sp .if n \{\ .RS 4 @@ -1029,7 +1196,8 @@ ROLLBACK; .RE .\} .PP -Hex dump output currently contains the following elements\&. This format is subject to change\&. +Hex dump output currently contains the elements in the following list\&. This format is subject to change\&. (For more information about binary log format, see +\m[blue]\fB\%http://forge.mysql.com/wiki/MySQL_Internals_Binary_Log\fR\m[]\&.) .sp .RS 4 .ie n \{\ @@ -1796,7 +1964,7 @@ option can be used to prevent this heade .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -1808,7 +1976,7 @@ You should have received a copy of the G .IP " 1." 4 Bug#42941 .RS 4 -\%http://bugs.mysql.com/42941 +\%http://bugs.mysql.com/bug.php?id=42941 .RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, === modified file 'man/mysqlbug.1' --- a/man/mysqlbug.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlbug.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlbug\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLBUG\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLBUG\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -26,7 +26,7 @@ mysqlbug \- generate bug report \fBmysqlbug\fR .SH "DESCRIPTION" .PP -This program enables you to generate a bug report and send it to Sun Microsystems, Inc\&. It is a shell script and runs on Unix\&. +This program enables you to generate a bug report and send it to Oracle Corporation\&. It is a shell script and runs on Unix\&. .PP The normal way to report bugs is to visit \m[blue]\fB\%http://bugs.mysql.com/\fR\m[], which is the address for our bugs database\&. This database is public and can be browsed and searched by anyone\&. If you log in to the system, you can enter new reports\&. If you have no Web access, you can generate a bug report by using the @@ -62,7 +62,7 @@ will send the report by email\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlcheck.1' --- a/man/mysqlcheck.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlcheck.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlcheck\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLCHECK\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLCHECK\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -37,7 +37,7 @@ client performs table maintenance: It ch Each table is locked and therefore unavailable to other sessions while it is being processed, although for check operations, the table is locked with a READ lock only (see -Section\ \&12.4.5, \(lqLOCK TABLES and UNLOCK TABLES Syntax\(rq, for more information about +Section\ \&12.3.5, \(lqLOCK TABLES and UNLOCK TABLES Syntax\(rq, for more information about READ and WRITE @@ -72,7 +72,7 @@ REPAIR TABLE, ANALYZE TABLE, and OPTIMIZE TABLE in a convenient way for the user\&. It determines which statements to use for the operation you want to perform, and then sends the statements to the server to be executed\&. For details about which storage engines each statement works with, see the descriptions for those statements in -Section\ \&12.5.2, \(lqTable Maintenance Statements\(rq\&. +Section\ \&12.4.2, \(lqTable Maintenance Statements\(rq\&. .PP The MyISAM @@ -135,8 +135,8 @@ There are three general ways to invoke .RS 4 .\} .nf -shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItables\fR\fR\fB]\fR -shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name1\fR\fR\fB [\fR\fB\fIdb_name2\fR\fR\fB \fR\fB\fIdb_name3\fR\fR\fB\&.\&.\&.]\fR +shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItbl_name\fR\fR\fB \&.\&.\&.]\fR +shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name\fR\fR\fB \&.\&.\&.\fR shell> \fBmysqlcheck [\fR\fB\fIoptions\fR\fR\fB] \-\-all\-databases\fR .fi .if n \{\ @@ -188,7 +188,13 @@ T} .sp 1 .PP \fBmysqlcheck\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlcheck] +and +[client] +option file groups\&. +\fBmysqlcheck\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -285,7 +291,7 @@ If a checked table is corrupted, automat \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -444,7 +450,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -598,10 +604,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlcheck\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -617,7 +625,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -734,7 +742,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -824,10 +832,37 @@ Verbose mode\&. Print information about .sp Display version information and exit\&. .RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysqlcheck: write-binlog option +.\" write-binlog option: mysqlcheck +\fB\-\-write\-binlog\fR +.sp +This option is enabled by default, so that +ANALYZE TABLE, +OPTIMIZE TABLE, and +REPAIR TABLE +statements generated by +\fBmysqlcheck\fR +are written to the binary log\&. Use +\fB\-\-skip\-write\-binlog\fR +to cause +NO_WRITE_TO_BINLOG +to be added to the statements so that they are not logged\&. Use the +\fB\-\-skip\-write\-binlog\fR +when these statements should not be sent to replication slaves or run when using the binary logs for recovery from backup\&. This option was added in MySQL 5\&.1\&.18\&. +.RE .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqld.8' --- a/man/mysqld.8 2009-12-01 07:24:05 +0000 +++ b/man/mysqld.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqld\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLD\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLD\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -53,7 +53,7 @@ Chapter\ \&2, Installing and Upgrading M .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqld_multi.1' --- a/man/mysqld_multi.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqld_multi.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqld_multi\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLD_MULTI\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLD_MULTI\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -227,7 +227,7 @@ pass the options to instances, so these techniques are inapplicable\&. .PP \fBmysqld_multi\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -258,6 +258,8 @@ Display a help message and exit\&. .sp As of MySQL 5\&.1\&.18, this option is deprecated\&. If given, it is treated the same way as \fB\-\-defaults\-extra\-file\fR, described earlier\&. +\fB\-\-config\-file\fR +is removed in MySQL 5\&.5\&. .sp Before MySQL 5\&.1\&.18, this option specifies the name of an extra option file\&. It affects where \fBmysqld_multi\fR @@ -534,7 +536,7 @@ use the Unix account for this, unless you \fIknow\fR what you are doing\&. See -Section\ \&5.3.5, \(lqHow to Run MySQL as a Normal User\(rq\&. +Section\ \&5.3.6, \(lqHow to Run MySQL as a Normal User\(rq\&. .sp .5v .RE .RE @@ -726,7 +728,7 @@ Section\ \&4.2.3.3, \(lqUsing Option Fil .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqld_safe.1' --- a/man/mysqld_safe.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqld_safe.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqld_safe\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLD_SAFE\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLD_SAFE\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -842,7 +842,7 @@ file in the data directory\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqldump.1' --- a/man/mysqldump.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqldump.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqldump\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLDUMP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLDUMP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -50,8 +50,8 @@ There are three general ways to invoke .RS 4 .\} .nf -shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItables\fR\fR\fB]\fR -shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name1\fR\fR\fB [\fR\fB\fIdb_name2\fR\fR\fB \fR\fB\fIdb_name3\fR\fR\fB\&.\&.\&.]\fR +shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \fR\fB\fIdb_name\fR\fR\fB [\fR\fB\fItbl_name\fR\fR\fB \&.\&.\&.]\fR +shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \-\-databases \fR\fB\fIdb_name\fR\fR\fB \&.\&.\&.\fR shell> \fBmysqldump [\fR\fB\fIoptions\fR\fR\fB] \-\-all\-databases\fR .fi .if n \{\ @@ -69,36 +69,70 @@ option, entire databases are dumped\&. \fBmysqldump\fR does not dump the INFORMATION_SCHEMA -database\&. If you name that database explicitly on the command line, +database by default\&. As of MySQL 5\&.1\&.38, \fBmysqldump\fR -silently ignores it\&. +dumps +INFORMATION_SCHEMA +if you name it explicitly on the command line, although currently you must also use the +\fB\-\-skip\-lock\-tables\fR +option\&. Before 5\&.1\&.38, +\fBmysqldump\fR +silently ignores +INFORMATION_SCHEMA +even if you name it explicitly on the command line\&. .PP -To get a list of the options your version of +To see a list of the options your version of \fBmysqldump\fR supports, execute \fBmysqldump \-\-help\fR\&. .PP Some \fBmysqldump\fR -options are shorthand for groups of other options\&. -\fB\-\-opt\fR -and -\fB\-\-compact\fR -fall into this category\&. For example, use of +options are shorthand for groups of other options: +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +Use of \fB\-\-opt\fR is the same as specifying -\fB\-\-add\-drop\-table\fR -\fB\-\-add\-locks\fR -\fB\-\-create\-options\fR -\fB\-\-disable\-keys\fR -\fB\-\-extended\-insert\fR -\fB\-\-lock\-tables\fR -\fB\-\-quick\fR -\fB\-\-set\-charset\fR\&. Note that all of the options that +\fB\-\-add\-drop\-table\fR, +\fB\-\-add\-locks\fR, +\fB\-\-create\-options\fR, +\fB\-\-disable\-keys\fR, +\fB\-\-extended\-insert\fR, +\fB\-\-lock\-tables\fR, +\fB\-\-quick\fR, and +\fB\-\-set\-charset\fR\&. All of the options that \fB\-\-opt\fR stands for also are on by default because \fB\-\-opt\fR is on by default\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +Use of +\fB\-\-compact\fR +is the same as specifying +\fB\-\-skip\-add\-drop\-table\fR, +\fB\-\-skip\-add\-locks\fR, +\fB\-\-skip\-comments\fR, +\fB\-\-skip\-disable\-keys\fR, and +\fB\-\-skip\-set\-charset\fR +options\&. +.RE .PP To reverse the effect of a group option, uses its \fB\-\-skip\-\fR\fB\fIxxx\fR\fR @@ -118,10 +152,10 @@ To select the effect of \fB\-\-opt\fR except for some features, use the \fB\-\-skip\fR -option for each feature\&. For example, to disable extended inserts and memory buffering, use +option for each feature\&. To disable extended inserts and memory buffering, use \fB\-\-opt\fR \fB\-\-skip\-extended\-insert\fR -\fB\-\-skip\-quick\fR\&. (As of MySQL 5\&.1, +\fB\-\-skip\-quick\fR\&. (Actually, \fB\-\-skip\-extended\-insert\fR \fB\-\-skip\-quick\fR is sufficient because @@ -161,7 +195,7 @@ option (or \fB\-\-quick\fR)\&. The \fB\-\-opt\fR option (and hence -\fB\-\-quick\fR) is enabled by default in MySQL 5\&.1; to enable memory buffering, use +\fB\-\-quick\fR) is enabled by default, so to enable memory buffering, use \fB\-\-skip\-quick\fR\&. .PP If you are using a recent version of @@ -187,12 +221,18 @@ instead\&. .br .PP \fBmysqldump\fR -from the MySQL 5\&.1\&.21 distribution cannot be used to create dumps from MySQL server versions 5\&.1\&.20 and older\&. This issue is fixed in MySQL 5\&.1\&.22\&. (\m[blue]\fBBug#30123\fR\m[]\&\s-2\u[1]\d\s+2) +from MySQL 5\&.1\&.21 cannot be used to create dumps from MySQL server 5\&.1\&.20 and older\&. This issue is fixed in MySQL 5\&.1\&.22\&. (\m[blue]\fBBug#30123\fR\m[]\&\s-2\u[1]\d\s+2) .sp .5v .RE .PP \fBmysqldump\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqldump] +and +[client] +option file groups\&. +\fBmysqldump\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -227,7 +267,13 @@ Add a DROP DATABASE statement before each CREATE DATABASE -statement\&. +statement\&. This option is typically used in conjunction with the +\fB\-\-all\-databases\fR +or +\fB\-\-databases\fR +option because no +CREATE DATABASE +statements are written unless one of those options is specified\&. .RE .sp .RS 4 @@ -336,7 +382,7 @@ Allow creation of column names that are \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -368,7 +414,7 @@ Write additional information in the dump .\" compact option: mysqldump \fB\-\-compact\fR .sp -Produce less verbose output\&. This option enables the +Produce more compact output\&. This option enables the \fB\-\-skip\-add\-drop\-table\fR, \fB\-\-skip\-add\-locks\fR, \fB\-\-skip\-comments\fR, @@ -387,7 +433,7 @@ options\&. \fBNote\fR .ps -1 .br -Prior to release 5\&.1\&.21, this option did not create valid SQL if the database dump contained views\&. The recreation of views requires the creation and removal of temporary tables and this option suppressed the removal of those temporary tables\&. As a workaround, use +Prior to MySQL 5\&.1\&.21, this option did not create valid SQL if the database dump contained views\&. The recreation of views requires the creation and removal of temporary tables and this option suppressed the removal of those temporary tables\&. As a workaround, use \fB\-\-compact\fR with the \fB\-\-add\-drop\-table\fR @@ -409,7 +455,7 @@ option and then manually adjust the dump \fB\-\-compatible=\fR\fB\fIname\fR\fR .sp Produce output that is more compatible with other database systems or with older MySQL servers\&. The value of -name +\fIname\fR can be ansi, mysql323, @@ -569,7 +615,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. If no character set is specified, +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. If no character set is specified, \fBmysqldump\fR uses utf8, and earlier versions use @@ -611,7 +657,9 @@ statements\&. .\" delete-master-logs option: mysqldump \fB\-\-delete\-master\-logs\fR .sp -On a master replication server, delete the binary logs after performing the dump operation\&. This option automatically enables +On a master replication server, delete the binary logs by sending a +PURGE BINARY LOGS +statement to the server after performing the dump operation\&. This option automatically enables \fB\-\-master\-data\fR\&. .RE .sp @@ -651,12 +699,23 @@ tables\&. .\" dump-date option: mysqldump \fB\-\-dump\-date\fR .sp +If the +\fB\-\-comments\fR +option is given, \fBmysqldump\fR -produces a +produces a comment at the end of the dump of the following form: +.sp +.if n \{\ +.RS 4 +.\} +.nf \-\- Dump completed on \fIDATE\fR -comment at the end of the dump if the -\fB\-\-comments\fR -option is given\&. However, the date causes dump files for identical data take at different times to appear to be different\&. +.fi +.if n \{\ +.RE +.\} +.sp +However, the date causes dump files taken at different times to appear to be different, even if the data are otherwise identical\&. \fB\-\-dump\-date\fR and \fB\-\-skip\-dump\-date\fR @@ -680,7 +739,7 @@ suppresses date printing\&. This option \fB\-\-events\fR, \fB\-E\fR .sp -Dump events from the dumped databases\&. This option was added in MySQL 5\&.1\&.8\&. +Include Event Scheduler events for the dumped databases in the output\&. This option was added in MySQL 5\&.1\&.8\&. .RE .sp .RS 4 @@ -725,8 +784,10 @@ lists\&. This results in a smaller dump \fB\-\-fields\-escaped\-by=\&.\&.\&.\fR .sp These options are used with the -\fB\-T\fR -option and have the same meaning as the corresponding clauses for +\fB\-\-tab\fR +option and have the same meaning as the corresponding +FIELDS +clauses for LOAD DATA INFILE\&. See Section\ \&12.2.6, \(lqLOAD DATA INFILE Syntax\(rq\&. .RE @@ -741,11 +802,13 @@ Section\ \&12.2.6, \(lqLOAD DATA INFILE .\} .\" mysqldump: first-slave option .\" first-slave option: mysqldump -\fB\-\-first\-slave\fR, -\fB\-x\fR +\fB\-\-first\-slave\fR .sp -Deprecated\&. Now renamed to -\fB\-\-lock\-all\-tables\fR\&. +Deprecated\&. Use +\fB\-\-lock\-all\-tables\fR +instead\&. +\fB\-\-first\-slave\fR +is removed in MySQL 5\&.5\&. .RE .sp .RS 4 @@ -763,10 +826,9 @@ Deprecated\&. Now renamed to .sp Flush the MySQL server log files before starting the dump\&. This option requires the RELOAD -privilege\&. Note that if you use this option in combination with the +privilege\&. If you use this option in combination with the \fB\-\-all\-databases\fR -(or -\fB\-A\fR) option, the logs are flushed +option, the logs are flushed \fIfor each database dumped\fR\&. The exception is when using \fB\-\-lock\-all\-tables\fR or @@ -790,9 +852,9 @@ or .\" flush-privileges option: mysqldump \fB\-\-flush\-privileges\fR .sp -Emit a +Send a FLUSH PRIVILEGES -statement after dumping the +statement to the server after dumping the mysql database\&. This option should be used any time the dump contains the mysql @@ -861,8 +923,9 @@ Dump binary columns using hexadecimal no becomes 0x616263)\&. The affected data types are BINARY, -VARBINARY, -BLOB, and +VARBINARY, the +BLOB +types, and BIT\&. .RE .sp @@ -894,10 +957,10 @@ Do not dump the given table, which must \fB\-\-insert\-ignore\fR .sp Write +INSERT IGNORE +statements rather than INSERT -statements with the -IGNORE -option\&. +statements\&. .RE .sp .RS 4 @@ -913,8 +976,10 @@ option\&. \fB\-\-lines\-terminated\-by=\&.\&.\&.\fR .sp This option is used with the -\fB\-T\fR -option and has the same meaning as the corresponding clause for +\fB\-\-tab\fR +option and has the same meaning as the corresponding +LINES +clause for LOAD DATA INFILE\&. See Section\ \&12.2.6, \(lqLOAD DATA INFILE Syntax\(rq\&. .RE @@ -951,18 +1016,20 @@ and \fB\-\-lock\-tables\fR, \fB\-l\fR .sp -Lock all tables before dumping them\&. The tables are locked with +For each dumped database, lock all tables to be dumped before dumping them\&. The tables are locked with READ LOCAL to allow concurrent inserts in the case of MyISAM tables\&. For transactional tables such as InnoDB, \fB\-\-single\-transaction\fR -is a much better option, because it does not need to lock the tables at all\&. +is a much better option than +\fB\-\-lock\-tables\fR +because it does not need to lock the tables at all\&. .sp -Please note that when dumping multiple databases, +Because \fB\-\-lock\-tables\fR -locks tables for each database separately\&. Therefore, this option does not guarantee that the tables in the dump file are logically consistent between databases\&. Tables in different databases may be dumped in completely different states\&. +locks tables for each database separately, this option does not guarantee that the tables in the dump file are logically consistent between databases\&. Tables in different databases may be dumped in completely different states\&. .RE .sp .RS 4 @@ -977,7 +1044,7 @@ locks tables for each database separatel .\" log-error option: mysqldump \fB\-\-log\-error=\fR\fB\fIfile_name\fR\fR .sp -Append warnings and errors to the named file\&. This option was added in MySQL 5\&.1\&.18\&. +Log warnings and errors by appending them to the named file\&. The default is to do no logging\&. This option was added in MySQL 5\&.1\&.18\&. .RE .sp .RS 4 @@ -994,11 +1061,11 @@ Append warnings and errors to the named .sp Use this option to dump a master replication server to produce a dump file that can be used to set up another server as a slave of the master\&. It causes the dump output to include a CHANGE MASTER TO -statement that indicates the binary log coordinates (file name and position) of the dumped server\&. These are the master server coordinates from which the slave should start replicating\&. +statement that indicates the binary log coordinates (file name and position) of the dumped server\&. These are the master server coordinates from which the slave should start replicating after you load the dump file into the slave\&. .sp If the option value is 2, the CHANGE MASTER TO -statement is written as an SQL comment, and thus is informative only; it has no effect when the dump file is reloaded\&. If the option value is 1, the statement takes effect when the dump file is reloaded\&. If the option value is not specified, the default value is 1\&. +statement is written as an SQL comment, and thus is informative only; it has no effect when the dump file is reloaded\&. If the option value is 1, the statement is not written as a comment and takes effect when the dump file is reloaded\&. If no option value is specified, the default value is 1\&. .sp This option requires the RELOAD @@ -1045,7 +1112,16 @@ mysql> \fBSHOW SLAVE STATUS;\fR .sp -1 .IP " 2." 4.2 .\} -From the output of the SHOW SLAVE STATUS statement, get the binary log coordinates of the master server from which the new slave should start replicating\&. These coordinates are the values of the Relay_Master_Log_File and Exec_Master_Log_Pos values\&. Denote those values as file_name and file_pos\&. +From the output of the +SHOW SLAVE STATUS +statement, the binary log coordinates of the master server from which the new slave should start replicating are the values of the +Relay_Master_Log_File +and +Exec_Master_Log_Pos +fields\&. Denote those values as +\fIfile_name\fR +and +\fIfile_pos\fR\&. .RE .sp .RS 4 @@ -1098,7 +1174,7 @@ mysql> \fBSTART SLAVE;\fR .sp -1 .IP " 5." 4.2 .\} -On the new slave, reload the dump file: +On the new slave, load the dump file: .sp .if n \{\ .RS 4 @@ -1126,7 +1202,7 @@ On the new slave, set the replication co .\} .nf mysql> \fBCHANGE MASTER TO\fR - \-> \fBMASTER_LOG_FILE = \'file_name\', MASTER_LOG_POS = file_pos;\fR + \-> \fBMASTER_LOG_FILE = \'\fR\fB\fIfile_name\fR\fR\fB\', MASTER_LOG_POS = \fR\fB\fIfile_pos\fR\fR\fB;\fR .fi .if n \{\ .RE @@ -1214,9 +1290,9 @@ statements that re\-create each dumped t \fB\-\-no\-data\fR, \fB\-d\fR .sp -Do not write any table row information (that is, do not dump table contents)\&. This is very useful if you want to dump only the +Do not write any table row information (that is, do not dump table contents)\&. This is useful if you want to dump only the CREATE TABLE -statement for the table\&. +statement for the table (for example, to create an empty copy of the table by loading the dump file)\&. .RE .sp .RS 4 @@ -1229,11 +1305,11 @@ statement for the table\&. .\} .\" mysqldump: no-set-names option .\" no-set-names option: mysqldump -\fB\-\-no\-set\-names\fR +\fB\-\-no\-set\-names\fR, +\fB\-N\fR .sp -This option is deprecated\&. Use -\fB\-\-skip\-set\-charset\fR -instead\&. +This has the same effect as +\fB\-\-skip\-set\-charset\fR\&. .RE .sp .RS 4 @@ -1248,7 +1324,7 @@ instead\&. .\" opt option: mysqldump \fB\-\-opt\fR .sp -This option is shorthand; it is the same as specifying +This option is shorthand\&. It is the same as specifying \fB\-\-add\-drop\-table\fR \fB\-\-add\-locks\fR \fB\-\-create\-options\fR @@ -1259,7 +1335,7 @@ This option is shorthand; it is the same \fB\-\-set\-charset\fR\&. It should give you a fast dump operation and produce a dump file that can be reloaded into a MySQL server quickly\&. .sp \fIThe \fR\fI\fB\-\-opt\fR\fR\fI option is enabled by default\&. Use \fR\fI\fB\-\-skip\-opt\fR\fR\fI to disable it\&.\fR -See the discussion at the beginning of this section for information about selectively enabling or disabling certain of the options affected by +See the discussion at the beginning of this section for information about selectively enabling or disabling a subset of the options affected by \fB\-\-opt\fR\&. .RE .sp @@ -1275,11 +1351,11 @@ See the discussion at the beginning of t .\" order-by-primary option: mysqldump \fB\-\-order\-by\-primary\fR .sp -Sort each table\'s rows by its primary key, or by its first unique index, if such an index exists\&. This is useful when dumping a +Dump each table\'s rows sorted by its primary key, or by its first unique index, if such an index exists\&. This is useful when dumping a MyISAM table to be loaded into an InnoDB -table, but will make the dump itself take considerably longer\&. +table, but will make the dump operation take considerably longer\&. .RE .sp .RS 4 @@ -1303,10 +1379,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqldump\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -1322,7 +1400,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -1388,11 +1466,11 @@ to retrieve rows for a table from the se \fB\-\-quote\-names\fR, \fB\-Q\fR .sp -Quote database, table, and column names within +Quote identifiers (such as database, table, and column names) within \(lq`\(rq characters\&. If the ANSI_QUOTES -SQL mode is enabled, names are quoted within +SQL mode is enabled, identifiers are quoted within \(lq"\(rq characters\&. This option is enabled by default\&. It can be disabled with \fB\-\-skip\-quote\-names\fR, but this option should be given after any option such as @@ -1417,7 +1495,7 @@ Write REPLACE statements rather than INSERT -statements\&. Available as of MySQL 5\&.1\&.3\&. +statements\&. This option was added in MySQL 5\&.1\&.3\&. .RE .sp .RS 4 @@ -1437,7 +1515,7 @@ Direct output to a given file\&. This op \(lq\en\(rq characters from being converted to \(lq\er\en\(rq -carriage return/newline sequences\&. The result file is created and its contents overwritten, even if an error occurs while generating the dump\&. The previous contents are lost\&. +carriage return/newline sequences\&. The result file is created and its previous contents overwritten, even if an error occurs while generating the dump\&. .RE .sp .RS 4 @@ -1453,7 +1531,7 @@ carriage return/newline sequences\&. The \fB\-\-routines\fR, \fB\-R\fR .sp -Dump stored routines (procedures and functions) from the dumped databases\&. Use of this option requires the +Included stored routines (procedures and functions) for the dumped databases in the output\&. Use of this option requires the SELECT privilege for the mysql\&.proc @@ -1511,9 +1589,9 @@ statement, use .\" single-transaction option: mysqldump \fB\-\-single\-transaction\fR .sp -This option issues a -BEGIN -SQL statement before dumping data from the server\&. It is useful only with transactional tables such as +This option sends a +START TRANSACTION +SQL statement to the server before dumping data\&. It is useful only with transactional tables such as InnoDB, because then it dumps the consistent state of the database at the time when BEGIN was issued without blocking any applications\&. @@ -1528,16 +1606,25 @@ tables dumped while using this option ma .sp While a \fB\-\-single\-transaction\fR -dump is in process, to ensure a valid dump file (correct table contents and binary log position), no other connection should use the following statements: +dump is in process, to ensure a valid dump file (correct table contents and binary log coordinates), no other connection should use the following statements: ALTER TABLE, +CREATE TABLE, DROP TABLE, RENAME TABLE, TRUNCATE TABLE\&. A consistent read is not isolated from those statements, so use of them on a table to be dumped can cause the SELECT -performed by +that is performed by \fBmysqldump\fR to retrieve the table contents to obtain incorrect contents or fail\&. .sp +The +\fB\-\-single\-transaction\fR +option and the +\fB\-\-lock\-tables\fR +option are mutually exclusive because +LOCK TABLES +causes any pending transactions to be committed implicitly\&. +.sp This option is not supported for MySQL Cluster tables; the results cannot be guaranteed to be consistent due to the fact that the NDBCLUSTER storage engine supports only the @@ -1546,15 +1633,9 @@ transaction isolation level\&. You shoul NDB backup and restore instead\&. .sp -The +To dump large tables, you should combine the \fB\-\-single\-transaction\fR -option and the -\fB\-\-lock\-tables\fR -option are mutually exclusive, because -LOCK TABLES -causes any pending transactions to be committed implicitly\&. -.sp -To dump large tables, you should combine this option with +option with \fB\-\-quick\fR\&. .RE .sp @@ -1624,7 +1705,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -1640,29 +1721,15 @@ Section\ \&5.5.7.3, \(lqSSL Command Opti \fB\-\-tab=\fR\fB\fIpath\fR\fR, \fB\-T \fR\fB\fIpath\fR\fR .sp -Produce tab\-separated data files\&. For each dumped table, +Produce tab\-separated text\-format data files\&. For each dumped table, \fBmysqldump\fR creates a \fItbl_name\fR\&.sql file that contains the CREATE TABLE -statement that creates the table, and a +statement that creates the table, and the server writes a \fItbl_name\fR\&.txt file that contains its data\&. The option value is the directory in which to write the files\&. -.sp -By default, the -\&.txt -data files are formatted using tab characters between column values and a newline at the end of each line\&. The format can be specified explicitly using the -\fB\-\-fields\-\fR\fB\fIxxx\fR\fR -and -\fB\-\-lines\-terminated\-by\fR -options\&. -.sp -As of MySQL 5\&.1\&.38, column values are written converted to the character set specified by the -\fB\-\-default\-character\-set\fR -option\&. Prior to 5\&.1\&.38 or if no such option is present, values are dumped using the -binary -character set\&. In effect, there is no character set conversion\&. If a table contains columns in several character sets, the output data file will as well and you may not be able to reload the file correctly\&. .if n \{\ .sp .\} @@ -1684,6 +1751,19 @@ FILE privilege, and the server must have permission to write files in the directory that you specify\&. .sp .5v .RE +By default, the +\&.txt +data files are formatted using tab characters between column values and a newline at the end of each line\&. The format can be specified explicitly using the +\fB\-\-fields\-\fR\fB\fIxxx\fR\fR +and +\fB\-\-lines\-terminated\-by\fR +options\&. +.sp +As of MySQL 5\&.1\&.38, column values are converted to the character set specified by the +\fB\-\-default\-character\-set\fR +option\&. Prior to 5\&.1\&.38 or if no such option is present, values are dumped using the +binary +character set\&. In effect, there is no character set conversion\&. If a table contains columns in several character sets, the output data file will as well and you may not be able to reload the file correctly\&. .RE .sp .RS 4 @@ -1719,7 +1799,7 @@ regards all name arguments following the .\" triggers option: mysqldump \fB\-\-triggers\fR .sp -Dump triggers for each dumped table\&. This option is enabled by default; disable it with +Include triggers for each dumped table in the output\&. This option is enabled by default; disable it with \fB\-\-skip\-triggers\fR\&. .RE .sp @@ -1743,7 +1823,7 @@ sets its connection time zone to UTC and SET TIME_ZONE=\'+00:00\' to the dump file\&. Without this option, TIMESTAMP -columns are dumped and reloaded in the time zones local to the source and destination servers, which can cause the values to change\&. +columns are dumped and reloaded in the time zones local to the source and destination servers, which can cause the values to change if the servers are in different time zones\&. \fB\-\-tz\-utc\fR also protects against changes due to daylight saving time\&. \fB\-\-tz\-utc\fR @@ -1846,7 +1926,7 @@ Examples: .sp Write dump output as well\-formed XML\&. .sp -\fBNULL\fR\fB, \fR\fB\'NULL\'\fR\fB, and Empty Values\fR: For some column named +\fBNULL\fR\fB, \fR\fB\'NULL\'\fR\fB, and Empty Values\fR: For a column named \fIcolumn_name\fR, the NULL value, an empty string, and the string value @@ -1884,7 +1964,7 @@ Beginning with MySQL 5\&.1\&.12, the out \fBmysql\fR client when run using the \fB\-\-xml\fR -option also follows these rules\&. (See +option also follows the preceding rules\&. (See the section called \(lqMYSQL OPTIONS\(rq\&.) .sp Beginning with MySQL 5\&.1\&.18, XML output from @@ -1905,11 +1985,13 @@ shell> \fBmysqldump \-\-xml \-u root wor <field Field="CountryCode" Type="char(3)" Null="NO" Key="" Default="" Extra="" /> <field Field="District" Type="char(20)" Null="NO" Key="" Default="" Extra="" /> <field Field="Population" Type="int(11)" Null="NO" Key="" Default="0" Extra="" /> -<key Table="City" Non_unique="0" Key_name="PRIMARY" Seq_in_index="1" Column_name="ID" Collation="A" Cardinality="4079" -Null="" Index_type="BTREE" Comment="" /> -<options Name="City" Engine="MyISAM" Version="10" Row_format="Fixed" Rows="4079" Avg_row_length="67" Data_length="27329 -3" Max_data_length="18858823439613951" Index_length="43008" Data_free="0" Auto_increment="4080" Create_time="2007\-03\-31 01:47:01" Updat -e_time="2007\-03\-31 01:47:02" Collation="latin1_swedish_ci" Create_options="" Comment="" /> +<key Table="City" Non_unique="0" Key_name="PRIMARY" Seq_in_index="1" Column_name="ID" +Collation="A" Cardinality="4079" Null="" Index_type="BTREE" Comment="" /> +<options Name="City" Engine="MyISAM" Version="10" Row_format="Fixed" Rows="4079" +Avg_row_length="67" Data_length="273293" Max_data_length="18858823439613951" +Index_length="43008" Data_free="0" Auto_increment="4080" +Create_time="2007\-03\-31 01:47:01" Update_time="2007\-03\-31 01:47:02" +Collation="latin1_swedish_ci" Create_options="" Comment="" /> </table_structure> <table_data name="City"> <row> @@ -1964,10 +2046,13 @@ The maximum size of the buffer for clien .\} net_buffer_length .sp -The initial size of the buffer for client/server communication\&. When creating multiple\-row\-insert statements (as with option +The initial size of the buffer for client/server communication\&. When creating multiple\-row +INSERT +statements (as with the \fB\-\-extended\-insert\fR or -\fB\-\-opt\fR), +\fB\-\-opt\fR +option), \fBmysqldump\fR creates rows up to net_buffer_length @@ -1976,9 +2061,9 @@ net_buffer_length variable in the MySQL server is at least this large\&. .RE .PP -The most common use of +A common use of \fBmysqldump\fR -is probably for making a backup of an entire database: +is for making a backup of an entire database: .sp .if n \{\ .RS 4 @@ -1990,7 +2075,7 @@ shell> \fBmysqldump \fR\fB\fIdb_name\fR\ .RE .\} .PP -You can read the dump file back into the server like this: +You can load the dump file back into the server like this: .sp .if n \{\ .RS 4 @@ -2072,7 +2157,7 @@ shell> \fBmysqldump \-\-all\-databases \ This backup acquires a global read lock on all tables (using FLUSH TABLES WITH READ LOCK) at the beginning of the dump\&. As soon as this lock has been acquired, the binary log coordinates are read and the lock is released\&. If long updating statements are running when the FLUSH -statement is issued, the MySQL server may get stalled until those statements finish\&. After that, the dump becomes lock\-free and does not disturb reads and writes on the tables\&. If the update statements that the MySQL server receives are short (in terms of execution time), the initial lock period should not be noticeable, even with many updates\&. +statement is issued, the MySQL server may get stalled until those statements finish\&. After that, the dump becomes lock free and does not disturb reads and writes on the tables\&. If the update statements that the MySQL server receives are short (in terms of execution time), the initial lock period should not be noticeable, even with many updates\&. .PP For point\-in\-time recovery (also known as \(lqroll\-forward,\(rq @@ -2106,13 +2191,13 @@ The \fB\-\-master\-data\fR and \fB\-\-single\-transaction\fR -options can be used simultaneously, which provides a convenient way to make an online backup suitable for point\-in\-time recovery if tables are stored using the +options can be used simultaneously, which provides a convenient way to make an online backup suitable for use prior to point\-in\-time recovery if tables are stored using the InnoDB storage engine\&. .PP For more information on making backups, see -Section\ \&6.1, \(lqDatabase Backup Methods\(rq, and -Section\ \&6.2, \(lqExample Backup and Recovery Strategy\(rq\&. +Section\ \&6.2, \(lqDatabase Backup Methods\(rq, and +Section\ \&6.3, \(lqExample Backup and Recovery Strategy\(rq\&. .\" mysqldump: views .\" mysqldump: problems .\" mysqldump: workarounds @@ -2122,7 +2207,7 @@ Section\ \&D.4, \(lqRestrictions on View .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -2134,7 +2219,7 @@ You should have received a copy of the G .IP " 1." 4 Bug#30123 .RS 4 -\%http://bugs.mysql.com/30123 +\%http://bugs.mysql.com/bug.php?id=30123 .RE .SH "SEE ALSO" For more information, please refer to the MySQL Reference Manual, === modified file 'man/mysqldumpslow.1' --- a/man/mysqldumpslow.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqldumpslow.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqldumpslow\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLDUMPSLOW\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLDUMPSLOW\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -60,7 +60,7 @@ shell> \fBmysqldumpslow [\fR\fB\fIoption .\} .PP \fBmysqldumpslow\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -309,7 +309,7 @@ Count: 3 Time=2\&.13s (6s) Lock=0\&.00 .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlhotcopy.1' --- a/man/mysqlhotcopy.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlhotcopy.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlhotcopy\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLHOTCOPY\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLHOTCOPY\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -32,18 +32,28 @@ mysqlhotcopy \- a database backup progra .PP \fBmysqlhotcopy\fR is a Perl script that was originally written and contributed by Tim Bunce\&. It uses -LOCK TABLES, -FLUSH TABLES, and +FLUSH TABLES, +LOCK TABLES, and cp or scp -to make a database backup quickly\&. It is the fastest way to make a backup of the database or single tables, but it can be run only on the same machine where the database directories are located\&. +to make a database backup\&. It is a fast way to make a backup of the database or single tables, but it can be run only on the same machine where the database directories are located\&. \fBmysqlhotcopy\fR works only for backing up MyISAM and ARCHIVE tables\&. It runs on Unix and NetWare\&. +.PP +To use +\fBmysqlhotcopy\fR, you must have read access to the files for the tables that you are backing up, the +SELECT +privilege for those tables, the +RELOAD +privilege (to be able to execute +FLUSH TABLES), and the +LOCK TABLES +privilege (to be able to lock the tables)\&. .sp .if n \{\ .RS 4 @@ -90,7 +100,11 @@ shell> \fBmysqlhotcopy \fR\fB\fIdb_name\ .\} .PP \fBmysqlhotcopy\fR -supports the following options: +supports the following options, which can be specified on the command line or in the +[mysqlhotcopy] +and +[client] +option file groups\&. .sp .RS 4 .ie n \{\ @@ -275,7 +289,8 @@ Do not delete previous (renamed) target .sp The method for copying files (cp or -scp)\&. +scp)\&. The default is +cp\&. .RE .sp .RS 4 @@ -290,7 +305,9 @@ scp)\&. .\" noindices option: mysqlhotcopy \fB\-\-noindices\fR .sp -Do not include full index files in the backup\&. This makes the backup smaller and faster\&. The indexes for reloaded tables can be reconstructed later with +Do not include full index files for +MyISAM +tables in the backup\&. This makes the backup smaller and faster\&. The indexes for reloaded tables can be reconstructed later with \fBmyisamchk \-rq\fR\&. .RE .sp @@ -307,10 +324,10 @@ Do not include full index files in the b \fB\-\-password=\fR\fB\fIpassword\fR\fR, \fB\-p\fR\fB\fIpassword\fR\fR .sp -The password to use when connecting to the server\&. Note that the password value is not optional for this option, unlike for other MySQL programs\&. You can use an option file to avoid giving the password on the command line\&. +The password to use when connecting to the server\&. The password value is not optional for this option, unlike for other MySQL programs\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -423,7 +440,8 @@ file after locking all the tables\&. \fB\-\-socket=\fR\fB\fIpath\fR\fR, \fB\-S \fR\fB\fIpath\fR\fR .sp -The Unix socket file to use for the connection\&. +The Unix socket file to use for connections to +localhost\&. .RE .sp .RS 4 @@ -438,7 +456,7 @@ The Unix socket file to use for the conn .\" suffix option: mysqlhotcopy \fB\-\-suffix=\fR\fB\fIstr\fR\fR .sp -The suffix for names of copied databases\&. +The suffix to use for names of copied databases\&. .RE .sp .RS 4 @@ -473,23 +491,6 @@ The temporary directory\&. The default i The MySQL user name to use when connecting to the server\&. .RE .PP -\fBmysqlhotcopy\fR -reads the -[client] -and -[mysqlhotcopy] -option groups from option files\&. -.PP -To execute -\fBmysqlhotcopy\fR, you must have access to the files for the tables that you are backing up, the -SELECT -privilege for those tables, the -RELOAD -privilege (to be able to execute -FLUSH TABLES), and the -LOCK TABLES -privilege (to be able to lock the tables)\&. -.PP Use perldoc for additional @@ -512,7 +513,7 @@ shell> \fBperldoc mysqlhotcopy\fR .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlimport.1' --- a/man/mysqlimport.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlimport.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlimport\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLIMPORT\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLIMPORT\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -64,8 +64,18 @@ patient all would be imported into a table named patient\&. .PP -\fBmysqlimport\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +For additional information about +\fBmysqldump\fR, see +Section\ \&6.4, \(lqUsing mysqldump for Backups\(rq\&. +.PP +\fBmysqldump\fR +supports the following options, which can be specified on the command line or in the +[mysqldump] +and +[client] +option file groups\&. +\fBmysqldump\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -97,7 +107,7 @@ Display a help message and exit\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -197,7 +207,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -379,8 +389,9 @@ tables for writing before processing any .sp Use LOW_PRIORITY -when loading the table\&. This affects only storage engines that use only table\-level locking (MyISAM, -MEMORY, +when loading the table\&. This affects only storage engines that use only table\-level locking (such as +MyISAM, +MEMORY, and MERGE)\&. .RE .sp @@ -405,10 +416,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlimport\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -424,7 +437,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -529,7 +542,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -633,7 +646,7 @@ shell> \fBmysql \-e \'SELECT * FROM impt .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlmanager.8' --- a/man/mysqlmanager.8 2009-12-01 07:24:05 +0000 +++ b/man/mysqlmanager.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlmanager\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLMANAGER\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLMANAGER\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -153,7 +153,7 @@ in the directory where Instance Manager option\&. .PP \fBmysqlmanager\fR -supports the options described in the following list\&. The options for managing entries in the password file are described further in +supports the following options\&. The options for managing entries in the password file are described further in the section called \(lqINSTANCE MANAGER USER AND PASSWORD MANAGEMENT\(rq\&. .sp .RS 4 @@ -2062,7 +2062,7 @@ Query OK, 0 rows affected (0\&.04 sec) .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlshow.1' --- a/man/mysqlshow.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlshow.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlshow\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLSHOW\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLSHOW\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -39,7 +39,7 @@ client can be used to quickly see which provides a command\-line interface to several SQL SHOW statements\&. See -Section\ \&12.5.5, \(lqSHOW Syntax\(rq\&. The same information can be obtained by using those statements directly\&. For example, you can issue them from the +Section\ \&12.4.5, \(lqSHOW Syntax\(rq\&. The same information can be obtained by using those statements directly\&. For example, you can issue them from the \fBmysql\fR client program\&. .PP @@ -112,7 +112,13 @@ shows you only the table names that matc last on the command line as a separate argument\&. .PP \fBmysqlshow\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlshow] +and +[client] +option file groups\&. +\fBmysqlshow\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -144,7 +150,7 @@ Display a help message and exit\&. \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp The directory where character sets are installed\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -244,7 +250,7 @@ Print debugging information and memory a Use \fIcharset_name\fR as the default character set\&. See -Section\ \&9.2, \(lqThe Character Set Used for Data and Sorting\(rq\&. +Section\ \&9.5, \(lqCharacter Set Configuration\(rq\&. .RE .sp .RS 4 @@ -300,10 +306,12 @@ value following the \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlshow\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -319,7 +327,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -406,7 +414,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -475,7 +483,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqlslap.1' --- a/man/mysqlslap.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqlslap.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqlslap\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBMYSQLSLAP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBMYSQLSLAP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -151,7 +151,13 @@ mysqlslap \-\-concurrency=5 \e .\} .PP \fBmysqlslap\fR -supports the options in the following list\&. It also reads option files and supports the options for processing them described at +supports the following options, which can be specified on the command line or in the +[mysqlslap] +and +[client] +option file groups\&. +\fBmysqlslap\fR +also supports the options for processing option files described at Section\ \&4.2.3.3.1, \(lqCommand-Line Options that Affect Option-File Handling\(rq\&. .sp .RS 4 @@ -615,7 +621,24 @@ is specified\&. .\" number-of-queries option: mysqlslap \fB\-\-number\-of\-queries=\fR\fB\fIN\fR\fR .sp -Limit each client to approximately this number of queries\&. This option was added in MySQL 5\&.1\&.5\&. +Limit each client to approximately this many queries\&. Query counting takes into account the statement delimiter\&. For example, if you invoke +\fBmysqlslap\fR +as follows, the +; +delimiter is recognized so that each instance of the query string counts as two queries\&. As a result, 5 rows (not 10) are inserted\&. +.sp +.if n \{\ +.RS 4 +.\} +.nf +shell> \fBmysqlslap \-\-delimiter=";" \-\-number\-of\-queries=10\fR + \fB\-\-query="use test;insert into t values(null)"\fR +.fi +.if n \{\ +.RE +.\} +.sp +This option was added in MySQL 5\&.1\&.5\&. .RE .sp .RS 4 @@ -653,15 +676,15 @@ The password to use when connecting to t have a space between the option and the password\&. If you omit the \fIpassword\fR value following the -.\" mysqlslap: password option -.\" password option: mysqlslap \fB\-\-password\fR or \fB\-p\fR -option on the command line, you are prompted for one\&. +option on the command line, +\fBmysqlslap\fR +prompts for one\&. .sp Specifying a password on the command line should be considered insecure\&. See -Section\ \&5.5.6.2, \(lqEnd-User Guidelines for Password Security\(rq\&. +Section\ \&5.3.2.2, \(lqEnd-User Guidelines for Password Security\(rq\&. You can use an option file to avoid giving the password on the command line\&. .RE .sp .RS 4 @@ -677,7 +700,7 @@ Section\ \&5.5.6.2, \(lqEnd-User Guideli \fB\-\-pipe\fR, \fB\-W\fR .sp -On Windows, connect to the server via a named pipe\&. This option applies only for connections to a local server, and only if the server supports named\-pipe connections\&. +On Windows, connect to the server via a named pipe\&. This option applies only if the server supports named\-pipe connections\&. .RE .sp .RS 4 @@ -897,7 +920,7 @@ localhost, the Unix socket file to use, Options that begin with \fB\-\-ssl\fR specify whether to connect to the server via SSL and indicate where to find SSL keys and certificates\&. See -Section\ \&5.5.7.3, \(lqSSL Command Options\(rq\&. +Section\ \&5.5.6.3, \(lqSSL Command Options\(rq\&. .RE .sp .RS 4 @@ -971,7 +994,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/mysqltest.1' --- a/man/mysqltest.1 2009-12-01 07:24:05 +0000 +++ b/man/mysqltest.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBmysqltest\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 10/29/2009 +.\" Date: 03/31/2010 .\" Manual: MySQL Database System .\" Source: MySQL .\" Language: English .\" -.TH "\FBMYSQLTEST\FR" "1" "10/29/2009" "MySQL" "MySQL Database System" +.TH "\FBMYSQLTEST\FR" "1" "03/31/2010" "MySQL" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -175,31 +175,11 @@ The base directory for tests\&. .sp -1 .IP \(bu 2.3 .\} -.\" mysqltest: big-test option -.\" big-test option: mysqltest -\fB\-\-big\-test\fR, -\fB\-B\fR -.sp -Define the -\fBmysqltest\fR -variable -$BIG_TEST -as 1\&. This option was removed in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.13\&. -.RE -.sp -.RS 4 -.ie n \{\ -\h'-04'\(bu\h'+03'\c -.\} -.el \{\ -.sp -1 -.IP \(bu 2.3 -.\} .\" mysqltest: character-sets-dir option .\" character-sets-dir option: mysqltest \fB\-\-character\-sets\-dir=\fR\fB\fIpath\fR\fR .sp -The directory where character sets are installed\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.32, and 5\&.1\&.13\&. +The directory where character sets are installed\&. .RE .sp .RS 4 @@ -226,12 +206,11 @@ Compress all information sent between th .sp -1 .IP \(bu 2.3 .\} -.\" mysqltest: cursor-protocol option +.\" mysqltest: currsor-protocol option .\" cursor-protocol option: mysqltest \fB\-\-cursor\-protocol\fR .sp -Use cursors for prepared statements (implies -\fB\-\-ps\-protocol\fR)\&. This option was added in MySQL 5\&.0\&.19\&. +Use cursors for prepared statements\&. .RE .sp .RS 4 @@ -281,7 +260,7 @@ value is .\" debug-check option: mysqltest \fB\-\-debug\-check\fR .sp -Print some debugging information when the program exits\&. This option was added in MySQL 5\&.1\&.21\&. +Print some debugging information when the program exits\&. .RE .sp .RS 4 @@ -296,7 +275,7 @@ Print some debugging information when th .\" debug-info option: mysqltest \fB\-\-debug\-info\fR .sp -Print debugging information and memory and CPU usage statistics when the program exits\&. This option was added in MySQL 5\&.1\&.14\&. +Print debugging information and memory and CPU usage statistics when the program exits\&. .RE .sp .RS 4 @@ -332,7 +311,7 @@ Include the contents of the given file b \fBmysqltest\fR test files\&. This option has the same effect as putting a \-\-source \fIfile_name\fR -command as the first line of the test file\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.7\&. +command as the first line of the test file\&. .RE .sp .RS 4 @@ -347,7 +326,7 @@ command as the first line of the test fi .\" logdir option: mysqltest \fB\-\-logdir=\fR\fB\fIdir_name\fR\fR .sp -The directory to use for log files\&. This option was added in MySQL 5\&.1\&.14\&. +The directory to use for log files\&. .RE .sp .RS 4 @@ -363,7 +342,7 @@ The directory to use for log files\&. Th \fB\-\-mark\-progress\fR .sp Write the line number and elapsed time to -\fItest_file\fR\&.progress\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.30, and 5\&.1\&.12\&. +\fItest_file\fR\&.progress\&. .RE .sp .RS 4 @@ -378,7 +357,24 @@ Write the line number and elapsed time t .\" max-connect-retries option: mysqltest \fB\-\-max\-connect\-retries=\fR\fB\fInum\fR\fR .sp -The maximum number of connection attempts when connecting to server\&. This option was added in MySQL 4\&.1\&.23, 5\&.0\&.23, and 5\&.1\&.11\&. +The maximum number of connection attempts when connecting to server\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysqltest: max-connections option +.\" max-connections option: mysqltest +\fB\-\-max\-connections=\fR\fB\fInum\fR\fR +.sp +The maximum number of simultaneous server connections per client (that is, per test)\&. If not set, the maximum is 128\&. Minimum allowed limit is 8, maximum is 5120\&. +.sp +This option is available from MySQL 5\&.1\&.45\&. .RE .sp .RS 4 @@ -393,7 +389,7 @@ The maximum number of connection attempt .\" no-defaults option: mysqltest \fB\-\-no\-defaults\fR .sp -Do not read default options from any option files\&. +Do not read default options from any option files\&. If used, this must be the first option\&. .RE .sp .RS 4 @@ -484,7 +480,8 @@ Suppress all normal output\&. This is a .sp Record the output that results from running the test file into the file named by the \fB\-\-result\-file\fR -option, if that option is given\&. +option, if that option is given\&. It is an error to use this option without also using +\fB\-\-result\-file\fR\&. .RE .sp .RS 4 @@ -516,7 +513,9 @@ treats the test actual and expected resu .\} If the test produces no results, \fBmysqltest\fR -exits with an error message to that effect\&. +exits with an error message to that effect, unless +\fB\-\-result\-file\fR +is given and the named file is an empty file\&. .RE .sp .RS 4 @@ -551,7 +550,7 @@ reads the expected results from the give \fBmysqltest\fR writes a \&.reject -file in the same directory as the result file and exits with an error\&. +file in the same directory as the result file, outputs a diff of the two files, and exits with an error\&. .RE .sp .RS 4 @@ -695,7 +694,22 @@ localhost .sp Execute DML statements within a stored procedure\&. For every DML statement, \fBmysqltest\fR -creates and invokes a stored procedure that executes the statement rather than executing the statement directly\&. This option was added in MySQL 5\&.0\&.19\&. +creates and invokes a stored procedure that executes the statement rather than executing the statement directly\&. +.RE +.sp +.RS 4 +.ie n \{\ +\h'-04'\(bu\h'+03'\c +.\} +.el \{\ +.sp -1 +.IP \(bu 2.3 +.\} +.\" mysqltest: tail-lines option +.\" tail-lines option: mysqltest +\fB\-\-tail\-lines=\fR\fB\fInn\fR\fR +.sp +Specify how many lines of the result to include in the output if the test fails because an SQL statement fails\&. The default is 0, meaning no lines of result printed\&. .RE .sp .RS 4 @@ -727,7 +741,9 @@ Read test input from this file\&. The de \fB\-\-timer\-file=\fR\fB\fIfile_name\fR\fR, \fB\-m \fR\fB\fIfile_name\fR\fR .sp -The file where the timing in microseconds is written\&. +If given, the number of millisecond spent running the test will be written to this file\&. This is used by +\fBmysql\-test\-run\&.pl\fR +for its reporting\&. .RE .sp .RS 4 @@ -743,7 +759,7 @@ The file where the timing in microsecond \fB\-\-tmpdir=\fR\fB\fIdir_name\fR\fR, \fB\-t \fR\fB\fIdir_name\fR\fR .sp -The temporary directory where socket files are put\&. +The temporary directory where socket files are created\&. .RE .sp .RS 4 @@ -775,7 +791,7 @@ The MySQL user name to use when connecti \fB\-\-verbose\fR, \fB\-v\fR .sp -Verbose mode\&. Print out more information what the program does\&. +Verbose mode\&. Print out more information about what the program does\&. .RE .sp .RS 4 @@ -813,7 +829,7 @@ statement is wrapped inside a view\&. Th .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright \(co 2007, 2010, Oracle and/or its affiliates. All rights reserved. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/ndbd.8' --- a/man/ndbd.8 2009-12-01 07:24:05 +0000 +++ b/man/ndbd.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBndbd\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBNDBD\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBNDBD\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -93,7 +93,7 @@ T}:T{ 5\&.1\&.12 T} T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-bind\-address=name T} @@ -137,7 +137,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-daemon T} @@ -189,7 +189,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-initial T} @@ -298,7 +298,7 @@ Backup files that have already been crea .IP \(bu 2.3 .\} MySQL Cluster Disk Data files (see -Section\ \&17.5.9, \(lqMySQL Cluster Disk Data Tables\(rq)\&. +Section\ \&17.5.10, \(lqMySQL Cluster Disk Data Tables\(rq)\&. .RE .RS 4 .sp @@ -334,7 +334,7 @@ T}:T{ 5\&.1\&.11 T} T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-initial\-start T} @@ -358,20 +358,27 @@ T} This option is used when performing a partial initial start of the cluster\&. Each node should be started with this option, as well as \fB\-\-nowait\-nodes\fR\&. .sp -For example, suppose you have a 4\-node cluster whose data nodes have the IDs 2, 3, 4, and 5, and you wish to perform a partial initial start using only nodes 2, 4, and 5 \(em that is, omitting node 3: +Suppose that you have a 4\-node cluster whose data nodes have the IDs 2, 3, 4, and 5, and you wish to perform a partial initial start using only nodes 2, 4, and 5 \(em that is, omitting node 3: .sp .if n \{\ .RS 4 .\} .nf -ndbd \-\-ndbd\-nodeid=2 \-\-nowait\-nodes=3 \-\-initial\-start -ndbd \-\-ndbd\-nodeid=4 \-\-nowait\-nodes=3 \-\-initial\-start -ndbd \-\-ndbd\-nodeid=5 \-\-nowait\-nodes=3 \-\-initial\-start +shell> \fBndbd \-\-ndb\-nodeid=2 \-\-nowait\-nodes=3 \-\-initial\-start\fR +shell> \fBndbd \-\-ndb\-nodeid=4 \-\-nowait\-nodes=3 \-\-initial\-start\fR +shell> \fBndbd \-\-ndb\-nodeid=5 \-\-nowait\-nodes=3 \-\-initial\-start\fR .fi .if n \{\ .RE .\} .sp +Prior to MySQL 5\&.1\&.19, it was not possible to perform DDL operations involving Disk Data tables on a partially started cluster\&. (See +\m[blue]\fBBug#24631\fR\m[]\&\s-2\u[1]\d\s+2\&.) +.sp +When using this option, you must also specify the node ID for the data node being started with the +\fB\-\-ndb\-nodeid\fR +option\&. +.sp This option was added in MySQL 5\&.1\&.11\&. .if n \{\ .sp @@ -385,8 +392,11 @@ This option was added in MySQL 5\&.1\&.1 \fBImportant\fR .ps -1 .br -Prior to MySQL 5\&.1\&.19, it was not possible to perform DDL operations involving Disk Data tables on a partially started cluster\&. (See -\m[blue]\fBBug#24631\fR\m[]\&\s-2\u[1]\d\s+2\&.) +Do not confuse this option with the +\fB\-\-nowait\-nodes\fR +option added for +\fBndb_mgmd\fR +in MySQL Cluster NDB 7\&.0\&.10, which can be used to allow a cluster configured with multiple management servers to be started without all management servers being online\&. .sp .5v .RE .RE @@ -412,10 +422,10 @@ l l s T{ \fBVersion Introduced\fR T}:T{ -5\&.1\&.11 +5\&.1\&.9 T} T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-nowait\-nodes=list T} @@ -470,9 +480,12 @@ allbox tab(:); l l s l l s ^ l l +^ l l +l l s +^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-nodaemon T} @@ -491,6 +504,21 @@ T} T}:T{ FALSE T} +T{ +\ \& +T}:T{ +\fBPermitted Values \fR +T} +:T{ +\fBType\fR (windows) +T}:T{ +boolean +T} +:T{ +\fBDefault\fR +T}:T{ +TRUE +T} .TE .sp 1 Instructs @@ -527,7 +555,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-\-nostart T} @@ -759,7 +787,7 @@ Section\ \&17.1.5, \(lqKnown Limitations .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -771,12 +799,12 @@ You should have received a copy of the G .IP " 1." 4 Bug#24631 .RS 4 -\%http://bugs.mysql.com/24631 +\%http://bugs.mysql.com/bug.php?id=24631 .RE .IP " 2." 4 Bug#45588 .RS 4 -\%http://bugs.mysql.com/45588 +\%http://bugs.mysql.com/bug.php?id=45588 .RE .IP " 3." 4 ndbd Error Messages === modified file 'man/ndbd_redo_log_reader.1' --- a/man/ndbd_redo_log_reader.1 2009-12-01 07:24:05 +0000 +++ b/man/ndbd_redo_log_reader.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBndbd_redo_log_reader\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBNDBD_REDO_LOG_REA" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBNDBD_REDO_LOG_REA" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -77,7 +77,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-noprint T} @@ -116,7 +116,7 @@ l l s ^ l l ^ l l. T{ -\fBCommand Line Format\fR +\fBCommand\-Line Format\fR T}:T{ \-nocheck T} @@ -154,7 +154,7 @@ must be run on a cluster data node, sinc .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/ndbmtd.8' --- a/man/ndbmtd.8 2009-12-01 07:24:05 +0000 +++ b/man/ndbmtd.8 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBndbmtd\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBNDBMTD\FR" "8" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBNDBMTD\FR" "8" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -74,7 +74,7 @@ Prior to MySQL Cluster NDB 7\&.0\&.6, th \fBndbmtd\fR with MySQL Cluster Disk Data tables\&. If you wish to use multi\-threaded data nodes with disk\-based NDB -tables, you should insure that you are running MySQL Cluster NDB 7\&.0\&.6 or later\&. (\m[blue]\fBBug#41915\fR\m[]\&\s-2\u[1]\d\s+2, +tables, you should ensure that you are running MySQL Cluster NDB 7\&.0\&.6 or later\&. (\m[blue]\fBBug#41915\fR\m[]\&\s-2\u[1]\d\s+2, \m[blue]\fBBug#44915\fR\m[]\&\s-2\u[2]\d\s+2) .PP Using @@ -356,7 +356,7 @@ concurrently on different data nodes in .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP @@ -368,12 +368,12 @@ You should have received a copy of the G .IP " 1." 4 Bug#41915 .RS 4 -\%http://bugs.mysql.com/41915 +\%http://bugs.mysql.com/bug.php?id=41915 .RE .IP " 2." 4 Bug#44915 .RS 4 -\%http://bugs.mysql.com/44915 +\%http://bugs.mysql.com/bug.php?id=44915 .RE .IP " 3." 4 ndbd Error Messages === modified file 'man/perror.1' --- a/man/perror.1 2009-12-01 07:24:05 +0000 +++ b/man/perror.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBperror\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBPERROR\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBPERROR\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -96,7 +96,7 @@ shell> \fBperror \-\-ndb \fR\fB\fIerrorc Note that the meaning of system error messages may be dependent on your operating system\&. A given error code may mean different things on different operating systems\&. .PP \fBperror\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -181,7 +181,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/replace.1' --- a/man/replace.1 2009-12-01 07:24:05 +0000 +++ b/man/replace.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBreplace\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBREPLACE\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBREPLACE\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -90,7 +90,7 @@ program is used by \fBmsql2mysql\fR(1)\&. .PP \fBreplace\fR -supports the following options: +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -160,7 +160,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/resolve_stack_dump.1' --- a/man/resolve_stack_dump.1 2009-12-01 07:24:05 +0000 +++ b/man/resolve_stack_dump.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBresolve_stack_dump\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBRESOLVE_STACK_DUM" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBRESOLVE_STACK_DUM" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -49,7 +49,7 @@ command\&. The numeric dump file should \fBmysqld\fR\&. If no numeric dump file is named on the command line, the stack trace is read from the standard input\&. .PP \fBresolve_stack_dump\fR -supports the options described in the following list\&. +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -117,7 +117,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'man/resolveip.1' --- a/man/resolveip.1 2009-12-01 07:24:05 +0000 +++ b/man/resolveip.1 2010-04-28 13:06:11 +0000 @@ -2,12 +2,12 @@ .\" Title: \fBresolveip\fR .\" Author: [FIXME: author] [see http://docbook.sf.net/el/author] .\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/> -.\" Date: 11/04/2009 +.\" Date: 04/06/2010 .\" Manual: MySQL Database System .\" Source: MySQL 5.1 .\" Language: English .\" -.TH "\FBRESOLVEIP\FR" "1" "11/04/2009" "MySQL 5\&.1" "MySQL Database System" +.TH "\FBRESOLVEIP\FR" "1" "04/06/2010" "MySQL 5\&.1" "MySQL Database System" .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- @@ -45,7 +45,7 @@ shell> \fBresolveip [\fR\fB\fIoptions\fR .\} .PP \fBresolveip\fR -supports the options described in the following list\&. +supports the following options\&. .sp .RS 4 .ie n \{\ @@ -99,7 +99,7 @@ Display version information and exit\&. .SH "COPYRIGHT" .br .PP -Copyright 2007-2008 MySQL AB, 2009 Sun Microsystems, Inc. +Copyright 2007-2008 MySQL AB, 2008-2010 Sun Microsystems, Inc. .PP This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. .PP === modified file 'scripts/fill_help_tables.sql' --- a/scripts/fill_help_tables.sql 2009-12-01 07:24:05 +0000 +++ b/scripts/fill_help_tables.sql 2010-04-28 13:06:11 +0000 @@ -1,4 +1,4 @@ --- Copyright (C) 2000-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc. +-- Copyright (c) 2003, &year, Oracle and/or its affiliates. All rights reserved. -- -- This program is free software; you can redistribute it and/or modify -- it under the terms of the GNU General Public License as published by @@ -11,7 +11,7 @@ -- -- You should have received a copy of the GNU General Public License -- along with this program; if not, write to the Free Software --- Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +-- Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -- DO NOT EDIT THIS FILE. It is generated automatically. @@ -87,24 +87,24 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (18,25,'SHOW CREATE PROCEDURE','Syntax:\nSHOW CREATE PROCEDURE proc_name\n\nThis statement is a MySQL extension. It returns the exact string that\ncan be used to re-create the named stored procedure. A similar\nstatement, SHOW CREATE FUNCTION, displays information about stored\nfunctions (see [HELP SHOW CREATE FUNCTION]).\n\nBoth statements require that you be the owner of the routine or have\nSELECT access to the mysql.proc table. If you do not have privileges\nfor the routine itself, the value displayed for the Create Procedure or\nCreate Function field will be NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-create-procedure.html\n\n','mysql> SHOW CREATE PROCEDURE test.simpleproc\\G\n*************************** 1. row ***************************\n Procedure: simpleproc\n sql_mode:\n Create Procedure: CREATE PROCEDURE `simpleproc`(OUT param1 INT)\n BEGIN\n SELECT COUNT(*) INTO param1 FROM t;\n END\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n\nmysql> SHOW CREATE FUNCTION test.hello\\G\n*************************** 1. row ***************************\n Function: hello\n sql_mode:\n Create Function: CREATE FUNCTION `hello`(s CHAR(20))\n RETURNS CHAR(50)\n RETURN CONCAT(\'Hello, \',s,\'!\')\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n','http://dev.mysql.com/doc/refman/5.1/en/show-create-procedure.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (19,20,'INTEGER','INTEGER[(M)] [UNSIGNED] [ZEROFILL]\n\nThis type is a synonym for INT.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (20,35,'LOWER','Syntax:\nLOWER(str)\n\nReturns the string str with all characters changed to lowercase\naccording to the current character set mapping. The default is latin1\n(cp1252 West European).\n\nmysql> SELECT LOWER(\'QUADRATICALLY\');\n -> \'quadratically\'\n\nLOWER() (and UPPER()) are ineffective when applied to binary strings\n(BINARY, VARBINARY, BLOB). To perform lettercase conversion, convert\nthe string to a nonbinary string:\n\nmysql> SET @str = BINARY \'New York\';\nmysql> SELECT LOWER(@str), LOWER(CONVERT(@str USING latin1));\n+-------------+-----------------------------------+\n| LOWER(@str) | LOWER(CONVERT(@str USING latin1)) |\n+-------------+-----------------------------------+\n| New York | new york |\n+-------------+-----------------------------------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (21,25,'SHOW COLUMNS','Syntax:\nSHOW [FULL] COLUMNS {FROM | IN} tbl_name [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW COLUMNS displays information about the columns in a given table.\nIt also works for views. The LIKE clause, if present, indicates which\ncolumn names to match. The WHERE clause can be given to select rows\nusing more general conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nmysql> SHOW COLUMNS FROM City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nIf the data types differ from what you expect them to be based on a\nCREATE TABLE statement, note that MySQL sometimes changes data types\nwhen you create or alter a table. The conditions under which this\noccurs are described in\nhttp://dev.mysql.com/doc/refman/5.1/en/silent-column-changes.html.\n\nT… FULL keyword causes the output to include the column collation and\ncomments, as well as the privileges you have for each column.\n\nYou can use db_name.tbl_name as an alternative to the tbl_name FROM\ndb_name syntax. In other words, these two statements are equivalent:\n\nmysql> SHOW COLUMNS FROM mytable FROM mydb;\nmysql> SHOW COLUMNS FROM mydb.mytable;\n\nSHOW COLUMNS displays the following values for each table column:\n\nField indicates the column name.\n\nType indicates the column data type.\n\nCollation indicates the collation for nonbinary string columns, or NULL\nfor other columns. This value is displayed only if you use the FULL\nkeyword.\n\nThe Null field contains YES if NULL values can be stored in the column,\nNO if not.\n\nThe Key field indicates whether the column is indexed:\n\no If Key is empty, the column either is not indexed or is indexed only\n as a secondary column in a multiple-column, nonunique index.\n\no If Key is PRI, the column is a PRIMARY KEY or is one of the columns\n in a multiple-column PRIMARY KEY.\n\no If Key is UNI, the column is the first column of a unique-valued\n index that cannot contain NULL values.\n\no If Key is MUL, multiple occurrences of a given value are allowed\n within the column. The column is the first column of a nonunique\n index or a unique-valued index that can contain NULL values.\n\nIf more than one of the Key values applies to a given column of a\ntable, Key displays the one with the highest priority, in the order\nPRI, UNI, MUL.\n\nA UNIQUE index may be displayed as PRI if it cannot contain NULL values\nand there is no PRIMARY KEY in the table. A UNIQUE index may display as\nMUL if several columns form a composite UNIQUE index; although the\ncombination of the columns is unique, each column can still hold\nmultiple occurrences of a given value.\n\nThe Default field indicates the default value that is assigned to the\ncolumn.\n\nThe Extra field contains any additional information that is available\nabout a given column. The value is nonempty in these cases:\nauto_increment for columns that have the AUTO_INCREMENT attribute; as\nof MySQL 5.1.23, on update CURRENT_TIMESTAMP for TIMESTAMP columns that\nhave the ON UPDATE CURRENT_TIMESTAMP attribute.\n\nPrivileges indicates the privileges you have for the column. This value\nis displayed only if you use the FULL keyword.\n\nComment indicates any comment the column has. This value is displayed\nonly if you use the FULL keyword.\n\nSHOW FIELDS is a synonym for SHOW COLUMNS. You can also list a table\'s\ncolumns with the mysqlshow db_name tbl_name command.\n\nThe DESCRIBE statement provides information similar to SHOW COLUMNS.\nSee [HELP DESCRIBE].\n\nThe SHOW CREATE TABLE, SHOW TABLE STATUS, and SHOW INDEX statements\nalso provide information about tables. See [HELP SHOW].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-columns.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-columns.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (22,37,'CREATE TRIGGER','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n TRIGGER trigger_name trigger_time trigger_event\n ON tbl_name FOR EACH ROW trigger_stmt\n\nThis statement creates a new trigger. A trigger is a named database\nobject that is associated with a table, and that activates when a\nparticular event occurs for the table. The trigger becomes associated\nwith the table named tbl_name, which must refer to a permanent table.\nYou cannot associate a trigger with a TEMPORARY table or a view.\n\nCREATE TRIGGER requires the TRIGGER privilege for the table associated\nwith the trigger. (Before MySQL 5.1.6, this statement requires the\nSUPER privilege.)\n\nThe DEFINER clause determines the security context to be used when\nchecking access privileges at trigger activation time.\n\ntrigger_time is the trigger action time. It can be BEFORE or AFTER to\nindicate that the trigger activates before or after each row to be\nmodified.\n\ntrigger_event indicates the kind of statement that activates the\ntrigger. The trigger_event can be one of the following:\n\no INSERT: The trigger is activated whenever a new row is inserted into\n the table; for example, through INSERT, LOAD DATA, and REPLACE\n statements.\n\no UPDATE: The trigger is activated whenever a row is modified; for\n example, through UPDATE statements.\n\no DELETE: The trigger is activated whenever a row is deleted from the\n table; for example, through DELETE and REPLACE statements. However,\n DROP TABLE and TRUNCATE statements on the table do not activate this\n trigger, because they do not use DELETE. Dropping a partition does\n not activate DELETE triggers, either. See [HELP TRUNCATE TABLE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (21,25,'SHOW COLUMNS','Syntax:\nSHOW [FULL] COLUMNS {FROM | IN} tbl_name [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW COLUMNS displays information about the columns in a given table.\nIt also works for views. The LIKE clause, if present, indicates which\ncolumn names to match. The WHERE clause can be given to select rows\nusing more general conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nmysql> SHOW COLUMNS FROM City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nIf the data types differ from what you expect them to be based on a\nCREATE TABLE statement, note that MySQL sometimes changes data types\nwhen you create or alter a table. The conditions under which this\noccurs are described in\nhttp://dev.mysql.com/doc/refman/5.1/en/silent-column-changes.html.\n\nT… FULL keyword causes the output to include the column collation and\ncomments, as well as the privileges you have for each column.\n\nYou can use db_name.tbl_name as an alternative to the tbl_name FROM\ndb_name syntax. In other words, these two statements are equivalent:\n\nmysql> SHOW COLUMNS FROM mytable FROM mydb;\nmysql> SHOW COLUMNS FROM mydb.mytable;\n\nSHOW COLUMNS displays the following values for each table column:\n\nField indicates the column name.\n\nType indicates the column data type.\n\nCollation indicates the collation for nonbinary string columns, or NULL\nfor other columns. This value is displayed only if you use the FULL\nkeyword.\n\nThe Null field contains YES if NULL values can be stored in the column,\nNO if not.\n\nThe Key field indicates whether the column is indexed:\n\no If Key is empty, the column either is not indexed or is indexed only\n as a secondary column in a multiple-column, nonunique index.\n\no If Key is PRI, the column is a PRIMARY KEY or is one of the columns\n in a multiple-column PRIMARY KEY.\n\no If Key is UNI, the column is the first column of a UNIQUE index. (A\n UNIQUE index allows multiple NULL values, but you can tell whether\n the column allows NULL by checking the Null field.)\n\no If Key is MUL, the column is the first column of a nonunique index in\n which multiple occurrences of a given value are allowed within the\n column.\n\nIf more than one of the Key values applies to a given column of a\ntable, Key displays the one with the highest priority, in the order\nPRI, UNI, MUL.\n\nA UNIQUE index may be displayed as PRI if it cannot contain NULL values\nand there is no PRIMARY KEY in the table. A UNIQUE index may display as\nMUL if several columns form a composite UNIQUE index; although the\ncombination of the columns is unique, each column can still hold\nmultiple occurrences of a given value.\n\nThe Default field indicates the default value that is assigned to the\ncolumn.\n\nThe Extra field contains any additional information that is available\nabout a given column. The value is nonempty in these cases:\nauto_increment for columns that have the AUTO_INCREMENT attribute; as\nof MySQL 5.1.23, on update CURRENT_TIMESTAMP for TIMESTAMP columns that\nhave the ON UPDATE CURRENT_TIMESTAMP attribute.\n\nPrivileges indicates the privileges you have for the column. This value\nis displayed only if you use the FULL keyword.\n\nComment indicates any comment the column has. This value is displayed\nonly if you use the FULL keyword.\n\nSHOW FIELDS is a synonym for SHOW COLUMNS. You can also list a table\'s\ncolumns with the mysqlshow db_name tbl_name command.\n\nThe DESCRIBE statement provides information similar to SHOW COLUMNS.\nSee [HELP DESCRIBE].\n\nThe SHOW CREATE TABLE, SHOW TABLE STATUS, and SHOW INDEX statements\nalso provide information about tables. See [HELP SHOW].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-columns.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-columns.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (22,37,'CREATE TRIGGER','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n TRIGGER trigger_name trigger_time trigger_event\n ON tbl_name FOR EACH ROW trigger_stmt\n\nThis statement creates a new trigger. A trigger is a named database\nobject that is associated with a table, and that activates when a\nparticular event occurs for the table. The trigger becomes associated\nwith the table named tbl_name, which must refer to a permanent table.\nYou cannot associate a trigger with a TEMPORARY table or a view.\n\nCREATE TRIGGER requires the TRIGGER privilege for the table associated\nwith the trigger. If binary logging is enabled, the CREATE TRIGGER\nstatement might also require the SUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\nS… may also be required depending on the DEFINER value, as described\nlater. (Before MySQL 5.1.6, there is no TRIGGER privilege and this\nstatement requires the SUPER privilege in all cases.)\n\nThe DEFINER clause determines the security context to be used when\nchecking access privileges at trigger activation time.\n\ntrigger_time is the trigger action time. It can be BEFORE or AFTER to\nindicate that the trigger activates before or after each row to be\nmodified.\n\ntrigger_event indicates the kind of statement that activates the\ntrigger. The trigger_event can be one of the following:\n\no INSERT: The trigger is activated whenever a new row is inserted into\n the table; for example, through INSERT, LOAD DATA, and REPLACE\n statements.\n\no UPDATE: The trigger is activated whenever a row is modified; for\n example, through UPDATE statements.\n\no DELETE: The trigger is activated whenever a row is deleted from the\n table; for example, through DELETE and REPLACE statements. However,\n DROP TABLE and TRUNCATE TABLE statements on the table do not activate\n this trigger, because they do not use DELETE. Dropping a partition\n does not activate DELETE triggers, either. See [HELP TRUNCATE TABLE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (23,30,'MONTH','Syntax:\nMONTH(date)\n\nReturns the month for date, in the range 1 to 12 for January to\nDecember, or 0 for dates such as \'0000-00-00\' or \'2008-00-00\' that have\na zero month part.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MONTH(\'2008-02-03\');\n -> 2\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (24,20,'TINYINT','TINYINT[(M)] [UNSIGNED] [ZEROFILL]\n\nA very small integer. The signed range is -128 to 127. The unsigned\nrange is 0 to 255.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (25,25,'SHOW TRIGGERS','Syntax:\nSHOW TRIGGERS [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW TRIGGERS lists the triggers currently defined for tables in a\ndatabase (the default database unless a FROM clause is given). This\nstatement requires the TRIGGER privilege (prior to MySQL 5.1.22, it\nrequires the SUPER privilege). The LIKE clause, if present, indicates\nwhich table names to match and causes the statement to display triggers\nfor those tables. The WHERE clause can be given to select rows using\nmore general conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nFor the trigger ins_sum as defined in\nhttp://dev.mysql.com/doc/refman/5.1/en/triggers.html, the output of\nthis statement is as shown here:\n\nmysql> SHOW TRIGGERS LIKE \'acc%\'\\G\n*************************** 1. row ***************************\n Trigger: ins_sum\n Event: INSERT\n Table: account\n Statement: SET @sum = @sum + NEW.amount\n Timing: BEFORE\n Created: NULL\n sql_mode:\n Definer: myname@localhost\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n\ncharacter_set_client is the session value of the character_set_client\nsystem variable when the trigger was created. collation_connection is\nthe session value of the collation_connection system variable when the\ntrigger was created. Database Collation is the collation of the\ndatabase with which the trigger is associated. These columns were added\nin MySQL 5.1.21.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-triggers.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-triggers.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (26,14,'MASTER_POS_WAIT','Syntax:\nMASTER_POS_WAIT(log_name,log_pos[,timeout])\n\nThis function is useful for control of master/slave synchronization. It\nblocks until the slave has read and applied all updates up to the\nspecified position in the master log. The return value is the number of\nlog events the slave had to wait for to advance to the specified\nposition. The function returns NULL if the slave SQL thread is not\nstarted, the slave\'s master information is not initialized, the\narguments are incorrect, or an error occurs. It returns -1 if the\ntimeout has been exceeded. If the slave SQL thread stops while\nMASTER_POS_WAIT() is waiting, the function returns NULL. If the slave\nis past the specified position, the function returns immediately.\n\nIf a timeout value is specified, MASTER_POS_WAIT() stops waiting when\ntimeout seconds have elapsed. timeout must be greater than 0; a zero or\nnegative timeout means no timeout.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (27,35,'REGEXP','Syntax:\nexpr REGEXP pat, expr RLIKE pat\n\nPerforms a pattern match of a string expression expr against a pattern\npat. The pattern can be an extended regular expression. The syntax for\nregular expressions is discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/regexp.html. Returns 1 if expr\nmatches pat; otherwise it returns 0. If either expr or pat is NULL, the\nresult is NULL. RLIKE is a synonym for REGEXP, provided for mSQL\ncompatibility.\n\nThe pattern need not be a literal string. For example, it can be\nspecified as a string expression or table column.\n\n*Note*: Because MySQL uses the C escape syntax in strings (for example,\n"\\n" to represent the newline character), you must double any "\\" that\nyou use in your REGEXP strings.\n\nREGEXP is not case sensitive, except when used with binary strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/regexp.html\n\n','mysql> SELECT \'Monty!\' REGEXP \'m%y%%\';\n -> 0\nmysql> SELECT \'Monty!\' REGEXP \'.*\';\n -> 1\nmysql> SELECT \'new*\\n*line\' REGEXP \'new\\\\*.\\\\*line\';\n -> 1\nmysql> SELECT \'a\' REGEXP \'A\', \'a\' REGEXP BINARY \'A\';\n -> 1 0\nmysql> SELECT \'a\' REGEXP \'^[a-d]\';\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/regexp.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (27,35,'REGEXP','Syntax:\nexpr REGEXP pat, expr RLIKE pat\n\nPerforms a pattern match of a string expression expr against a pattern\npat. The pattern can be an extended regular expression. The syntax for\nregular expressions is discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/regexp.html. Returns 1 if expr\nmatches pat; otherwise it returns 0. If either expr or pat is NULL, the\nresult is NULL. RLIKE is a synonym for REGEXP, provided for mSQL\ncompatibility.\n\nThe pattern need not be a literal string. For example, it can be\nspecified as a string expression or table column.\n\n*Note*: Because MySQL uses the C escape syntax in strings (for example,\n"\\n" to represent the newline character), you must double any "\\" that\nyou use in your REGEXP strings.\n\nREGEXP is not case sensitive, except when used with binary strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/regexp.html\n\n','mysql> SELECT \'Monty!\' REGEXP \'.*\';\n -> 1\nmysql> SELECT \'new*\\n*line\' REGEXP \'new\\\\*.\\\\*line\';\n -> 1\nmysql> SELECT \'a\' REGEXP \'A\', \'a\' REGEXP BINARY \'A\';\n -> 1 0\nmysql> SELECT \'a\' REGEXP \'^[a-d]\';\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/regexp.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (28,22,'IF STATEMENT','Syntax:\nIF search_condition THEN statement_list\n [ELSEIF search_condition THEN statement_list] ...\n [ELSE statement_list]\nEND IF\n\nIF implements a basic conditional construct. If the search_condition\nevaluates to true, the corresponding SQL statement list is executed. If\nno search_condition matches, the statement list in the ELSE clause is\nexecuted. Each statement_list consists of one or more statements.\n\n*Note*: There is also an IF() function, which differs from the IF\nstatement described here. See\nhttp://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html.\n\…: http://dev.mysql.com/doc/refman/5.1/en/if-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/if-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (29,18,'^','Syntax:\n^\n\nBitwise XOR:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html\n\n','mysql> SELECT 1 ^ 1;\n -> 0\nmysql> SELECT 1 ^ 0;\n -> 1\nmysql> SELECT 11 ^ 3;\n -> 8\n','http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (30,37,'DROP VIEW','Syntax:\nDROP VIEW [IF EXISTS]\n view_name [, view_name] ...\n [RESTRICT | CASCADE]\n\nDROP VIEW removes one or more views. You must have the DROP privilege\nfor each view. If any of the views named in the argument list do not\nexist, MySQL returns an error indicating by name which nonexisting\nviews it was unable to drop, but it also drops all of the views in the\nlist that do exist.\n\nThe IF EXISTS clause prevents an error from occurring for views that\ndon\'t exist. When this clause is given, a NOTE is generated for each\nnonexistent view. See [HELP SHOW WARNINGS].\n\nRESTRICT and CASCADE, if given, are parsed and ignored.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-view.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-view.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (31,29,'WITHIN','Within(g1,g2)\n\nReturns 1 or 0 to indicate whether g1 is spatially within g2. This\ntests the opposite relationship as Contains().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (32,30,'WEEK','Syntax:\nWEEK(date[,mode])\n\nThis function returns the week number for date. The two-argument form\nof WEEK() allows you to specify whether the week starts on Sunday or\nMonday and whether the return value should be in the range from 0 to 53\nor from 1 to 53. If the mode argument is omitted, the value of the\ndefault_week_format system variable is used. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT WEEK(\'2008-02-20\');\n -> 7\nmysql> SELECT WEEK(\'2008-02-20\',0);\n -> 7\nmysql> SELECT WEEK(\'2008-02-20\',1);\n -> 8\nmysql> SELECT WEEK(\'2008-12-31\',1);\n -> 53\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (33,25,'SHOW PLUGINS','Syntax:\nSHOW PLUGINS\n\nSHOW PLUGINS displays information about known plugins.\n\nmysql> SHOW PLUGINS;\n+------------+--------+----------------+---------+\n| Name | Status | Type | Library |\n+------------+--------+----------------+---------+\n| MEMORY | ACTIVE | STORAGE ENGINE | NULL |\n| MyISAM | ACTIVE | STORAGE ENGINE | NULL |\n| InnoDB | ACTIVE | STORAGE ENGINE | NULL |\n| ARCHIVE | ACTIVE | STORAGE ENGINE | NULL |\n| CSV | ACTIVE | STORAGE ENGINE | NULL |\n| BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL |\n| FEDERATED | ACTIVE | STORAGE ENGINE | NULL |\n| MRG_MYISAM | ACTIVE | STORAGE ENGINE | NULL |\n+------------+--------+----------------+---------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (33,25,'SHOW PLUGINS','Syntax:\nSHOW PLUGINS\n\nSHOW PLUGINS displays information about server plugins.\n\nmysql> SHOW PLUGINS\\G\n*************************** 1. row ***************************\n Name: binlog\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n*************************** 2. row ***************************\n Name: CSV\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n*************************** 3. row ***************************\n Name: MEMORY\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n*************************** 4. row ***************************\n Name: MyISAM\n Status: ACTIVE\n Type: STORAGE ENGINE\nLibrary: NULL\nLicense: GPL\n...\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-plugins.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (34,21,'DROP FUNCTION UDF','Syntax:\nDROP FUNCTION function_name\n\nThis statement drops the user-defined function (UDF) named\nfunction_name.\n\nTo drop a function, you must have the DELETE privilege for the mysql\ndatabase. This is because DROP FUNCTION removes a row from the\nmysql.func system table that records the function\'s name, type, and\nshared library name.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-function-udf.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-function-udf.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (35,25,'PREPARE','Syntax:\nPREPARE stmt_name FROM preparable_stmt\n\nThe PREPARE statement prepares a statement and assigns it a name,\nstmt_name, by which to refer to the statement later. Statement names\nare not case sensitive. preparable_stmt is either a string literal or a\nuser variable that contains the text of the statement. The text must\nrepresent a single SQL statement, not multiple statements. Within the\nstatement, "?" characters can be used as parameter markers to indicate\nwhere data values are to be bound to the query later when you execute\nit. The "?" characters should not be enclosed within quotes, even if\nyou intend to bind them to string values. Parameter markers can be used\nonly where data values should appear, not for SQL keywords,\nidentifiers, and so forth.\n\nIf a prepared statement with the given name already exists, it is\ndeallocated implicitly before the new statement is prepared. This means\nthat if the new statement contains an error and cannot be prepared, an\nerror is returned and no statement with the given name exists.\n\nA prepared statement is executed with EXECUTE and released with\nDEALLOCATE PREPARE.\n\nThe scope of a prepared statement is the session within which it is\ncreated. Other sessions cannot see it.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/prepare.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/prepare.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (36,8,'LOCK','Syntax:\nLOCK TABLES\n tbl_name [[AS] alias] lock_type\n [, tbl_name [[AS] alias] lock_type] ...\n\nlock_type:\n READ [LOCAL]\n | [LOW_PRIORITY] WRITE\n\nUNLOCK TABLES\n\nMySQL enables client sessions to acquire table locks explicitly for the\npurpose of cooperating with other sessions for access to tables, or to\nprevent other sessions from modifying tables during periods when a\nsession requires exclusive access to them. A session can acquire or\nrelease locks only for itself. One session cannot acquire locks for\nanother session or release locks held by another session.\n\nLocks may be used to emulate transactions or to get more speed when\nupdating tables. This is explained in more detail later in this\nsection.\n\nLOCK TABLES explicitly acquires table locks for the current client\nsession. Table locks can be acquired for base tables or views. You must\nhave the LOCK TABLES privilege, and the SELECT privilege for each\nobject to be locked.\n\nFor view locking, LOCK TABLES adds all base tables used in the view to\nthe set of tables to be locked and locks them automatically. If you\nlock a table explicitly with LOCK TABLES, any tables used in triggers\nare also locked implicitly, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/lock-tables-and-triggers.html.\n… TABLES explicitly releases any table locks held by the current\nsession.\n\nAnother use for UNLOCK TABLES is to release the global read lock\nacquired with the FLUSH TABLES WITH READ LOCK statement, which enables\nyou to lock all tables in all databases. See [HELP FLUSH]. (This is a\nvery convenient way to get backups if you have a file system such as\nVeritas that can take snapshots in time.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/lock-tables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/lock-tables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (37,35,'UPDATEXML','Syntax:\nUpdateXML(xml_target, xpath_expr, new_xml)\n\nThis function replaces a single portion of a given fragment of XML\nmarkup xml_target with a new XML fragment new_xml, and then returns the\nchanged XML. The portion of xml_target that is replaced matches an\nXPath expression xpath_expr supplied by the user. If no expression\nmatching xpath_expr is found, or if multiple matches are found, the\nfunction returns the original xml_target XML fragment. All three\narguments should be strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html\n\n','mysql> SELECT\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'/a\', \'<e>fff</e>\') AS val1,\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'/b\', \'<e>fff</e>\') AS val2,\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'//b\', \'<e>fff</e>\') AS val3,\n -> UpdateXML(\'<a>ccc<d></d></a>\', \'/a/d\', \'<e>fff</e>\') AS val4,\n -> UpdateXML(\'<a><d></d>ccc<d></d></a>\', \'/a/d\', \'<e>fff</e>\') AS val5\n -> \\G\n\n*************************** 1. row ***************************\nval1: <e>fff</e>\nval2: <a>ccc<d></d></a>\nval3: <a><e>fff</e><d></d></a>\nval4: <a>ccc<e>fff</e></a>\nval5: <a><d></d>ccc<d></d></a>\n','http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (38,25,'RESET SLAVE','Syntax:\nRESET SLAVE\n\nRESET SLAVE makes the slave forget its replication position in the\nmaster\'s binary logs. This statement is meant to be used for a clean\nstart: It deletes the master.info and relay-log.info files, all the\nrelay logs, and starts a new relay log.\n\n*Note*: All relay logs are deleted, even if they have not been\ncompletely executed by the slave SQL thread. (This is a condition\nlikely to exist on a replication slave if you have issued a STOP SLAVE\nstatement or if the slave is highly loaded.)\n\nConnection information stored in the master.info file is immediately\nreset using any values specified in the corresponding startup options.\nThis information includes values such as master host, master port,\nmaster user, and master password. If the slave SQL thread was in the\nmiddle of replicating temporary tables when it was stopped, and RESET\nSLAVE is issued, these replicated temporary tables are deleted on the\nslave.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (38,25,'RESET SLAVE','Syntax:\nRESET SLAVE\n\nRESET SLAVE makes the slave forget its replication position in the\nmaster\'s binary log. This statement is meant to be used for a clean\nstart: It deletes the master.info and relay-log.info files, all the\nrelay log files, and starts a new relay log file. To use RESET SLAVE,\nthe slave replication threads must be stopped (use STOP SLAVE if\nnecessary).\n\n*Note*: All relay log files are deleted, even if they have not been\ncompletely executed by the slave SQL thread. (This is a condition\nlikely to exist on a replication slave if you have issued a STOP SLAVE\nstatement or if the slave is highly loaded.)\n\nConnection information stored in the master.info file is immediately\nreset using any values specified in the corresponding startup options.\nThis information includes values such as master host, master port,\nmaster user, and master password. If the slave SQL thread was in the\nmiddle of replicating temporary tables when it was stopped, and RESET\nSLAVE is issued, these replicated temporary tables are deleted on the\nslave.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-slave.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (39,25,'SHOW BINARY LOGS','Syntax:\nSHOW BINARY LOGS\nSHOW MASTER LOGS\n\nLists the binary log files on the server. This statement is used as\npart of the procedure described in [HELP PURGE BINARY LOGS], that shows\nhow to determine which logs can be purged.\n\nmysql> SHOW BINARY LOGS;\n+---------------+-----------+\n| Log_name | File_size |\n+---------------+-----------+\n| binlog.000015 | 724935 |\n| binlog.000016 | 733481 |\n+---------------+-----------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-binary-logs.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-binary-logs.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (40,23,'POLYGON','Polygon(ls1,ls2,...)\n\nConstructs a Polygon value from a number of LineString or WKB\nLineString arguments. If any argument does not represent a LinearRing\n(that is, not a closed and simple LineString), the return value is\nNULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (41,30,'MINUTE','Syntax:\nMINUTE(time)\n\nReturns the minute for time, in the range 0 to 59.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MINUTE(\'2008-02-03 10:05:03\');\n -> 5\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); @@ -118,7 +118,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (49,4,'ROUND','Syntax:\nROUND(X), ROUND(X,D)\n\nRounds the argument X to D decimal places. The rounding algorithm\ndepends on the data type of X. D defaults to 0 if not specified. D can\nbe negative to cause D digits left of the decimal point of the value X\nto become zero.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT ROUND(-1.23);\n -> -1\nmysql> SELECT ROUND(-1.58);\n -> -2\nmysql> SELECT ROUND(1.58);\n -> 2\nmysql> SELECT ROUND(1.298, 1);\n -> 1.3\nmysql> SELECT ROUND(1.298, 0);\n -> 1\nmysql> SELECT ROUND(23.298, -1);\n -> 20\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (50,7,'NULLIF','Syntax:\nNULLIF(expr1,expr2)\n\nReturns NULL if expr1 = expr2 is true, otherwise returns expr1. This is\nthe same as CASE WHEN expr1 = expr2 THEN NULL ELSE expr1 END.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html\n\n','mysql> SELECT NULLIF(1,1);\n -> NULL\nmysql> SELECT NULLIF(1,2);\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (51,22,'CLOSE','Syntax:\nCLOSE cursor_name\n\nThis statement closes a previously opened cursor.\n\nIf not closed explicitly, a cursor is closed at the end of the compound\nstatement in which it was declared.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/close.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/close.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (52,25,'STOP SLAVE','Syntax:\nSTOP SLAVE [thread_type [, thread_type] ... ]\n\nthread_type: IO_THREAD | SQL_THREAD\n\nStops the slave threads. STOP SLAVE requires the SUPER privilege.\n\nLike START SLAVE, this statement may be used with the IO_THREAD and\nSQL_THREAD options to name the thread or threads to be stopped.\n\n*Note*: The transactional behavior of STOP SLAVE changed in MySQL\n5.1.35. Previously, it took effect immediately; beginning with MySQL\n5.1.35, it waits until the current replication event group (if any) has\nfinished executing, or until the user issues a KILL QUERY or KILL\nCONNECTION statement. (Bug#319 (http://bugs.mysql.com/319) Bug#38205\n(http://bugs.mysql.com/38205))\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (52,25,'STOP SLAVE','Syntax:\nSTOP SLAVE [thread_type [, thread_type] ... ]\n\nthread_type: IO_THREAD | SQL_THREAD\n\nStops the slave threads. STOP SLAVE requires the SUPER privilege.\n\nLike START SLAVE, this statement may be used with the IO_THREAD and\nSQL_THREAD options to name the thread or threads to be stopped.\n\n*Note*: The transactional behavior of STOP SLAVE changed in MySQL\n5.1.35. Previously, it took effect immediately. Beginning with MySQL\n5.1.35, it waits until any current replication event group affecting\none or more non-transactional tables has finished executing (if there\nis any such replication group), or until the user issues a KILL QUERY\nor KILL CONNECTION statement. (Bug#319\n(http://bugs.mysql.com/bug.php?id=319), Bug#38205\n(http://bugs.mysql.com/bug.php?id=38205))\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/stop-slave.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (53,30,'TIMEDIFF','Syntax:\nTIMEDIFF(expr1,expr2)\n\nTIMEDIFF() returns expr1 - expr2 expressed as a time value. expr1 and\nexpr2 are time or date-and-time expressions, but both must be of the\nsame type.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMEDIFF(\'2000:01:01 00:00:00\',\n -> \'2000:01:01 00:00:00.000001\');\n -> \'-00:00:00.000001\'\nmysql> SELECT TIMEDIFF(\'2008-12-31 23:59:59.000001\',\n -> \'2008-12-30 01:01:01.000002\');\n -> \'46:58:57.999999\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (54,35,'REPLACE FUNCTION','Syntax:\nREPLACE(str,from_str,to_str)\n\nReturns the string str with all occurrences of the string from_str\nreplaced by the string to_str. REPLACE() performs a case-sensitive\nmatch when searching for from_str.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT REPLACE(\'www.mysql.com\', \'w\', \'Ww\');\n -> \'WwWwWw.mysql.com\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (55,27,'USE','Syntax:\nUSE db_name\n\nThe USE db_name statement tells MySQL to use the db_name database as\nthe default (current) database for subsequent statements. The database\nremains the default until the end of the session or another USE\nstatement is issued:\n\nUSE db1;\nSELECT COUNT(*) FROM mytable; # selects from db1.mytable\nUSE db2;\nSELECT COUNT(*) FROM mytable; # selects from db2.mytable\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/use.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/use.html'); @@ -147,13 +147,13 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (78,35,'LCASE','Syntax:\nLCASE(str)\n\nLCASE() is a synonym for LOWER().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (79,17,'<=','Syntax:\n<=\n\nLess than or equal:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 0.1 <= 2;\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (80,25,'SHOW PROFILES','Syntax:\nSHOW PROFILE [type [, type] ... ]\n [FOR QUERY n]\n [LIMIT row_count [OFFSET offset]]\n\ntype:\n ALL\n | BLOCK IO\n | CONTEXT SWITCHES\n | CPU\n | IPC\n | MEMORY\n | PAGE FAULTS\n | SOURCE\n | SWAPS\n\nThe SHOW PROFILES and SHOW PROFILE statements display profiling\ninformation that indicates resource usage for statements executed\nduring the course of the current session.\n\nProfiling is controlled by the profiling session variable, which has a\ndefault value of 0 (OFF). Profiling is enabled by setting profiling to\n1 or ON:\n\nmysql> SET profiling = 1;\n\nSHOW PROFILES displays a list of the most recent statements sent to the\nmaster. The size of the list is controlled by the\nprofiling_history_size session variable, which has a default value of\n15. The maximum value is 100. Setting the value to 0 has the practical\neffect of disabling profiling.\n\nAll statements are profiled except SHOW PROFILES and SHOW PROFILE, so\nyou will find neither of those statements in the profile list.\nMalformed statements are profiled. For example, SHOW PROFILING is an\nillegal statement, and a syntax error occurs if you try to execute it,\nbut it will show up in the profiling list.\n\nSHOW PROFILE displays detailed information about a single statement.\nWithout the FOR QUERY n clause, the output pertains to the most\nrecently executed statement. If FOR QUERY n is included, SHOW PROFILE\ndisplays information for statement n. The values of n correspond to the\nQuery_ID values displayed by SHOW PROFILES.\n\nThe LIMIT row_count clause may be given to limit the output to\nrow_count rows. If LIMIT is given, OFFSET offset may be added to begin\nthe output offset rows into the full set of rows.\n\nBy default, SHOW PROFILE displays Status and Duration columns. The\nStatus values are like the State values displayed by SHOW PROCESSLIST,\nalthought there might be some minor differences in interpretion for the\ntwo statements for some status values (see\nhttp://dev.mysql.com/doc/refman/5.1/en/thread-information.html).\n\nOp… type values may be specified to display specific additional\ntypes of information:\n\no ALL displays all information\n\no BLOCK IO displays counts for block input and output operations\n\no CONTEXT SWITCHES displays counts for voluntary and involuntary\n context switches\n\no CPU displays user and system CPU usage times\n\no IPC displays counts for messages sent and received\n\no MEMORY is not currently implemented\n\no PAGE FAULTS displays counts for major and minor page faults\n\no SOURCE displays the names of functions from the source code, together\n with the name and line number of the file in which the function\n occurs\n\no SWAPS displays swap counts\n\nProfiling is enabled per session. When a session ends, its profiling\ninformation is lost.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-profiles.html\n\n','mysql> SELECT @@profiling;\n+-------------+\n| @@profiling |\n+-------------+\n| 0 |\n+-------------+\n1 row in set (0.00 sec)\n\nmysql> SET profiling = 1;\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> DROP TABLE IF EXISTS t1;\nQuery OK, 0 rows affected, 1 warning (0.00 sec)\n\nmysql> CREATE TABLE T1 (id INT);\nQuery OK, 0 rows affected (0.01 sec)\n\nmysql> SHOW PROFILES;\n+----------+----------+--------------------------+\n| Query_ID | Duration | Query |\n+----------+----------+--------------------------+\n| 0 | 0.000088 | SET PROFILING = 1 |\n| 1 | 0.000136 | DROP TABLE IF EXISTS t1 |\n| 2 | 0.011947 | CREATE TABLE t1 (id INT) |\n+----------+----------+--------------------------+\n3 rows in set (0.00 sec)\n\nmysql> SHOW PROFILE;\n+----------------------+----------+\n| Status | Duration |\n+----------------------+----------+\n| checking permissions | 0.000040 |\n| creating table | 0.000056 |\n| After create | 0.011363 |\n| query end | 0.000375 |\n| freeing items | 0.000089 |\n| logging slow query | 0.000019 |\n| cleaning up | 0.000005 |\n+----------------------+----------+\n7 rows in set (0.00 sec)\n\nmysql> SHOW PROFILE FOR QUERY 1;\n+--------------------+----------+\n| Status | Duration |\n+--------------------+----------+\n| query end | 0.000107 |\n| freeing items | 0.000008 |\n| logging slow query | 0.000015 |\n| cleaning up | 0.000006 |\n+--------------------+----------+\n4 rows in set (0.00 sec)\n\nmysql> SHOW PROFILE CPU FOR QUERY 2;\n+----------------------+----------+----------+------------+\n| Status | Duration | CPU_user | CPU_system |\n+----------------------+----------+----------+------------+\n| checking permissions | 0.000040 | 0.000038 | 0.000002 |\n| creating table | 0.000056 | 0.000028 | 0.000028 |\n| After create | 0.011363 | 0.000217 | 0.001571 |\n| query end | 0.000375 | 0.000013 | 0.000028 |\n| freeing items | 0.000089 | 0.000010 | 0.000014 |\n| logging slow query | 0.000019 | 0.000009 | 0.000010 |\n| cleaning up | 0.000005 | 0.000003 | 0.000002 |\n+----------------------+----------+----------+------------+\n7 rows in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/show-profiles.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (81,26,'UPDATE','Syntax:\nSingle-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_reference\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n [ORDER BY ...]\n [LIMIT row_count]\n\nMultiple-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_references\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n\nFor the single-table syntax, the UPDATE statement updates columns of\nexisting rows in the named table with new values. The SET clause\nindicates which columns to modify and the values they should be given.\nEach value can be given as an expression, or the keyword DEFAULT to set\na column explicitly to its default value. The WHERE clause, if given,\nspecifies the conditions that identify which rows to update. With no\nWHERE clause, all rows are updated. If the ORDER BY clause is\nspecified, the rows are updated in the order that is specified. The\nLIMIT clause places a limit on the number of rows that can be updated.\n\nFor the multiple-table syntax, UPDATE updates rows in each table named\nin table_references that satisfy the conditions. In this case, ORDER BY\nand LIMIT cannot be used.\n\nwhere_condition is an expression that evaluates to true for each row to\nbe updated.\n\ntable_references and where_condition are is specified as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nYou need the UPDATE privilege only for columns referenced in an UPDATE\nthat are actually updated. You need only the SELECT privilege for any\ncolumns that are read but not modified.\n\nThe UPDATE statement supports the following modifiers:\n\no If you use the LOW_PRIORITY keyword, execution of the UPDATE is\n delayed until no other clients are reading from the table. This\n affects only storage engines that use only table-level locking\n (MyISAM, MEMORY, MERGE).\n\no If you use the IGNORE keyword, the update statement does not abort\n even if errors occur during the update. Rows for which duplicate-key\n conflicts occur are not updated. Rows for which columns are updated\n to values that would cause data conversion errors are updated to the\n closest valid values instead.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/update.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/update.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (81,26,'UPDATE','Syntax:\nSingle-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_reference\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n [ORDER BY ...]\n [LIMIT row_count]\n\nMultiple-table syntax:\n\nUPDATE [LOW_PRIORITY] [IGNORE] table_references\n SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...\n [WHERE where_condition]\n\nFor the single-table syntax, the UPDATE statement updates columns of\nexisting rows in the named table with new values. The SET clause\nindicates which columns to modify and the values they should be given.\nEach value can be given as an expression, or the keyword DEFAULT to set\na column explicitly to its default value. The WHERE clause, if given,\nspecifies the conditions that identify which rows to update. With no\nWHERE clause, all rows are updated. If the ORDER BY clause is\nspecified, the rows are updated in the order that is specified. The\nLIMIT clause places a limit on the number of rows that can be updated.\n\nFor the multiple-table syntax, UPDATE updates rows in each table named\nin table_references that satisfy the conditions. In this case, ORDER BY\nand LIMIT cannot be used.\n\nwhere_condition is an expression that evaluates to true for each row to\nbe updated.\n\ntable_references and where_condition are is specified as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nYou need the UPDATE privilege only for columns referenced in an UPDATE\nthat are actually updated. You need only the SELECT privilege for any\ncolumns that are read but not modified.\n\nThe UPDATE statement supports the following modifiers:\n\no If you use the LOW_PRIORITY keyword, execution of the UPDATE is\n delayed until no other clients are reading from the table. This\n affects only storage engines that use only table-level locking (such\n as MyISAM, MEMORY, and MERGE).\n\no If you use the IGNORE keyword, the update statement does not abort\n even if errors occur during the update. Rows for which duplicate-key\n conflicts occur are not updated. Rows for which columns are updated\n to values that would cause data conversion errors are updated to the\n closest valid values instead.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/update.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/update.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (82,17,'IS NOT NULL','Syntax:\nIS NOT NULL\n\nTests whether a value is not NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 1 IS NOT NULL, 0 IS NOT NULL, NULL IS NOT NULL;\n -> 1, 1, 0\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (83,22,'CASE STATEMENT','Syntax:\nCASE case_value\n WHEN when_value THEN statement_list\n [WHEN when_value THEN statement_list] ...\n [ELSE statement_list]\nEND CASE\n\nOr:\n\nCASE\n WHEN search_condition THEN statement_list\n [WHEN search_condition THEN statement_list] ...\n [ELSE statement_list]\nEND CASE\n\nThe CASE statement for stored programs implements a complex conditional\nconstruct. If a search_condition evaluates to true, the corresponding\nSQL statement list is executed. If no search condition matches, the\nstatement list in the ELSE clause is executed. Each statement_list\nconsists of one or more statements.\n\nIf no when_value or search_condition matches the value tested and the\nCASE statement contains no ELSE clause, a Case not found for CASE\nstatement error results.\n\nEach statement_list consists of one or more statements; an empty\nstatement_list is not allowed. To handle situations where no value is\nmatched by any WHEN clause, use an ELSE containing an empty BEGIN ...\nEND block, as shown in this example: DELIMITER | CREATE PROCEDURE p()\nBEGIN DECLARE v INT DEFAULT 1; CASE v WHEN 2 THEN SELECT v; WHEN 3 THEN\nSELECT 0; ELSE BEGIN END; END CASE; END; | (The indentation used here\nin the ELSE clause is for purposes of clarity only, and is not\notherwise significant.)\n\n*Note*: The syntax of the CASE statement used inside stored programs\ndiffers slightly from that of the SQL CASE expression described in\nhttp://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html. The\nCASE statement cannot have an ELSE NULL clause, and it is terminated\nwith END CASE instead of END.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/case-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/case-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (84,25,'EXECUTE STATEMENT','Syntax:\nEXECUTE stmt_name\n [USING @var_name [, @var_name] ...]\n\nAfter preparing a statement with PREPARE, you execute it with an\nEXECUTE statement that refers to the prepared statement name. If the\nprepared statement contains any parameter markers, you must supply a\nUSING clause that lists user variables containing the values to be\nbound to the parameters. Parameter values can be supplied only by user\nvariables, and the USING clause must name exactly as many variables as\nthe number of parameter markers in the statement.\n\nYou can execute a given prepared statement multiple times, passing\ndifferent variables to it or setting the variables to different values\nbefore each execution.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/execute.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/execute.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (85,37,'DROP INDEX','Syntax:\nDROP [ONLINE|OFFLINE] INDEX index_name ON tbl_name\n\nDROP INDEX drops the index named index_name from the table tbl_name.\nThis statement is mapped to an ALTER TABLE statement to drop the index.\nSee [HELP ALTER TABLE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-index.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-index.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (86,35,'MATCH AGAINST','Syntax:\nMATCH (col1,col2,...) AGAINST (expr [search_modifier])\n\nMySQL has support for full-text indexing and searching:\n\no A full-text index in MySQL is an index of type FULLTEXT.\n\no Full-text indexes can be used only with MyISAM tables, and can be\n created only for CHAR, VARCHAR, or TEXT columns.\n\no A FULLTEXT index definition can be given in the CREATE TABLE\n statement when a table is created, or added later using ALTER TABLE\n or CREATE INDEX.\n\no For large data sets, it is much faster to load your data into a table\n that has no FULLTEXT index and then create the index after that, than\n to load data into a table that has an existing FULLTEXT index.\n\nFull-text searching is performed using MATCH() ... AGAINST syntax.\nMATCH() takes a comma-separated list that names the columns to be\nsearched. AGAINST takes a string to search for, and an optional\nmodifier that indicates what type of search to perform. The search\nstring must be a literal string, not a variable or a column name. There\nare three types of full-text searches:\n\no A boolean search interprets the search string using the rules of a\n special query language. The string contains the words to search for.\n It can also contain operators that specify requirements such that a\n word must be present or absent in matching rows, or that it should be\n weighted higher or lower than usual. Common words such as "some" or\n "then" are stopwords and do not match if present in the search\n string. The IN BOOLEAN MODE modifier specifies a boolean search. For\n more information, see\n http://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html.\n\no A natural language search interprets the search string as a phrase in\n natural human language (a phrase in free text). There are no special\n operators. The stopword list applies. In addition, words that are\n present in 50% or more of the rows are considered common and do not\n match. Full-text searches are natural language searches if the IN\n NATURAL LANGUAGE MODE modifier is given or if no modifier is given.\n\no A query expansion search is a modification of a natural language\n search. The search string is used to perform a natural language\n search. Then words from the most relevant rows returned by the search\n are added to the search string and the search is done again. The\n query returns the rows from the second search. The IN NATURAL\n LANGUAGE MODE WITH QUERY EXPANSION or WITH QUERY EXPANSION modifier\n specifies a query expansion search. For more information, see\n http://dev.mysql.com/doc/refman/5.1/en/fulltext-query-expansion.html.\n\nThe IN NATURAL LANGUAGE MODE and IN NATURAL LANGUAGE MODE WITH QUERY\nEXPANSION modifiers were added in MySQL 5.1.7.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html\n\n','mysql> SELECT id, body, MATCH (title,body) AGAINST\n -> (\'Security implications of running MySQL as root\'\n -> IN NATURAL LANGUAGE MODE) AS score\n -> FROM articles WHERE MATCH (title,body) AGAINST\n -> (\'Security implications of running MySQL as root\'\n -> IN NATURAL LANGUAGE MODE);\n+----+-------------------------------------+-----------------+\n| id | body | score |\n+----+-------------------------------------+-----------------+\n| 4 | 1. Never run mysqld as root. 2. ... | 1.5219271183014 |\n| 6 | When configured properly, MySQL ... | 1.3114095926285 |\n+----+-------------------------------------+-----------------+\n2 rows in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (87,37,'CREATE EVENT','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n EVENT\n [IF NOT EXISTS]\n event_name\n ON SCHEDULE schedule\n [ON COMPLETION [NOT] PRESERVE]\n [ENABLE | DISABLE | DISABLE ON SLAVE]\n [COMMENT \'comment\']\n DO sql_statement;\n\nschedule:\n AT timestamp [+ INTERVAL interval] ...\n | EVERY interval\n [STARTS timestamp [+ INTERVAL interval] ...]\n [ENDS timestamp [+ INTERVAL interval] ...]\n\ninterval:\n quantity {YEAR | QUARTER | MONTH | DAY | HOUR | MINUTE |\n WEEK | SECOND | YEAR_MONTH | DAY_HOUR | DAY_MINUTE |\n DAY_SECOND | HOUR_MINUTE | HOUR_SECOND | MINUTE_SECOND}\n\nThis statement creates and schedules a new event. It requires the EVENT\nprivilege for the schema in which the event is to be created.\n\nThe minimum requirements for a valid CREATE EVENT statement are as\nfollows:\n\no The keywords CREATE EVENT plus an event name, which uniquely\n identifies the event within a database schema. (Prior to MySQL\n 5.1.12, the event name needed to be unique only among events created\n by the same user within a schema.)\n\no An ON SCHEDULE clause, which determines when and how often the event\n executes.\n\no A DO clause, which contains the SQL statement to be executed by an\n event.\n\nThis is an example of a minimal CREATE EVENT statement:\n\nCREATE EVENT myevent\n ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 HOUR\n DO\n UPDATE myschema.mytable SET mycol = mycol + 1;\n\nThe previous statement creates an event named myevent. This event\nexecutes once --- one hour following its creation --- by running an SQL\nstatement that increments the value of the myschema.mytable table\'s\nmycol column by 1.\n\nThe event_name must be a valid MySQL identifier with a maximum length\nof 64 characters. Event names are not case sensitive, so you cannot\nhave two events named myevent and MyEvent in the same schema. In\ngeneral, the rules governing event names are the same as those for\nnames of stored routines. See\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\nAn event is associated with a schema. If no schema is indicated as part\nof event_name, the default (current) schema is assumed. To create an\nevent in a specific schema, qualify the event name with a schema using\nschema_name.event_name syntax.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-event.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-event.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (87,37,'CREATE EVENT','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n EVENT\n [IF NOT EXISTS]\n event_name\n ON SCHEDULE schedule\n [ON COMPLETION [NOT] PRESERVE]\n [ENABLE | DISABLE | DISABLE ON SLAVE]\n [COMMENT \'comment\']\n DO sql_statement;\n\nschedule:\n AT timestamp [+ INTERVAL interval] ...\n | EVERY interval\n [STARTS timestamp [+ INTERVAL interval] ...]\n [ENDS timestamp [+ INTERVAL interval] ...]\n\ninterval:\n quantity {YEAR | QUARTER | MONTH | DAY | HOUR | MINUTE |\n WEEK | SECOND | YEAR_MONTH | DAY_HOUR | DAY_MINUTE |\n DAY_SECOND | HOUR_MINUTE | HOUR_SECOND | MINUTE_SECOND}\n\nThis statement creates and schedules a new event. It requires the EVENT\nprivilege for the schema in which the event is to be created (and\nperhaps SUPER depending on the DEFINER value, as described later). The\nevent will not run unless the Event Scheduler is enabled. For\ninformation about checking Event Scheduler status and enabling it if\nnecessary, see\nhttp://dev.mysql.com/doc/refman/5.1/en/events-configuration.html.\n\nT… minimum requirements for a valid CREATE EVENT statement are as\nfollows:\n\no The keywords CREATE EVENT plus an event name, which uniquely\n identifies the event within a database schema. (Prior to MySQL\n 5.1.12, the event name needed to be unique only among events created\n by the same user within a schema.)\n\no An ON SCHEDULE clause, which determines when and how often the event\n executes.\n\no A DO clause, which contains the SQL statement to be executed by an\n event.\n\nThis is an example of a minimal CREATE EVENT statement:\n\nCREATE EVENT myevent\n ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 1 HOUR\n DO\n UPDATE myschema.mytable SET mycol = mycol + 1;\n\nThe previous statement creates an event named myevent. This event\nexecutes once --- one hour following its creation --- by running an SQL\nstatement that increments the value of the myschema.mytable table\'s\nmycol column by 1.\n\nThe event_name must be a valid MySQL identifier with a maximum length\nof 64 characters. Event names are not case sensitive, so you cannot\nhave two events named myevent and MyEvent in the same schema. In\ngeneral, the rules governing event names are the same as those for\nnames of stored routines. See\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\nAn event is associated with a schema. If no schema is indicated as part\nof event_name, the default (current) schema is assumed. To create an\nevent in a specific schema, qualify the event name with a schema using\nschema_name.event_name syntax.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-event.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-event.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (88,4,'ABS','Syntax:\nABS(X)\n\nReturns the absolute value of X.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT ABS(2);\n -> 2\nmysql> SELECT ABS(-32);\n -> 32\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (89,31,'POLYFROMWKB','PolyFromWKB(wkb[,srid]), PolygonFromWKB(wkb[,srid])\n\nConstructs a POLYGON value using its WKB representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (90,35,'NOT LIKE','Syntax:\nexpr NOT LIKE pat [ESCAPE \'escape_char\']\n\nThis is the same as NOT (expr LIKE pat [ESCAPE \'escape_char\']).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-comparison-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-comparison-functions.html'); @@ -170,7 +170,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (101,11,'COMPRESS','Syntax:\nCOMPRESS(string_to_compress)\n\nCompresses a string and returns the result as a binary string. This\nfunction requires MySQL to have been compiled with a compression\nlibrary such as zlib. Otherwise, the return value is always NULL. The\ncompressed string can be uncompressed with UNCOMPRESS().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','mysql> SELECT LENGTH(COMPRESS(REPEAT(\'a\',1000)));\n -> 21\nmysql> SELECT LENGTH(COMPRESS(\'\'));\n -> 0\nmysql> SELECT LENGTH(COMPRESS(\'a\'));\n -> 13\nmysql> SELECT LENGTH(COMPRESS(REPEAT(\'a\',16)));\n -> 15\n','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (102,26,'INSERT','Syntax:\nINSERT [LOW_PRIORITY | DELAYED | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name [(col_name,...)]\n {VALUES | VALUE} ({expr | DEFAULT},...),(...),...\n [ ON DUPLICATE KEY UPDATE\n col_name=expr\n [, col_name=expr] ... ]\n\nOr:\n\nINSERT [LOW_PRIORITY | DELAYED | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name\n SET col_name={expr | DEFAULT}, ...\n [ ON DUPLICATE KEY UPDATE\n col_name=expr\n [, col_name=expr] ... ]\n\nOr:\n\nINSERT [LOW_PRIORITY | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name [(col_name,...)]\n SELECT ...\n [ ON DUPLICATE KEY UPDATE\n col_name=expr\n [, col_name=expr] ... ]\n\nINSERT inserts new rows into an existing table. The INSERT ... VALUES\nand INSERT ... SET forms of the statement insert rows based on\nexplicitly specified values. The INSERT ... SELECT form inserts rows\nselected from another table or tables. INSERT ... SELECT is discussed\nfurther in [HELP INSERT SELECT].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/insert.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/insert.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (103,16,'COUNT','Syntax:\nCOUNT(expr)\n\nReturns a count of the number of non-NULL values of expr in the rows\nretrieved by a SELECT statement. The result is a BIGINT value.\n\nCOUNT() returns 0 if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','mysql> SELECT student.student_name,COUNT(*)\n -> FROM student,course\n -> WHERE student.student_id=course.student_id\n -> GROUP BY student_name;\n','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (104,26,'HANDLER','Syntax:\nHANDLER tbl_name OPEN [ [AS] alias]\nHANDLER tbl_name READ index_name { = | >= | <= | < } (value1,value2,...)\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ index_name { FIRST | NEXT | PREV | LAST }\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ { FIRST | NEXT }\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name CLOSE\n\nThe HANDLER statement provides direct access to table storage engine\ninterfaces. It is available for MyISAM and InnoDB tables.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/handler.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/handler.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (104,26,'HANDLER','Syntax:\nHANDLER tbl_name OPEN [ [AS] alias]\n\nHANDLER tbl_name READ index_name { = | <= | >= | < | > } (value1,value2,...)\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ index_name { FIRST | NEXT | PREV | LAST }\n [ WHERE where_condition ] [LIMIT ... ]\nHANDLER tbl_name READ { FIRST | NEXT }\n [ WHERE where_condition ] [LIMIT ... ]\n\nHANDLER tbl_name CLOSE\n\nThe HANDLER statement provides direct access to table storage engine\ninterfaces. It is available for MyISAM and InnoDB tables.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/handler.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/handler.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (105,3,'MLINEFROMTEXT','MLineFromText(wkt[,srid]), MultiLineStringFromText(wkt[,srid])\n\nConstructs a MULTILINESTRING value using its WKT representation and\nSRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (106,31,'GEOMCOLLFROMWKB','GeomCollFromWKB(wkb[,srid]), GeometryCollectionFromWKB(wkb[,srid])\n\nConstructs a GEOMETRYCOLLECTION value using its WKB representation and\nSRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (107,37,'RENAME TABLE','Syntax:\nRENAME TABLE tbl_name TO new_tbl_name\n [, tbl_name2 TO new_tbl_name2] ...\n\nThis statement renames one or more tables.\n\nThe rename operation is done atomically, which means that no other\nsession can access any of the tables while the rename is running. For\nexample, if you have an existing table old_table, you can create\nanother table new_table that has the same structure but is empty, and\nthen replace the existing table with the empty one as follows (assuming\nthat backup_table does not already exist):\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/rename-table.html\n\n','CREATE TABLE new_table (...);\nRENAME TABLE old_table TO backup_table, new_table TO old_table;\n','http://dev.mysql.com/doc/refman/5.1/en/rename-table.html'); @@ -181,9 +181,9 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (112,19,'OPTIMIZE TABLE','Syntax:\nOPTIMIZE [NO_WRITE_TO_BINLOG | LOCAL] TABLE\n tbl_name [, tbl_name] ...\n\nOPTIMIZE TABLE should be used if you have deleted a large part of a\ntable or if you have made many changes to a table with variable-length\nrows (tables that have VARCHAR, VARBINARY, BLOB, or TEXT columns).\nDeleted rows are maintained in a linked list and subsequent INSERT\noperations reuse old row positions. You can use OPTIMIZE TABLE to\nreclaim the unused space and to defragment the data file.\n\nThis statement requires SELECT and INSERT privileges for the table.\n\nBeginning with MySQL 5.1.27, OPTIMIZE TABLE is also supported for\npartitioned tables. Also beginning with MySQL 5.1.27, you can use ALTER\nTABLE ... OPTIMIZE PARTITION to optimize one or more partitions; for\nmore information, see [HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/optimize-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/optimize-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (113,11,'DECODE','Syntax:\nDECODE(crypt_str,pass_str)\n\nDecrypts the encrypted string crypt_str using pass_str as the password.\ncrypt_str should be a string returned from ENCODE().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (114,17,'<=>','Syntax:\n<=>\n\nNULL-safe equal. This operator performs an equality comparison like the\n= operator, but returns 1 rather than NULL if both operands are NULL,\nand 0 rather than NULL if one operand is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 1 <=> 1, NULL <=> NULL, 1 <=> NULL;\n -> 1, 1, 0\nmysql> SELECT 1 = 1, NULL = NULL, 1 = NULL;\n -> 1, NULL, NULL\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (115,25,'LOAD DATA FROM MASTER','Syntax:\nLOAD DATA FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated in\nversions 4.1 of MySQL and above. We will introduce a more advanced\ntechnique (called "online backup") in a future version. That technique\nwill have the additional advantage of working with more storage\nengines.\n\nFor MySQL 5.1 and earlier, the recommended alternative solution to\nusing LOAD DATA FROM MASTER or LOAD TABLE FROM MASTER is using\nmysqldump or mysqlhotcopy. The latter requires Perl and two Perl\nmodules (DBI and DBD:mysql) and works for MyISAM and ARCHIVE tables\nonly. With mysqldump, you can create SQL dumps on the master and pipe\n(or copy) these to a mysql client on the slave. This has the advantage\nof working for all storage engines, but can be quite slow, since it\nworks using SELECT.\n\nThis statement takes a snapshot of the master and copies it to the\nslave. It updates the values of MASTER_LOG_FILE and MASTER_LOG_POS so\nthat the slave starts replicating from the correct position. Any table\nand database exclusion rules specified with the --replicate-*-do-* and\n--replicate-*-ignore-* options are honored. --replicate-rewrite-db is\nnot taken into account because a user could use this option to set up a\nnonunique mapping such as --replicate-rewrite-db="db1->db3" and\n--replicate-rewrite-db="db2->db3", which would confuse the slave when\nloading tables from the master.\n\nUse of this statement is subject to the following conditions:\n\no It works only for MyISAM tables. Attempting to load a non-MyISAM\n table results in the following error:\n\nERROR 1189 (08S01): Net error reading from master\n\no It acquires a global read lock on the master while taking the\n snapshot, which prevents updates on the master during the load\n operation.\n\nIf you are loading large tables, you might have to increase the values\nof net_read_timeout and net_write_timeout on both the master and slave\nservers. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html.\n… that LOAD DATA FROM MASTER does not copy any tables from the mysql\ndatabase. This makes it easy to have different users and privileges on\nthe master and the slave.\n\nTo use LOAD DATA FROM MASTER, the replication account that is used to\nconnect to the master must have the RELOAD and SUPER privileges on the\nmaster and the SELECT privilege for all master tables you want to load.\nAll master tables for which the user does not have the SELECT privilege\nare ignored by LOAD DATA FROM MASTER. This is because the master hides\nthem from the user: LOAD DATA FROM MASTER calls SHOW DATABASES to know\nthe master databases to load, but SHOW DATABASES returns only databases\nfor which the user has some privilege. See [HELP SHOW DATABASES]. On\nthe slave side, the user that issues LOAD DATA FROM MASTER must have\nprivileges for dropping and creating the databases and tables that are\ncopied.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (116,25,'RESET','Syntax:\nRESET reset_option [, reset_option] ...\n\nThe RESET statement is used to clear the state of various server\noperations. You must have the RELOAD privilege to execute RESET.\n\nRESET acts as a stronger version of the FLUSH statement. See [HELP\nFLUSH].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (117,27,'HELP STATEMENT','Syntax:\nHELP \'search_string\'\n\nThe HELP statement returns online information from the MySQL Reference\nmanual. Its proper operation requires that the help tables in the mysql\ndatabase be initialized with help topic information (see\nhttp://dev.mysql.com/doc/refman/5.1/en/server-side-help-support.html).… HELP statement searches the help tables for the given search string\nand displays the result of the search. The search string is not case\nsensitive.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/help.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/help.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (115,27,'HELP STATEMENT','Syntax:\nHELP \'search_string\'\n\nThe HELP statement returns online information from the MySQL Reference\nmanual. Its proper operation requires that the help tables in the mysql\ndatabase be initialized with help topic information (see\nhttp://dev.mysql.com/doc/refman/5.1/en/server-side-help-support.html).… HELP statement searches the help tables for the given search string\nand displays the result of the search. The search string is not case\nsensitive.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/help.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/help.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (116,25,'LOAD DATA FROM MASTER','Syntax:\nLOAD DATA FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated as\nof MySQL 4.1 and removed in MySQL 5.5.\n\nThe recommended alternative solution to using LOAD DATA FROM MASTER or\nLOAD TABLE FROM MASTER is using mysqldump or mysqlhotcopy. The latter\nrequires Perl and two Perl modules (DBI and DBD:mysql) and works for\nMyISAM and ARCHIVE tables only. With mysqldump, you can create SQL\ndumps on the master and pipe (or copy) these to a mysql client on the\nslave. This has the advantage of working for all storage engines, but\ncan be quite slow, since it works using SELECT.\n\nThis statement takes a snapshot of the master and copies it to the\nslave. It updates the values of MASTER_LOG_FILE and MASTER_LOG_POS so\nthat the slave starts replicating from the correct position. Any table\nand database exclusion rules specified with the --replicate-*-do-* and\n--replicate-*-ignore-* options are honored. --replicate-rewrite-db is\nnot taken into account because a user could use this option to set up a\nnonunique mapping such as --replicate-rewrite-db="db1->db3" and\n--replicate-rewrite-db="db2->db3", which would confuse the slave when\nloading tables from the master.\n\nUse of this statement is subject to the following conditions:\n\no It works only for MyISAM tables. Attempting to load a non-MyISAM\n table results in the following error:\n\nERROR 1189 (08S01): Net error reading from master\n\no It acquires a global read lock on the master while taking the\n snapshot, which prevents updates on the master during the load\n operation.\n\nIf you are loading large tables, you might have to increase the values\nof net_read_timeout and net_write_timeout on both the master and slave\nservers. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html.\n… that LOAD DATA FROM MASTER does not copy any tables from the mysql\ndatabase. This makes it easy to have different users and privileges on\nthe master and the slave.\n\nTo use LOAD DATA FROM MASTER, the replication account that is used to\nconnect to the master must have the RELOAD and SUPER privileges on the\nmaster and the SELECT privilege for all master tables you want to load.\nAll master tables for which the user does not have the SELECT privilege\nare ignored by LOAD DATA FROM MASTER. This is because the master hides\nthem from the user: LOAD DATA FROM MASTER calls SHOW DATABASES to know\nthe master databases to load, but SHOW DATABASES returns only databases\nfor which the user has some privilege. See [HELP SHOW DATABASES]. On\nthe slave side, the user that issues LOAD DATA FROM MASTER must have\nprivileges for dropping and creating the databases and tables that are\ncopied.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data-from-master.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (117,25,'RESET','Syntax:\nRESET reset_option [, reset_option] ...\n\nThe RESET statement is used to clear the state of various server\noperations. You must have the RELOAD privilege to execute RESET.\n\nRESET acts as a stronger version of the FLUSH statement. See [HELP\nFLUSH].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (118,14,'GET_LOCK','Syntax:\nGET_LOCK(str,timeout)\n\nTries to obtain a lock with a name given by the string str, using a\ntimeout of timeout seconds. Returns 1 if the lock was obtained\nsuccessfully, 0 if the attempt timed out (for example, because another\nclient has previously locked the name), or NULL if an error occurred\n(such as running out of memory or the thread was killed with mysqladmin\nkill). If you have a lock obtained with GET_LOCK(), it is released when\nyou execute RELEASE_LOCK(), execute a new GET_LOCK(), or your\nconnection terminates (either normally or abnormally). Locks obtained\nwith GET_LOCK() do not interact with transactions. That is, committing\na transaction does not release any such locks obtained during the\ntransaction.\n\nThis function can be used to implement application locks or to simulate\nrecord locks. Names are locked on a server-wide basis. If a name has\nbeen locked by one client, GET_LOCK() blocks any request by another\nclient for a lock with the same name. This allows clients that agree on\na given lock name to use the name to perform cooperative advisory\nlocking. But be aware that it also allows a client that is not among\nthe set of cooperating clients to lock a name, either inadvertently or\ndeliberately, and thus prevent any of the cooperating clients from\nlocking that name. One way to reduce the likelihood of this is to use\nlock names that are database-specific or application-specific. For\nexample, use lock names of the form db_name.str or app_name.str.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','mysql> SELECT GET_LOCK(\'lock1\',10);\n -> 1\nmysql> SELECT IS_FREE_LOCK(\'lock2\');\n -> 1\nmysql> SELECT GET_LOCK(\'lock2\',10);\n -> 1\nmysql> SELECT RELEASE_LOCK(\'lock2\');\n -> 1\nmysql> SELECT RELEASE_LOCK(\'lock1\');\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (119,35,'UCASE','Syntax:\nUCASE(str)\n\nUCASE() is a synonym for UPPER().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (120,25,'SHOW BINLOG EVENTS','Syntax:\nSHOW BINLOG EVENTS\n [IN \'log_name\'] [FROM pos] [LIMIT [offset,] row_count]\n\nShows the events in the binary log. If you do not specify \'log_name\',\nthe first binary log is displayed.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-binlog-events.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-binlog-events.html'); @@ -199,7 +199,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (130,25,'SHOW OPEN TABLES','Syntax:\nSHOW OPEN TABLES [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW OPEN TABLES lists the non-TEMPORARY tables that are currently open\nin the table cache. See\nhttp://dev.mysql.com/doc/refman/5.1/en/table-cache.html. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nThe FROM and LIKE clauses may be used beginning with MySQL 5.1.24. The\nLIKE clause, if present, indicates which table names to match. The FROM\nclause, if present, restricts the tables shown to those present in the\ndb_name database.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-open-tables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-open-tables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (131,30,'EXTRACT','Syntax:\nEXTRACT(unit FROM date)\n\nThe EXTRACT() function uses the same kinds of unit specifiers as\nDATE_ADD() or DATE_SUB(), but extracts parts from the date rather than\nperforming date arithmetic.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT EXTRACT(YEAR FROM \'2009-07-02\');\n -> 2009\nmysql> SELECT EXTRACT(YEAR_MONTH FROM \'2009-07-02 01:02:03\');\n -> 200907\nmysql> SELECT EXTRACT(DAY_MINUTE FROM \'2009-07-02 01:02:03\');\n -> 20102\nmysql> SELECT EXTRACT(MICROSECOND\n -> FROM \'2003-01-02 10:30:00.000123\');\n -> 123\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (132,11,'ENCRYPT','Syntax:\nENCRYPT(str[,salt])\n\nEncrypts str using the Unix crypt() system call and returns a binary\nstring. The salt argument should be a string with at least two\ncharacters. If no salt argument is given, a random value is used.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','mysql> SELECT ENCRYPT(\'hello\');\n -> \'VxuFAJXVARROc\'\n','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (133,25,'SHOW STATUS','Syntax:\nSHOW [GLOBAL | SESSION] STATUS\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW STATUS provides server status information. This information also\ncan be obtained using the mysqladmin extended-status command. The LIKE\nclause, if present, indicates which variable names to match. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\nThis statement does not require any privilege. It requires only the\nability to connect to the server.\nWith a LIKE clause, the statement displays only rows for those\nvariables with names that match the pattern:\n\nmysql> SHOW STATUS LIKE \'Key%\';\n+--------------------+----------+\n| Variable_name | Value |\n+--------------------+----------+\n| Key_blocks_used | 14955 |\n| Key_read_requests | 96854827 |\n| Key_reads | 162040 |\n| Key_write_requests | 7589728 |\n| Key_writes | 3813196 |\n+--------------------+----------+\n\nWith the GLOBAL modifier, SHOW STATUS displays the status values for\nall connections to MySQL. With SESSION, it displays the status values\nfor the current connection. If no modifier is present, the default is\nSESSION. LOCAL is a synonym for SESSION.\n\nSome status variables have only a global value. For these, you get the\nsame value for both GLOBAL and SESSION. The scope for each status\nvariable is listed at\nhttp://dev.mysql.com/doc/refman/5.1/en/server-status-variables.html.\n\…: http://dev.mysql.com/doc/refman/5.1/en/show-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-status.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (133,25,'SHOW STATUS','Syntax:\nSHOW [GLOBAL | SESSION] STATUS\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW STATUS provides server status information. This information also\ncan be obtained using the mysqladmin extended-status command. The LIKE\nclause, if present, indicates which variable names to match. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\nThis statement does not require any privilege. It requires only the\nability to connect to the server.\nWith a LIKE clause, the statement displays only rows for those\nvariables with names that match the pattern:\n\nmysql> SHOW STATUS LIKE \'Key%\';\n+--------------------+----------+\n| Variable_name | Value |\n+--------------------+----------+\n| Key_blocks_used | 14955 |\n| Key_read_requests | 96854827 |\n| Key_reads | 162040 |\n| Key_write_requests | 7589728 |\n| Key_writes | 3813196 |\n+--------------------+----------+\n\nWith the GLOBAL modifier, SHOW STATUS displays the status values for\nall connections to MySQL. With SESSION, it displays the status values\nfor the current connection. If no modifier is present, the default is\nSESSION. LOCAL is a synonym for SESSION.\n\nSome status variables have only a global value. For these, you get the\nsame value for both GLOBAL and SESSION. The scope for each status\nvariable is listed at\nhttp://dev.mysql.com/doc/refman/5.1/en/server-status-variables.html.\n\… invocation of the SHOW STATUS statement uses an internal temporary\ntable and increments the global Created_tmp_tables value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-status.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (134,35,'EXTRACTVALUE','Syntax:\nExtractValue(xml_frag, xpath_expr)\n\nExtractValue() takes two string arguments, a fragment of XML markup\nxml_frag and an XPath expression xpath_expr (also known as a locator);\nit returns the text (CDATA) of the first text node which is a child of\nthe element(s) matched by the XPath expression. It is the equivalent of\nperforming a match using the xpath_expr after appending /text(). In\nother words, ExtractValue(\'<a>Sakila</a>\', \'/a/b\') and\nExtractValue(\'<a>Sakila</a>\', \'/a/b/text()\') produce the same\nresult.\n\nIf multiple matches are found, then the content of the first child text\nnode of each matching element is returned (in the order matched) as a\nsingle, space-delimited string.\n\nIf no matching text node is found for the expression (including the\nimplicit /text()) --- for whatever reason, as long as xpath_expr is\nvalid, and xml_frag consists of elements which are properly nested and\nclosed --- an empty string is returned. No distinction is made between\na match on an empty element and no match at all. This is by design.\n\nIf you need to determine whether no matching element was found in\nxml_frag or such an element was found but contained no child text\nnodes, you should test the result of an expression that uses the XPath\ncount() function. For example, both of these statements return an empty\nstring, as shown here:\n\nmysql> SELECT ExtractValue(\'<a></a>\', \'/a/b\');\n+-------------------------------------+\n| ExtractValue(\'<a></a>\', \'/a/b\') |\n+-------------------------------------+\n| |\n+-------------------------------------+\n1 row in set (0.00 sec)\n\nmysql> SELECT ExtractValue(\'<a><c/></a>\', \'/a/b\');\n+-------------------------------------+\n| ExtractValue(\'<a><c/></a>\', \'/a/b\') |\n+-------------------------------------+\n| |\n+-------------------------------------+\n1 row in set (0.00 sec)\n\nHowever, you can determine whether there was actually a matching\nelement using the following:\n\nmysql> SELECT ExtractValue(\'<a></a>\', \'count(/a/b)\');\n+-------------------------------------+\n| ExtractValue(\'<a></a>\', \'count(/a/b)\') |\n+-------------------------------------+\n| 1 |\n+-------------------------------------+\n1 row in set (0.00 sec)\n\nmysql> SELECT ExtractValue(\'<a><c/></a>\', \'count(/a/b)\');\n+-------------------------------------+\n| ExtractValue(\'<a><c/></a>\', \'count(/a/b)\') |\n+-------------------------------------+\n| 0 |\n+-------------------------------------+\n1 row in set (0.01 sec)\n\n*Important*: ExtractValue() returns only CDATA, and does not return any\ntags that might be contained within a matching tag, nor any of their\ncontent (see the result returned as val1 in the following example).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html\n\n','mysql> SELECT\n -> ExtractValue(\'<a>cccddd</a>\', \'/a\') AS val1,\n -> ExtractValue(\'<a>cccddd</a>\', \'/a/b\') AS val2,\n -> ExtractValue(\'<a>cccddd</a>\', \'//b\') AS val3,\n -> ExtractValue(\'<a>cccddd</a>\', \'/b\') AS val4,\n -> ExtractValue(\'<a>cccdddeee</a>\', \'//b\') AS val5;\n\n+------+------+------+------+---------+\n| val1 | val2 | val3 | val4 | val5 |\n+------+------+------+------+---------+\n| ccc | ddd | ddd | | ddd eee |\n+------+------+------+------+---------+\n','http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (135,11,'OLD_PASSWORD','Syntax:\nOLD_PASSWORD(str)\n\nOLD_PASSWORD() was added to MySQL when the implementation of PASSWORD()\nwas changed to improve security. OLD_PASSWORD() returns the value of\nthe old (pre-4.1) implementation of PASSWORD() as a binary string, and\nis intended to permit you to reset passwords for any pre-4.1 clients\nthat need to connect to your version 5.1 MySQL server without locking\nthem out. See\nhttp://dev.mysql.com/doc/refman/5.1/en/password-hashing.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (136,22,'SET VARIABLE','Syntax:\nSET var_name = expr [, var_name = expr] ...\n\nThe SET statement in stored programs is an extended version of the\ngeneral SET statement (see [HELP SET]). Each var_name may refer to a\nlocal variable declared inside a stored program, a system variable, or\na user-defined variable.\n\nThe SET statement in stored programs is implemented as part of the\npre-existing SET syntax. This allows an extended syntax of SET a=x,\nb=y, ... where different variable types (locally declared variables,\nglobal and session system variables, user-defined variables) can be\nmixed. This also allows combinations of local variables and some\noptions that make sense only for system variables; in that case, the\noptions are recognized but ignored.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-statement.html'); @@ -239,31 +239,31 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (170,30,'DAYOFYEAR','Syntax:\nDAYOFYEAR(date)\n\nReturns the day of the year for date, in the range 1 to 366.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DAYOFYEAR(\'2007-02-03\');\n -> 34\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (171,20,'LONGTEXT','LONGTEXT [CHARACTER SET charset_name] [COLLATE collation_name]\n\nA TEXT column with a maximum length of 4,294,967,295 or 4GB (232 - 1)\ncharacters. The effective maximum length is less if the value contains\nmulti-byte characters. The effective maximum length of LONGTEXT columns\nalso depends on the configured maximum packet size in the client/server\nprotocol and available memory. Each LONGTEXT value is stored using a\nfour-byte length prefix that indicates the number of bytes in the\nvalue.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (172,4,'%','Syntax:\nN % M\n\nModulo operation. Returns the remainder of N divided by M. For more\ninformation, see the description for the MOD() function in\nhttp://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html.\n\n…: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (173,25,'KILL','Syntax:\nKILL [CONNECTION | QUERY] thread_id\n\nEach connection to mysqld runs in a separate thread. You can see which\nthreads are running with the SHOW PROCESSLIST statement and kill a\nthread with the KILL thread_id statement.\n\nKILL allows the optional CONNECTION or QUERY modifier:\n\no KILL CONNECTION is the same as KILL with no modifier: It terminates\n the connection associated with the given thread_id.\n\no KILL QUERY terminates the statement that the connection is currently\n executing, but leaves the connection itself intact.\n\nIf you have the PROCESS privilege, you can see all threads. If you have\nthe SUPER privilege, you can kill all threads and statements.\nOtherwise, you can see and kill only your own threads and statements.\n\nYou can also use the mysqladmin processlist and mysqladmin kill\ncommands to examine and kill threads.\n\n*Note*: You cannot use KILL with the Embedded MySQL Server library,\nbecause the embedded server merely runs inside the threads of the host\napplication. It does not create any connection threads of its own.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/kill.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/kill.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (173,25,'KILL','Syntax:\nKILL [CONNECTION | QUERY] thread_id\n\nEach connection to mysqld runs in a separate thread. You can see which\nthreads are running with the SHOW PROCESSLIST statement and kill a\nthread with the KILL thread_id statement.\n\nKILL allows the optional CONNECTION or QUERY modifier:\n\no KILL CONNECTION is the same as KILL with no modifier: It terminates\n the connection associated with the given thread_id.\n\no KILL QUERY terminates the statement that the connection is currently\n executing, but leaves the connection itself intact.\n\nIf you have the PROCESS privilege, you can see all threads. If you have\nthe SUPER privilege, you can kill all threads and statements.\nOtherwise, you can see and kill only your own threads and statements.\n\nYou can also use the mysqladmin processlist and mysqladmin kill\ncommands to examine and kill threads.\n\n*Note*: You cannot use KILL with the Embedded MySQL Server library\nbecause the embedded server merely runs inside the threads of the host\napplication. It does not create any connection threads of its own.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/kill.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/kill.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (174,29,'DISJOINT','Disjoint(g1,g2)\n\nReturns 1 or 0 to indicate whether g1 is spatially disjoint from (does\nnot intersect) g2.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (175,3,'ASTEXT','AsText(g), AsWKT(g)\n\nConverts a value in internal geometry format to its WKT representation\nand returns the string result.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…','mysql> SET @g = \'LineString(1 1,2 2,3 3)\';\nmysql> SELECT AsText(GeomFromText(@g));\n+--------------------------+\n| AsText(GeomFromText(@g)) |\n+--------------------------+\n| LINESTRING(1 1,2 2,3 3) |\n+--------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (176,35,'LPAD','Syntax:\nLPAD(str,len,padstr)\n\nReturns the string str, left-padded with the string padstr to a length\nof len characters. If str is longer than len, the return value is\nshortened to len characters.\n\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT LPAD(\'hi\',4,\'??\');\n -> \'??hi\'\nmysql> SELECT LPAD(\'hi\',1,\'??\');\n -> \'h\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (177,19,'RESTORE TABLE','Syntax:\nRESTORE TABLE tbl_name [, tbl_name] ... FROM \'/path/to/backup/directory\'\n\nRESTORE TABLE restores the table or tables from a backup that was made\nwith BACKUP TABLE. The directory should be specified as a full path\nname.\n\nExisting tables are not overwritten; if you try to restore over an\nexisting table, an error occurs. Just as for BACKUP TABLE, RESTORE\nTABLE currently works only for MyISAM tables. Restored tables are not\nreplicated from master to slave.\n\nThe backup for each table consists of its .frm format file and .MYD\ndata file. The restore operation restores those files, and then uses\nthem to rebuild the .MYI index file. Restoring takes longer than\nbacking up due to the need to rebuild the indexes. The more indexes the\ntable has, the longer it takes.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/restore-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/restore-table.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (178,22,'DECLARE CONDITION','Syntax:\nDECLARE condition_name CONDITION FOR condition_value\n\ncondition_value:\n SQLSTATE [VALUE] sqlstate_value\n | mysql_error_code\n\nThe DECLARE ... CONDITION statement defines a named error condition. It\nspecifies a condition that needs specific handling and associates a\nname with that condition. The name can be referred to in a subsequence\nDECLARE ... HANDLER statement. See [HELP DECLARE HANDLER].\n\nA condition_value for DECLARE ... CONDITION can be an SQLSTATE value (a\n5-character string literal) or a MySQL error code (a number). You\nshould not use SQLSTATE value \'00000\' or MySQL error code 0, because\nthose indicate sucess rather than an error condition. For a list of\nSQLSTATE values and MySQL error codes, see\nhttp://dev.mysql.com/doc/refman/5.1/en/error-messages-server.html.\n\n…: http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (177,19,'RESTORE TABLE','Syntax:\nRESTORE TABLE tbl_name [, tbl_name] ... FROM \'/path/to/backup/directory\'\n\n*Note*: This statement is deprecated and is removed in MySQL 5.5.\n\nRESTORE TABLE restores the table or tables from a backup that was made\nwith BACKUP TABLE. The directory should be specified as a full path\nname.\n\nExisting tables are not overwritten; if you try to restore over an\nexisting table, an error occurs. Just as for BACKUP TABLE, RESTORE\nTABLE currently works only for MyISAM tables. Restored tables are not\nreplicated from master to slave.\n\nThe backup for each table consists of its .frm format file and .MYD\ndata file. The restore operation restores those files, and then uses\nthem to rebuild the .MYI index file. Restoring takes longer than\nbacking up due to the need to rebuild the indexes. The more indexes the\ntable has, the longer it takes.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/restore-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/restore-table.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (178,22,'DECLARE CONDITION','Syntax:\nDECLARE condition_name CONDITION FOR condition_value\n\ncondition_value:\n SQLSTATE [VALUE] sqlstate_value\n | mysql_error_code\n\nThe DECLARE ... CONDITION statement defines a named error condition. It\nspecifies a condition that needs specific handling and associates a\nname with that condition. The name can be referred to in a subsequent\nDECLARE ... HANDLER statement. For an example, see [HELP DECLARE\nHANDLER].\n\nA condition_value for DECLARE ... CONDITION can be an SQLSTATE value (a\n5-character string literal) or a MySQL error code (a number). You\nshould not use SQLSTATE value \'00000\' or MySQL error code 0, because\nthose indicate success rather than an error condition. For a list of\nSQLSTATE values and MySQL error codes, see\nhttp://dev.mysql.com/doc/refman/5.1/en/error-messages-server.html.\n\n…: http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-condition.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (179,29,'OVERLAPS','Overlaps(g1,g2)\n\nReturns 1 or 0 to indicate whether g1 spatially overlaps g2. The term\nspatially overlaps is used if two geometries intersect and their\nintersection results in a geometry of the same dimension but not equal\nto either of the given geometries.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (180,25,'SET GLOBAL SQL_SLAVE_SKIP_COUNTER','Syntax:\nSET GLOBAL SQL_SLAVE_SKIP_COUNTER = N\n\nThis statement skips the next N events from the master. This is useful\nfor recovering from replication stops caused by a statement.\n\nThis statement is valid only when the slave thread is not running.\nOtherwise, it produces an error.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…','','http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (180,25,'SET GLOBAL SQL_SLAVE_SKIP_COUNTER','Syntax:\nSET GLOBAL sql_slave_skip_counter = N\n\nThis statement skips the next N events from the master. This is useful\nfor recovering from replication stops caused by a statement.\n\nThis statement is valid only when the slave threads are not running.\nOtherwise, it produces an error.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…','','http://dev.mysql.com/doc/refman/5.1/en/set-global-sql-slave-skip-counter.ht…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (181,24,'NUMGEOMETRIES','NumGeometries(gc)\n\nReturns the number of geometries in the GeometryCollection value gc.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#geo…','mysql> SET @gc = \'GeometryCollection(Point(1 1),LineString(2 2, 3 3))\';\nmysql> SELECT NumGeometries(GeomFromText(@gc));\n+----------------------------------+\n| NumGeometries(GeomFromText(@gc)) |\n+----------------------------------+\n| 2 |\n+----------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#geo…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (182,30,'MONTHNAME','Syntax:\nMONTHNAME(date)\n\nReturns the full name of the month for date. As of MySQL 5.1.12, the\nlanguage used for the name is controlled by the value of the\nlc_time_names system variable\n(http://dev.mysql.com/doc/refman/5.1/en/locale-support.html).\n\n…: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MONTHNAME(\'2008-02-03\');\n -> \'February\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (183,36,'PROCEDURE ANALYSE','Syntax:\nANALYSE([max_elements[,max_memory]])\n\nANALYSE() is defined in the sql/sql_analyse.cc source file, which\nserves as an example of how to create a procedure for use with the\nPROCEDURE clause of SELECT statements. ANALYSE() is built in and is\navailable by default; other procedures can be created using the format\ndemonstrated in the source file.\n\nANALYSE() examines the result from a query and returns an analysis of\nthe results that suggests optimal data types for each column that may\nhelp reduce table sizes. To obtain this analysis, append PROCEDURE\nANALYSE to the end of a SELECT statement:\n\nSELECT ... FROM ... WHERE ... PROCEDURE ANALYSE([max_elements,[max_memory]])\n\nFor example:\n\nSELECT col1, col2 FROM table1 PROCEDURE ANALYSE(10, 2000);\n\nThe results show some statistics for the values returned by the query,\nand propose an optimal data type for the columns. This can be helpful\nfor checking your existing tables, or after importing new data. You may\nneed to try different settings for the arguments so that PROCEDURE\nANALYSE() does not suggest the ENUM data type when it is not\nappropriate.\n\nThe arguments are optional and are used as follows:\n\no max_elements (default 256) is the maximum number of distinct values\n that ANALYSE() notices per column. This is used by ANALYSE() to check\n whether the optimal data type should be of type ENUM; if there are\n more than max_elements distinct values, then ENUM is not a suggested\n type.\n\no max_memory (default 8192) is the maximum amount of memory that\n ANALYSE() should allocate per column while trying to find all\n distinct values.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/procedure-analyse.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/procedure-analyse.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (184,25,'CHANGE MASTER TO','Syntax:\nCHANGE MASTER TO master_def [, master_def] ...\n\nmaster_def:\n MASTER_BIND = \'interface_name\'\n | MASTER_HOST = \'host_name\'\n | MASTER_USER = \'user_name\'\n | MASTER_PASSWORD = \'password\'\n | MASTER_PORT = port_num\n | MASTER_CONNECT_RETRY = interval\n | MASTER_HEARTBEAT_PERIOD = interval\n | MASTER_LOG_FILE = \'master_log_name\'\n | MASTER_LOG_POS = master_log_pos\n | RELAY_LOG_FILE = \'relay_log_name\'\n | RELAY_LOG_POS = relay_log_pos\n | MASTER_SSL = {0|1}\n | MASTER_SSL_CA = \'ca_file_name\'\n | MASTER_SSL_CAPATH = \'ca_directory_name\'\n | MASTER_SSL_CERT = \'cert_file_name\'\n | MASTER_SSL_KEY = \'key_file_name\'\n | MASTER_SSL_CIPHER = \'cipher_list\'\n | MASTER_SSL_VERIFY_SERVER_CERT = {0|1}\n\nCHANGE MASTER TO changes the parameters that the slave server uses for\nconnecting to and communicating with the master server. It also updates\nthe contents of the master.info and relay-log.info files.\n\nMASTER_USER, MASTER_PASSWORD, MASTER_SSL, MASTER_SSL_CA,\nMASTER_SSL_CAPATH, MASTER_SSL_CERT, MASTER_SSL_KEY, MASTER_SSL_CIPHER,\nand MASTER_SSL_VERIFY_SERVER_CERT provide information to the slave\nabout how to connect to its master. MASTER_SSL_VERIFY_SERVER_CERT was\nadded in MySQL 5.1.18. It is used as described for the\n--ssl-verify-server-cert option in\nhttp://dev.mysql.com/doc/refman/5.1/en/ssl-options.html.\n\nMASTER_CONN… specifies how many seconds to wait between connect\nretries. The default is 60. The number of reconnection attempts is\nlimited by the --master-retry-count server option; for more\ninformation, see\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-options.html.\n\nThe SSL options (MASTER_SSL, MASTER_SSL_CA, MASTER_SSL_CAPATH,\nMASTER_SSL_CERT, MASTER_SSL_KEY, MASTER_SSL_CIPHER), and\nMASTER_SSL_VERIFY_SERVER_CERT can be changed even on slaves that are\ncompiled without SSL support. They are saved to the master.info file,\nbut are ignored unless you use a server that has SSL support enabled.\n\nIf you do not specify a given parameter, it keeps its old value, except\nas indicated in the following discussion. For example, if the password\nto connect to your MySQL master has changed, you just need to issue\nthese statements to tell the slave about the new password:\n\nSTOP SLAVE; -- if replication was running\nCHANGE MASTER TO MASTER_PASSWORD=\'new3cret\';\nSTART SLAVE; -- if you want to restart replication\n\nThere is no need to specify the parameters that do not change (host,\nport, user, and so forth).\n\nMASTER_HOST and MASTER_PORT are the host name (or IP address) of the\nmaster host and its TCP/IP port.\n\nThe next two options (MASTER_BIND and MASTER_HEARTBEAT_PERIOD) are\navailable in MySQL Cluster NDB 6.3 and later, but are not supported in\nmainline MySQL 5.1:\n\no MASTER_BIND is for use on replication slaves having multiple network\n interfaces, and determines which of the slave\'s network interfaces is\n chosen for connecting to the master. It is also possible to determine\n which network interface is to be used in such cases by starting the\n slave mysqld process with the --master-bind option.\n\n The ability to bind a replication slave to specific network interface\n was added in MySQL Cluster NDB 6.3.4.\n\no MASTER_HEARTBEAT_PERIOD is used to set the interval in seconds\n between replication heartbeats. Whenever the master\'s binlog is\n updated with an event, the waiting period for the next heartbeat is\n reset. interval is a decimal value having the range 0 to 4294967\n seconds and a resolution to hundredths of a second; the smallest\n nonzero value is 0.001. Heartbeats are sent by the master only if\n there are no unsent events in the binlog file for a period longer\n than interval.\n\n Setting interval to 0 disables heartbeats altogether. The default\n value for interval is equal to the value of slave_net_timeout divided\n by 2.\n\n Setting @@global.slave_net_timeout to a value less than that of the\n current heartbeat interval results in a warning being issued. The\n effect of issuing RESET SLAVE on the heartbeat interval is to reset\n it to the default value.\n\n MASTER_HEARTBEAT_PERIOD was added in MySQL Cluster NDB 6.3.4.\n\n*Note*: Replication cannot use Unix socket files. You must be able to\nconnect to the master MySQL server using TCP/IP.\n\nIf you specify MASTER_HOST or MASTER_PORT, the slave assumes that the\nmaster server is different from before (even if you specify a host or\nport value that is the same as the current value.) In this case, the\nold values for the master binary log name and position are considered\nno longer applicable, so if you do not specify MASTER_LOG_FILE and\nMASTER_LOG_POS in the statement, MASTER_LOG_FILE=\'\' and\nMASTER_LOG_POS=4 are silently appended to it.\n\nSetting MASTER_HOST=\'\' --- that is, setting its value explicitly to an\nempty string --- is not the same as not setting it at all. Setting this\noption to an empty string causes START SLAVE subsequently to fail. This\nissue is addressed in MySQL 5.5. (Bug#28796\n(http://bugs.mysql.com/28796))\n\nMASTER_LOG_FILE and MASTER_LOG_POS are the coordinates at which the\nslave I/O thread should begin reading from the master the next time the\nthread starts. If you specify either of them, you cannot specify\nRELAY_LOG_FILE or RELAY_LOG_POS. If neither of MASTER_LOG_FILE or\nMASTER_LOG_POS are specified, the slave uses the last coordinates of\nthe slave SQL thread before CHANGE MASTER TO was issued. This ensures\nthat there is no discontinuity in replication, even if the slave SQL\nthread was late compared to the slave I/O thread, when you merely want\nto change, say, the password to use.\n\nCHANGE MASTER TO deletes all relay log files and starts a new one,\nunless you specify RELAY_LOG_FILE or RELAY_LOG_POS. In that case, relay\nlogs are kept; the relay_log_purge global variable is set silently to\n0.\n\nCHANGE MASTER TO is useful for setting up a slave when you have the\nsnapshot of the master and have recorded the log and the offset\ncorresponding to it. After loading the snapshot into the slave, you can\nrun CHANGE MASTER TO MASTER_LOG_FILE=\'log_name_on_master\',\nMASTER_LOG_POS=log_offset_on_master on the slave.\n\nThe following example changes the master and master\'s binary log\ncoordinates. This is used when you want to set up the slave to\nreplicate the master:\n\nCHANGE MASTER TO\n MASTER_HOST=\'master2.mycompany.com\',\n MASTER_USER=\'replication\',\n MASTER_PASSWORD=\'bigs3cret\',\n MASTER_PORT=3306,\n MASTER_LOG_FILE=\'master2-bin.001\',\n MASTER_LOG_POS=4,\n MASTER_CONNECT_RETRY=10;\n\nThe next example shows an operation that is less frequently employed.\nIt is used when the slave has relay logs that you want it to execute\nagain for some reason. To do this, the master need not be reachable.\nYou need only use CHANGE MASTER TO and start the SQL thread (START\nSLAVE SQL_THREAD):\n\nCHANGE MASTER TO\n RELAY_LOG_FILE=\'slave-relay-bin.006\',\n RELAY_LOG_POS=4025;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (184,25,'CHANGE MASTER TO','Syntax:\nCHANGE MASTER TO option [, option] ...\n\noption:\n MASTER_BIND = \'interface_name\'\n | MASTER_HOST = \'host_name\'\n | MASTER_USER = \'user_name\'\n | MASTER_PASSWORD = \'password\'\n | MASTER_PORT = port_num\n | MASTER_CONNECT_RETRY = interval\n | MASTER_HEARTBEAT_PERIOD = interval\n | MASTER_LOG_FILE = \'master_log_name\'\n | MASTER_LOG_POS = master_log_pos\n | RELAY_LOG_FILE = \'relay_log_name\'\n | RELAY_LOG_POS = relay_log_pos\n | MASTER_SSL = {0|1}\n | MASTER_SSL_CA = \'ca_file_name\'\n | MASTER_SSL_CAPATH = \'ca_directory_name\'\n | MASTER_SSL_CERT = \'cert_file_name\'\n | MASTER_SSL_KEY = \'key_file_name\'\n | MASTER_SSL_CIPHER = \'cipher_list\'\n | MASTER_SSL_VERIFY_SERVER_CERT = {0|1}\n\nCHANGE MASTER TO changes the parameters that the slave server uses for\nconnecting to the master server, for reading the master binary log, and\nreading the slave relay log. It also updates the contents of the\nmaster.info and relay-log.info files. To use CHANGE MASTER TO, the\nslave replication threads must be stopped (use STOP SLAVE if\nnecessary).\n\nOptions not specified retain their value, except as indicated in the\nfollowing discussion. Thus, in most cases, there is no need to specify\noptions that do not change. For example, if the password to connect to\nyour MySQL master has changed, you just need to issue these statements\nto tell the slave about the new password:\n\nSTOP SLAVE; -- if replication was running\nCHANGE MASTER TO MASTER_PASSWORD=\'new3cret\';\nSTART SLAVE; -- if you want to restart replication\n\nMASTER_HOST, MASTER_USER, MASTER_PASSWORD, and MASTER_PORT provide\ninformation to the slave about how to connect to its master:\n\no MASTER_HOST and MASTER_PORT are the host name (or IP address) of the\n master host and its TCP/IP port.\n\n *Note*: Replication cannot use Unix socket files. You must be able to\n connect to the master MySQL server using TCP/IP.\n\n If you specify the MASTER_HOST or MASTER_PORT option, the slave\n assumes that the master server is different from before (even if the\n option value is the same as its current value.) In this case, the old\n values for the master binary log file name and position are\n considered no longer applicable, so if you do not specify\n MASTER_LOG_FILE and MASTER_LOG_POS in the statement,\n MASTER_LOG_FILE=\'\' and MASTER_LOG_POS=4 are silently appended to it.\n\n Setting MASTER_HOST=\'\' (that is, setting its value explicitly to an\n empty string) is not the same as not setting MASTER_HOST at all.\n Setting this option to an empty string causes START SLAVE\n subsequently to fail. This issue is addressed in MySQL 5.5.\n (Bug#28796 (http://bugs.mysql.com/bug.php?id=28796))\n\no MASTER_USER and MASTER_PASSWORD are the user name and password of the\n account to use for connecting to the master.\n\nThe MASTER_SSL_xxx options provide information about using SSL for the\nconnection. They correspond to the --ssl-xxx options described in\nhttp://dev.mysql.com/doc/refman/5.1/en/ssl-options.html, and\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-solutions-ssl.html.… was added in MySQL 5.1.18. These options\ncan be changed even on slaves that are compiled without SSL support.\nThey are saved to the master.info file, but are ignored if the slave\ndoes not have SSL support enabled.\n\nMASTER_CONNECT_RETRY specifies how many seconds to wait between connect\nretries. The default is 60. The number of reconnection attempts is\nlimited by the --master-retry-count server option; for more\ninformation, see\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-options.html.\n\nThe next two options (MASTER_BIND and MASTER_HEARTBEAT_PERIOD) are\navailable in MySQL Cluster NDB 6.3 and later, but are not supported in\nmainline MySQL 5.1:\n\nMASTER_BIND is for use on replication slaves having multiple network\ninterfaces, and determines which of the slave\'s network interfaces is\nchosen for connecting to the master. It is also possible to determine\nwhich network interface is to be used in such cases by starting the\nslave mysqld process with the --master-bind option.\n\nThe ability to bind a replication slave to specific network interface\nwas added in MySQL Cluster NDB 6.3.4.\n\nMASTER_HEARTBEAT_PERIOD is used to set the interval in seconds between\nreplication heartbeats. Whenever the master\'s binary log is updated\nwith an event, the waiting period for the next heartbeat is reset.\ninterval is a decimal value having the range 0 to 4294967 seconds and a\nresolution to hundredths of a second; the smallest nonzero value is\n0.01. Heartbeats are sent by the master only if there are no unsent\nevents in the binary log file for a period longer than interval.\n\nSetting interval to 0 disables heartbeats altogether. The default value\nfor interval is equal to the value of slave_net_timeout divided by 2.\n\nSetting @@global.slave_net_timeout to a value less than that of the\ncurrent heartbeat interval results in a warning being issued. The\neffect of issuing RESET SLAVE on the heartbeat interval is to reset it\nto the default value.\n\nMASTER_HEARTBEAT_PERIOD was added in MySQL Cluster NDB 6.3.4.\n\nMASTER_LOG_FILE and MASTER_LOG_POS are the coordinates at which the\nslave I/O thread should begin reading from the master the next time the\nthread starts. RELAY_LOG_FILE and RELAY_LOG_POS are the coordinates at\nwhich the slave SQL thread should begin reading from the relay log the\nnext time the thread starts. If you specify either of MASTER_LOG_FILE\nor MASTER_LOG_POS, you cannot specify RELAY_LOG_FILE or RELAY_LOG_POS.\nIf neither of MASTER_LOG_FILE or MASTER_LOG_POS is specified, the slave\nuses the last coordinates of the slave SQL thread before CHANGE MASTER\nTO was issued. This ensures that there is no discontinuity in\nreplication, even if the slave SQL thread was late compared to the\nslave I/O thread, when you merely want to change, say, the password to\nuse.\n\nCHANGE MASTER TO deletes all relay log files and starts a new one,\nunless you specify RELAY_LOG_FILE or RELAY_LOG_POS. In that case, relay\nlog files are kept; the relay_log_purge global variable is set silently\nto 0.\n\nCHANGE MASTER TO is useful for setting up a slave when you have the\nsnapshot of the master and have recorded the master binary log\ncoordinates corresponding to the time of the snapshot. After loading\nthe snapshot into the slave to synchronize it to the slave, you can run\nCHANGE MASTER TO MASTER_LOG_FILE=\'log_name\', MASTER_LOG_POS=log_pos on\nthe slave to specify the coodinates at which the slave should begin\nreading the master binary log.\n\nThe following example changes the master server the slave uses and\nestablishes the master binary log coordinates from which the slave\nbegins reading. This is used when you want to set up the slave to\nreplicate the master:\n\nCHANGE MASTER TO\n MASTER_HOST=\'master2.mycompany.com\',\n MASTER_USER=\'replication\',\n MASTER_PASSWORD=\'bigs3cret\',\n MASTER_PORT=3306,\n MASTER_LOG_FILE=\'master2-bin.001\',\n MASTER_LOG_POS=4,\n MASTER_CONNECT_RETRY=10;\n\nThe next example shows an operation that is less frequently employed.\nIt is used when the slave has relay log files that you want it to\nexecute again for some reason. To do this, the master need not be\nreachable. You need only use CHANGE MASTER TO and start the SQL thread\n(START SLAVE SQL_THREAD):\n\nCHANGE MASTER TO\n RELAY_LOG_FILE=\'slave-relay-bin.006\',\n RELAY_LOG_POS=4025;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/change-master-to.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (185,37,'DROP DATABASE','Syntax:\nDROP {DATABASE | SCHEMA} [IF EXISTS] db_name\n\nDROP DATABASE drops all tables in the database and deletes the\ndatabase. Be very careful with this statement! To use DROP DATABASE,\nyou need the DROP privilege on the database. DROP SCHEMA is a synonym\nfor DROP DATABASE.\n\n*Important*: When a database is dropped, user privileges on the\ndatabase are not automatically dropped. See [HELP GRANT].\n\nIF EXISTS is used to prevent an error from occurring if the database\ndoes not exist.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-database.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-database.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (186,6,'MBREQUAL','MBREqual(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 are the same.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (187,30,'TIMESTAMP FUNCTION','Syntax:\nTIMESTAMP(expr), TIMESTAMP(expr1,expr2)\n\nWith a single argument, this function returns the date or datetime\nexpression expr as a datetime value. With two arguments, it adds the\ntime expression expr2 to the date or datetime expression expr1 and\nreturns the result as a datetime value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMESTAMP(\'2003-12-31\');\n -> \'2003-12-31 00:00:00\'\nmysql> SELECT TIMESTAMP(\'2003-12-31 12:00:00\',\'12:00:00\');\n -> \'2004-01-01 00:00:00\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (188,35,'CHARACTER_LENGTH','Syntax:\nCHARACTER_LENGTH(str)\n\nCHARACTER_LENGTH() is a synonym for CHAR_LENGTH().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (189,25,'SHOW GRANTS','Syntax:\nSHOW GRANTS [FOR user]\n\nThis statement lists the GRANT statement or statements that must be\nissued to duplicate the privileges that are granted to a MySQL user\naccount. The account is named using the same format as for the GRANT\nstatement; for example, \'jeffrey\'@\'localhost\'. If you specify only the\nuser name part of the account name, a host name part of \'%\' is used.\nFor additional information about specifying account names, see [HELP\nGRANT].\n\nmysql> SHOW GRANTS FOR \'root\'@\'localhost\';\n+---------------------------------------------------------------------+\n| Grants for root@localhost |\n+---------------------------------------------------------------------+\n| GRANT ALL PRIVILEGES ON *.* TO \'root\'@\'localhost\' WITH GRANT OPTION |\n+---------------------------------------------------------------------+\n\nTo list the privileges granted to the account that you are using to\nconnect to the server, you can use any of the following statements:\n\nSHOW GRANTS;\nSHOW GRANTS FOR CURRENT_USER;\nSHOW GRANTS FOR CURRENT_USER();\n\nAs of MySQL 5.1.12, if SHOW GRANTS FOR CURRENT_USER (or any of the\nequivalent syntaxes) is used in DEFINER context, such as within a\nstored procedure that is defined with SQL SECURITY DEFINER), the grants\ndisplayed are those of the definer and not the invoker.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-grants.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-grants.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (190,25,'SHOW PRIVILEGES','Syntax:\nSHOW PRIVILEGES\n\nSHOW PRIVILEGES shows the list of system privileges that the MySQL\nserver supports. The exact list of privileges depends on the version of\nyour server.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-privileges.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-privileges.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (191,37,'CREATE TABLESPACE','Syntax:\nCREATE TABLESPACE tablespace_name\n ADD DATAFILE \'file_name\'\n USE LOGFILE GROUP logfile_group\n [EXTENT_SIZE [=] extent_size]\n [INITIAL_SIZE [=] initial_size]\n [AUTOEXTEND_SIZE [=] autoextend_size]\n [MAX_SIZE [=] max_size]\n [NODEGROUP [=] nodegroup_id]\n [WAIT]\n [COMMENT [=] comment_text]\n ENGINE [=] engine_name\n\nThis statement is used to create a tablespace, which can contain one or\nmore data files, providing storage space for tables. One data file is\ncreated and added to the tablespace using this statement. Additional\ndata files may be added to the tablespace by using the ALTER TABLESPACE\nstatement (see [HELP ALTER TABLESPACE]). For rules covering the naming\nof tablespaces, see\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and a log file group with the same name, or a\ntablespace and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/31770))\n\nA log file group of one or more UNDO log files must be assigned to the\ntablespace to be created with the USE LOGFILE GROUP clause.\nlogfile_group must be an existing log file group created with CREATE\nLOGFILE GROUP (see\nhttp://dev.mysql.com/doc/refman/5.1/en/create-logfile-group.html).\nMu… tablespaces may use the same log file group for UNDO logging.\n\nThe EXTENT_SIZE sets the size, in bytes, of the extents used by any\nfiles belonging to the tablespace. The default value is 1M. The minimum\nsize is 32K, and theoretical maximum is 2G, although the practical\nmaximum size depends on a number of factors. In most cases, changing\nthe extent size does not have any measurable effect on performance, and\nthe default value is recommended for all but the most unusual\nsituations.\n\nAn extent is a unit of disk space allocation. One extent is filled with\nas much data as that extent can contain before another extent is used.\nIn theory, up to 65,535 (64K) extents may used per data file; however,\nthe recommended maximum is 32,768 (32K). The recommended maximum size\nfor a single data file is 32G --- that is, 32K extents x 1 MB per\nextent. In addition, once an extent is allocated to a given partition,\nit cannot be used to store data from a different partition; an extent\ncannot store data from more than one partition. This means, for example\nthat a tablespace having a single datafile whose INITIAL_SIZE is 256 MB\nand whose EXTENT_SIZE is 128M has just two extents, and so can be used\nto store data from at most two different disk data table partitions.\n\nYou can see how many extents remain free in a given data file by\nquerying the INFORMATION_SCHEMA.FILES table, and so derive an estimate\nfor how much space remains free in the file. For further discussion and\nexamples, see http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nThe INITIAL_SIZE parameter sets the data file\'s total size in bytes.\nOnce the file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using ALTER TABLESPACE\n... ADD DATAFILE. See [HELP ALTER TABLESPACE].\n\nINITIAL_SIZE is optional; its default value is 128M.\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/29186))\n\nWhen setting EXTENT_SIZE or INITIAL_SIZE (either or both), you may\noptionally follow the number with a one-letter abbreviation for an\norder of magnitude, similar to those used in my.cnf. Generally, this is\none of the letters M (for megabytes) or G (for gigabytes).\n\nAUTOEXTEND_SIZE, MAX_SIZE, NODEGROUP, WAIT, and COMMENT are parsed but\nignored, and so have no effect in MySQL 5.1. These options are intended\nfor future expansion.\n\nThe ENGINE parameter determines the storage engine which uses this\ntablespace, with engine_name being the name of the storage engine. In\nMySQL 5.1, engine_name must be one of the values NDB or NDBCLUSTER.\n\nWhen CREATE TABLESPACE is used with ENGINE = NDB, a tablespace and\nassociated data file are created on each Cluster data node. You can\nverify that the data files were created and obtain information about\nthem by querying the INFORMATION_SCHEMA.FILES table. For example:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+-------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+-------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n+--------------------+-------------+----------------+\n2 rows in set (0.01 sec)\n\n(See http://dev.mysql.com/doc/refman/5.1/en/files-table.html.)\n\nCREATE TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (191,37,'CREATE TABLESPACE','Syntax:\nCREATE TABLESPACE tablespace_name\n ADD DATAFILE \'file_name\'\n USE LOGFILE GROUP logfile_group\n [EXTENT_SIZE [=] extent_size]\n [INITIAL_SIZE [=] initial_size]\n [AUTOEXTEND_SIZE [=] autoextend_size]\n [MAX_SIZE [=] max_size]\n [NODEGROUP [=] nodegroup_id]\n [WAIT]\n [COMMENT [=] comment_text]\n ENGINE [=] engine_name\n\nThis statement is used to create a tablespace, which can contain one or\nmore data files, providing storage space for tables. One data file is\ncreated and added to the tablespace using this statement. Additional\ndata files may be added to the tablespace by using the ALTER TABLESPACE\nstatement (see [HELP ALTER TABLESPACE]). For rules covering the naming\nof tablespaces, see\nhttp://dev.mysql.com/doc/refman/5.1/en/identifiers.html.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and a log file group with the same name, or a\ntablespace and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/bug.php?id=31770))\n\nA log file group of one or more UNDO log files must be assigned to the\ntablespace to be created with the USE LOGFILE GROUP clause.\nlogfile_group must be an existing log file group created with CREATE\nLOGFILE GROUP (see\nhttp://dev.mysql.com/doc/refman/5.1/en/create-logfile-group.html).\nMu… tablespaces may use the same log file group for UNDO logging.\n\nThe EXTENT_SIZE sets the size, in bytes, of the extents used by any\nfiles belonging to the tablespace. The default value is 1M. The minimum\nsize is 32K, and theoretical maximum is 2G, although the practical\nmaximum size depends on a number of factors. In most cases, changing\nthe extent size does not have any measurable effect on performance, and\nthe default value is recommended for all but the most unusual\nsituations.\n\nAn extent is a unit of disk space allocation. One extent is filled with\nas much data as that extent can contain before another extent is used.\nIn theory, up to 65,535 (64K) extents may used per data file; however,\nthe recommended maximum is 32,768 (32K). The recommended maximum size\nfor a single data file is 32G --- that is, 32K extents x 1 MB per\nextent. In addition, once an extent is allocated to a given partition,\nit cannot be used to store data from a different partition; an extent\ncannot store data from more than one partition. This means, for example\nthat a tablespace having a single datafile whose INITIAL_SIZE is 256 MB\nand whose EXTENT_SIZE is 128M has just two extents, and so can be used\nto store data from at most two different disk data table partitions.\n\nYou can see how many extents remain free in a given data file by\nquerying the INFORMATION_SCHEMA.FILES table, and so derive an estimate\nfor how much space remains free in the file. For further discussion and\nexamples, see http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nThe INITIAL_SIZE parameter sets the data file\'s total size in bytes.\nOnce the file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using ALTER TABLESPACE\n... ADD DATAFILE. See [HELP ALTER TABLESPACE].\n\nINITIAL_SIZE is optional; its default value is 128M.\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/bug.php?id=29186))\n\nWhen setting EXTENT_SIZE or INITIAL_SIZE (either or both), you may\noptionally follow the number with a one-letter abbreviation for an\norder of magnitude, similar to those used in my.cnf. Generally, this is\none of the letters M (for megabytes) or G (for gigabytes).\n\nINITIAL_SIZE, EXTENT_SIZE, and UNDO_BUFFER_SIZE are subject to rounding\nas follows:\n\no EXTENT_SIZE and UNDO_BUFFER_SIZE are each rounded up to the nearest\n whole multiple of 32K.\n\no INITIAL_SIZE is rounded down to the nearest whole multiple of 32K.\n\n For data files, INITIAL_SIZE is subject to further rounding; the\n result just obtained is rounded up to the nearest whole multiple of\n EXTENT_SIZE (after any rounding).\n\nThe rounding just described has always (since Disk Data tablespaces\nwere introduced in MySQL 5.1.6) been performed implicitly, but\nbeginning with MySQL Cluster NDB 6.2.19, MySQL Cluster NDB 6.3.32,\nMySQL Cluster NDB 7.0.13, and MySQL Cluster NDB 7.1.2, this rounding is\ndone explicitly, and a warning is issued by the MySQL Server when any\nsuch rounding is performed. The rounded values are also used by the NDB\nkernel for calculating INFORMATION_SCHEMA.FILES column values and other\npurposes. However, to avoid an unexpected result, we suggest that you\nalways use whole multiples of 32K in specifying these options.\n\nAUTOEXTEND_SIZE, MAX_SIZE, NODEGROUP, WAIT, and COMMENT are parsed but\nignored, and so currently have no effect. These options are intended\nfor future expansion.\n\nThe ENGINE parameter determines the storage engine which uses this\ntablespace, with engine_name being the name of the storage engine. In\nMySQL 5.1, engine_name must be one of the values NDB or NDBCLUSTER.\n\nWhen CREATE TABLESPACE is used with ENGINE = NDB, a tablespace and\nassociated data file are created on each Cluster data node. You can\nverify that the data files were created and obtain information about\nthem by querying the INFORMATION_SCHEMA.FILES table. For example:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+-------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+-------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n+--------------------+-------------+----------------+\n2 rows in set (0.01 sec)\n\n(See http://dev.mysql.com/doc/refman/5.1/en/files-table.html.)\n\nCREATE TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (192,35,'INSERT FUNCTION','Syntax:\nINSERT(str,pos,len,newstr)\n\nReturns the string str, with the substring beginning at position pos\nand len characters long replaced by the string newstr. Returns the\noriginal string if pos is not within the length of the string. Replaces\nthe rest of the string from position pos if len is not within the\nlength of the rest of the string. Returns NULL if any argument is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT INSERT(\'Quadratic\', 3, 4, \'What\');\n -> \'QuWhattic\'\nmysql> SELECT INSERT(\'Quadratic\', -1, 4, \'What\');\n -> \'Quadratic\'\nmysql> SELECT INSERT(\'Quadratic\', 3, 100, \'What\');\n -> \'QuWhat\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (193,4,'CRC32','Syntax:\nCRC32(expr)\n\nComputes a cyclic redundancy check value and returns a 32-bit unsigned\nvalue. The result is NULL if the argument is NULL. The argument is\nexpected to be a string and (if possible) is treated as one if it is\nnot.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT CRC32(\'MySQL\');\n -> 3259397556\nmysql> SELECT CRC32(\'mysql\');\n -> 2501908538\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (194,13,'XOR','Syntax:\nXOR\n\nLogical XOR. Returns NULL if either operand is NULL. For non-NULL\noperands, evaluates to 1 if an odd number of operands is nonzero,\notherwise 0 is returned.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/logical-operators.html\n\n','mysql> SELECT 1 XOR 1;\n -> 0\nmysql> SELECT 1 XOR 0;\n -> 1\nmysql> SELECT 1 XOR NULL;\n -> NULL\nmysql> SELECT 1 XOR 1 XOR 1;\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/logical-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (195,12,'STARTPOINT','StartPoint(ls)\n\nReturns the Point that is the start point of the LineString value ls.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…','mysql> SET @ls = \'LineString(1 1,2 2,3 3)\';\nmysql> SELECT AsText(StartPoint(GeomFromText(@ls)));\n+---------------------------------------+\n| AsText(StartPoint(GeomFromText(@ls))) |\n+---------------------------------------+\n| POINT(1 1) |\n+---------------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (196,22,'DECLARE VARIABLE','Syntax:\nDECLARE var_name [, var_name] ... type [DEFAULT value]\n\nThis statement is used to declare local variables within stored\nprograms. To provide a default value for the variable, include a\nDEFAULT clause. The value can be specified as an expression; it need\nnot be a constant. If the DEFAULT clause is missing, the initial value\nis NULL.\n\nLocal variables are treated like stored routine parameters with respect\nto data type and overflow checking. See [HELP CREATE PROCEDURE].\n\nLocal variable names are not case sensitive.\n\nThe scope of a local variable is within the BEGIN ... END block where\nit is declared. The variable can be referred to in blocks nested within\nthe declaring block, except those blocks that declare a variable with\nthe same name.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/declare-local-variable.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-local-variable.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (197,9,'GRANT','Syntax:\nGRANT\n priv_type [(column_list)]\n [, priv_type [(column_list)]] ...\n ON [object_type] priv_level\n TO user [IDENTIFIED BY [PASSWORD] \'password\']\n [, user [IDENTIFIED BY [PASSWORD] \'password\']] ...\n [REQUIRE {NONE | ssl_option [[AND] ssl_option] ...}]\n [WITH with_option [with_option] ...]\n\nobject_type:\n TABLE\n | FUNCTION\n | PROCEDURE\n\npriv_level:\n *\n | *.*\n | db_name.*\n | db_name.tbl_name\n | tbl_name\n | db_name.routine_name\n\nwith_option:\n GRANT OPTION\n | MAX_QUERIES_PER_HOUR count\n | MAX_UPDATES_PER_HOUR count\n | MAX_CONNECTIONS_PER_HOUR count\n | MAX_USER_CONNECTIONS count\n\nssl_option:\n SSL\n | X509\n | CIPHER \'cipher\'\n | ISSUER \'issuer\'\n | SUBJECT \'subject\'\n\nThe GRANT statement enables system administrators to create MySQL user\naccounts and to grant rights to accounts. To use GRANT, you must have\nthe GRANT OPTION privilege, and you must have the privileges that you\nare granting. The REVOKE statement is related and enables\nadministrators to remove account privileges. To determine what\nprivileges an account has, use SHOW GRANTS. See [HELP REVOKE], and\n[HELP SHOW GRANTS].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/grant.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/grant.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (197,9,'GRANT','Syntax:\nGRANT\n priv_type [(column_list)]\n [, priv_type [(column_list)]] ...\n ON [object_type] priv_level\n TO user [IDENTIFIED BY [PASSWORD] \'password\']\n [, user [IDENTIFIED BY [PASSWORD] \'password\']] ...\n [REQUIRE {NONE | ssl_option [[AND] ssl_option] ...}]\n [WITH with_option ...]\n\nobject_type:\n TABLE\n | FUNCTION\n | PROCEDURE\n\npriv_level:\n *\n | *.*\n | db_name.*\n | db_name.tbl_name\n | tbl_name\n | db_name.routine_name\n\nssl_option:\n SSL\n | X509\n | CIPHER \'cipher\'\n | ISSUER \'issuer\'\n | SUBJECT \'subject\'\n\nwith_option:\n GRANT OPTION\n | MAX_QUERIES_PER_HOUR count\n | MAX_UPDATES_PER_HOUR count\n | MAX_CONNECTIONS_PER_HOUR count\n | MAX_USER_CONNECTIONS count\n\nThe GRANT statement enables system administrators to grant privileges\nto MySQL user accounts. GRANT also serves to specify other account\ncharacteristics such as use of secure connections and limits on access\nto server resources. To use GRANT, you must have the GRANT OPTION\nprivilege, and you must have the privileges that you are granting.\n\nNormally, CREATE USER is used to create an account and GRANT to define\nits privileges. However, if an account named in a GRANT statement does\nnot already exist, GRANT may create it under the conditions described\nlater in the discussion of the NO_AUTO_CREATE_USER SQL mode.\n\nThe REVOKE statement is related to GRANT and enables administrators to\nremove account privileges. To determine what privileges an account has,\nuse SHOW GRANTS. See [HELP REVOKE], and [HELP SHOW GRANTS].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/grant.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/grant.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (198,3,'MPOLYFROMTEXT','MPolyFromText(wkt[,srid]), MultiPolygonFromText(wkt[,srid])\n\nConstructs a MULTIPOLYGON value using its WKT representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (199,6,'MBRINTERSECTS','MBRIntersects(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 intersect.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (200,16,'BIT_OR','Syntax:\nBIT_OR(expr)\n\nReturns the bitwise OR of all bits in expr. The calculation is\nperformed with 64-bit (BIGINT) precision.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); @@ -286,15 +286,15 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (217,35,'ASCII','Syntax:\nASCII(str)\n\nReturns the numeric value of the leftmost character of the string str.\nReturns 0 if str is the empty string. Returns NULL if str is NULL.\nASCII() works for 8-bit characters.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT ASCII(\'2\');\n -> 50\nmysql> SELECT ASCII(2);\n -> 50\nmysql> SELECT ASCII(\'dx\');\n -> 100\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (218,4,'DIV','Syntax:\nDIV\n\nInteger division. Similar to FLOOR(), but is safe with BIGINT values.\nIncorrect results may occur for noninteger operands that exceed BIGINT\nrange.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html\n\n','mysql> SELECT 5 DIV 2;\n -> 2\n','http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (219,9,'RENAME USER','Syntax:\nRENAME USER old_user TO new_user\n [, old_user TO new_user] ...\n\nThe RENAME USER statement renames existing MySQL accounts. To use it,\nyou must have the global CREATE USER privilege or the UPDATE privilege\nfor the mysql database. An error occurs if any old account does not\nexist or any new account exists. Each account is named using the same\nformat as for the GRANT statement; for example, \'jeffrey\'@\'localhost\'.\nIf you specify only the user name part of the account name, a host name\npart of \'%\' is used. For additional information about specifying\naccount names, see [HELP GRANT].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/rename-user.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/rename-user.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (220,25,'SHOW SLAVE STATUS','Syntax:\nSHOW SLAVE STATUS\n\nThis statement provides status information on essential parameters of\nthe slave threads. It requires either the SUPER or REPLICATION CLIENT\nprivilege.\n\nIf you issue this statement using the mysql client, you can use a \\G\nstatement terminator rather than a semicolon to obtain a more readable\nvertical layout:\n\nmysql> SHOW SLAVE STATUS\\G\n*************************** 1. row ***************************\n Slave_IO_State: Waiting for master to send event\n Master_Host: localhost\n Master_User: root\n Master_Port: 3306\n Connect_Retry: 3\n Master_Log_File: gbichot-bin.005\n Read_Master_Log_Pos: 79\n Relay_Log_File: gbichot-relay-bin.005\n Relay_Log_Pos: 548\n Relay_Master_Log_File: gbichot-bin.005\n Slave_IO_Running: Yes\n Slave_SQL_Running: Yes\n Replicate_Do_DB:\n Replicate_Ignore_DB:\n Last_Errno: 0\n Last_Error:\n Skip_Counter: 0\n Exec_Master_Log_Pos: 79\n Relay_Log_Space: 552\n Until_Condition: None\n Until_Log_File:\n Until_Log_Pos: 0\n Master_SSL_Allowed: No\n Master_SSL_CA_File:\n Master_SSL_CA_Path:\n Master_SSL_Cert:\n Master_SSL_Cipher:\n Master_SSL_Key:\n Seconds_Behind_Master: 8\nMaster_SSL_Verify_Server_Cert: No\n Last_IO_Errno: 0\n Last_IO_Error:\n Last_SQL_Errno: 0\n Last_SQL_Error:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (220,25,'SHOW SLAVE STATUS','Syntax:\nSHOW SLAVE STATUS\n\nThis statement provides status information on essential parameters of\nthe slave threads. It requires either the SUPER or REPLICATION CLIENT\nprivilege.\n\nIf you issue this statement using the mysql client, you can use a \\G\nstatement terminator rather than a semicolon to obtain a more readable\nvertical layout:\n\nmysql> SHOW SLAVE STATUS\\G\n*************************** 1. row ***************************\n Slave_IO_State: Waiting for master to send event\n Master_Host: localhost\n Master_User: root\n Master_Port: 3306\n Connect_Retry: 3\n Master_Log_File: gbichot-bin.005\n Read_Master_Log_Pos: 79\n Relay_Log_File: gbichot-relay-bin.005\n Relay_Log_Pos: 548\n Relay_Master_Log_File: gbichot-bin.005\n Slave_IO_Running: Yes\n Slave_SQL_Running: Yes\n Replicate_Do_DB:\n Replicate_Ignore_DB:\n Replicate_Do_Table:\n Replicate_Ignore_Table:\n Replicate_Wild_Do_Table:\n Replicate_Wild_Ignore_Table:\n Last_Errno: 0\n Last_Error:\n Skip_Counter: 0\n Exec_Master_Log_Pos: 79\n Relay_Log_Space: 552\n Until_Condition: None\n Until_Log_File:\n Until_Log_Pos: 0\n Master_SSL_Allowed: No\n Master_SSL_CA_File:\n Master_SSL_CA_Path:\n Master_SSL_Cert:\n Master_SSL_Cipher:\n Master_SSL_Key:\n Seconds_Behind_Master: 8\nMaster_SSL_Verify_Server_Cert: No\n Last_IO_Errno: 0\n Last_IO_Error:\n Last_SQL_Errno: 0\n Last_SQL_Error:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (221,32,'GEOMETRY','MySQL provides a standard way of creating spatial columns for geometry\ntypes, for example, with CREATE TABLE or ALTER TABLE. Currently,\nspatial columns are supported for MyISAM, InnoDB, NDB, and ARCHIVE\ntables. See also the annotations about spatial indexes under [HELP\nSPATIAL].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-columns.html\n\n','CREATE TABLE geom (g GEOMETRY);\n','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-columns.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (222,12,'NUMPOINTS','NumPoints(ls)\n\nReturns the number of Point objects in the LineString value ls.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…','mysql> SET @ls = \'LineString(1 1,2 2,3 3)\';\nmysql> SELECT NumPoints(GeomFromText(@ls));\n+------------------------------+\n| NumPoints(GeomFromText(@ls)) |\n+------------------------------+\n| 3 |\n+------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#lin…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (223,37,'ALTER LOGFILE GROUP','Syntax:\nALTER LOGFILE GROUP logfile_group\n ADD UNDOFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement adds an UNDO file named \'file_name\' to an existing log\nfile group logfile_group. An ALTER LOGFILE GROUP statement has one and\nonly one ADD UNDOFILE clause. No DROP UNDOFILE clause is currently\nsupported.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an undo log file with the same name, or an undo\nlog file and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for undo log files could not be longer than 128 characters.\n(Bug#31769 (http://bugs.mysql.com/31769))\n\nThe optional INITIAL_SIZE parameter sets the UNDO file\'s initial size\nin bytes; if not specified, the initial size default to 128M (128\nmegabytes). You may optionally follow size with a one-letter\nabbreviation for an order of magnitude, similar to those used in\nmy.cnf. Generally, this is one of the letters M (for megabytes) or G\n(for gigabytes).\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/29186))\n\nBeginning with MySQL Cluster NDB 2.1.18, 6.3.24, and 7.0.4, the minimum\nallowed value for INITIAL_SIZE is 1M. (Bug#29574\n(http://bugs.mysql.com/29574))\n\n*Note*: WAIT is parsed but otherwise ignored, and so has no effect in\nMySQL 5.1 and MySQL Cluster NDB 6.x. It is intended for future\nexpansion.\n\nThe ENGINE parameter (required) determines the storage engine which is\nused by this log file group, with engine_name being the name of the\nstorage engine. In MySQL 5.1 and MySQL Cluster NDB 6.x, the only\naccepted values for engine_name are "NDBCLUSTER" and "NDB". The two\nvalues are equivalent.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (223,37,'ALTER LOGFILE GROUP','Syntax:\nALTER LOGFILE GROUP logfile_group\n ADD UNDOFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement adds an UNDO file named \'file_name\' to an existing log\nfile group logfile_group. An ALTER LOGFILE GROUP statement has one and\nonly one ADD UNDOFILE clause. No DROP UNDOFILE clause is currently\nsupported.\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an undo log file with the same name, or an undo\nlog file and a data file with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for undo log files could not be longer than 128 characters.\n(Bug#31769 (http://bugs.mysql.com/bug.php?id=31769))\n\nThe optional INITIAL_SIZE parameter sets the UNDO file\'s initial size\nin bytes; if not specified, the initial size default to 128M (128\nmegabytes). You may optionally follow size with a one-letter\nabbreviation for an order of magnitude, similar to those used in\nmy.cnf. Generally, this is one of the letters M (for megabytes) or G\n(for gigabytes).\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/bug.php?id=29186))\n\nBeginning with MySQL Cluster NDB 2.1.18, 6.3.24, and 7.0.4, the minimum\nallowed value for INITIAL_SIZE is 1M. (Bug#29574\n(http://bugs.mysql.com/bug.php?id=29574))\n\n*Note*: WAIT is parsed but otherwise ignored, and so has no effect in\nMySQL 5.1 and MySQL Cluster NDB 6.x. It is intended for future\nexpansion.\n\nThe ENGINE parameter (required) determines the storage engine which is\nused by this log file group, with engine_name being the name of the\nstorage engine. In MySQL 5.1 and MySQL Cluster NDB 6.x, the only\naccepted values for engine_name are "NDBCLUSTER" and "NDB". The two\nvalues are equivalent.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-logfile-group.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (224,18,'&','Syntax:\n&\n\nBitwise AND:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html\n\n','mysql> SELECT 29 & 15;\n -> 13\n','http://dev.mysql.com/doc/refman/5.1/en/bit-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (225,30,'LOCALTIMESTAMP','Syntax:\nLOCALTIMESTAMP, LOCALTIMESTAMP()\n\nLOCALTIMESTAMP and LOCALTIMESTAMP() are synonyms for NOW().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (226,35,'CONVERT','Syntax:\nCONVERT(expr,type), CONVERT(expr USING transcoding_name)\n\nThe CONVERT() and CAST() functions take a value of one type and produce\na value of another type.\n\nThe type can be one of the following values:\n\no BINARY[(N)]\n\no CHAR[(N)]\n\no DATE\n\no DATETIME\n\no DECIMAL[(M[,D])]\n\no SIGNED [INTEGER]\n\no TIME\n\no UNSIGNED [INTEGER]\n\nBINARY produces a string with the BINARY data type. See\nhttp://dev.mysql.com/doc/refman/5.1/en/binary-varbinary.html for a\ndescription of how this affects comparisons. If the optional length N\nis given, BINARY(N) causes the cast to use no more than N bytes of the\nargument. Values shorter than N bytes are padded with 0x00 bytes to a\nlength of N.\n\nCHAR(N) causes the cast to use no more than N characters of the\nargument.\n\nCAST() and CONVERT(... USING ...) are standard SQL syntax. The\nnon-USING form of CONVERT() is ODBC syntax.\n\nCONVERT() with USING is used to convert data between different\ncharacter sets. In MySQL, transcoding names are the same as the\ncorresponding character set names. For example, this statement converts\nthe string \'abc\' in the default character set to the corresponding\nstring in the utf8 character set:\n\nSELECT CONVERT(\'abc\' USING utf8);\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html\n\n','SELECT enum_col FROM tbl_name ORDER BY CAST(enum_col AS CHAR);\n','http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (227,30,'ADDDATE','Syntax:\nADDDATE(date,INTERVAL expr unit), ADDDATE(expr,days)\n\nWhen invoked with the INTERVAL form of the second argument, ADDDATE()\nis a synonym for DATE_ADD(). The related function SUBDATE() is a\nsynonym for DATE_SUB(). For information on the INTERVAL unit argument,\nsee the discussion for DATE_ADD().\n\nmysql> SELECT DATE_ADD(\'2008-01-02\', INTERVAL 31 DAY);\n -> \'2008-02-02\'\nmysql> SELECT ADDDATE(\'2008-01-02\', INTERVAL 31 DAY);\n -> \'2008-02-02\'\n\nWhen invoked with the days form of the second argument, MySQL treats it\nas an integer number of days to be added to expr.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT ADDDATE(\'2008-01-02\', 31);\n -> \'2008-02-02\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (228,22,'REPEAT LOOP','Syntax:\n[begin_label:] REPEAT\n statement_list\nUNTIL search_condition\nEND REPEAT [end_label]\n\nThe statement list within a REPEAT statement is repeated until the\nsearch_condition is true. Thus, a REPEAT always enters the loop at\nleast once. statement_list consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter.\n\nA REPEAT statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html\n\n','mysql> delimiter //\n\nmysql> CREATE PROCEDURE dorepeat(p1 INT)\n -> BEGIN\n -> SET @x = 0;\n -> REPEAT SET @x = @x + 1; UNTIL @x > p1 END REPEAT;\n -> END\n -> //\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> CALL dorepeat(1000)//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @x//\n+------+\n| @x |\n+------+\n| 1001 |\n+------+\n1 row in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (228,22,'REPEAT LOOP','Syntax:\n[begin_label:] REPEAT\n statement_list\nUNTIL search_condition\nEND REPEAT [end_label]\n\nThe statement list within a REPEAT statement is repeated until the\nsearch_condition is true. Thus, a REPEAT always enters the loop at\nleast once. statement_list consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter.\n\nA REPEAT statement can be labeled. See [HELP BEGIN END] for the rules\nregarding label use.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html\n\n','mysql> delimiter //\n\nmysql> CREATE PROCEDURE dorepeat(p1 INT)\n -> BEGIN\n -> SET @x = 0;\n -> REPEAT SET @x = @x + 1; UNTIL @x > p1 END REPEAT;\n -> END\n -> //\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> CALL dorepeat(1000)//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @x//\n+------+\n| @x |\n+------+\n| 1001 |\n+------+\n1 row in set (0.00 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/repeat-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (229,37,'ALTER FUNCTION','Syntax:\nALTER FUNCTION func_name [characteristic ...]\n\ncharacteristic:\n { CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }\n | SQL SECURITY { DEFINER | INVOKER }\n | COMMENT \'string\'\n\nThis statement can be used to change the characteristics of a stored\nfunction. More than one change may be specified in an ALTER FUNCTION\nstatement. However, you cannot change the parameters or body of a\nstored function using this statement; to make such changes, you must\ndrop and re-create the function using DROP FUNCTION and CREATE\nFUNCTION.\n\nYou must have the ALTER ROUTINE privilege for the function. (That\nprivilege is granted automatically to the function creator.) If binary\nlogging is enabled, the ALTER FUNCTION statement might also require the\nSUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\n\…: http://dev.mysql.com/doc/refman/5.1/en/alter-function.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-function.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (230,20,'SMALLINT','SMALLINT[(M)] [UNSIGNED] [ZEROFILL]\n\nA small integer. The signed range is -32768 to 32767. The unsigned\nrange is 0 to 65535.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (231,20,'DOUBLE PRECISION','DOUBLE PRECISION[(M,D)] [UNSIGNED] [ZEROFILL], REAL[(M,D)] [UNSIGNED]\n[ZEROFILL]\n\nThese types are synonyms for DOUBLE. Exception: If the REAL_AS_FLOAT\nSQL mode is enabled, REAL is a synonym for FLOAT rather than DOUBLE.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html'); @@ -327,12 +327,12 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (258,4,'COS','Syntax:\nCOS(X)\n\nReturns the cosine of X, where X is given in radians.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT COS(PI());\n -> -1\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (259,30,'DATE FUNCTION','Syntax:\nDATE(expr)\n\nExtracts the date part of the date or datetime expression expr.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DATE(\'2003-12-31 01:02:03\');\n -> \'2003-12-31\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (260,37,'DROP TRIGGER','Syntax:\nDROP TRIGGER [IF EXISTS] [schema_name.]trigger_name\n\nThis statement drops a trigger. The schema (database) name is optional.\nIf the schema is omitted, the trigger is dropped from the default\nschema. DROP TRIGGER was added in MySQL 5.0.2. Its use requires the\nTRIGGER privilege for the table associated with the trigger. (This\nstatement requires the SUPER privilege prior to MySQL 5.1.6.)\n\nUse IF EXISTS to prevent an error from occurring for a trigger that\ndoes not exist. A NOTE is generated for a nonexistent trigger when\nusing IF EXISTS. See [HELP SHOW WARNINGS]. The IF EXISTS clause was\nadded in MySQL 5.1.14.\n\nTriggers for a table are also dropped if you drop the table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-trigger.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-trigger.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (261,25,'RESET MASTER','Syntax:\nRESET MASTER\n\nDeletes all binary logs listed in the index file, resets the binary log\nindex file to be empty, and creates a new binary log file. It is\nintended to be used only when the master is started for the first time.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-master.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (261,25,'RESET MASTER','Syntax:\nRESET MASTER\n\nDeletes all binary log files listed in the index file, resets the\nbinary log index file to be empty, and creates a new binary log file.\nThis statement is intended to be used only when the master is started\nfor the first time.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/reset-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/reset-master.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (262,4,'TAN','Syntax:\nTAN(X)\n\nReturns the tangent of X, where X is given in radians.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT TAN(PI());\n -> -1.2246063538224e-16\nmysql> SELECT TAN(PI()+1);\n -> 1.5574077246549\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (263,4,'PI','Syntax:\nPI()\n\nReturns the value of π (pi). The default number of decimal places\ndisplayed is seven, but MySQL uses the full double-precision value\ninternally.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT PI();\n -> 3.141593\nmysql> SELECT PI()+0.000000000000000000;\n -> 3.141592653589793116\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (264,30,'WEEKOFYEAR','Syntax:\nWEEKOFYEAR(date)\n\nReturns the calendar week of the date as a number in the range from 1\nto 53. WEEKOFYEAR() is a compatibility function that is equivalent to\nWEEK(date,3).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT WEEKOFYEAR(\'2008-02-20\');\n -> 8\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (265,4,'/','Syntax:\n/\n\nDivision:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html\n\n','mysql> SELECT 3/5;\n -> 0.60\n','http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (266,25,'PURGE BINARY LOGS','Syntax:\nPURGE { BINARY | MASTER } LOGS\n { TO \'log_name\' | BEFORE datetime_expr }\n\nThe binary log is a set of files that contain information about data\nmodifications made by the MySQL server. The log consists of a set of\nbinary log files, plus an index file.\n\nThe PURGE BINARY LOGS statement deletes all the binary log files listed\nin the log index file prior to the specified log file name or date. The\nlog files also are removed from the list recorded in the index file, so\nthat the given log file becomes the first.\n\nThis statement has no effect if the --log-bin option has not been\nenabled.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html\n\n','PURGE BINARY LOGS TO \'mysql-bin.010\';\nPURGE BINARY LOGS BEFORE \'2008-04-02 22:46:26\';\n','http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (266,25,'PURGE BINARY LOGS','Syntax:\nPURGE { BINARY | MASTER } LOGS\n { TO \'log_name\' | BEFORE datetime_expr }\n\nThe binary log is a set of files that contain information about data\nmodifications made by the MySQL server. The log consists of a set of\nbinary log files, plus an index file (see\nhttp://dev.mysql.com/doc/refman/5.1/en/binary-log.html).\n\nThe PURGE BINARY LOGS statement deletes all the binary log files listed\nin the log index file prior to the specified log file name or date.\nBINARY and MASTER are synonyms. Deleted log files also are removed from\nthe list recorded in the index file, so that the given log file becomes\nthe first in the list.\n\nThis statement has no effect if the server was not started with the\n--log-bin option to enable binary logging.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html\n\n','PURGE BINARY LOGS TO \'mysql-bin.010\';\nPURGE BINARY LOGS BEFORE \'2008-04-02 22:46:26\';\n','http://dev.mysql.com/doc/refman/5.1/en/purge-binary-logs.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (267,16,'STDDEV_SAMP','Syntax:\nSTDDEV_SAMP(expr)\n\nReturns the sample standard deviation of expr (the square root of\nVAR_SAMP().\n\nSTDDEV_SAMP() returns NULL if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (268,15,'SCHEMA','Syntax:\nSCHEMA()\n\nThis function is a synonym for DATABASE().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (269,31,'MLINEFROMWKB','MLineFromWKB(wkb[,srid]), MultiLineStringFromWKB(wkb[,srid])\n\nConstructs a MULTILINESTRING value using its WKB representation and\nSRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); @@ -345,7 +345,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (276,26,'DUAL','You are allowed to specify DUAL as a dummy table name in situations\nwhere no tables are referenced:\n\nmysql> SELECT 1 + 1 FROM DUAL;\n -> 2\n\nDUAL is purely for the convenience of people who require that all\nSELECT statements should have FROM and possibly other clauses. MySQL\nmay ignore the clauses. MySQL does not require FROM DUAL if no tables\nare referenced.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/select.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/select.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (277,35,'INSTR','Syntax:\nINSTR(str,substr)\n\nReturns the position of the first occurrence of substring substr in\nstring str. This is the same as the two-argument form of LOCATE(),\nexcept that the order of the arguments is reversed.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT INSTR(\'foobarbar\', \'bar\');\n -> 4\nmysql> SELECT INSTR(\'xbar\', \'foobar\');\n -> 0\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (278,30,'NOW','Syntax:\nNOW()\n\nReturns the current date and time as a value in \'YYYY-MM-DD HH:MM:SS\'\nor YYYYMMDDHHMMSS.uuuuuu format, depending on whether the function is\nused in a string or numeric context. The value is expressed in the\ncurrent time zone.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT NOW();\n -> \'2007-12-15 23:50:26\'\nmysql> SELECT NOW() + 0;\n -> 20071215235026.000000\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (279,25,'SHOW ENGINES','Syntax:\nSHOW [STORAGE] ENGINES\n\nSHOW ENGINES displays status information about the server\'s storage\nengines. This is particularly useful for checking whether a storage\nengine is supported, or to see what the default engine is. SHOW TABLE\nTYPES is a deprecated synonym.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engines.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engines.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (279,25,'SHOW ENGINES','Syntax:\nSHOW [STORAGE] ENGINES\n\nSHOW ENGINES displays status information about the server\'s storage\nengines. This is particularly useful for checking whether a storage\nengine is supported, or to see what the default engine is. SHOW TABLE\nTYPES is a synonym, but is deprecated and is removed in MySQL 5.5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engines.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engines.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (280,17,'>=','Syntax:\n>=\n\nGreater than or equal:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 2 >= 2;\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (281,4,'EXP','Syntax:\nEXP(X)\n\nReturns the value of e (the base of natural logarithms) raised to the\npower of X. The inverse of this function is LOG() (using a single\nargument only) or LN().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT EXP(2);\n -> 7.3890560989307\nmysql> SELECT EXP(-2);\n -> 0.13533528323661\nmysql> SELECT EXP(0);\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (282,20,'LONGBLOB','LONGBLOB\n\nA BLOB column with a maximum length of 4,294,967,295 or 4GB (232 - 1)\nbytes. The effective maximum length of LONGBLOB columns depends on the\nconfigured maximum packet size in the client/server protocol and\navailable memory. Each LONGBLOB value is stored using a four-byte\nlength prefix that indicates the number of bytes in the value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); @@ -353,19 +353,19 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (284,20,'YEAR DATA TYPE','YEAR[(2|4)]\n\nA year in two-digit or four-digit format. The default is four-digit\nformat. In four-digit format, the allowable values are 1901 to 2155,\nand 0000. In two-digit format, the allowable values are 70 to 69,\nrepresenting years from 1970 to 2069. MySQL displays YEAR values in\nYYYY format, but allows you to assign values to YEAR columns using\neither strings or numbers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (285,16,'SUM','Syntax:\nSUM([DISTINCT] expr)\n\nReturns the sum of expr. If the return set has no rows, SUM() returns\nNULL. The DISTINCT keyword can be used in MySQL 5.1 to sum only the\ndistinct values of expr.\n\nSUM() returns NULL if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (286,4,'OCT','Syntax:\nOCT(N)\n\nReturns a string representation of the octal value of N, where N is a\nlonglong (BIGINT) number. This is equivalent to CONV(N,10,8). Returns\nNULL if N is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT OCT(12);\n -> \'14\'\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (287,30,'SYSDATE','Syntax:\nSYSDATE()\n\nReturns the current date and time as a value in \'YYYY-MM-DD HH:MM:SS\'\nor YYYYMMDDHHMMSS.uuuuuu format, depending on whether the function is\nused in a string or numeric context.\n\nSYSDATE() returns the time at which it executes. This differs from the\nbehavior for NOW(), which returns a constant time that indicates the\ntime at which the statement began to execute. (Within a stored function\nor trigger, NOW() returns the time at which the function or triggering\nstatement began to execute.)\n\nmysql> SELECT NOW(), SLEEP(2), NOW();\n+---------------------+----------+---------------------+\n| NOW() | SLEEP(2) | NOW() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:36 | 0 | 2006-04-12 13:47:36 |\n+---------------------+----------+---------------------+\n\nmysql> SELECT SYSDATE(), SLEEP(2), SYSDATE();\n+---------------------+----------+---------------------+\n| SYSDATE() | SLEEP(2) | SYSDATE() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:44 | 0 | 2006-04-12 13:47:46 |\n+---------------------+----------+---------------------+\n\nIn addition, the SET TIMESTAMP statement affects the value returned by\nNOW() but not by SYSDATE(). This means that timestamp settings in the\nbinary log have no effect on invocations of SYSDATE().\n\nBecause SYSDATE() can return different values even within the same\nstatement, and is not affected by SET TIMESTAMP, it is nondeterministic\nand therefore unsafe for replication if statement-based binary logging\nis used. If that is a problem, you can use row-based logging, or start\nthe server with the --sysdate-is-now option to cause SYSDATE() to be an\nalias for NOW(). The nondeterministic nature of SYSDATE() also means\nthat indexes cannot be used for evaluating expressions that refer to\nit.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (288,5,'UNINSTALL PLUGIN','Syntax:\nUNINSTALL PLUGIN plugin_name\n\nThis statement removes an installed plugin. You cannot uninstall a\nplugin if any table that uses it is open.\n\nplugin_name must be the name of some plugin that is listed in the\nmysql.plugin table. The server executes the plugin\'s deinitialization\nfunction and removes the row for the plugin from the mysql.plugin\ntable, so that subsequent server restarts will not load and initialize\nthe plugin. UNINSTALL PLUGIN does not remove the plugin\'s shared\nlibrary file.\n\nTo use UNINSTALL PLUGIN, you must have the DELETE privilege for the\nmysql.plugin table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (287,30,'SYSDATE','Syntax:\nSYSDATE()\n\nReturns the current date and time as a value in \'YYYY-MM-DD HH:MM:SS\'\nor YYYYMMDDHHMMSS.uuuuuu format, depending on whether the function is\nused in a string or numeric context.\n\nSYSDATE() returns the time at which it executes. This differs from the\nbehavior for NOW(), which returns a constant time that indicates the\ntime at which the statement began to execute. (Within a stored function\nor trigger, NOW() returns the time at which the function or triggering\nstatement began to execute.)\n\nmysql> SELECT NOW(), SLEEP(2), NOW();\n+---------------------+----------+---------------------+\n| NOW() | SLEEP(2) | NOW() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:36 | 0 | 2006-04-12 13:47:36 |\n+---------------------+----------+---------------------+\n\nmysql> SELECT SYSDATE(), SLEEP(2), SYSDATE();\n+---------------------+----------+---------------------+\n| SYSDATE() | SLEEP(2) | SYSDATE() |\n+---------------------+----------+---------------------+\n| 2006-04-12 13:47:44 | 0 | 2006-04-12 13:47:46 |\n+---------------------+----------+---------------------+\n\nIn addition, the SET TIMESTAMP statement affects the value returned by\nNOW() but not by SYSDATE(). This means that timestamp settings in the\nbinary log have no effect on invocations of SYSDATE().\n\nBecause SYSDATE() can return different values even within the same\nstatement, and is not affected by SET TIMESTAMP, it is nondeterministic\nand therefore unsafe for replication if statement-based binary logging\nis used. If that is a problem, you can use row-based logging.\n\nAlternatively, you can use the --sysdate-is-now option to cause\nSYSDATE() to be an alias for NOW(). This works if the option is used on\nboth the master and the slave.\n\nThe nondeterministic nature of SYSDATE() also means that indexes cannot\nbe used for evaluating expressions that refer to it.\n\nBeginning with MySQL 5.1.42, a warning is logged if you use this\nfunction when binlog_format is set to STATEMENT. (Bug#47995\n(http://bugs.mysql.com/bug.php?id=47995))\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (288,5,'UNINSTALL PLUGIN','Syntax:\nUNINSTALL PLUGIN plugin_name\n\nThis statement removes an installed server plugin. It requires the\nDELETE privilege for the mysql.plugin table.\n\nplugin_name must be the name of some plugin that is listed in the\nmysql.plugin table. The server executes the plugin\'s deinitialization\nfunction and removes the row for the plugin from the mysql.plugin\ntable, so that subsequent server restarts will not load and initialize\nthe plugin. UNINSTALL PLUGIN does not remove the plugin\'s shared\nlibrary file.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/uninstall-plugin.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (289,31,'ASBINARY','AsBinary(g), AsWKB(g)\n\nConverts a value in internal geometry format to its WKB representation\nand returns the binary result.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…','SELECT AsBinary(g) FROM geom;\n','http://dev.mysql.com/doc/refman/5.1/en/functions-to-convert-geometries-betw…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (290,35,'REPEAT FUNCTION','Syntax:\nREPEAT(str,count)\n\nReturns a string consisting of the string str repeated count times. If\ncount is less than 1, returns an empty string. Returns NULL if str or\ncount are NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT REPEAT(\'MySQL\', 3);\n -> \'MySQLMySQLMySQL\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (291,25,'SHOW TABLES','Syntax:\nSHOW [FULL] TABLES [{FROM | IN} db_name]\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW TABLES lists the non-TEMPORARY tables in a given database. You can\nalso get this list using the mysqlshow db_name command. The LIKE\nclause, if present, indicates which table names to match. The WHERE\nclause can be given to select rows using more general conditions, as\ndiscussed in http://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nThis statement also lists any views in the database. The FULL modifier\nis supported such that SHOW FULL TABLES displays a second output\ncolumn. Values for the second column are BASE TABLE for a table and\nVIEW for a view.\n\nIf you have no privileges for a base table or view, it does not show up\nin the output from SHOW TABLES or mysqlshow db_name.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-tables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-tables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (292,30,'MAKEDATE','Syntax:\nMAKEDATE(year,dayofyear)\n\nReturns a date, given year and day-of-year values. dayofyear must be\ngreater than 0 or the result is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MAKEDATE(2011,31), MAKEDATE(2011,32);\n -> \'2011-01-31\', \'2011-02-01\'\nmysql> SELECT MAKEDATE(2011,365), MAKEDATE(2014,365);\n -> \'2011-12-31\', \'2014-12-31\'\nmysql> SELECT MAKEDATE(2011,0);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (293,35,'BINARY OPERATOR','Syntax:\nBINARY\n\nThe BINARY operator casts the string following it to a binary string.\nThis is an easy way to force a column comparison to be done byte by\nbyte rather than character by character. This causes the comparison to\nbe case sensitive even if the column isn\'t defined as BINARY or BLOB.\nBINARY also causes trailing spaces to be significant.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html\n\n','mysql> SELECT \'a\' = \'A\';\n -> 1\nmysql> SELECT BINARY \'a\' = \'A\';\n -> 0\nmysql> SELECT \'a\' = \'a \';\n -> 1\nmysql> SELECT BINARY \'a\' = \'a \';\n -> 0\n','http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (294,6,'MBROVERLAPS','MBROverlaps(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 overlap. The term spatially overlaps is\nused if two geometries intersect and their intersection results in a\ngeometry of the same dimension but not equal to either of the given\ngeometries.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (295,35,'SOUNDEX','Syntax:\nSOUNDEX(str)\n\nReturns a soundex string from str. Two strings that sound almost the\nsame should have identical soundex strings. A standard soundex string\nis four characters long, but the SOUNDEX() function returns an\narbitrarily long string. You can use SUBSTRING() on the result to get a\nstandard soundex string. All nonalphabetic characters in str are\nignored. All international alphabetic characters outside the A-Z range\nare treated as vowels.\n\n*Important*: When using SOUNDEX(), you should be aware of the following\nlimitations:\n\no This function, as currently implemented, is intended to work well\n with strings that are in the English language only. Strings in other\n languages may not produce reliable results.\n\no This function is not guaranteed to provide consistent results with\n strings that use multi-byte character sets, including utf-8.\n\n We hope to remove these limitations in a future release. See\n Bug#22638 (http://bugs.mysql.com/22638) for more information.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT SOUNDEX(\'Hello\');\n -> \'H400\'\nmysql> SELECT SOUNDEX(\'Quadratically\');\n -> \'Q36324\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (295,35,'SOUNDEX','Syntax:\nSOUNDEX(str)\n\nReturns a soundex string from str. Two strings that sound almost the\nsame should have identical soundex strings. A standard soundex string\nis four characters long, but the SOUNDEX() function returns an\narbitrarily long string. You can use SUBSTRING() on the result to get a\nstandard soundex string. All nonalphabetic characters in str are\nignored. All international alphabetic characters outside the A-Z range\nare treated as vowels.\n\n*Important*: When using SOUNDEX(), you should be aware of the following\nlimitations:\n\no This function, as currently implemented, is intended to work well\n with strings that are in the English language only. Strings in other\n languages may not produce reliable results.\n\no This function is not guaranteed to provide consistent results with\n strings that use multi-byte character sets, including utf-8.\n\n We hope to remove these limitations in a future release. See\n Bug#22638 (http://bugs.mysql.com/bug.php?id=22638) for more\n information.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT SOUNDEX(\'Hello\');\n -> \'H400\'\nmysql> SELECT SOUNDEX(\'Quadratically\');\n -> \'Q36324\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (296,6,'MBRTOUCHES','MBRTouches(g1,g2)\n\nReturns 1 or 0 to indicate whether the Minimum Bounding Rectangles of\nthe two geometries g1 and g2 touch. Two geometries spatially touch if\nthe interiors of the geometries do not intersect, but the boundary of\none of the geometries intersects either the boundary or the interior of\nthe other.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/relations-on-geometry-mbr.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (297,37,'DROP EVENT','Syntax:\nDROP EVENT [IF EXISTS] event_name\n\nThis statement drops the event named event_name. The event immediately\nceases being active, and is deleted completely from the server.\n\nIf the event does not exist, the error ERROR 1517 (HY000): Unknown\nevent \'event_name\' results. You can override this and cause the\nstatement to generate a warning for nonexistent events instead using IF\nEXISTS.\n\nBeginning with MySQL 5.1.12, this statement requires the EVENT\nprivilege for the schema to which the event to be dropped belongs. (In\nMySQL 5.1.11 and earlier, an event could be dropped only by its\ndefiner, or by a user having the SUPER privilege.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-event.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-event.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (298,26,'INSERT SELECT','Syntax:\nINSERT [LOW_PRIORITY | HIGH_PRIORITY] [IGNORE]\n [INTO] tbl_name [(col_name,...)]\n SELECT ...\n [ ON DUPLICATE KEY UPDATE col_name=expr, ... ]\n\nWith INSERT ... SELECT, you can quickly insert many rows into a table\nfrom one or many tables. For example:\n\nINSERT INTO tbl_temp2 (fld_id)\n SELECT tbl_temp1.fld_order_id\n FROM tbl_temp1 WHERE tbl_temp1.fld_order_id > 100;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/insert-select.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/insert-select.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (299,37,'CREATE PROCEDURE','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n PROCEDURE sp_name ([proc_parameter[,...]])\n [characteristic ...] routine_body\n\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n FUNCTION sp_name ([func_parameter[,...]])\n RETURNS type\n [characteristic ...] routine_body\n\nproc_parameter:\n [ IN | OUT | INOUT ] param_name type\n\nfunc_parameter:\n param_name type\n\ntype:\n Any valid MySQL data type\n\ncharacteristic:\n LANGUAGE SQL\n | [NOT] DETERMINISTIC\n | { CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }\n | SQL SECURITY { DEFINER | INVOKER }\n | COMMENT \'string\'\n\nroutine_body:\n Valid SQL procedure statement\n\nThese statements create stored routines. By default, a routine is\nassociated with the default database. To associate the routine\nexplicitly with a given database, specify the name as db_name.sp_name\nwhen you create it.\n\nThe CREATE FUNCTION statement is also used in MySQL to support UDFs\n(user-defined functions). See\nhttp://dev.mysql.com/doc/refman/5.1/en/adding-functions.html. A UDF can\nbe regarded as an external stored function. However, do note that\nstored functions share their namespace with UDFs. See\nhttp://dev.mysql.com/doc/refman/5.1/en/function-resolution.html, for\nthe rules describing how the server interprets references to different\nkinds of functions.\n\nTo invoke a stored procedure, use the CALL statement (see [HELP CALL]).\nTo invoke a stored function, refer to it in an expression. The function\nreturns a value during expression evaluation.\n\nTo execute the CREATE PROCEDURE or CREATE FUNCTION statement, it is\nnecessary to have the CREATE ROUTINE privilege. By default, MySQL\nautomatically grants the ALTER ROUTINE and EXECUTE privileges to the\nroutine creator. This behavior can be changed by disabling the\nautomatic_sp_privileges system variable. See\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-routines-privileges.html… binary logging is enabled, the CREATE FUNCTION statement might also\nrequire the SUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\n\… DEFINER and SQL SECURITY clauses specify the security context to be\nused when checking access privileges at routine execution time, as\ndescribed later.\n\nIf the routine name is the same as the name of a built-in SQL function,\na syntax error occurs unless you use a space between the name and the\nfollowing parenthesis when defining the routine or invoking it later.\nFor this reason, avoid using the names of existing SQL functions for\nyour own stored routines.\n\nThe IGNORE_SPACE SQL mode applies to built-in functions, not to stored\nroutines. It is always allowable to have spaces after a stored routine\nname, regardless of whether IGNORE_SPACE is enabled.\n\nThe parameter list enclosed within parentheses must always be present.\nIf there are no parameters, an empty parameter list of () should be\nused. Parameter names are not case sensitive.\n\nEach parameter is an IN parameter by default. To specify otherwise for\na parameter, use the keyword OUT or INOUT before the parameter name.\n\n*Note*: Specifying a parameter as IN, OUT, or INOUT is valid only for a\nPROCEDURE. (FUNCTION parameters are always regarded as IN parameters.)\n\nAn IN parameter passes a value into a procedure. The procedure might\nmodify the value, but the modification is not visible to the caller\nwhen the procedure returns. An OUT parameter passes a value from the\nprocedure back to the caller. Its initial value is NULL within the\nprocedure, and its value is visible to the caller when the procedure\nreturns. An INOUT parameter is initialized by the caller, can be\nmodified by the procedure, and any change made by the procedure is\nvisible to the caller when the procedure returns.\n\nFor each OUT or INOUT parameter, pass a user-defined variable in the\nCALL statement that invokes the procedure so that you can obtain its\nvalue when the procedure returns. If you are calling the procedure from\nwithin another stored procedure or function, you can also pass a\nroutine parameter or local routine variable as an IN or INOUT\nparameter.\n\nThe following example shows a simple stored procedure that uses an OUT\nparameter:\n\nmysql> delimiter //\n\nmysql> CREATE PROCEDURE simpleproc (OUT param1 INT)\n -> BEGIN\n -> SELECT COUNT(*) INTO param1 FROM t;\n -> END//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> delimiter ;\n\nmysql> CALL simpleproc(@a);\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @a;\n+------+\n| @a |\n+------+\n| 3 |\n+------+\n1 row in set (0.00 sec)\n\nThe example uses the mysql client delimiter command to change the\nstatement delimiter from ; to // while the procedure is being defined.\nThis allows the ; delimiter used in the procedure body to be passed\nthrough to the server rather than being interpreted by mysql itself.\nSee\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defini… RETURNS clause may be specified only for a FUNCTION, for which it\nis mandatory. It indicates the return type of the function, and the\nfunction body must contain a RETURN value statement. If the RETURN\nstatement returns a value of a different type, the value is coerced to\nthe proper type. For example, if a function specifies an ENUM or SET\nvalue in the RETURNS clause, but the RETURN statement returns an\ninteger, the value returned from the function is the string for the\ncorresponding ENUM member of set of SET members.\n\nThe following example function takes a parameter, performs an operation\nusing an SQL function, and returns the result. In this case, it is\nunnecessary to use delimiter because the function definition contains\nno internal ; statement delimiters:\n\nmysql> CREATE FUNCTION hello (s CHAR(20))\nmysql> RETURNS CHAR(50) DETERMINISTIC\n -> RETURN CONCAT(\'Hello, \',s,\'!\');\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT hello(\'world\');\n+----------------+\n| hello(\'world\') |\n+----------------+\n| Hello, world! |\n+----------------+\n1 row in set (0.00 sec)\n\nParameter types and function return types can be declared to use any\nvalid data type, except that the COLLATE attribute cannot be used.\n\nThe routine_body consists of a valid SQL procedure statement. This can\nbe a simple statement such as SELECT or INSERT, or it can be a compound\nstatement written using BEGIN and END. Compound statements can contain\ndeclarations, loops, and other control structure statements. The syntax\nfor these statements is described in\nhttp://dev.mysql.com/doc/refman/5.1/en/sql-syntax-compound-statements.h… allows routines to contain DDL statements, such as CREATE and\nDROP. MySQL also allows stored procedures (but not stored functions) to\ncontain SQL transaction statements such as COMMIT. Stored functions may\nnot contain statements that perform explicit or implicit commit or\nrollback. Support for these statements is not required by the SQL\nstandard, which states that each DBMS vendor may decide whether to\nallow them.\n\nStatements that return a result set can be used within a stored\nprocedcure but not within a stored function. This prohibition includes\nSELECT statements that do not have an INTO var_list clause and other\nstatements such as SHOW, EXPLAIN, and CHECK TABLE. For statements that\ncan be determined at function definition time to return a result set, a\nNot allowed to return a result set from a function error occurs\n(ER_SP_NO_RETSET). For statements that can be determined only at\nruntime to return a result set, a PROCEDURE %s can\'t return a result\nset in the given context error occurs (ER_SP_BADSELECT).\n\nUSE statements within stored routines are disallowed. When a routine is\ninvoked, an implicit USE db_name is performed (and undone when the\nroutine terminates). The causes the routine to have the given default\ndatabase while it executes. References to objects in databases other\nthan the routine default database should be qualified with the\nappropriate database name.\n\nFor additional information about statements that are not allowed in\nstored routines, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-program-restrictions.htm… information about invoking stored procedures from within programs\nwritten in a language that has a MySQL interface, see [HELP CALL].\n\nMySQL stores the sql_mode system variable setting that is in effect at\nthe time a routine is created, and always executes the routine with\nthis setting in force, regardless of the server SQL mode in effect when\nthe routine is invoked.\n\nThe switch from the SQL mode of the invoker to that of the routine\noccurs after evaluation of arguments and assignment of the resulting\nvalues to routine parameters. If you define a routine in strict SQL\nmode but invoke it in nonstrict mode, assignment of arguments to\nroutine parameters does not take place in strict mode. If you require\nthat expressions passed to a routine be assigned in strict SQL mode,\nyou should invoke the routine with strict mode in effect.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (299,37,'CREATE PROCEDURE','Syntax:\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n PROCEDURE sp_name ([proc_parameter[,...]])\n [characteristic ...] routine_body\n\nCREATE\n [DEFINER = { user | CURRENT_USER }]\n FUNCTION sp_name ([func_parameter[,...]])\n RETURNS type\n [characteristic ...] routine_body\n\nproc_parameter:\n [ IN | OUT | INOUT ] param_name type\n\nfunc_parameter:\n param_name type\n\ntype:\n Any valid MySQL data type\n\ncharacteristic:\n LANGUAGE SQL\n | [NOT] DETERMINISTIC\n | { CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }\n | SQL SECURITY { DEFINER | INVOKER }\n | COMMENT \'string\'\n\nroutine_body:\n Valid SQL procedure statement\n\nThese statements create stored routines. By default, a routine is\nassociated with the default database. To associate the routine\nexplicitly with a given database, specify the name as db_name.sp_name\nwhen you create it.\n\nThe CREATE FUNCTION statement is also used in MySQL to support UDFs\n(user-defined functions). See\nhttp://dev.mysql.com/doc/refman/5.1/en/adding-functions.html. A UDF can\nbe regarded as an external stored function. However, do note that\nstored functions share their namespace with UDFs. See\nhttp://dev.mysql.com/doc/refman/5.1/en/function-resolution.html, for\nthe rules describing how the server interprets references to different\nkinds of functions.\n\nTo invoke a stored procedure, use the CALL statement (see [HELP CALL]).\nTo invoke a stored function, refer to it in an expression. The function\nreturns a value during expression evaluation.\n\nTo execute the CREATE PROCEDURE or CREATE FUNCTION statement, it is\nnecessary to have the CREATE ROUTINE privilege. By default, MySQL\nautomatically grants the ALTER ROUTINE and EXECUTE privileges to the\nroutine creator. This behavior can be changed by disabling the\nautomatic_sp_privileges system variable. See\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-routines-privileges.html… binary logging is enabled, the CREATE FUNCTION statement might also\nrequire the SUPER privilege, as described in\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-logging.html.\nS… may also be required depending on the DEFINER value, as described\nlater.\n\nThe DEFINER and SQL SECURITY clauses specify the security context to be\nused when checking access privileges at routine execution time, as\ndescribed later.\n\nIf the routine name is the same as the name of a built-in SQL function,\na syntax error occurs unless you use a space between the name and the\nfollowing parenthesis when defining the routine or invoking it later.\nFor this reason, avoid using the names of existing SQL functions for\nyour own stored routines.\n\nThe IGNORE_SPACE SQL mode applies to built-in functions, not to stored\nroutines. It is always allowable to have spaces after a stored routine\nname, regardless of whether IGNORE_SPACE is enabled.\n\nThe parameter list enclosed within parentheses must always be present.\nIf there are no parameters, an empty parameter list of () should be\nused. Parameter names are not case sensitive.\n\nEach parameter is an IN parameter by default. To specify otherwise for\na parameter, use the keyword OUT or INOUT before the parameter name.\n\n*Note*: Specifying a parameter as IN, OUT, or INOUT is valid only for a\nPROCEDURE. (FUNCTION parameters are always regarded as IN parameters.)\n\nAn IN parameter passes a value into a procedure. The procedure might\nmodify the value, but the modification is not visible to the caller\nwhen the procedure returns. An OUT parameter passes a value from the\nprocedure back to the caller. Its initial value is NULL within the\nprocedure, and its value is visible to the caller when the procedure\nreturns. An INOUT parameter is initialized by the caller, can be\nmodified by the procedure, and any change made by the procedure is\nvisible to the caller when the procedure returns.\n\nFor each OUT or INOUT parameter, pass a user-defined variable in the\nCALL statement that invokes the procedure so that you can obtain its\nvalue when the procedure returns. If you are calling the procedure from\nwithin another stored procedure or function, you can also pass a\nroutine parameter or local routine variable as an IN or INOUT\nparameter.\n\nThe following example shows a simple stored procedure that uses an OUT\nparameter:\n\nmysql> delimiter //\n\nmysql> CREATE PROCEDURE simpleproc (OUT param1 INT)\n -> BEGIN\n -> SELECT COUNT(*) INTO param1 FROM t;\n -> END//\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> delimiter ;\n\nmysql> CALL simpleproc(@a);\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT @a;\n+------+\n| @a |\n+------+\n| 3 |\n+------+\n1 row in set (0.00 sec)\n\nThe example uses the mysql client delimiter command to change the\nstatement delimiter from ; to // while the procedure is being defined.\nThis allows the ; delimiter used in the procedure body to be passed\nthrough to the server rather than being interpreted by mysql itself.\nSee\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defini… RETURNS clause may be specified only for a FUNCTION, for which it\nis mandatory. It indicates the return type of the function, and the\nfunction body must contain a RETURN value statement. If the RETURN\nstatement returns a value of a different type, the value is coerced to\nthe proper type. For example, if a function specifies an ENUM or SET\nvalue in the RETURNS clause, but the RETURN statement returns an\ninteger, the value returned from the function is the string for the\ncorresponding ENUM member of set of SET members.\n\nThe following example function takes a parameter, performs an operation\nusing an SQL function, and returns the result. In this case, it is\nunnecessary to use delimiter because the function definition contains\nno internal ; statement delimiters:\n\nmysql> CREATE FUNCTION hello (s CHAR(20))\nmysql> RETURNS CHAR(50) DETERMINISTIC\n -> RETURN CONCAT(\'Hello, \',s,\'!\');\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql> SELECT hello(\'world\');\n+----------------+\n| hello(\'world\') |\n+----------------+\n| Hello, world! |\n+----------------+\n1 row in set (0.00 sec)\n\nParameter types and function return types can be declared to use any\nvalid data type, except that the COLLATE attribute cannot be used.\n\nThe routine_body consists of a valid SQL procedure statement. This can\nbe a simple statement such as SELECT or INSERT, or it can be a compound\nstatement written using BEGIN and END. Compound statements can contain\ndeclarations, loops, and other control structure statements. The syntax\nfor these statements is described in\nhttp://dev.mysql.com/doc/refman/5.1/en/sql-syntax-compound-statements.h… allows routines to contain DDL statements, such as CREATE and\nDROP. MySQL also allows stored procedures (but not stored functions) to\ncontain SQL transaction statements such as COMMIT. Stored functions may\nnot contain statements that perform explicit or implicit commit or\nrollback. Support for these statements is not required by the SQL\nstandard, which states that each DBMS vendor may decide whether to\nallow them.\n\nStatements that return a result set can be used within a stored\nprocedcure but not within a stored function. This prohibition includes\nSELECT statements that do not have an INTO var_list clause and other\nstatements such as SHOW, EXPLAIN, and CHECK TABLE. For statements that\ncan be determined at function definition time to return a result set, a\nNot allowed to return a result set from a function error occurs\n(ER_SP_NO_RETSET). For statements that can be determined only at\nruntime to return a result set, a PROCEDURE %s can\'t return a result\nset in the given context error occurs (ER_SP_BADSELECT).\n\nUSE statements within stored routines are disallowed. When a routine is\ninvoked, an implicit USE db_name is performed (and undone when the\nroutine terminates). This causes the routine to have the given default\ndatabase while it executes. References to objects in databases other\nthan the routine default database should be qualified with the\nappropriate database name.\n\nFor additional information about statements that are not allowed in\nstored routines, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-program-restrictions.htm… information about invoking stored procedures from within programs\nwritten in a language that has a MySQL interface, see [HELP CALL].\n\nMySQL stores the sql_mode system variable setting that is in effect at\nthe time a routine is created, and always executes the routine with\nthis setting in force, regardless of the server SQL mode in effect when\nthe routine is invoked.\n\nThe switch from the SQL mode of the invoker to that of the routine\noccurs after evaluation of arguments and assignment of the resulting\nvalues to routine parameters. If you define a routine in strict SQL\nmode but invoke it in nonstrict mode, assignment of arguments to\nroutine parameters does not take place in strict mode. If you require\nthat expressions passed to a routine be assigned in strict SQL mode,\nyou should invoke the routine with strict mode in effect.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/create-procedure.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (300,20,'VARBINARY','VARBINARY(M)\n\nThe VARBINARY type is similar to the VARCHAR type, but stores binary\nbyte strings rather than nonbinary character strings. M represents the\nmaximum column length in bytes.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (301,25,'LOAD INDEX','Syntax:\nLOAD INDEX INTO CACHE\n tbl_index_list [, tbl_index_list] ...\n\ntbl_index_list:\n tbl_name\n [[INDEX|KEY] (index_name[, index_name] ...)]\n [IGNORE LEAVES]\n\nThe LOAD INDEX INTO CACHE statement preloads a table index into the key\ncache to which it has been assigned by an explicit CACHE INDEX\nstatement, or into the default key cache otherwise. LOAD INDEX INTO\nCACHE is used only for MyISAM tables. It is not supported for tables\nhaving user-defined partitioning (see\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-limitations.html.)… IGNORE LEAVES modifier causes only blocks for the nonleaf nodes of\nthe index to be preloaded.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-index.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-index.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (302,26,'UNION','Syntax:\nSELECT ...\nUNION [ALL | DISTINCT] SELECT ...\n[UNION [ALL | DISTINCT] SELECT ...]\n\nUNION is used to combine the result from multiple SELECT statements\ninto a single result set.\n\nThe column names from the first SELECT statement are used as the column\nnames for the results returned. Selected columns listed in\ncorresponding positions of each SELECT statement should have the same\ndata type. (For example, the first column selected by the first\nstatement should have the same type as the first column selected by the\nother statements.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/union.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/union.html'); @@ -384,16 +384,16 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (315,15,'FOUND_ROWS','Syntax:\nFOUND_ROWS()\n\nA SELECT statement may include a LIMIT clause to restrict the number of\nrows the server returns to the client. In some cases, it is desirable\nto know how many rows the statement would have returned without the\nLIMIT, but without running the statement again. To obtain this row\ncount, include a SQL_CALC_FOUND_ROWS option in the SELECT statement,\nand then invoke FOUND_ROWS() afterward:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name\n -> WHERE id > 100 LIMIT 10;\nmysql> SELECT FOUND_ROWS();\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (316,15,'SYSTEM_USER','Syntax:\nSYSTEM_USER()\n\nSYSTEM_USER() is a synonym for USER().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (317,29,'CROSSES','Crosses(g1,g2)\n\nReturns 1 if g1 spatially crosses g2. Returns NULL if g1 is a Polygon\nor a MultiPolygon, or if g2 is a Point or a MultiPoint. Otherwise,\nreturns 0.\n\nThe term spatially crosses denotes a spatial relation between two given\ngeometries that has the following properties:\n\no The two geometries intersect\n\no Their intersection results in a geometry that has a dimension that is\n one less than the maximum dimension of the two given geometries\n\no Their intersection is not equal to either of the two given geometries\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…','','http://dev.mysql.com/doc/refman/5.1/en/functions-that-test-spatial-relation…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (318,26,'TRUNCATE TABLE','Syntax:\nTRUNCATE [TABLE] tbl_name\n\nTRUNCATE TABLE empties a table completely. It requires the DROP\nprivilege as of MySQL 5.1.16. (Before 5.1.16, it requires the DELETE\nprivilege.\n\nLogically, TRUNCATE TABLE is equivalent to a DELETE statement that\ndeletes all rows, but there are practical differences under some\ncircumstances.\n\nFor an InnoDB table, InnoDB processes TRUNCATE TABLE by deleting rows\none by one if there are any FOREIGN KEY constraints that reference the\ntable. If there are no FOREIGN KEY constraints, InnoDB performs fast\ntruncation by dropping the original table and creating an empty one\nwith the same definition, which is much faster than deleting rows one\nby one. The AUTO_INCREMENT counter is reset by TRUNCATE TABLE,\nregardless of whether there is a FOREIGN KEY constraint.\n\nIn the case that FOREIGN KEY constraints reference the table, InnoDB\ndeletes rows one by one and processes the constraints on each one. If\nthe FOREIGN KEY constraint specifies DELETE CASCADE, rows from the\nchild (referenced) table are deleted, and the truncated table becomes\nempty. If the FOREIGN KEY constraint does not specify CASCADE, the\nTRUNCATE statement deletes rows one by one and stops if it encounters a\nparent row that is referenced by the child, returning this error:\n\nERROR 1451 (23000): Cannot delete or update a parent row: a foreign\nkey constraint fails (`test`.`child`, CONSTRAINT `child_ibfk_1`\nFOREIGN KEY (`parent_id`) REFERENCES `parent` (`id`))\n\nThis is the same as a DELETE statement with no WHERE clause.\n\nThe count of rows affected by TRUNCATE TABLE is accurate only when it\nis mapped to a DELETE statement.\n\nFor other storage engines, TRUNCATE TABLE differs from DELETE in the\nfollowing ways in MySQL 5.1:\n\no Truncate operations drop and re-create the table, which is much\n faster than deleting rows one by one, particularly for large tables.\n\no Truncate operations cause an implicit commit.\n\no Truncation operations cannot be performed if the session holds an\n active table lock.\n\no Truncation operations do not return a meaningful value for the number\n of deleted rows. The usual result is "0 rows affected," which should\n be interpreted as "no information."\n\no As long as the table format file tbl_name.frm is valid, the table can\n be re-created as an empty table with TRUNCATE TABLE, even if the data\n or index files have become corrupted.\n\no The table handler does not remember the last used AUTO_INCREMENT\n value, but starts counting from the beginning. This is true even for\n MyISAM and InnoDB, which normally do not reuse sequence values.\n\no When used with partitioned tables, TRUNCATE TABLE preserves the\n partitioning; that is, the data and index files are dropped and\n re-created, while the partition definitions (.par) file is\n unaffected.\n\no Since truncation of a table does not make any use of DELETE, the\n TRUNCATE statement does not invoke ON DELETE triggers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/truncate.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/truncate.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (318,26,'TRUNCATE TABLE','Syntax:\nTRUNCATE [TABLE] tbl_name\n\nTRUNCATE TABLE empties a table completely. It requires the DROP\nprivilege as of MySQL 5.1.16. (Before 5.1.16, it requires the DELETE\nprivilege).\n\nLogically, TRUNCATE TABLE is equivalent to a DELETE statement that\ndeletes all rows, but there are practical differences under some\ncircumstances.\n\nFor an InnoDB table, InnoDB processes TRUNCATE TABLE by deleting rows\none by one if there are any FOREIGN KEY constraints that reference the\ntable. If there are no FOREIGN KEY constraints, InnoDB performs fast\ntruncation by dropping the original table and creating an empty one\nwith the same definition, which is much faster than deleting rows one\nby one. The AUTO_INCREMENT counter is reset to zero by TRUNCATE TABLE,\nregardless of whether there is a FOREIGN KEY constraint.\n\nIn the case that FOREIGN KEY constraints reference the table, InnoDB\ndeletes rows one by one and processes the constraints on each one. If\nthe FOREIGN KEY constraint specifies DELETE CASCADE, rows from the\nchild (referenced) table are deleted, and the truncated table becomes\nempty. If the FOREIGN KEY constraint does not specify CASCADE, the\nTRUNCATE TABLE statement deletes rows one by one and stops if it\nencounters a parent row that is referenced by the child, returning this\nerror:\n\nERROR 1451 (23000): Cannot delete or update a parent row: a foreign\nkey constraint fails (`test`.`child`, CONSTRAINT `child_ibfk_1`\nFOREIGN KEY (`parent_id`) REFERENCES `parent` (`id`))\n\nThis is the same as a DELETE statement with no WHERE clause.\n\nThe count of rows affected by TRUNCATE TABLE is accurate only when it\nis mapped to a DELETE statement.\n\nFor other storage engines, TRUNCATE TABLE differs from DELETE in the\nfollowing ways in MySQL 5.1:\n\no Truncate operations drop and re-create the table, which is much\n faster than deleting rows one by one, particularly for large tables.\n\no Truncate operations cause an implicit commit.\n\no Truncation operations cannot be performed if the session holds an\n active table lock.\n\no Truncation operations do not return a meaningful value for the number\n of deleted rows. The usual result is "0 rows affected," which should\n be interpreted as "no information."\n\no As long as the table format file tbl_name.frm is valid, the table can\n be re-created as an empty table with TRUNCATE TABLE, even if the data\n or index files have become corrupted.\n\no The table handler does not remember the last used AUTO_INCREMENT\n value, but starts counting from the beginning. This is true even for\n MyISAM and InnoDB, which normally do not reuse sequence values.\n\no When used with partitioned tables, TRUNCATE TABLE preserves the\n partitioning; that is, the data and index files are dropped and\n re-created, while the partition definitions (.par) file is\n unaffected.\n\no Since truncation of a table does not make any use of DELETE, the\n TRUNCATE TABLE statement does not invoke ON DELETE triggers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/truncate-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/truncate-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (319,16,'BIT_XOR','Syntax:\nBIT_XOR(expr)\n\nReturns the bitwise XOR of all bits in expr. The calculation is\nperformed with 64-bit (BIGINT) precision.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (320,30,'CURRENT_DATE','Syntax:\nCURRENT_DATE, CURRENT_DATE()\n\nCURRENT_DATE and CURRENT_DATE() are synonyms for CURDATE().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (321,25,'START SLAVE','Syntax:\nSTART SLAVE [thread_type [, thread_type] ... ]\nSTART SLAVE [SQL_THREAD] UNTIL\n MASTER_LOG_FILE = \'log_name\', MASTER_LOG_POS = log_pos\nSTART SLAVE [SQL_THREAD] UNTIL\n RELAY_LOG_FILE = \'log_name\', RELAY_LOG_POS = log_pos\n\nthread_type: IO_THREAD | SQL_THREAD\n\nSTART SLAVE with no thread_type options starts both of the slave\nthreads. The I/O thread reads queries from the master server and stores\nthem in the relay log. The SQL thread reads the relay log and executes\nthe queries. START SLAVE requires the SUPER privilege.\n\nIf START SLAVE succeeds in starting the slave threads, it returns\nwithout any error. However, even in that case, it might be that the\nslave threads start and then later stop (for example, because they do\nnot manage to connect to the master or read its binary logs, or some\nother problem). START SLAVE does not warn you about this. You must\ncheck the slave\'s error log for error messages generated by the slave\nthreads, or check that they are running satisfactorily with SHOW SLAVE\nSTATUS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/start-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/start-slave.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (321,25,'START SLAVE','Syntax:\nSTART SLAVE [thread_type [, thread_type] ... ]\nSTART SLAVE [SQL_THREAD] UNTIL\n MASTER_LOG_FILE = \'log_name\', MASTER_LOG_POS = log_pos\nSTART SLAVE [SQL_THREAD] UNTIL\n RELAY_LOG_FILE = \'log_name\', RELAY_LOG_POS = log_pos\n\nthread_type: IO_THREAD | SQL_THREAD\n\nSTART SLAVE with no thread_type options starts both of the slave\nthreads. The I/O thread reads events from the master server and stores\nthem in the relay log. The SQL thread reads events from the relay log\nand executes them. START SLAVE requires the SUPER privilege.\n\nIf START SLAVE succeeds in starting the slave threads, it returns\nwithout any error. However, even in that case, it might be that the\nslave threads start and then later stop (for example, because they do\nnot manage to connect to the master or read its binary log, or some\nother problem). START SLAVE does not warn you about this. You must\ncheck the slave\'s error log for error messages generated by the slave\nthreads, or check that they are running satisfactorily with SHOW SLAVE\nSTATUS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/start-slave.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/start-slave.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (322,2,'AREA','Area(poly)\n\nReturns as a double-precision number the area of the Polygon value\npoly, as measured in its spatial reference system.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…','mysql> SET @poly = \'Polygon((0 0,0 3,3 0,0 0),(1 1,1 2,2 1,1 1))\';\nmysql> SELECT Area(GeomFromText(@poly));\n+---------------------------+\n| Area(GeomFromText(@poly)) |\n+---------------------------+\n| 4 |\n+---------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (323,22,'BEGIN END','Syntax:\n[begin_label:] BEGIN\n [statement_list]\nEND [end_label]\n\nBEGIN ... END syntax is used for writing compound statements, which can\nappear within stored programs. A compound statement can contain\nmultiple statements, enclosed by the BEGIN and END keywords.\nstatement_list represents a list of one or more statements, each\nterminated by a semicolon (;) statement delimiter. statement_list is\noptional, which means that the empty compound statement (BEGIN END) is\nlegal.\n\nUse of multiple statements requires that a client is able to send\nstatement strings containing the ; statement delimiter. This is handled\nin the mysql command-line client with the delimiter command. Changing\nthe ; end-of-statement delimiter (for example, to //) allows ; to be\nused in a program body. For an example, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defining.html.\… compound statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/begin-end.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/begin-end.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (324,25,'FLUSH','Syntax:\nFLUSH [NO_WRITE_TO_BINLOG | LOCAL]\n flush_option [, flush_option] ...\n\nThe FLUSH statement clears or reloads various internal caches used by\nMySQL. To execute FLUSH, you must have the RELOAD privilege.\n\nThe RESET statement is similar to FLUSH. See [HELP RESET].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/flush.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/flush.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (323,22,'BEGIN END','Syntax:\n[begin_label:] BEGIN\n [statement_list]\nEND [end_label]\n\nBEGIN ... END syntax is used for writing compound statements, which can\nappear within stored programs. A compound statement can contain\nmultiple statements, enclosed by the BEGIN and END keywords.\nstatement_list represents a list of one or more statements, each\nterminated by a semicolon (;) statement delimiter. statement_list is\noptional, which means that the empty compound statement (BEGIN END) is\nlegal.\n\nUse of multiple statements requires that a client is able to send\nstatement strings containing the ; statement delimiter. This is handled\nin the mysql command-line client with the delimiter command. Changing\nthe ; end-of-statement delimiter (for example, to //) allows ; to be\nused in a program body. For an example, see\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-programs-defining.html.\… BEGIN ... END block can be labeled. Labels follow these rules:\n\no end_label cannot be given unless begin_label is also present.\n\no If both begin_label and end_label are present, they must be the same.\n\no Labels can be up to 16 characters long.\n\nLabels are also allowed for the LOOP, REPEAT, and WHILE statements.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/begin-end.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/begin-end.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (324,25,'FLUSH','Syntax:\nFLUSH [NO_WRITE_TO_BINLOG | LOCAL]\n flush_option [, flush_option] ...\n\nThe FLUSH statement clears or reloads various internal caches used by\nMySQL. One variant acquires a lock. To execute FLUSH, you must have the\nRELOAD privilege.\n\nBy default, FLUSH statements are written to the binary log so that they\nwill be replicated to replication slaves. Logging can be suppressed\nwith the optional NO_WRITE_TO_BINLOG keyword or its alias LOCAL.\n\n*Note*: FLUSH LOGS, FLUSH MASTER, FLUSH SLAVE, and FLUSH TABLES WITH\nREAD LOCK are not written to the binary log in any case because they\nwould cause problems if replicated to a slave.\n\nThe RESET statement is similar to FLUSH. See [HELP RESET], for\ninformation about using the RESET statement with replication.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/flush.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/flush.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (325,25,'SHOW PROCEDURE STATUS','Syntax:\nSHOW PROCEDURE STATUS\n [LIKE \'pattern\' | WHERE expr]\n\nThis statement is a MySQL extension. It returns characteristics of a\nstored procedure, such as the database, name, type, creator, creation\nand modification dates, and character set information. A similar\nstatement, SHOW FUNCTION STATUS, displays information about stored\nfunctions (see [HELP SHOW FUNCTION STATUS]).\n\nThe LIKE clause, if present, indicates which procedure or function\nnames to match. The WHERE clause can be given to select rows using more\ngeneral conditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-procedure-status.html\n\n','mysql> SHOW PROCEDURE STATUS LIKE \'sp1\'\\G\n*************************** 1. row ***************************\n Db: test\n Name: sp1\n Type: PROCEDURE\n Definer: testuser@localhost\n Modified: 2004-08-03 15:29:37\n Created: 2004-08-03 15:29:37\n Security_type: DEFINER\n Comment:\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n','http://dev.mysql.com/doc/refman/5.1/en/show-procedure-status.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (326,25,'SHOW WARNINGS','Syntax:\nSHOW WARNINGS [LIMIT [offset,] row_count]\nSHOW COUNT(*) WARNINGS\n\nSHOW WARNINGS shows the error, warning, and note messages that resulted\nfrom the last statement that generated messages in the current session.\nIt shows nothing if the last statement used a table and generated no\nmessages. (That is, a statement that uses a table but generates no\nmessages clears the message list.) Statements that do not use tables\nand do not generate messages have no effect on the message list.\n\nWarnings are generated for DML statements such as INSERT, UPDATE, and\nLOAD DATA INFILE as well as DDL statements such as CREATE TABLE and\nALTER TABLE.\n\nA related statement, SHOW ERRORS, shows only the errors. See [HELP SHOW\nERRORS].\n\nThe SHOW COUNT(*) WARNINGS statement displays the total number of\nerrors, warnings, and notes. You can also retrieve this number from the\nwarning_count variable:\n\nSHOW COUNT(*) WARNINGS;\nSELECT @@warning_count;\n\nThe value of warning_count might be greater than the number of messages\ndisplayed by SHOW WARNINGS if the max_error_count system variable is\nset so low that not all messages are stored. An example shown later in\nthis section demonstrates how this can happen.\n\nThe LIMIT clause has the same syntax as for the SELECT statement. See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (327,27,'DESCRIBE','Syntax:\n{DESCRIBE | DESC} tbl_name [col_name | wild]\n\nDESCRIBE provides information about the columns in a table. It is a\nshortcut for SHOW COLUMNS FROM. These statements also display\ninformation for views. (See [HELP SHOW COLUMNS].)\n\ncol_name can be a column name, or a string containing the SQL "%" and\n"_" wildcard characters to obtain output only for the columns with\nnames matching the string. There is no need to enclose the string\nwithin quotes unless it contains spaces or other special characters.\n\nmysql> DESCRIBE City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nThe description for SHOW COLUMNS provides more information about the\noutput columns (see [HELP SHOW COLUMNS]).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/describe.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/describe.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (326,27,'DESCRIBE','Syntax:\n{DESCRIBE | DESC} tbl_name [col_name | wild]\n\nDESCRIBE provides information about the columns in a table. It is a\nshortcut for SHOW COLUMNS FROM. These statements also display\ninformation for views. (See [HELP SHOW COLUMNS].)\n\ncol_name can be a column name, or a string containing the SQL "%" and\n"_" wildcard characters to obtain output only for the columns with\nnames matching the string. There is no need to enclose the string\nwithin quotes unless it contains spaces or other special characters.\n\nmysql> DESCRIBE City;\n+------------+----------+------+-----+---------+----------------+\n| Field | Type | Null | Key | Default | Extra |\n+------------+----------+------+-----+---------+----------------+\n| Id | int(11) | NO | PRI | NULL | auto_increment |\n| Name | char(35) | NO | | | |\n| Country | char(3) | NO | UNI | | |\n| District | char(20) | YES | MUL | | |\n| Population | int(11) | NO | | 0 | |\n+------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n\nThe description for SHOW COLUMNS provides more information about the\noutput columns (see [HELP SHOW COLUMNS]).\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/describe.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/describe.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (327,25,'SHOW WARNINGS','Syntax:\nSHOW WARNINGS [LIMIT [offset,] row_count]\nSHOW COUNT(*) WARNINGS\n\nSHOW WARNINGS shows the error, warning, and note messages that resulted\nfrom the last statement that generated messages in the current session.\nIt shows nothing if the last statement used a table and generated no\nmessages. (That is, a statement that uses a table but generates no\nmessages clears the message list.) Statements that do not use tables\nand do not generate messages have no effect on the message list.\n\nWarnings are generated for DML statements such as INSERT, UPDATE, and\nLOAD DATA INFILE as well as DDL statements such as CREATE TABLE and\nALTER TABLE.\n\nA related statement, SHOW ERRORS, shows only the errors. See [HELP SHOW\nERRORS].\n\nThe SHOW COUNT(*) WARNINGS statement displays the total number of\nerrors, warnings, and notes. You can also retrieve this number from the\nwarning_count variable:\n\nSHOW COUNT(*) WARNINGS;\nSELECT @@warning_count;\n\nThe value of warning_count might be greater than the number of messages\ndisplayed by SHOW WARNINGS if the max_error_count system variable is\nset so low that not all messages are stored. An example shown later in\nthis section demonstrates how this can happen.\n\nThe LIMIT clause has the same syntax as for the SELECT statement. See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (328,9,'DROP USER','Syntax:\nDROP USER user [, user] ...\n\nThe DROP USER statement removes one or more MySQL accounts. It removes\nprivilege rows for the account from all grant tables. To use this\nstatement, you must have the global CREATE USER privilege or the DELETE\nprivilege for the mysql database. Each account is named using the same\nformat as for the GRANT statement; for example, \'jeffrey\'@\'localhost\'.\nIf you specify only the user name part of the account name, a host name\npart of \'%\' is used. For additional information about specifying\naccount names, see [HELP GRANT].\n\nWith DROP USER, you can remove an account and its privileges as\nfollows:\n\nDROP USER user;\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-user.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-user.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (329,16,'STDDEV_POP','Syntax:\nSTDDEV_POP(expr)\n\nReturns the population standard deviation of expr (the square root of\nVAR_POP()). You can also use STD() or STDDEV(), which are equivalent\nbut not standard SQL.\n\nSTDDEV_POP() returns NULL if there were no matching rows.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (330,25,'SHOW CHARACTER SET','Syntax:\nSHOW CHARACTER SET\n [LIKE \'pattern\' | WHERE expr]\n\nThe SHOW CHARACTER SET statement shows all available character sets.\nThe LIKE clause, if present, indicates which character set names to\nmatch. The WHERE clause can be given to select rows using more general\nconditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html. For example:\n\nmysql> SHOW CHARACTER SET LIKE \'latin%\';\n+---------+-----------------------------+-------------------+--------+\n| Charset | Description | Default collation | Maxlen |\n+---------+-----------------------------+-------------------+--------+\n| latin1 | cp1252 West European | latin1_swedish_ci | 1 |\n| latin2 | ISO 8859-2 Central European | latin2_general_ci | 1 |\n| latin5 | ISO 8859-9 Turkish | latin5_turkish_ci | 1 |\n| latin7 | ISO 8859-13 Baltic | latin7_general_ci | 1 |\n+---------+-----------------------------+-------------------+--------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-character-set.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-character-set.html'); @@ -407,10 +407,10 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (338,14,'VALUES','Syntax:\nVALUES(col_name)\n\nIn an INSERT ... ON DUPLICATE KEY UPDATE statement, you can use the\nVALUES(col_name) function in the UPDATE clause to refer to column\nvalues from the INSERT portion of the statement. In other words,\nVALUES(col_name) in the UPDATE clause refers to the value of col_name\nthat would be inserted, had no duplicate-key conflict occurred. This\nfunction is especially useful in multiple-row inserts. The VALUES()\nfunction is meaningful only in INSERT ... ON DUPLICATE KEY UPDATE\nstatements and returns NULL otherwise.\nhttp://dev.mysql.com/doc/refman/5.1/en/insert-on-duplicate.html…: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','mysql> INSERT INTO table (a,b,c) VALUES (1,2,3),(4,5,6)\n -> ON DUPLICATE KEY UPDATE c=VALUES(a)+VALUES(b);\n','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (339,35,'SUBSTRING_INDEX','Syntax:\nSUBSTRING_INDEX(str,delim,count)\n\nReturns the substring from string str before count occurrences of the\ndelimiter delim. If count is positive, everything to the left of the\nfinal delimiter (counting from the left) is returned. If count is\nnegative, everything to the right of the final delimiter (counting from\nthe right) is returned. SUBSTRING_INDEX() performs a case-sensitive\nmatch when searching for delim.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT SUBSTRING_INDEX(\'www.mysql.com\', \'.\', 2);\n -> \'www.mysql\'\nmysql> SELECT SUBSTRING_INDEX(\'www.mysql.com\', \'.\', -2);\n -> \'mysql.com\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (340,11,'ENCODE','Syntax:\nENCODE(str,pass_str)\n\nEncrypt str using pass_str as the password. To decrypt the result, use\nDECODE().\n\nThe result is a binary string of the same length as str.\n\nThe strength of the encryption is based on how good the random\ngenerator is. It should suffice for short strings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (341,22,'LOOP','Syntax:\n[begin_label:] LOOP\n statement_list\nEND LOOP [end_label]\n\nLOOP implements a simple loop construct, enabling repeated execution of\nthe statement list, which consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter. The statements\nwithin the loop are repeated until the loop is exited; usually this is\naccomplished with a LEAVE statement.\n\nA LOOP statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (341,22,'LOOP','Syntax:\n[begin_label:] LOOP\n statement_list\nEND LOOP [end_label]\n\nLOOP implements a simple loop construct, enabling repeated execution of\nthe statement list, which consists of one or more statements, each\nterminated by a semicolon (;) statement delimiter. The statements\nwithin the loop are repeated until the loop is exited; usually this is\naccomplished with a LEAVE statement.\n\nA LOOP statement can be labeled. See [HELP BEGIN END] for the rules\nregarding label use.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/loop-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (342,4,'TRUNCATE','Syntax:\nTRUNCATE(X,D)\n\nReturns the number X, truncated to D decimal places. If D is 0, the\nresult has no decimal point or fractional part. D can be negative to\ncause D digits left of the decimal point of the value X to become zero.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT TRUNCATE(1.223,1);\n -> 1.2\nmysql> SELECT TRUNCATE(1.999,1);\n -> 1.9\nmysql> SELECT TRUNCATE(1.999,0);\n -> 1\nmysql> SELECT TRUNCATE(-1.999,1);\n -> -1.9\nmysql> SELECT TRUNCATE(122,-2);\n -> 100\nmysql> SELECT TRUNCATE(10.28*100,0);\n -> 1028\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (343,30,'TIMESTAMPADD','Syntax:\nTIMESTAMPADD(unit,interval,datetime_expr)\n\nAdds the integer expression interval to the date or datetime expression\ndatetime_expr. The unit for interval is given by the unit argument,\nwhich should be one of the following values: FRAC_SECOND\n(microseconds), SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER, or\nYEAR.\n\nBeginning with MySQL 5.1.24, it is possible to use MICROSECOND in place\nof FRAC_SECOND with this function, and FRAC_SECOND is deprecated.\n\nThe unit value may be specified using one of keywords as shown, or with\na prefix of SQL_TSI_. For example, DAY and SQL_TSI_DAY both are legal.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMESTAMPADD(MINUTE,1,\'2003-01-02\');\n -> \'2003-01-02 00:01:00\'\nmysql> SELECT TIMESTAMPADD(WEEK,1,\'2003-01-02\');\n -> \'2003-01-09\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (344,25,'SHOW','SHOW has many forms that provide information about databases, tables,\ncolumns, or status information about the server. This section describes\nthose following:\n\nSHOW AUTHORS\nSHOW CHARACTER SET [like_or_where]\nSHOW COLLATION [like_or_where]\nSHOW [FULL] COLUMNS FROM tbl_name [FROM db_name] [like_or_where]\nSHOW CONTRIBUTORS\nSHOW CREATE DATABASE db_name\nSHOW CREATE EVENT event_name\nSHOW CREATE FUNCTION func_name\nSHOW CREATE PROCEDURE proc_name\nSHOW CREATE TABLE tbl_name\nSHOW CREATE TRIGGER trigger_name\nSHOW CREATE VIEW view_name\nSHOW DATABASES [like_or_where]\nSHOW ENGINE engine_name {STATUS | MUTEX}\nSHOW [STORAGE] ENGINES\nSHOW ERRORS [LIMIT [offset,] row_count]\nSHOW EVENTS\nSHOW FUNCTION CODE func_name\nSHOW FUNCTION STATUS [like_or_where]\nSHOW GRANTS FOR user\nSHOW INDEX FROM tbl_name [FROM db_name]\nSHOW INNODB STATUS\nSHOW OPEN TABLES [FROM db_name] [like_or_where]\nSHOW PLUGINS\nSHOW PROCEDURE CODE proc_name\nSHOW PROCEDURE STATUS [like_or_where]\nSHOW PRIVILEGES\nSHOW [FULL] PROCESSLIST\nSHOW PROFILE [types] [FOR QUERY n] [OFFSET n] [LIMIT n]\nSHOW PROFILES\nSHOW SCHEDULER STATUS\nSHOW [GLOBAL | SESSION] STATUS [like_or_where]\nSHOW TABLE STATUS [FROM db_name] [like_or_where]\nSHOW TABLES [FROM db_name] [like_or_where]\nSHOW TRIGGERS [FROM db_name] [like_or_where]\nSHOW [GLOBAL | SESSION] VARIABLES [like_or_where]\nSHOW WARNINGS [LIMIT [offset,] row_count]\n\nlike_or_where:\n LIKE \'pattern\'\n | WHERE expr\n\nIf the syntax for a given SHOW statement includes a LIKE \'pattern\'\npart, \'pattern\' is a string that can contain the SQL "%" and "_"\nwildcard characters. The pattern is useful for restricting statement\noutput to matching values.\n\nSeveral SHOW statements also accept a WHERE clause that provides more\nflexibility in specifying which rows to display. See\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (343,30,'TIMESTAMPADD','Syntax:\nTIMESTAMPADD(unit,interval,datetime_expr)\n\nAdds the integer expression interval to the date or datetime expression\ndatetime_expr. The unit for interval is given by the unit argument,\nwhich should be one of the following values: FRAC_SECOND\n(microseconds), SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER, or\nYEAR.\n\nBeginning with MySQL 5.1.24, it is possible to use MICROSECOND in place\nof FRAC_SECOND with this function, and FRAC_SECOND is deprecated.\nFRAC_SECOND is removed in MySQL 5.5.\n\nThe unit value may be specified using one of keywords as shown, or with\na prefix of SQL_TSI_. For example, DAY and SQL_TSI_DAY both are legal.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT TIMESTAMPADD(MINUTE,1,\'2003-01-02\');\n -> \'2003-01-02 00:01:00\'\nmysql> SELECT TIMESTAMPADD(WEEK,1,\'2003-01-02\');\n -> \'2003-01-09\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (344,25,'SHOW','SHOW has many forms that provide information about databases, tables,\ncolumns, or status information about the server. This section describes\nthose following:\n\nSHOW AUTHORS\nSHOW CHARACTER SET [like_or_where]\nSHOW COLLATION [like_or_where]\nSHOW [FULL] COLUMNS FROM tbl_name [FROM db_name] [like_or_where]\nSHOW CONTRIBUTORS\nSHOW CREATE DATABASE db_name\nSHOW CREATE EVENT event_name\nSHOW CREATE FUNCTION func_name\nSHOW CREATE PROCEDURE proc_name\nSHOW CREATE TABLE tbl_name\nSHOW CREATE TRIGGER trigger_name\nSHOW CREATE VIEW view_name\nSHOW DATABASES [like_or_where]\nSHOW ENGINE engine_name {STATUS | MUTEX}\nSHOW [STORAGE] ENGINES\nSHOW ERRORS [LIMIT [offset,] row_count]\nSHOW EVENTS\nSHOW FUNCTION CODE func_name\nSHOW FUNCTION STATUS [like_or_where]\nSHOW GRANTS FOR user\nSHOW INDEX FROM tbl_name [FROM db_name]\nSHOW INNODB STATUS\nSHOW OPEN TABLES [FROM db_name] [like_or_where]\nSHOW PLUGINS\nSHOW PROCEDURE CODE proc_name\nSHOW PROCEDURE STATUS [like_or_where]\nSHOW PRIVILEGES\nSHOW [FULL] PROCESSLIST\nSHOW PROFILE [types] [FOR QUERY n] [OFFSET n] [LIMIT n]\nSHOW PROFILES\nSHOW SCHEDULER STATUS\nSHOW [GLOBAL | SESSION] STATUS [like_or_where]\nSHOW TABLE STATUS [FROM db_name] [like_or_where]\nSHOW [FULL] TABLES [FROM db_name] [like_or_where]\nSHOW TRIGGERS [FROM db_name] [like_or_where]\nSHOW [GLOBAL | SESSION] VARIABLES [like_or_where]\nSHOW WARNINGS [LIMIT [offset,] row_count]\n\nlike_or_where:\n LIKE \'pattern\'\n | WHERE expr\n\nIf the syntax for a given SHOW statement includes a LIKE \'pattern\'\npart, \'pattern\' is a string that can contain the SQL "%" and "_"\nwildcard characters. The pattern is useful for restricting statement\noutput to matching values.\n\nSeveral SHOW statements also accept a WHERE clause that provides more\nflexibility in specifying which rows to display. See\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (345,17,'GREATEST','Syntax:\nGREATEST(value1,value2,...)\n\nWith two or more arguments, returns the largest (maximum-valued)\nargument. The arguments are compared using the same rules as for\nLEAST().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT GREATEST(2,0);\n -> 2\nmysql> SELECT GREATEST(34.0,3.0,5.0,767.0);\n -> 767.0\nmysql> SELECT GREATEST(\'B\',\'A\',\'C\');\n -> \'C\'\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (346,25,'SHOW VARIABLES','Syntax:\nSHOW [GLOBAL | SESSION] VARIABLES\n [LIKE \'pattern\' | WHERE expr]\n\nSHOW VARIABLES shows the values of MySQL system variables. This\ninformation also can be obtained using the mysqladmin variables\ncommand. The LIKE clause, if present, indicates which variable names to\nmatch. The WHERE clause can be given to select rows using more general\nconditions, as discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html. This\nstatement does not require any privilege. It requires only the ability\nto connect to the server.\n\nWith the GLOBAL modifier, SHOW VARIABLES displays the values that are\nused for new connections to MySQL. With SESSION, it displays the values\nthat are in effect for the current connection. If no modifier is\npresent, the default is SESSION. LOCAL is a synonym for SESSION.\nWith a LIKE clause, the statement displays only rows for those\nvariables with names that match the pattern. To obtain the row for a\nspecific variable, use a LIKE clause as shown:\n\nSHOW VARIABLES LIKE \'max_join_size\';\nSHOW SESSION VARIABLES LIKE \'max_join_size\';\n\nTo get a list of variables whose name match a pattern, use the "%"\nwildcard character in a LIKE clause:\n\nSHOW VARIABLES LIKE \'%size%\';\nSHOW GLOBAL VARIABLES LIKE \'%size%\';\n\nWildcard characters can be used in any position within the pattern to\nbe matched. Strictly speaking, because "_" is a wildcard that matches\nany single character, you should escape it as "\\_" to match it\nliterally. In practice, this is rarely necessary.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-variables.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-variables.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (347,25,'BINLOG','Syntax:\nBINLOG \'str\'\n\nBINLOG is an internal-use statement. It is generated by the mysqlbinlog\nprogram as the printable representation of certain events in binary log\nfiles. (See http://dev.mysql.com/doc/refman/5.1/en/mysqlbinlog.html.)\nThe \'str\' value is a base 64-encoded string the that server decodes to\ndetermine the data change indicated by the corresponding event. This\nstatement requires the SUPER privilege. It was added in MySQL 5.1.5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/binlog.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/binlog.html'); @@ -422,15 +422,15 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (353,26,'SELECT','Syntax:\nSELECT\n [ALL | DISTINCT | DISTINCTROW ]\n [HIGH_PRIORITY]\n [STRAIGHT_JOIN]\n [SQL_SMALL_RESULT] [SQL_BIG_RESULT] [SQL_BUFFER_RESULT]\n [SQL_CACHE | SQL_NO_CACHE] [SQL_CALC_FOUND_ROWS]\n select_expr [, select_expr ...]\n [FROM table_references\n [WHERE where_condition]\n [GROUP BY {col_name | expr | position}\n [ASC | DESC], ... [WITH ROLLUP]]\n [HAVING where_condition]\n [ORDER BY {col_name | expr | position}\n [ASC | DESC], ...]\n [LIMIT {[offset,] row_count | row_count OFFSET offset}]\n [PROCEDURE procedure_name(argument_list)]\n [INTO OUTFILE \'file_name\'\n [CHARACTER SET charset_name]\n export_options\n | INTO DUMPFILE \'file_name\'\n | INTO var_name [, var_name]]\n [FOR UPDATE | LOCK IN SHARE MODE]]\n\nSELECT is used to retrieve rows selected from one or more tables, and\ncan include UNION statements and subqueries. See [HELP UNION], and\nhttp://dev.mysql.com/doc/refman/5.1/en/subqueries.html.\n\nThe most commonly used clauses of SELECT statements are these:\n\no Each select_expr indicates a column that you want to retrieve. There\n must be at least one select_expr.\n\no table_references indicates the table or tables from which to retrieve\n rows. Its syntax is described in [HELP JOIN].\n\no The WHERE clause, if given, indicates the condition or conditions\n that rows must satisfy to be selected. where_condition is an\n expression that evaluates to true for each row to be selected. The\n statement selects all rows if there is no WHERE clause.\n\n In the WHERE clause, you can use any of the functions and operators\n that MySQL supports, except for aggregate (summary) functions. See\n http://dev.mysql.com/doc/refman/5.1/en/functions.html.\n\nSELECT can also be used to retrieve rows computed without reference to\nany table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/select.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/select.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (354,4,'COT','Syntax:\nCOT(X)\n\nReturns the cotangent of X.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT COT(12);\n -> -1.5726734063977\nmysql> SELECT COT(0);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (355,25,'SHOW CREATE EVENT','Syntax:\nSHOW CREATE EVENT event_name\n\nThis statement displays the CREATE EVENT statement needed to re-create\na given event. For example (using the same event e_daily defined and\nthen altered in [HELP SHOW EVENTS]):\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-create-event.html\n\n','mysql> SHOW CREATE EVENT test.e_daily\\G\n*************************** 1. row ***************************\n Event: e_daily\n sql_mode:\n time_zone: SYSTEM\n Create Event: CREATE EVENT `e_daily`\n ON SCHEDULE EVERY 1 DAY\n STARTS CURRENT_TIMESTAMP + INTERVAL 6 HOUR\n ON COMPLETION NOT PRESERVE\n ENABLE\n COMMENT \'Saves total number of sessions then\n clears the table each day\'\n DO BEGIN\n INSERT INTO site_activity.totals (time, total)\n SELECT CURRENT_TIMESTAMP, COUNT(*)\n FROM site_activity.sessions;\n DELETE FROM site_activity.sessions;\n END\ncharacter_set_client: latin1\ncollation_connection: latin1_swedish_ci\n Database Collation: latin1_swedish_ci\n','http://dev.mysql.com/doc/refman/5.1/en/show-create-event.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (356,19,'BACKUP TABLE','Syntax:\nBACKUP TABLE tbl_name [, tbl_name] ... TO \'/path/to/backup/directory\'\n\n*Note*: This statement is deprecated. We are working on a better\nreplacement for it that will provide online backup capabilities. In the\nmeantime, the mysqlhotcopy script can be used instead.\n\nBACKUP TABLE copies to the backup directory the minimum number of table\nfiles needed to restore the table, after flushing any buffered changes\nto disk. The statement works only for MyISAM tables. It copies the .frm\ndefinition and .MYD data files. The .MYI index file can be rebuilt from\nthose two files. The directory should be specified as a full path name.\nTo restore the table, use RESTORE TABLE.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/backup-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/backup-table.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (356,19,'BACKUP TABLE','Syntax:\nBACKUP TABLE tbl_name [, tbl_name] ... TO \'/path/to/backup/directory\'\n\n*Note*: This statement is deprecated and is removed in MySQL 5.5. As an\nalternative, mysqldump or mysqlhotcopy can be used instead.\n\nBACKUP TABLE copies to the backup directory the minimum number of table\nfiles needed to restore the table, after flushing any buffered changes\nto disk. The statement works only for MyISAM tables. It copies the .frm\ndefinition and .MYD data files. The .MYI index file can be rebuilt from\nthose two files. The directory should be specified as a full path name.\nTo restore the table, use RESTORE TABLE.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/backup-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/backup-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (357,35,'LOAD_FILE','Syntax:\nLOAD_FILE(file_name)\n\nReads the file and returns the file contents as a string. To use this\nfunction, the file must be located on the server host, you must specify\nthe full path name to the file, and you must have the FILE privilege.\nThe file must be readable by all and its size less than\nmax_allowed_packet bytes. If the secure_file_priv system variable is\nset to a nonempty directory name, the file to be loaded must be located\nin that directory.\n\nIf the file does not exist or cannot be read because one of the\npreceding conditions is not satisfied, the function returns NULL.\n\nAs of MySQL 5.1.6, the character_set_filesystem system variable\ncontrols interpretation of file names that are given as literal\nstrings.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> UPDATE t\n SET blob_col=LOAD_FILE(\'/tmp/picture\')\n WHERE id=1;\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (358,25,'LOAD TABLE FROM MASTER','Syntax:\nLOAD TABLE tbl_name FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated in\nversions 4.1 of MySQL and above. We will introduce a more advanced\ntechnique (called "online backup") in a future version. That technique\nwill have the additional advantage of working with more storage\nengines.\n\nFor MySQL 5.1 and earlier, the recommended alternative solution to\nusing LOAD DATA FROM MASTER or LOAD TABLE FROM MASTER is using\nmysqldump or mysqlhotcopy. The latter requires Perl and two Perl\nmodules (DBI and DBD:mysql) and works for MyISAM and ARCHIVE tables\nonly. With mysqldump, you can create SQL dumps on the master and pipe\n(or copy) these to a mysql client on the slave. This has the advantage\nof working for all storage engines, but can be quite slow, since it\nworks using SELECT.\n\nTransfers a copy of the table from the master to the slave. This\nstatement is implemented mainly debugging LOAD DATA FROM MASTER\noperations. To use LOAD TABLE, the account used for connecting to the\nmaster server must have the RELOAD and SUPER privileges on the master\nand the SELECT privilege for the master table to load. On the slave\nside, the user that issues LOAD TABLE FROM MASTER must have privileges\nfor dropping and creating the table.\n\nThe conditions for LOAD DATA FROM MASTER apply here as well. For\nexample, LOAD TABLE FROM MASTER works only for MyISAM tables. The\ntimeout notes for LOAD DATA FROM MASTER apply as well.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (358,25,'LOAD TABLE FROM MASTER','Syntax:\nLOAD TABLE tbl_name FROM MASTER\n\nThis feature is deprecated and should be avoided. It is subject to\nremoval in a future version of MySQL.\n\nSince the current implementation of LOAD DATA FROM MASTER and LOAD\nTABLE FROM MASTER is very limited, these statements are deprecated as\nof MySQL 4.1 and removed in MySQL 5.5.\n\nThe recommended alternative solution to using LOAD DATA FROM MASTER or\nLOAD TABLE FROM MASTER is using mysqldump or mysqlhotcopy. The latter\nrequires Perl and two Perl modules (DBI and DBD:mysql) and works for\nMyISAM and ARCHIVE tables only. With mysqldump, you can create SQL\ndumps on the master and pipe (or copy) these to a mysql client on the\nslave. This has the advantage of working for all storage engines, but\ncan be quite slow, since it works using SELECT.\n\nTransfers a copy of the table from the master to the slave. This\nstatement is implemented mainly debugging LOAD DATA FROM MASTER\noperations. To use LOAD TABLE, the account used for connecting to the\nmaster server must have the RELOAD and SUPER privileges on the master\nand the SELECT privilege for the master table to load. On the slave\nside, the user that issues LOAD TABLE FROM MASTER must have privileges\nfor dropping and creating the table.\n\nThe conditions for LOAD DATA FROM MASTER apply here as well. For\nexample, LOAD TABLE FROM MASTER works only for MyISAM tables. The\ntimeout notes for LOAD DATA FROM MASTER apply as well.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-table-from-master.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (359,3,'POINTFROMTEXT','PointFromText(wkt[,srid])\n\nConstructs a POINT value using its WKT representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (360,16,'GROUP_CONCAT','Syntax:\nGROUP_CONCAT(expr)\n\nThis function returns a string result with the concatenated non-NULL\nvalues from a group. It returns NULL if there are no non-NULL values.\nThe full syntax is as follows:\n\nGROUP_CONCAT([DISTINCT] expr [,expr ...]\n [ORDER BY {unsigned_integer | col_name | expr}\n [ASC | DESC] [,col_name ...]]\n [SEPARATOR str_val])\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html\n\n','mysql> SELECT student_name,\n -> GROUP_CONCAT(test_score)\n -> FROM student\n -> GROUP BY student_name;\n','http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (361,30,'DATE_FORMAT','Syntax:\nDATE_FORMAT(date,format)\n\nFormats the date value according to the format string.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DATE_FORMAT(\'2009-10-04 22:23:00\', \'%W %M %Y\');\n -> \'Sunday October 2009\'\nmysql> SELECT DATE_FORMAT(\'2007-10-04 22:23:00\', \'%H:%i:%s\');\n -> \'22:23:00\'\nmysql> SELECT DATE_FORMAT(\'1900-10-04 22:23:00\',\n -> \'%D %y %a %d %m %b %j\');\n -> \'4th 00 Thu 04 10 Oct 277\'\nmysql> SELECT DATE_FORMAT(\'1997-10-04 22:23:00\',\n -> \'%H %k %I %r %T %S %w\');\n -> \'22 22 10 10:23:00 PM 22:23:00 00 6\'\nmysql> SELECT DATE_FORMAT(\'1999-01-01\', \'%X %V\');\n -> \'1998 52\'\nmysql> SELECT DATE_FORMAT(\'2006-06-00\', \'%d\');\n -> \'00\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (362,15,'BENCHMARK','Syntax:\nBENCHMARK(count,expr)\n\nThe BENCHMARK() function executes the expression expr repeatedly count\ntimes. It may be used to time how quickly MySQL processes the\nexpression. The result value is always 0. The intended use is from\nwithin the mysql client, which reports query execution times:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT BENCHMARK(1000000,ENCODE(\'hello\',\'goodbye\'));\n+----------------------------------------------+\n| BENCHMARK(1000000,ENCODE(\'hello\',\'goodbye\')) |\n+----------------------------------------------+\n| 0 |\n+----------------------------------------------+\n1 row in set (4.74 sec)\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (363,30,'YEAR','Syntax:\nYEAR(date)\n\nReturns the year for date, in the range 1000 to 9999, or 0 for the\n"zero" date.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT YEAR(\'1987-01-01\');\n -> 1987\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (364,25,'SHOW ENGINE','Syntax:\nSHOW ENGINE engine_name {STATUS | MUTEX}\n\nSHOW ENGINE displays operational information about a storage engine.\nThe following statements currently are supported:\n\nSHOW ENGINE INNODB STATUS\nSHOW ENGINE INNODB MUTEX\nSHOW ENGINE {NDB | NDBCLUSTER} STATUS\n\nOlder (and now deprecated) synonyms are SHOW INNODB STATUS for SHOW\nENGINE INNODB STATUS and SHOW MUTEX STATUS for SHOW ENGINE INNODB\nMUTEX.\n\nIn MySQL 5.0, SHOW ENGINE INNODB MUTEX is invoked as SHOW MUTEX STATUS.\nThe latter statement displays similar information but in a somewhat\ndifferent output format.\n\nSHOW ENGINE BDB LOGS formerly displayed status information about BDB\nlog files. As of MySQL 5.1.12, the BDB storage engine is not supported,\nand this statement produces a warning.\n\nSHOW ENGINE INNODB STATUS displays extensive information from the\nstandard InnoDB Monitor about the state of the InnoDB storage engine.\nFor information about the standard monitor and other InnoDB Monitors\nthat provide information about InnoDB processing, see\nhttp://dev.mysql.com/doc/refman/5.1/en/innodb-monitors.html.\n\nSHOW ENGINE INNODB MUTEX displays InnoDB mutex statistics. From MySQL\n5.1.2 to 5.1.14, the statement displays the following output fields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The mutex name and the source file where it is implemented. Example:\n &pool->mutex:mem0pool.c\n\n The mutex name indicates its purpose. For example, the log_sys mutex\n is used by the InnoDB logging subsystem and indicates how intensive\n logging activity is. The buf_pool mutex protects the InnoDB buffer\n pool.\n\no Status\n\n The mutex status. The fields contains several values:\n\n o count indicates how many times the mutex was requested.\n\n o spin_waits indicates how many times the spinlock had to run.\n\n o spin_rounds indicates the number of spinlock rounds. (spin_rounds\n divided by spin_waits provides the average round count.)\n\n o os_waits indicates the number of operating system waits. This\n occurs when the spinlock did not work (the mutex was not locked\n during the spinlock and it was necessary to yield to the operating\n system and wait).\n\n o os_yields indicates the number of times a the thread trying to lock\n a mutex gave up its timeslice and yielded to the operating system\n (on the presumption that allowing other threads to run will free\n the mutex so that it can be locked).\n\n o os_wait_times indicates the amount of time (in ms) spent in\n operating system waits, if the timed_mutexes system variable is 1\n (ON). If timed_mutexes is 0 (OFF), timing is disabled, so\n os_wait_times is 0. timed_mutexes is off by default.\n\nFrom MySQL 5.1.15 on, the statement displays the following output\nfields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The source file where the mutex is implemented, and the line number\n in the file where the mutex is created. The line number may change\n depending on your version of MySQL.\n\no Status\n\n This field displays the same values as previously described (count,\n spin_waits, spin_rounds, os_waits, os_yields, os_wait_times), but\n only if UNIV_DEBUG was defined at MySQL compilation time (for\n example, in include/univ.h in the InnoDB part of the MySQL source\n tree). If UNIV_DEBUG was not defined, the statement displays only the\n os_waits value. In the latter case (without UNIV_DEBUG), the\n information on which the output is based is insufficient to\n distinguish regular mutexes and mutexes that protect rw-locks (which\n allow multiple readers or a single writer). Consequently, the output\n may appear to contain multiple rows for the same mutex.\n\nInformation from this statement can be used to diagnose system\nproblems. For example, large values of spin_waits and spin_rounds may\nindicate scalability problems.\n\nIf the server has the NDBCLUSTER storage engine enabled, SHOW ENGINE\nNDB STATUS displays cluster status information such as the number of\nconnected data nodes, the cluster connectstring, and cluster binlog\nepochs, as well as counts of various Cluster API objects created by the\nMySQL Server when connected to the cluster. Sample output from this\nstatement is shown here:\n\nmysql> SHOW ENGINE NDB STATUS;\n+------------+-----------------------+--------------------------------------------------+\n| Type | Name | Status |\n+------------+-----------------------+--------------------------------------------------+\n| ndbcluster | connection | cluster_node_id=7,\n connected_host=192.168.0.103, connected_port=1186, number_of_data_nodes=4,\n number_of_ready_data_nodes=3, connect_count=0 |\n| ndbcluster | NdbTransaction | created=6, free=0, sizeof=212 |\n| ndbcluster | NdbOperation | created=8, free=8, sizeof=660 |\n| ndbcluster | NdbIndexScanOperation | created=1, free=1, sizeof=744 |\n| ndbcluster | NdbIndexOperation | created=0, free=0, sizeof=664 |\n| ndbcluster | NdbRecAttr | created=1285, free=1285, sizeof=60 |\n| ndbcluster | NdbApiSignal | created=16, free=16, sizeof=136 |\n| ndbcluster | NdbLabel | created=0, free=0, sizeof=196 |\n| ndbcluster | NdbBranch | created=0, free=0, sizeof=24 |\n| ndbcluster | NdbSubroutine | created=0, free=0, sizeof=68 |\n| ndbcluster | NdbCall | created=0, free=0, sizeof=16 |\n| ndbcluster | NdbBlob | created=1, free=1, sizeof=264 |\n| ndbcluster | NdbReceiver | created=4, free=0, sizeof=68 |\n| ndbcluster | binlog | latest_epoch=155467, latest_trans_epoch=148126,\n latest_received_binlog_epoch=0, latest_handled_binlog_epoch=0,\n latest_applied_binlog_epoch=0 |\n+------------+-----------------------+--------------------------------------------------+\n\nThe rows with connection and binlog in the Name column were added to\nthe output of this statement in MySQL 5.1. The Status column in each of\nthese rows provides information about the MySQL server\'s connection to\nthe cluster and about the cluster binary log\'s status, respectively.\nThe Status information is in the form of comma-delimited set of\nname/value pairs.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engine.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engine.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (364,25,'SHOW ENGINE','Syntax:\nSHOW ENGINE engine_name {STATUS | MUTEX}\n\nSHOW ENGINE displays operational information about a storage engine.\nThe following statements currently are supported:\n\nSHOW ENGINE INNODB STATUS\nSHOW ENGINE INNODB MUTEX\nSHOW ENGINE {NDB | NDBCLUSTER} STATUS\n\nOlder (and now deprecated) synonyms are SHOW INNODB STATUS for SHOW\nENGINE INNODB STATUS and SHOW MUTEX STATUS for SHOW ENGINE INNODB\nMUTEX. SHOW INNODB STATUS and SHOW MUTEX STATUS are removed in MySQL\n5.5.\n\nIn MySQL 5.0, SHOW ENGINE INNODB MUTEX is invoked as SHOW MUTEX STATUS.\nThe latter statement displays similar information but in a somewhat\ndifferent output format.\n\nSHOW ENGINE BDB LOGS formerly displayed status information about BDB\nlog files. As of MySQL 5.1.12, the BDB storage engine is not supported,\nand this statement produces a warning.\n\nSHOW ENGINE INNODB STATUS displays extensive information from the\nstandard InnoDB Monitor about the state of the InnoDB storage engine.\nFor information about the standard monitor and other InnoDB Monitors\nthat provide information about InnoDB processing, see\nhttp://dev.mysql.com/doc/refman/5.1/en/innodb-monitors.html.\n\nSHOW ENGINE INNODB MUTEX displays InnoDB mutex statistics. From MySQL\n5.1.2 to 5.1.14, the statement displays the following output fields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The mutex name and the source file where it is implemented. Example:\n &pool->mutex:mem0pool.c\n\n The mutex name indicates its purpose. For example, the log_sys mutex\n is used by the InnoDB logging subsystem and indicates how intensive\n logging activity is. The buf_pool mutex protects the InnoDB buffer\n pool.\n\no Status\n\n The mutex status. The fields contains several values:\n\n o count indicates how many times the mutex was requested.\n\n o spin_waits indicates how many times the spinlock had to run.\n\n o spin_rounds indicates the number of spinlock rounds. (spin_rounds\n divided by spin_waits provides the average round count.)\n\n o os_waits indicates the number of operating system waits. This\n occurs when the spinlock did not work (the mutex was not locked\n during the spinlock and it was necessary to yield to the operating\n system and wait).\n\n o os_yields indicates the number of times a the thread trying to lock\n a mutex gave up its timeslice and yielded to the operating system\n (on the presumption that allowing other threads to run will free\n the mutex so that it can be locked).\n\n o os_wait_times indicates the amount of time (in ms) spent in\n operating system waits, if the timed_mutexes system variable is 1\n (ON). If timed_mutexes is 0 (OFF), timing is disabled, so\n os_wait_times is 0. timed_mutexes is off by default.\n\nFrom MySQL 5.1.15 on, the statement displays the following output\nfields:\n\no Type\n\n Always InnoDB.\n\no Name\n\n The source file where the mutex is implemented, and the line number\n in the file where the mutex is created. The line number may change\n depending on your version of MySQL.\n\no Status\n\n This field displays the same values as previously described (count,\n spin_waits, spin_rounds, os_waits, os_yields, os_wait_times), but\n only if UNIV_DEBUG was defined at MySQL compilation time (for\n example, in include/univ.h in the InnoDB part of the MySQL source\n tree). If UNIV_DEBUG was not defined, the statement displays only the\n os_waits value. In the latter case (without UNIV_DEBUG), the\n information on which the output is based is insufficient to\n distinguish regular mutexes and mutexes that protect rw-locks (which\n allow multiple readers or a single writer). Consequently, the output\n may appear to contain multiple rows for the same mutex.\n\nInformation from this statement can be used to diagnose system\nproblems. For example, large values of spin_waits and spin_rounds may\nindicate scalability problems.\n\nIf the server has the NDBCLUSTER storage engine enabled, SHOW ENGINE\nNDB STATUS displays cluster status information such as the number of\nconnected data nodes, the cluster connectstring, and cluster binlog\nepochs, as well as counts of various Cluster API objects created by the\nMySQL Server when connected to the cluster. Sample output from this\nstatement is shown here:\n\nmysql> SHOW ENGINE NDB STATUS;\n+------------+-----------------------+--------------------------------------------------+\n| Type | Name | Status |\n+------------+-----------------------+--------------------------------------------------+\n| ndbcluster | connection | cluster_node_id=7,\n connected_host=192.168.0.103, connected_port=1186, number_of_data_nodes=4,\n number_of_ready_data_nodes=3, connect_count=0 |\n| ndbcluster | NdbTransaction | created=6, free=0, sizeof=212 |\n| ndbcluster | NdbOperation | created=8, free=8, sizeof=660 |\n| ndbcluster | NdbIndexScanOperation | created=1, free=1, sizeof=744 |\n| ndbcluster | NdbIndexOperation | created=0, free=0, sizeof=664 |\n| ndbcluster | NdbRecAttr | created=1285, free=1285, sizeof=60 |\n| ndbcluster | NdbApiSignal | created=16, free=16, sizeof=136 |\n| ndbcluster | NdbLabel | created=0, free=0, sizeof=196 |\n| ndbcluster | NdbBranch | created=0, free=0, sizeof=24 |\n| ndbcluster | NdbSubroutine | created=0, free=0, sizeof=68 |\n| ndbcluster | NdbCall | created=0, free=0, sizeof=16 |\n| ndbcluster | NdbBlob | created=1, free=1, sizeof=264 |\n| ndbcluster | NdbReceiver | created=4, free=0, sizeof=68 |\n| ndbcluster | binlog | latest_epoch=155467, latest_trans_epoch=148126,\n latest_received_binlog_epoch=0, latest_handled_binlog_epoch=0,\n latest_applied_binlog_epoch=0 |\n+------------+-----------------------+--------------------------------------------------+\n\nThe rows with connection and binlog in the Name column were added to\nthe output of this statement in MySQL 5.1. The Status column in each of\nthese rows provides information about the MySQL server\'s connection to\nthe cluster and about the cluster binary log\'s status, respectively.\nThe Status information is in the form of comma-delimited set of\nname/value pairs.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-engine.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-engine.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (365,14,'NAME_CONST','Syntax:\nNAME_CONST(name,value)\n\nReturns the given value. When used to produce a result set column,\nNAME_CONST() causes the column to have the given name. The arguments\nshould be constants.\n\nmysql> SELECT NAME_CONST(\'myname\', 14);\n+--------+\n| myname |\n+--------+\n| 14 |\n+--------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (366,14,'RELEASE_LOCK','Syntax:\nRELEASE_LOCK(str)\n\nReleases the lock named by the string str that was obtained with\nGET_LOCK(). Returns 1 if the lock was released, 0 if the lock was not\nestablished by this thread (in which case the lock is not released),\nand NULL if the named lock did not exist. The lock does not exist if it\nwas never obtained by a call to GET_LOCK() or if it has previously been\nreleased.\n\nThe DO statement is convenient to use with RELEASE_LOCK(). See [HELP\nDO].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (367,17,'IS NULL','Syntax:\nIS NULL\n\nTests whether a value is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT 1 IS NULL, 0 IS NULL, NULL IS NULL;\n -> 0, 0, 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); @@ -465,7 +465,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (396,35,'LENGTH','Syntax:\nLENGTH(str)\n\nReturns the length of the string str, measured in bytes. A multi-byte\ncharacter counts as multiple bytes. This means that for a string\ncontaining five two-byte characters, LENGTH() returns 10, whereas\nCHAR_LENGTH() returns 5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT LENGTH(\'text\');\n -> 4\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (397,30,'STR_TO_DATE','Syntax:\nSTR_TO_DATE(str,format)\n\nThis is the inverse of the DATE_FORMAT() function. It takes a string\nstr and a format string format. STR_TO_DATE() returns a DATETIME value\nif the format string contains both date and time parts, or a DATE or\nTIME value if the string contains only date or time parts. If the date,\ntime, or datetime value extracted from str is illegal, STR_TO_DATE()\nreturns NULL and produces a warning.\n\nThe server scans str attempting to match format to it. The format\nstring can contain literal characters and format specifiers beginning\nwith %. Literal characters in format must match literally in str.\nFormat specifiers in format must match a date or time part in str. For\nthe specifiers that can be used in format, see the DATE_FORMAT()\nfunction description.\n\nmysql> SELECT STR_TO_DATE(\'01,5,2013\',\'%d,%m,%Y\');\n -> \'2013-05-01\'\nmysql> SELECT STR_TO_DATE(\'May 1, 2013\',\'%M %d,%Y\');\n -> \'2013-05-01\'\n\nScanning starts at the beginning of str and fails if format is found\nnot to match. Extra characters at the end of str are ignored.\n\nmysql> SELECT STR_TO_DATE(\'a09:30:17\',\'a%h:%i:%s\');\n -> \'09:30:17\'\nmysql> SELECT STR_TO_DATE(\'a09:30:17\',\'%h:%i:%s\');\n -> NULL\nmysql> SELECT STR_TO_DATE(\'09:30:17a\',\'%h:%i:%s\');\n -> \'09:30:17\'\n\nUnspecified date or time parts have a value of 0, so incompletely\nspecified values in str produce a result with some or all parts set to\n0:\n\nmysql> SELECT STR_TO_DATE(\'abc\',\'abc\');\n -> \'0000-00-00\'\nmysql> SELECT STR_TO_DATE(\'9\',\'%m\');\n -> \'0000-09-00\'\nmysql> SELECT STR_TO_DATE(\'9\',\'%s\');\n -> \'00:00:09\'\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (398,10,'Y','Y(p)\n\nReturns the Y-coordinate value for the point p as a double-precision\nnumber.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#poi…','mysql> SET @pt = \'Point(56.7 53.34)\';\nmysql> SELECT Y(GeomFromText(@pt));\n+----------------------+\n| Y(GeomFromText(@pt)) |\n+----------------------+\n| 53.34 |\n+----------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#poi…'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (399,25,'SHOW INNODB STATUS','Syntax:\nSHOW INNODB STATUS\n\nIn MySQL 5.1, this is a deprecated synonym for SHOW ENGINE INNODB\nSTATUS. See [HELP SHOW ENGINE].\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (399,25,'SHOW INNODB STATUS','Syntax:\nSHOW INNODB STATUS\n\nIn MySQL 5.1, this is a deprecated synonym for SHOW ENGINE INNODB\nSTATUS. See [HELP SHOW ENGINE]. SHOW INNODB STATUS is removed in MySQL\n5.5.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-innodb-status.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (400,19,'CHECKSUM TABLE','Syntax:\nCHECKSUM TABLE tbl_name [, tbl_name] ... [ QUICK | EXTENDED ]\n\nCHECKSUM TABLE reports a table checksum.\n\nWith QUICK, the live table checksum is reported if it is available, or\nNULL otherwise. This is very fast. A live checksum is enabled by\nspecifying the CHECKSUM=1 table option when you create the table;\ncurrently, this is supported only for MyISAM tables. See [HELP CREATE\nTABLE].\n\nWith EXTENDED, the entire table is read row by row and the checksum is\ncalculated. This can be very slow for large tables.\n\nIf neither QUICK nor EXTENDED is specified, MySQL returns a live\nchecksum if the table storage engine supports it and scans the table\notherwise.\n\nFor a nonexistent table, CHECKSUM TABLE returns NULL and generates a\nwarning.\n\nThe checksum value depends on the table row format. If the row format\nchanges, the checksum also changes. For example, the storage format for\nVARCHAR changed between MySQL 4.1 and 5.0, so if a 4.1 table is\nupgraded to MySQL 5.0, the checksum value may change.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/checksum-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/checksum-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (401,2,'NUMINTERIORRINGS','NumInteriorRings(poly)\n\nReturns the number of interior rings in the Polygon value poly.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…','mysql> SET @poly =\n -> \'Polygon((0 0,0 3,3 3,3 0,0 0),(1 1,1 2,2 2,2 1,1 1))\';\nmysql> SELECT NumInteriorRings(GeomFromText(@poly));\n+---------------------------------------+\n| NumInteriorRings(GeomFromText(@poly)) |\n+---------------------------------------+\n| 1 |\n+---------------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (402,2,'INTERIORRINGN','InteriorRingN(poly,N)\n\nReturns the N-th interior ring for the Polygon value poly as a\nLineString. Rings are numbered beginning with 1.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…','mysql> SET @poly =\n -> \'Polygon((0 0,0 3,3 3,3 0,0 0),(1 1,1 2,2 2,2 1,1 1))\';\nmysql> SELECT AsText(InteriorRingN(GeomFromText(@poly),1));\n+----------------------------------------------+\n| AsText(InteriorRingN(GeomFromText(@poly),1)) |\n+----------------------------------------------+\n| LINESTRING(1 1,1 2,2 2,2 1,1 1) |\n+----------------------------------------------+\n','http://dev.mysql.com/doc/refman/5.1/en/geometry-property-functions.html#pol…'); @@ -481,12 +481,12 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (412,35,'RIGHT','Syntax:\nRIGHT(str,len)\n\nReturns the rightmost len characters from the string str, or NULL if\nany argument is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT RIGHT(\'foobarbar\', 4);\n -> \'rbar\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (413,30,'DATEDIFF','Syntax:\nDATEDIFF(expr1,expr2)\n\nDATEDIFF() returns expr1 - expr2 expressed as a value in days from one\ndate to the other. expr1 and expr2 are date or date-and-time\nexpressions. Only the date parts of the values are used in the\ncalculation.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DATEDIFF(\'2007-12-31 23:59:59\',\'2007-12-30\');\n -> 1\nmysql> SELECT DATEDIFF(\'2010-11-30 23:59:59\',\'2010-12-31\');\n -> -31\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (414,37,'DROP TABLESPACE','Syntax:\nDROP TABLESPACE tablespace_name\n ENGINE [=] engine_name\n\nThis statement drops a tablespace that was previously created using\nCREATE TABLESPACE (see [HELP CREATE TABLESPACE]).\n\n*Important*: The tablespace to be dropped must not contain any data\nfiles; in other words, before you can drop a tablespace, you must first\ndrop each of its data files using ALTER TABLESPACE ... DROP DATAFILE\n(see [HELP ALTER TABLESPACE]).\n\nThe ENGINE clause (required) specifies the storage engine used by the\ntablespace. In MySQL 5.1, the only accepted values for engine_name are\nNDB and NDBCLUSTER.\n\nDROP TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/drop-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-tablespace.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (415,37,'DROP PROCEDURE','Syntax:\nDROP {PROCEDURE | FUNCTION} [IF EXISTS] sp_name\n\nThis statement is used to drop a stored procedure or function. That is,\nthe specified routine is removed from the server. You must have the\nALTER ROUTINE privilege for the routine. (That privilege is granted\nautomatically to the routine creator.)\n\nThe IF EXISTS clause is a MySQL extension. It prevents an error from\noccurring if the procedure or function does not exist. A warning is\nproduced that can be viewed with SHOW WARNINGS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (415,37,'DROP PROCEDURE','Syntax:\nDROP {PROCEDURE | FUNCTION} [IF EXISTS] sp_name\n\nThis statement is used to drop a stored procedure or function. That is,\nthe specified routine is removed from the server. You must have the\nALTER ROUTINE privilege for the routine. (If the\nautomatic_sp_privileges system variable is enabled, that privilege and\nEXECUTE are granted automatically to the routine creator when the\nroutine is created and dropped from the creator when the routine is\ndropped. See\nhttp://dev.mysql.com/doc/refman/5.1/en/stored-routines-privileges.html… IF EXISTS clause is a MySQL extension. It prevents an error from\noccurring if the procedure or function does not exist. A warning is\nproduced that can be viewed with SHOW WARNINGS.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/drop-procedure.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (416,19,'CHECK TABLE','Syntax:\nCHECK TABLE tbl_name [, tbl_name] ... [option] ...\n\noption = {FOR UPGRADE | QUICK | FAST | MEDIUM | EXTENDED | CHANGED}\n\nCHECK TABLE checks a table or tables for errors. CHECK TABLE works for\nMyISAM, InnoDB, and ARCHIVE tables. Starting with MySQL 5.1.9, CHECK\nTABLE is also valid for CSV tables, see\nhttp://dev.mysql.com/doc/refman/5.1/en/csv-storage-engine.html. For\nMyISAM tables, the key statistics are updated as well.\n\nCHECK TABLE can also check views for problems, such as tables that are\nreferenced in the view definition that no longer exist.\n\nBeginning with MySQL 5.1.27, CHECK TABLE is also supported for\npartitioned tables. Also beginning with MySQL 5.1.27, you can use ALTER\nTABLE ... CHECK PARTITION to check one or more partitions; for more\ninformation, see [HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/check-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/check-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (417,35,'BIN','Syntax:\nBIN(N)\n\nReturns a string representation of the binary value of N, where N is a\nlonglong (BIGINT) number. This is equivalent to CONV(N,10,2). Returns\nNULL if N is NULL.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT BIN(12);\n -> \'1100\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (418,5,'INSTALL PLUGIN','Syntax:\nINSTALL PLUGIN plugin_name SONAME \'plugin_library\'\n\nThis statement installs a plugin.\n\nplugin_name is the name of the plugin as defined in the plugin\ndeclaration structure contained in the library file. Plugin names are\nnot case sensitive. For maximal compatibility, plugin names should be\nlimited to ASCII letters, digits, and underscore, because they are used\nin C source files, shell command lines, M4 and Bourne shell scripts,\nand SQL environments.\n\nplugin_library is the name of the shared library that contains the\nplugin code. The name includes the file name extension (for example,\nlibmyplugin.so or libmyplugin.dylib).\n\nThe shared library must be located in the plugin directory (that is,\nthe directory named by the plugin_dir system variable). The library\nmust be in the plugin directory itself, not in a subdirectory. By\ndefault, plugin_dir is plugin directory under the directory named by\nthe pkglibdir configuration variable, but it can be changed by setting\nthe value of plugin_dir at server startup. For example, set its value\nin a my.cnf file:\n\n[mysqld]\nplugin_dir=/path/to/plugin/directory\n\nIf the value of plugin_dir is a relative path name, it is taken to be\nrelative to the MySQL base directory (the value of the basedir system\nvariable).\n\nINSTALL PLUGIN adds a line to the mysql.plugin table that describes the\nplugin. This table contains the plugin name and library file name.\n\nAs of MySQL 5.1.33, INSTALL PLUGIN causes the server to read option\n(my.cnf) files just as during server startup. This enables the plugin\nto pick up any relevant options from those files. It is possible to add\nplugin options to an option file even before loading a plugin (if the\nloose prefix is used). It is also possible to uninstall a plugin, edit\nmy.cnf, and install the plugin again. Restarting the plugin this way\nenables it to the new option values without a server restart.\n\nBefore MySQL 5.1.33, a plugin is started with each option set to its\ndefault value.\n\nINSTALL PLUGIN also loads and initializes the plugin code to make the\nplugin available for use. A plugin is initialized by executing its\ninitialization function, which handles any setup that the plugin must\nperform before it can be used.\n\nTo use INSTALL PLUGIN, you must have the INSERT privilege for the\nmysql.plugin table.\n\nAt server startup, the server loads and initializes any plugin that is\nlisted in the mysql.plugin table. This means that a plugin is installed\nwith INSTALL PLUGIN only once, not every time the server starts. Plugin\nloading at startup does not occur if the server is started with the\n--skip-grant-tables option.\n\nWhen the server shuts down, it executes the deinitialization function\nfor each plugin that is loaded so that the plugin has a change to\nperform any final cleanup.\n\nFor options that control individual plugin loading at server startup,\nsee http://dev.mysql.com/doc/refman/5.1/en/server-plugin-options.html.\nIf you need to load plugins for a single server startup when the\n--skip-grant-tables option is given (which tells the server not to read\nsystem tables), use the --plugin-load option. See\nhttp://dev.mysql.com/doc/refman/5.1/en/server-options.html.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (419,22,'DECLARE CURSOR','Syntax:\nDECLARE cursor_name CURSOR FOR select_statement\n\nThis statement declares a cursor. Multiple cursors may be declared in a\nstored program, but each cursor in a given block must have a unique\nname.\n\nThe SELECT statement cannot have an INTO clause.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (420,26,'LOAD DATA','Syntax:\nLOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE \'file_name\'\n [REPLACE | IGNORE]\n INTO TABLE tbl_name\n [CHARACTER SET charset_name]\n [{FIELDS | COLUMNS}\n [TERMINATED BY \'string\']\n [[OPTIONALLY] ENCLOSED BY \'char\']\n [ESCAPED BY \'char\']\n ]\n [LINES\n [STARTING BY \'string\']\n [TERMINATED BY \'string\']\n ]\n [IGNORE number LINES]\n [(col_name_or_user_var,...)]\n [SET col_name = expr,...]\n\nThe LOAD DATA INFILE statement reads rows from a text file into a table\nat a very high speed. The file name must be given as a literal string.\n\nLOAD DATA INFILE is the complement of SELECT ... INTO OUTFILE. (See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.) To write data from\na table to a file, use SELECT ... INTO OUTFILE. To read the file back\ninto a table, use LOAD DATA INFILE. The syntax of the FIELDS and LINES\nclauses is the same for both statements. Both clauses are optional, but\nFIELDS must precede LINES if both are specified.\n\nFor more information about the efficiency of INSERT versus LOAD DATA\nINFILE and speeding up LOAD DATA INFILE, see\nhttp://dev.mysql.com/doc/refman/5.1/en/insert-speed.html.\n\nThe character set indicated by the character_set_database system\nvariable is used to interpret the information in the file. SET NAMES\nand the setting of character_set_client do not affect interpretation of\ninput. If the contents of the input file use a character set that\ndiffers from the default, it is usually preferable to specify the\ncharacter set of the file by using the CHARACTER SET clause, which is\navailable as of MySQL 5.1.17. A character set of binary specifies "no\nconversion."\n\nLOAD DATA INFILE interprets all fields in the file as having the same\ncharacter set, regardless of the data types of the columns into which\nfield values are loaded. For proper interpretation of file contents,\nyou must ensure that it was written with the correct character set. For\nexample, if you write a data file with mysqldump -T or by issuing a\nSELECT ... INTO OUTFILE statement in mysql, be sure to use a\n--default-character-set option with mysqldump or mysql so that output\nis written in the character set to be used when the file is loaded with\nLOAD DATA INFILE.\n\nNote that it is currently not possible to load data files that use the\nucs2, utf16, or utf32 character set.\n\nAs of MySQL 5.1.6, the character_set_filesystem system variable\ncontrols the interpretation of the file name.\n\nYou can also load data files by using the mysqlimport utility; it\noperates by sending a LOAD DATA INFILE statement to the server. The\n--local option causes mysqlimport to read data files from the client\nhost. You can specify the --compress option to get better performance\nover slow networks if the client and server support the compressed\nprotocol. See http://dev.mysql.com/doc/refman/5.1/en/mysqlimport.html.\n\nIf you use LOW_PRIORITY, execution of the LOAD DATA statement is\ndelayed until no other clients are reading from the table. This affects\nonly storage engines that use only table-level locking (MyISAM, MEMORY,\nMERGE).\n\nIf you specify CONCURRENT with a MyISAM table that satisfies the\ncondition for concurrent inserts (that is, it contains no free blocks\nin the middle), other threads can retrieve data from the table while\nLOAD DATA is executing. Using this option affects the performance of\nLOAD DATA a bit, even if no other thread is using the table at the same\ntime.\n\nCONCURRENT is not replicated when using statement-based replication;\nhowever, it is replicated when using row-based replication. See\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-features-load-data.…, for more information.\n\n*Note*: Prior to MySQL 5.1.23, LOAD DATA performed very poorly when\nimporting into partitioned tables. The statement now uses buffering to\nimprove performance; however, the buffer uses 130 KB memory per\npartition to achieve this. (Bug#26527 (http://bugs.mysql.com/26527))\n\nThe LOCAL keyword, if specified, is interpreted with respect to the\nclient end of the connection:\n\no If LOCAL is specified, the file is read by the client program on the\n client host and sent to the server. The file can be given as a full\n path name to specify its exact location. If given as a relative path\n name, the name is interpreted relative to the directory in which the\n client program was started.\n\no If LOCAL is not specified, the file must be located on the server\n host and is read directly by the server. The server uses the\n following rules to locate the file:\n\n o If the file name is an absolute path name, the server uses it as\n given.\n\n o If the file name is a relative path name with one or more leading\n components, the server searches for the file relative to the\n server\'s data directory.\n\n o If a file name with no leading components is given, the server\n looks for the file in the database directory of the default\n database.\n\nNote that, in the non-LOCAL case, these rules mean that a file named as\n./myfile.txt is read from the server\'s data directory, whereas the file\nnamed as myfile.txt is read from the database directory of the default\ndatabase. For example, if db1 is the default database, the following\nLOAD DATA statement reads the file data.txt from the database directory\nfor db1, even though the statement explicitly loads the file into a\ntable in the db2 database:\n\nLOAD DATA INFILE \'data.txt\' INTO TABLE db2.my_table;\n\nWindows path names are specified using forward slashes rather than\nbackslashes. If you do use backslashes, you must double them.\n\n*Note*: A regression in MySQL 5.1.40 caused the database referenced in\na fully qualified table name to be ignored by LOAD DATA when using\nreplication with either STATEMENT or MIXED as the binary logging\nformat; this could lead to problems if the table was not in the current\ndatabase. As a workaround, you can specify the correct database with\nthe USE statement prior to executing LOAD DATA. If necessary, you can\nreset the current database with a second USE statement following the\nLOAD DATA statement. This issue was fixed in MySQL 5.1.41. (Bug#48297\n(http://bugs.mysql.com/48297))\n\nFor security reasons, when reading text files located on the server,\nthe files must either reside in the database directory or be readable\nby all. Also, to use LOAD DATA INFILE on server files, you must have\nthe FILE privilege. See\nhttp://dev.mysql.com/doc/refman/5.1/en/privileges-provided.html. For\nnon-LOCAL load operations, if the secure_file_priv system variable is\nset to a nonempty directory name, the file to be loaded must be located\nin that directory.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (418,5,'INSTALL PLUGIN','Syntax:\nINSTALL PLUGIN plugin_name SONAME \'shared_library_name\'\n\nThis statement installs a server plugin. It requires the INSERT\nprivilege for the mysql.plugin table.\n\nplugin_name is the name of the plugin as defined in the plugin\ndescriptor structure contained in the library file (see\nhttp://dev.mysql.com/doc/refman/5.1/en/plugin-data-structures.html).\n… names are not case sensitive. For maximal compatibility, plugin\nnames should be limited to ASCII letters, digits, and underscore\nbecause they are used in C source files, shell command lines, M4 and\nBourne shell scripts, and SQL environments.\n\nshared_library_name is the name of the shared library that contains the\nplugin code. The name includes the file name extension (for example,\nlibmyplugin.so, libmyplugin.dll, or libmyplugin.dylib).\n\nThe shared library must be located in the plugin directory (the\ndirectory named by the plugin_dir system variable). The library must be\nin the plugin directory itself, not in a subdirectory. By default,\nplugin_dir is the plugin directory under the directory named by the\npkglibdir configuration variable, but it can be changed by setting the\nvalue of plugin_dir at server startup. For example, set its value in a\nmy.cnf file:\n\n[mysqld]\nplugin_dir=/path/to/plugin/directory\n\nIf the value of plugin_dir is a relative path name, it is taken to be\nrelative to the MySQL base directory (the value of the basedir system\nvariable).\n\nINSTALL PLUGIN loads and initializes the plugin code to make the plugin\navailable for use. A plugin is initialized by executing its\ninitialization function, which handles any setup that the plugin must\nperform before it can be used. When the server shuts down, it executes\nthe deinitialization function for each plugin that is loaded so that\nthe plugin has a change to perform any final cleanup.\n\nINSTALL PLUGIN also registers the plugin by adding a line that\nindicates the plugin name and library file name to the mysql.plugin\ntable. At server startup, the server loads and initializes any plugin\nthat is listed in the mysql.plugin table. This means that a plugin is\ninstalled with INSTALL PLUGIN only once, not every time the server\nstarts. Plugin loading at startup does not occur if the server is\nstarted with the --skip-grant-tables option.\n\nA plugin library can contain multiple plugins. For each of them to be\ninstalled, use a separate INSTALL PLUGIN statement. Each statement\nnames a different plugin, but all of them specify the same library\nname.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/install-plugin.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (419,22,'DECLARE CURSOR','Syntax:\nDECLARE cursor_name CURSOR FOR select_statement\n\nThis statement declares a cursor. Multiple cursors may be declared in a\nstored program, but each cursor in a given block must have a unique\nname.\n\nThe SELECT statement cannot have an INTO clause.\n\nFor information available through SHOW statements, it is possible in\nmany cases to obtain equivalent information by using a cursor with an\nINFORMATION_SCHEMA table.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/declare-cursor.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (420,26,'LOAD DATA','Syntax:\nLOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE \'file_name\'\n [REPLACE | IGNORE]\n INTO TABLE tbl_name\n [CHARACTER SET charset_name]\n [{FIELDS | COLUMNS}\n [TERMINATED BY \'string\']\n [[OPTIONALLY] ENCLOSED BY \'char\']\n [ESCAPED BY \'char\']\n ]\n [LINES\n [STARTING BY \'string\']\n [TERMINATED BY \'string\']\n ]\n [IGNORE number LINES]\n [(col_name_or_user_var,...)]\n [SET col_name = expr,...]\n\nThe LOAD DATA INFILE statement reads rows from a text file into a table\nat a very high speed. The file name must be given as a literal string.\n\nLOAD DATA INFILE is the complement of SELECT ... INTO OUTFILE. (See\nhttp://dev.mysql.com/doc/refman/5.1/en/select.html.) To write data from\na table to a file, use SELECT ... INTO OUTFILE. To read the file back\ninto a table, use LOAD DATA INFILE. The syntax of the FIELDS and LINES\nclauses is the same for both statements. Both clauses are optional, but\nFIELDS must precede LINES if both are specified.\n\nFor more information about the efficiency of INSERT versus LOAD DATA\nINFILE and speeding up LOAD DATA INFILE, see\nhttp://dev.mysql.com/doc/refman/5.1/en/insert-speed.html.\n\nThe character set indicated by the character_set_database system\nvariable is used to interpret the information in the file. SET NAMES\nand the setting of character_set_client do not affect interpretation of\ninput. If the contents of the input file use a character set that\ndiffers from the default, it is usually preferable to specify the\ncharacter set of the file by using the CHARACTER SET clause, which is\navailable as of MySQL 5.1.17. A character set of binary specifies "no\nconversion."\n\nLOAD DATA INFILE interprets all fields in the file as having the same\ncharacter set, regardless of the data types of the columns into which\nfield values are loaded. For proper interpretation of file contents,\nyou must ensure that it was written with the correct character set. For\nexample, if you write a data file with mysqldump -T or by issuing a\nSELECT ... INTO OUTFILE statement in mysql, be sure to use a\n--default-character-set option with mysqldump or mysql so that output\nis written in the character set to be used when the file is loaded with\nLOAD DATA INFILE.\n\nNote that it is currently not possible to load data files that use the\nucs2 character set.\n\nAs of MySQL 5.1.6, the character_set_filesystem system variable\ncontrols the interpretation of the file name.\n\nYou can also load data files by using the mysqlimport utility; it\noperates by sending a LOAD DATA INFILE statement to the server. The\n--local option causes mysqlimport to read data files from the client\nhost. You can specify the --compress option to get better performance\nover slow networks if the client and server support the compressed\nprotocol. See http://dev.mysql.com/doc/refman/5.1/en/mysqlimport.html.\n\nIf you use LOW_PRIORITY, execution of the LOAD DATA statement is\ndelayed until no other clients are reading from the table. This affects\nonly storage engines that use only table-level locking (such as MyISAM,\nMEMORY, and MERGE).\n\nIf you specify CONCURRENT with a MyISAM table that satisfies the\ncondition for concurrent inserts (that is, it contains no free blocks\nin the middle), other threads can retrieve data from the table while\nLOAD DATA is executing. Using this option affects the performance of\nLOAD DATA a bit, even if no other thread is using the table at the same\ntime.\n\nPrior to MySQL 5.1.43, CONCURRENT was not replicated when using\nstatement-based replication (see Bug#34628\n(http://bugs.mysql.com/bug.php?id=34628)) However, it is replicated\nwhen using row-based replication, regardless of the version. See\nhttp://dev.mysql.com/doc/refman/5.1/en/replication-features-load-data.…, for more information.\n\n*Note*: Prior to MySQL 5.1.23, LOAD DATA performed very poorly when\nimporting into partitioned tables. The statement now uses buffering to\nimprove performance; however, the buffer uses 130KB memory per\npartition to achieve this. (Bug#26527\n(http://bugs.mysql.com/bug.php?id=26527))\n\nThe LOCAL keyword, if specified, is interpreted with respect to the\nclient end of the connection:\n\no If LOCAL is specified, the file is read by the client program on the\n client host and sent to the server. The file can be given as a full\n path name to specify its exact location. If given as a relative path\n name, the name is interpreted relative to the directory in which the\n client program was started.\n\no If LOCAL is not specified, the file must be located on the server\n host and is read directly by the server. The server uses the\n following rules to locate the file:\n\n o If the file name is an absolute path name, the server uses it as\n given.\n\n o If the file name is a relative path name with one or more leading\n components, the server searches for the file relative to the\n server\'s data directory.\n\n o If a file name with no leading components is given, the server\n looks for the file in the database directory of the default\n database.\n\nNote that, in the non-LOCAL case, these rules mean that a file named as\n./myfile.txt is read from the server\'s data directory, whereas the file\nnamed as myfile.txt is read from the database directory of the default\ndatabase. For example, if db1 is the default database, the following\nLOAD DATA statement reads the file data.txt from the database directory\nfor db1, even though the statement explicitly loads the file into a\ntable in the db2 database:\n\nLOAD DATA INFILE \'data.txt\' INTO TABLE db2.my_table;\n\nWindows path names are specified using forward slashes rather than\nbackslashes. If you do use backslashes, you must double them.\n\n*Note*: A regression in MySQL 5.1.40 caused the database referenced in\na fully qualified table name to be ignored by LOAD DATA when using\nreplication with either STATEMENT or MIXED as the binary logging\nformat; this could lead to problems if the table was not in the current\ndatabase. As a workaround, you can specify the correct database with\nthe USE statement prior to executing LOAD DATA. If necessary, you can\nreset the default database with a second USE statement following the\nLOAD DATA statement. This issue was fixed in MySQL 5.1.41. (Bug#48297\n(http://bugs.mysql.com/bug.php?id=48297))\n\nFor security reasons, when reading text files located on the server,\nthe files must either reside in the database directory or be readable\nby all. Also, to use LOAD DATA INFILE on server files, you must have\nthe FILE privilege. See\nhttp://dev.mysql.com/doc/refman/5.1/en/privileges-provided.html. For\nnon-LOCAL load operations, if the secure_file_priv system variable is\nset to a nonempty directory name, the file to be loaded must be located\nin that directory.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/load-data.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/load-data.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (421,23,'MULTILINESTRING','MultiLineString(ls1,ls2,...)\n\nConstructs a MultiLineString value using LineString or WKB LineString\narguments.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-mys…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (422,30,'LOCALTIME','Syntax:\nLOCALTIME, LOCALTIME()\n\nLOCALTIME and LOCALTIME() are synonyms for NOW().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (423,3,'MPOINTFROMTEXT','MPointFromText(wkt[,srid]), MultiPointFromText(wkt[,srid])\n\nConstructs a MULTIPOINT value using its WKT representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkt…'); @@ -525,7 +525,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (456,20,'TINYBLOB','TINYBLOB\n\nA BLOB column with a maximum length of 255 (28 - 1) bytes. Each\nTINYBLOB value is stored using a one-byte length prefix that indicates\nthe number of bytes in the value.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (457,8,'SAVEPOINT','Syntax:\nSAVEPOINT identifier\nROLLBACK [WORK] TO [SAVEPOINT] identifier\nRELEASE SAVEPOINT identifier\n\nInnoDB supports the SQL statements SAVEPOINT, ROLLBACK TO SAVEPOINT,\nRELEASE SAVEPOINT and the optional WORK keyword for ROLLBACK.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/savepoint.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/savepoint.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (458,15,'USER','Syntax:\nUSER()\n\nReturns the current MySQL user name and host name as a string in the\nutf8 character set.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT USER();\n -> \'davida@localhost\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (459,37,'ALTER TABLE','Syntax:\nALTER [ONLINE | OFFLINE] [IGNORE] TABLE tbl_name\n alter_specification [, alter_specification] ...\n\nalter_specification:\n table_options\n | ADD [COLUMN] col_name column_definition\n [FIRST | AFTER col_name ]\n | ADD [COLUMN] (col_name column_definition,...)\n | ADD {INDEX|KEY} [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]] PRIMARY KEY\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n UNIQUE [INDEX|KEY] [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD FULLTEXT [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD SPATIAL [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n FOREIGN KEY [index_name] (index_col_name,...)\n reference_definition\n | ALTER [COLUMN] col_name {SET DEFAULT literal | DROP DEFAULT}\n | CHANGE [COLUMN] old_col_name new_col_name column_definition\n [FIRST|AFTER col_name]\n | MODIFY [COLUMN] col_name column_definition\n [FIRST | AFTER col_name]\n | DROP [COLUMN] col_name\n | DROP PRIMARY KEY\n | DROP {INDEX|KEY} index_name\n | DROP FOREIGN KEY fk_symbol\n | DISABLE KEYS\n | ENABLE KEYS\n | RENAME [TO] new_tbl_name\n | ORDER BY col_name [, col_name] ...\n | CONVERT TO CHARACTER SET charset_name [COLLATE collation_name]\n | [DEFAULT] CHARACTER SET [=] charset_name [COLLATE [=] collation_name]\n | DISCARD TABLESPACE\n | IMPORT TABLESPACE\n | partition_options\n | ADD PARTITION (partition_definition)\n | DROP PARTITION partition_names\n | COALESCE PARTITION number\n | REORGANIZE PARTITION [partition_names INTO (partition_definitions)]\n | ANALYZE PARTITION partition_names\n | CHECK PARTITION partition_names\n | OPTIMIZE PARTITION partition_names\n | REBUILD PARTITION partition_names\n | REPAIR PARTITION partition_names\n | REMOVE PARTITIONING\n\nindex_col_name:\n col_name [(length)] [ASC | DESC]\n\nindex_type:\n USING {BTREE | HASH | RTREE}\n\nindex_option:\n KEY_BLOCK_SIZE [=] value\n | index_type\n | WITH PARSER parser_name\n | COMMENT \'string\'\n\ntable_options:\n table_option [[,] table_option] ...\n\nALTER TABLE enables you to change the structure of an existing table.\nFor example, you can add or delete columns, create or destroy indexes,\nchange the type of existing columns, or rename columns or the table\nitself. You can also change the comment for the table and type of the\ntable.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-table.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (459,37,'ALTER TABLE','Syntax:\nALTER [ONLINE | OFFLINE] [IGNORE] TABLE tbl_name\n {table_options | partitioning_specification}\n\ntable_options:\n table_option [, table_option] ...\n\ntable_option:\n ADD [COLUMN] col_name column_definition\n [FIRST | AFTER col_name ]\n | ADD [COLUMN] (col_name column_definition,...)\n | ADD {INDEX|KEY} [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]] PRIMARY KEY\n [index_type] (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n UNIQUE [INDEX|KEY] [index_name]\n [index_type] (index_col_name,...) [index_option] ...\n | ADD FULLTEXT [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD SPATIAL [INDEX|KEY] [index_name]\n (index_col_name,...) [index_option] ...\n | ADD [CONSTRAINT [symbol]]\n FOREIGN KEY [index_name] (index_col_name,...)\n reference_definition\n | ALTER [COLUMN] col_name {SET DEFAULT literal | DROP DEFAULT}\n | CHANGE [COLUMN] old_col_name new_col_name column_definition\n [FIRST|AFTER col_name]\n | MODIFY [COLUMN] col_name column_definition\n [FIRST | AFTER col_name]\n | DROP [COLUMN] col_name\n | DROP PRIMARY KEY\n | DROP {INDEX|KEY} index_name\n | DROP FOREIGN KEY fk_symbol\n | DISABLE KEYS\n | ENABLE KEYS\n | RENAME [TO] new_tbl_name\n | ORDER BY col_name [, col_name] ...\n | CONVERT TO CHARACTER SET charset_name [COLLATE collation_name]\n | [DEFAULT] CHARACTER SET [=] charset_name [COLLATE [=] collation_name]\n | DISCARD TABLESPACE\n | IMPORT TABLESPACE\n\npartitioning_specification:\n ADD PARTITION (partition_definition)\n | DROP PARTITION partition_names\n | COALESCE PARTITION number\n | REORGANIZE PARTITION [partition_names INTO (partition_definitions)]\n | ANALYZE PARTITION {partition_names | ALL }\n | CHECK PARTITION {partition_names | ALL }\n | OPTIMIZE PARTITION {partition_names | ALL }\n | REBUILD PARTITION {partition_names | ALL }\n | REPAIR PARTITION {partition_names | ALL }\n | PARTITION BY partitioning_expression\n | REMOVE PARTITIONING\n\nindex_col_name:\n col_name [(length)] [ASC | DESC]\n\nindex_type:\n USING {BTREE | HASH | RTREE}\n\nindex_option:\n KEY_BLOCK_SIZE [=] value\n | index_type\n | WITH PARSER parser_name\n\ntable_options:\n table_option [[,] table_option] ... (see CREATE TABLE options)\n\nALTER TABLE enables you to change the structure of an existing table.\nFor example, you can add or delete columns, create or destroy indexes,\nchange the type of existing columns, or rename columns or the table\nitself. You can also change the comment for the table and type of the\ntable.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/alter-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (460,31,'MPOINTFROMWKB','MPointFromWKB(wkb[,srid]), MultiPointFromWKB(wkb[,srid])\n\nConstructs a MULTIPOINT value using its WKB representation and SRID.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…','','http://dev.mysql.com/doc/refman/5.1/en/creating-spatial-values.html#gis-wkb…'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (461,20,'CHAR BYTE','The CHAR BYTE data type is an alias for the BINARY data type. This is a\ncompatibility feature.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (462,19,'REPAIR TABLE','Syntax:\nREPAIR [NO_WRITE_TO_BINLOG | LOCAL] TABLE\n tbl_name [, tbl_name] ...\n [QUICK] [EXTENDED] [USE_FRM]\n\nREPAIR TABLE repairs a possibly corrupted table. By default, it has the\nsame effect as myisamchk --recover tbl_name. REPAIR TABLE works for\nMyISAM and for ARCHIVE tables. Starting with MySQL 5.1.9, REPAIR is\nalso valid for CSV tables. See\nhttp://dev.mysql.com/doc/refman/5.1/en/myisam-storage-engine.html, and\nhttp://dev.mysql.com/doc/refman/5.1/en/archive-storage-engine.html, and\nhttp://dev.mysql.com/doc/refman/5.1/en/csv-storage-engine.html\n\nThis statement requires SELECT and INSERT privileges for the table.\n\nBeginning with MySQL 5.1.27, REPAIR TABLE is also supported for\npartitioned tables. However, the USE_FRM option cannot be used with\nthis statement on a partitioned table.\n\nAlso beginning with MySQL 5.1.27, you can use ALTER TABLE ... REPAIR\nPARTITION to repair one or more partitions; for more information, see\n[HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/repair-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/repair-table.html'); @@ -535,12 +535,12 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (466,19,'ANALYZE TABLE','Syntax:\nANALYZE [NO_WRITE_TO_BINLOG | LOCAL] TABLE\n tbl_name [, tbl_name] ...\n\nANALYZE TABLE analyzes and stores the key distribution for a table.\nDuring the analysis, the table is locked with a read lock for MyISAM.\nFor InnoDB the table is locked with a write lock. This statement works\nwith MyISAM and InnoDB tables. For MyISAM tables, this statement is\nequivalent to using myisamchk --analyze.\n\nFor more information on how the analysis works within InnoDB, see\nhttp://dev.mysql.com/doc/refman/5.1/en/innodb-restrictions.html.\n\nMy… uses the stored key distribution to decide the order in which\ntables should be joined when you perform a join on something other than\na constant. In addition, key distributions can be used when deciding\nwhich indexes to use for a specific table within a query.\n\nThis statement requires SELECT and INSERT privileges for the table.\n\nBeginning with MySQL 5.1.27, ANALYZE TABLE is also supported for\npartitioned tables. Also beginning with MySQL 5.1.27, you can use ALTER\nTABLE ... ANALYZE PARTITION to analyze one or more partitions; for more\ninformation, see [HELP ALTER TABLE], and\nhttp://dev.mysql.com/doc/refman/5.1/en/partitioning-maintenance.html.\…: http://dev.mysql.com/doc/refman/5.1/en/analyze-table.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/analyze-table.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (467,30,'MICROSECOND','Syntax:\nMICROSECOND(expr)\n\nReturns the microseconds from the time or datetime expression expr as a\nnumber in the range from 0 to 999999.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MICROSECOND(\'12:00:00.123456\');\n -> 123456\nmysql> SELECT MICROSECOND(\'2009-12-31 23:59:59.000010\');\n -> 10\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (468,37,'CONSTRAINT','InnoDB supports foreign key constraints. The syntax for a foreign key\nconstraint definition in InnoDB looks like this:\n\n[CONSTRAINT [symbol]] FOREIGN KEY\n [index_name] (index_col_name, ...)\n REFERENCES tbl_name (index_col_name,...)\n [ON DELETE reference_option]\n [ON UPDATE reference_option]\n\nreference_option:\n RESTRICT | CASCADE | SET NULL | NO ACTION\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/innodb-foreign-key-constraints.html\…','CREATE TABLE product (category INT NOT NULL, id INT NOT NULL,\n price DECIMAL,\n PRIMARY KEY(category, id)) ENGINE=INNODB;\nCREATE TABLE customer (id INT NOT NULL,\n PRIMARY KEY (id)) ENGINE=INNODB;\nCREATE TABLE product_order (no INT NOT NULL AUTO_INCREMENT,\n product_category INT NOT NULL,\n product_id INT NOT NULL,\n customer_id INT NOT NULL,\n PRIMARY KEY(no),\n INDEX (product_category, product_id),\n FOREIGN KEY (product_category, product_id)\n REFERENCES product(category, id)\n ON UPDATE CASCADE ON DELETE RESTRICT,\n INDEX (customer_id),\n FOREIGN KEY (customer_id)\n REFERENCES customer(id)) ENGINE=INNODB;\n','http://dev.mysql.com/doc/refman/5.1/en/innodb-foreign-key-constraints.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (469,37,'CREATE SERVER','Syntax:\nCREATE SERVER server_name\n FOREIGN DATA WRAPPER wrapper_name\n OPTIONS (option [, option] ...)\n\noption:\n { HOST character-literal\n | DATABASE character-literal\n | USER character-literal\n | PASSWORD character-literal\n | SOCKET character-literal\n | OWNER character-literal\n | PORT numeric-literal }\n\nThis statement creates the definition of a server for use with the\nFEDERATED storage engine. The CREATE SERVER statement creates a new row\nwithin the servers table within the mysql database. This statement\nrequires the SUPER privilege.\n\nThe server_name should be a unique reference to the server. Server\ndefinitions are global within the scope of the server, it is not\npossible to qualify the server definition to a specific database.\nserver_name has a maximum length of 64 characters (names longer than 64\ncharacters are silently truncated), and is case insensitive. You may\nspecify the name as a quoted string.\n\nThe wrapper_name should be mysql, and may be quoted with single quotes.\nOther values for wrapper_name are not currently supported.\n\nFor each option you must specify either a character literal or numeric\nliteral. Character literals are UTF-8, support a maximum length of 64\ncharacters and default to a blank (empty) string. String literals are\nsilently truncated to 64 characters. Numeric literals must be a number\nbetween 0 and 9999, default value is 0.\n\n*Note*: Note that the OWNER option is currently not applied, and has no\neffect on the ownership or operation of the server connection that is\ncreated.\n\nThe CREATE SERVER statement creates an entry in the mysql.server table\nthat can later be used with the CREATE TABLE statement when creating a\nFEDERATED table. The options that you specify will be used to populate\nthe columns in the mysql.server table. The table columns are\nServer_name, Host, Db, Username, Password, Port and Socket.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-server.html\n\n','CREATE SERVER s\nFOREIGN DATA WRAPPER mysql\nOPTIONS (USER \'Remote\', HOST \'192.168.1.106\', DATABASE \'test\');\n','http://dev.mysql.com/doc/refman/5.1/en/create-server.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (469,37,'CREATE SERVER','Syntax:\nCREATE SERVER server_name\n FOREIGN DATA WRAPPER wrapper_name\n OPTIONS (option [, option] ...)\n\noption:\n { HOST character-literal\n | DATABASE character-literal\n | USER character-literal\n | PASSWORD character-literal\n | SOCKET character-literal\n | OWNER character-literal\n | PORT numeric-literal }\n\nThis statement creates the definition of a server for use with the\nFEDERATED storage engine. The CREATE SERVER statement creates a new row\nwithin the servers table within the mysql database. This statement\nrequires the SUPER privilege.\n\nThe server_name should be a unique reference to the server. Server\ndefinitions are global within the scope of the server, it is not\npossible to qualify the server definition to a specific database.\nserver_name has a maximum length of 64 characters (names longer than 64\ncharacters are silently truncated), and is case insensitive. You may\nspecify the name as a quoted string.\n\nThe wrapper_name should be mysql, and may be quoted with single quotes.\nOther values for wrapper_name are not currently supported.\n\nFor each option you must specify either a character literal or numeric\nliteral. Character literals are UTF-8, support a maximum length of 64\ncharacters and default to a blank (empty) string. String literals are\nsilently truncated to 64 characters. Numeric literals must be a number\nbetween 0 and 9999, default value is 0.\n\n*Note*: Note that the OWNER option is currently not applied, and has no\neffect on the ownership or operation of the server connection that is\ncreated.\n\nThe CREATE SERVER statement creates an entry in the mysql.servers table\nthat can later be used with the CREATE TABLE statement when creating a\nFEDERATED table. The options that you specify will be used to populate\nthe columns in the mysql.servers table. The table columns are\nServer_name, Host, Db, Username, Password, Port and Socket.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/create-server.html\n\n','CREATE SERVER s\nFOREIGN DATA WRAPPER mysql\nOPTIONS (USER \'Remote\', HOST \'192.168.1.106\', DATABASE \'test\');\n','http://dev.mysql.com/doc/refman/5.1/en/create-server.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (470,35,'FIELD','Syntax:\nFIELD(str,str1,str2,str3,...)\n\nReturns the index (position) of str in the str1, str2, str3, ... list.\nReturns 0 if str is not found.\n\nIf all arguments to FIELD() are strings, all arguments are compared as\nstrings. If all arguments are numbers, they are compared as numbers.\nOtherwise, the arguments are compared as double.\n\nIf str is NULL, the return value is 0 because NULL fails equality\ncomparison with any value. FIELD() is the complement of ELT().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT FIELD(\'ej\', \'Hej\', \'ej\', \'Heja\', \'hej\', \'foo\');\n -> 2\nmysql> SELECT FIELD(\'fo\', \'Hej\', \'ej\', \'Heja\', \'hej\', \'foo\');\n -> 0\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (471,30,'MAKETIME','Syntax:\nMAKETIME(hour,minute,second)\n\nReturns a time value calculated from the hour, minute, and second\narguments.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT MAKETIME(12,15,30);\n -> \'12:15:30\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (472,30,'CURDATE','Syntax:\nCURDATE()\n\nReturns the current date as a value in \'YYYY-MM-DD\' or YYYYMMDD format,\ndepending on whether the function is used in a string or numeric\ncontext.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT CURDATE();\n -> \'2008-06-13\'\nmysql> SELECT CURDATE() + 0;\n -> 20080613\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (473,9,'SET PASSWORD','Syntax:\nSET PASSWORD [FOR user] =\n {\n PASSWORD(\'some password\')\n | OLD_PASSWORD(\'some password\')\n | \'encrypted password\'\n }\n\nThe SET PASSWORD statement assigns a password to an existing MySQL user\naccount.\n\nIf the password is specified using the PASSWORD() or OLD_PASSWORD()\nfunction, the literal text of the password should be given. If the\npassword is specified without using either function, the password\nshould be the already-encrypted password value as returned by\nPASSWORD().\n\nWith no FOR clause, this statement sets the password for the current\nuser. Any client that has connected to the server using a nonanonymous\naccount can change the password for that account.\n\nWith a FOR clause, this statement sets the password for a specific\naccount on the current server host. Only clients that have the UPDATE\nprivilege for the mysql database can do this. The user value should be\ngiven in user_name@host_name format, where user_name and host_name are\nexactly as they are listed in the User and Host columns of the\nmysql.user table entry. For example, if you had an entry with User and\nHost column values of \'bob\' and \'%.loc.gov\', you would write the\nstatement like this:\n\nSET PASSWORD FOR \'bob\'@\'%.loc.gov\' = PASSWORD(\'newpass\');\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-password.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-password.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (474,37,'ALTER TABLESPACE','Syntax:\nALTER TABLESPACE tablespace_name\n {ADD|DROP} DATAFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement can be used either to add a new data file, or to drop a\ndata file from a tablespace.\n\nThe ADD DATAFILE variant allows you to specify an initial size using an\nINITIAL_SIZE clause, where size is measured in bytes; the default value\nis 128M (128 megabytes). You may optionally follow this integer value\nwith a one-letter abbreviation for an order of magnitude, similar to\nthose used in my.cnf. Generally, this is one of the letters M (for\nmegabytes) or G (for gigabytes).\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an data file with the same name, or an undo log\nfile and a with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/31770))\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/29186))\n\nOnce a data file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using additional ALTER\nTABLESPACE ... ADD DATAFILE statements.\n\nUsing DROP DATAFILE with ALTER TABLESPACE drops the data file\n\'file_name\' from the tablespace. This file must already have been added\nto the tablespace using CREATE TABLESPACE or ALTER TABLESPACE;\notherwise an error will result.\n\nBoth ALTER TABLESPACE ... ADD DATAFILE and ALTER TABLESPACE ... DROP\nDATAFILE require an ENGINE clause which specifies the storage engine\nused by the tablespace. In MySQL 5.1, the only accepted values for\nengine_name are NDB and NDBCLUSTER.\n\nWAIT is parsed but otherwise ignored, and so has no effect in MySQL\n5.1. It is intended for future expansion.\n\nWhen ALTER TABLESPACE ... ADD DATAFILE is used with ENGINE = NDB, a\ndata file is created on each Cluster data node. You can verify that the\ndata files were created and obtain information about them by querying\nthe INFORMATION_SCHEMA.FILES table. For example, the following query\nshows all data files belonging to the tablespace named newts:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+--------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+--------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=4 |\n+--------------------+--------------+----------------+\n2 rows in set (0.03 sec)\n\nSee http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nALTER TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (474,37,'ALTER TABLESPACE','Syntax:\nALTER TABLESPACE tablespace_name\n {ADD|DROP} DATAFILE \'file_name\'\n [INITIAL_SIZE [=] size]\n [WAIT]\n ENGINE [=] engine_name\n\nThis statement can be used either to add a new data file, or to drop a\ndata file from a tablespace.\n\nThe ADD DATAFILE variant allows you to specify an initial size using an\nINITIAL_SIZE clause, where size is measured in bytes; the default value\nis 128M (128 megabytes). You may optionally follow this integer value\nwith a one-letter abbreviation for an order of magnitude, similar to\nthose used in my.cnf. Generally, this is one of the letters M (for\nmegabytes) or G (for gigabytes).\n\n*Note*: All MySQL Cluster Disk Data objects share the same namespace.\nThis means that each Disk Data object must be uniquely named (and not\nmerely each Disk Data object of a given type). For example, you cannot\nhave a tablespace and an data file with the same name, or an undo log\nfile and a with the same name.\n\nPrior to MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3, path and file\nnames for data files could not be longer than 128 characters.\n(Bug#31770 (http://bugs.mysql.com/bug.php?id=31770))\n\nOn 32-bit systems, the maximum supported value for INITIAL_SIZE is 4G.\n(Bug#29186 (http://bugs.mysql.com/bug.php?id=29186))\n\nINITIAL_SIZE is rounded as for CREATE TABLESPACE. Beginning with MySQL\nCluster NDB 6.2.19, MySQL Cluster NDB 6.3.32, MySQL Cluster NDB 7.0.13,\nand MySQL Cluster NDB 7.1.2, this rounding is done explicitly (also as\nwith CREATE TABLESPACE).\n\nOnce a data file has been created, its size cannot be changed; however,\nyou can add more data files to the tablespace using additional ALTER\nTABLESPACE ... ADD DATAFILE statements.\n\nUsing DROP DATAFILE with ALTER TABLESPACE drops the data file\n\'file_name\' from the tablespace. This file must already have been added\nto the tablespace using CREATE TABLESPACE or ALTER TABLESPACE;\notherwise an error will result.\n\nBoth ALTER TABLESPACE ... ADD DATAFILE and ALTER TABLESPACE ... DROP\nDATAFILE require an ENGINE clause which specifies the storage engine\nused by the tablespace. In MySQL 5.1, the only accepted values for\nengine_name are NDB and NDBCLUSTER.\n\nWAIT is parsed but otherwise ignored, and so has no effect in MySQL\n5.1. It is intended for future expansion.\n\nWhen ALTER TABLESPACE ... ADD DATAFILE is used with ENGINE = NDB, a\ndata file is created on each Cluster data node. You can verify that the\ndata files were created and obtain information about them by querying\nthe INFORMATION_SCHEMA.FILES table. For example, the following query\nshows all data files belonging to the tablespace named newts:\n\nmysql> SELECT LOGFILE_GROUP_NAME, FILE_NAME, EXTRA\n -> FROM INFORMATION_SCHEMA.FILES\n -> WHERE TABLESPACE_NAME = \'newts\' AND FILE_TYPE = \'DATAFILE\';\n+--------------------+--------------+----------------+\n| LOGFILE_GROUP_NAME | FILE_NAME | EXTRA |\n+--------------------+--------------+----------------+\n| lg_3 | newdata.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata.dat | CLUSTER_NODE=4 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=3 |\n| lg_3 | newdata2.dat | CLUSTER_NODE=4 |\n+--------------------+--------------+----------------+\n2 rows in set (0.03 sec)\n\nSee http://dev.mysql.com/doc/refman/5.1/en/files-table.html.\n\nALTER TABLESPACE was added in MySQL 5.1.6. In MySQL 5.1, it is useful\nonly with Disk Data storage for MySQL Cluster. See\nhttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html.\n…: http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/alter-tablespace.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (475,20,'ENUM','ENUM(\'value1\',\'value2\',...) [CHARACTER SET charset_name] [COLLATE\ncollation_name]\n\nAn enumeration. A string object that can have only one value, chosen\nfrom the list of values \'value1\', \'value2\', ..., NULL or the special \'\'\nerror value. An ENUM column can have a maximum of 65,535 distinct\nvalues. ENUM values are represented internally as integers.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/string-type-overview.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (476,7,'IF FUNCTION','Syntax:\nIF(expr1,expr2,expr3)\n\nIf expr1 is TRUE (expr1 <> 0 and expr1 <> NULL) then IF() returns\nexpr2; otherwise it returns expr3. IF() returns a numeric or string\nvalue, depending on the context in which it is used.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html\n\n','mysql> SELECT IF(1>2,2,3);\n -> 3\nmysql> SELECT IF(1<2,\'yes\',\'no\');\n -> \'yes\'\nmysql> SELECT IF(STRCMP(\'test\',\'test1\'),\'no\',\'yes\');\n -> \'no\'\n','http://dev.mysql.com/doc/refman/5.1/en/control-flow-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (477,15,'DATABASE','Syntax:\nDATABASE()\n\nReturns the default (current) database name as a string in the utf8\ncharacter set. If there is no default database, DATABASE() returns\nNULL. Within a stored routine, the default database is the database\nthat the routine is associated with, which is not necessarily the same\nas the database that is the default in the calling context.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT DATABASE();\n -> \'test\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); @@ -556,9 +556,9 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (487,22,'RETURN','Syntax:\nRETURN expr\n\nThe RETURN statement terminates execution of a stored function and\nreturns the value expr to the function caller. There must be at least\none RETURN statement in a stored function. There may be more than one\nif the function has multiple exit points.\n\nThis statement is not used in stored procedures, triggers, or events.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/return.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/return.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (488,25,'SHOW COLLATION','Syntax:\nSHOW COLLATION\n [LIKE \'pattern\' | WHERE expr]\n\nThis statement lists collations supported by the server. By default,\nthe output from SHOW COLLATION includes all available collations. The\nLIKE clause, if present, indicates which collation names to match. The\nWHERE clause can be given to select rows using more general conditions,\nas discussed in\nhttp://dev.mysql.com/doc/refman/5.1/en/extended-show.html. For example:\n\nmysql> SHOW COLLATION LIKE \'latin1%\';\n+-------------------+---------+----+---------+----------+---------+\n| Collation | Charset | Id | Default | Compiled | Sortlen |\n+-------------------+---------+----+---------+----------+---------+\n| latin1_german1_ci | latin1 | 5 | | | 0 |\n| latin1_swedish_ci | latin1 | 8 | Yes | Yes | 0 |\n| latin1_danish_ci | latin1 | 15 | | | 0 |\n| latin1_german2_ci | latin1 | 31 | | Yes | 2 |\n| latin1_bin | latin1 | 47 | | Yes | 0 |\n| latin1_general_ci | latin1 | 48 | | | 0 |\n| latin1_general_cs | latin1 | 49 | | | 0 |\n| latin1_spanish_ci | latin1 | 94 | | | 0 |\n+-------------------+---------+----+---------+----------+---------+\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/show-collation.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/show-collation.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (489,4,'LOG','Syntax:\nLOG(X), LOG(B,X)\n\nIf called with one parameter, this function returns the natural\nlogarithm of X. If X is less than or equal to 0, then NULL is returned.\n\nThe inverse of this function (when called with a single argument) is\nthe EXP() function.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT LOG(2);\n -> 0.69314718055995\nmysql> SELECT LOG(-2);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (490,25,'SET SQL_LOG_BIN','Syntax:\nSET sql_log_bin = {0|1}\n\nDisables or enables binary logging for the current connection\n(sql_log_bin is a session variable) if the client has the SUPER\nprivilege. The statement is refused with an error if the client does\nnot have that privilege.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (490,25,'SET SQL_LOG_BIN','Syntax:\nSET sql_log_bin = {0|1}\n\nDisables or enables binary logging for the current session (sql_log_bin\nis a session variable) if the client has the SUPER privilege. The\nstatement fails with an error if the client does not have that\nprivilege.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/set-sql-log-bin.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (491,17,'!=','Syntax:\n<>, !=\n\nNot equal:\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT \'.01\' <> \'0.01\';\n -> 1\nmysql> SELECT .01 <> \'0.01\';\n -> 0\nmysql> SELECT \'zapp\' <> \'zappp\';\n -> 1\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (492,22,'WHILE','Syntax:\n[begin_label:] WHILE search_condition DO\n statement_list\nEND WHILE [end_label]\n\nThe statement list within a WHILE statement is repeated as long as the\nsearch_condition is true. statement_list consists of one or more\nstatements.\n\nA WHILE statement can be labeled. end_label cannot be given unless\nbegin_label also is present. If both are present, they must be the\nsame.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/while-statement.html\n\n','CREATE PROCEDURE dowhile()\nBEGIN\n DECLARE v1 INT DEFAULT 5;\n\n WHILE v1 > 0 DO\n ...\n SET v1 = v1 - 1;\n END WHILE;\nEND\n','http://dev.mysql.com/doc/refman/5.1/en/while-statement.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (492,22,'WHILE','Syntax:\n[begin_label:] WHILE search_condition DO\n statement_list\nEND WHILE [end_label]\n\nThe statement list within a WHILE statement is repeated as long as the\nsearch_condition is true. statement_list consists of one or more\nstatements.\n\nA WHILE statement can be labeled. See [HELP BEGIN END] for the rules\nregarding label use.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/while-statement.html\n\n','CREATE PROCEDURE dowhile()\nBEGIN\n DECLARE v1 INT DEFAULT 5;\n\n WHILE v1 > 0 DO\n ...\n SET v1 = v1 - 1;\n END WHILE;\nEND\n','http://dev.mysql.com/doc/refman/5.1/en/while-statement.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (493,11,'AES_DECRYPT','Syntax:\nAES_DECRYPT(crypt_str,key_str)\n\nThis function allows decryption of data using the official AES\n(Advanced Encryption Standard) algorithm. For more information, see the\ndescription of AES_ENCRYPT().\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html\n\n','','http://dev.mysql.com/doc/refman/5.1/en/encryption-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (494,30,'DAYNAME','Syntax:\nDAYNAME(date)\n\nReturns the name of the weekday for date. As of MySQL 5.1.12, the\nlanguage used for the name is controlled by the value of the\nlc_time_names system variable\n(http://dev.mysql.com/doc/refman/5.1/en/locale-support.html).\n\n…: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html\n\n','mysql> SELECT DAYNAME(\'2007-02-03\');\n -> \'Saturday\'\n','http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (495,15,'COERCIBILITY','Syntax:\nCOERCIBILITY(str)\n\nReturns the collation coercibility value of the string argument.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT COERCIBILITY(\'abc\' COLLATE latin1_swedish_ci);\n -> 0\nmysql> SELECT COERCIBILITY(USER());\n -> 3\nmysql> SELECT COERCIBILITY(\'abc\');\n -> 4\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); @@ -567,7 +567,7 @@ insert into help_topic (help_topic_id,he insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (498,4,'RADIANS','Syntax:\nRADIANS(X)\n\nReturns the argument X, converted from degrees to radians. (Note that\nπ radians equals 180 degrees.)\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html\n\n','mysql> SELECT RADIANS(90);\n -> 1.5707963267949\n','http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (499,15,'COLLATION','Syntax:\nCOLLATION(str)\n\nReturns the collation of the string argument.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT COLLATION(\'abc\');\n -> \'latin1_swedish_ci\'\nmysql> SELECT COLLATION(_utf8\'abc\');\n -> \'utf8_general_ci\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (500,17,'COALESCE','Syntax:\nCOALESCE(value,...)\n\nReturns the first non-NULL value in the list, or NULL if there are no\nnon-NULL values.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html\n\n','mysql> SELECT COALESCE(NULL,1);\n -> 1\nmysql> SELECT COALESCE(NULL,NULL,NULL);\n -> NULL\n','http://dev.mysql.com/doc/refman/5.1/en/comparison-operators.html'); -insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (501,15,'VERSION','Syntax:\nVERSION()\n\nReturns a string that indicates the MySQL server version. The string\nuses the utf8 character set.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT VERSION();\n -> \'5.1.41-standard\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); +insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (501,15,'VERSION','Syntax:\nVERSION()\n\nReturns a string that indicates the MySQL server version. The string\nuses the utf8 character set.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/information-functions.html\n\n','mysql> SELECT VERSION();\n -> \'5.1.46-standard\'\n','http://dev.mysql.com/doc/refman/5.1/en/information-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (502,35,'MAKE_SET','Syntax:\nMAKE_SET(bits,str1,str2,...)\n\nReturns a set value (a string containing substrings separated by ","\ncharacters) consisting of the strings that have the corresponding bit\nin bits set. str1 corresponds to bit 0, str2 to bit 1, and so on. NULL\nvalues in str1, str2, ... are not appended to the result.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT MAKE_SET(1,\'a\',\'b\',\'c\');\n -> \'a\'\nmysql> SELECT MAKE_SET(1 | 4,\'hello\',\'nice\',\'world\');\n -> \'hello,world\'\nmysql> SELECT MAKE_SET(1 | 4,\'hello\',\'nice\',NULL,\'world\');\n -> \'hello\'\nmysql> SELECT MAKE_SET(0,\'a\',\'b\',\'c\');\n -> \'\'\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); insert into help_topic (help_topic_id,help_category_id,name,description,example,url) values (503,35,'FIND_IN_SET','Syntax:\nFIND_IN_SET(str,strlist)\n\nReturns a value in the range of 1 to N if the string str is in the\nstring list strlist consisting of N substrings. A string list is a\nstring composed of substrings separated by "," characters. If the first\nargument is a constant string and the second is a column of type SET,\nthe FIND_IN_SET() function is optimized to use bit arithmetic. Returns\n0 if str is not in strlist or if strlist is the empty string. Returns\nNULL if either argument is NULL. This function does not work properly\nif the first argument contains a comma (",") character.\n\nURL: http://dev.mysql.com/doc/refman/5.1/en/string-functions.html\n\n','mysql> SELECT FIND_IN_SET(\'b\',\'a,b,c,d\');\n -> 2\n','http://dev.mysql.com/doc/refman/5.1/en/string-functions.html'); @@ -1115,7 +1115,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (464,43); insert into help_relation (help_topic_id,help_keyword_id) values (459,43); insert into help_relation (help_topic_id,help_keyword_id) values (469,44); -insert into help_relation (help_topic_id,help_keyword_id) values (115,44); +insert into help_relation (help_topic_id,help_keyword_id) values (116,44); insert into help_relation (help_topic_id,help_keyword_id) values (464,44); insert into help_relation (help_topic_id,help_keyword_id) values (209,44); insert into help_relation (help_topic_id,help_keyword_id) values (420,44); @@ -1222,7 +1222,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (464,92); insert into help_relation (help_topic_id,help_keyword_id) values (67,93); insert into help_relation (help_topic_id,help_keyword_id) values (431,93); -insert into help_relation (help_topic_id,help_keyword_id) values (326,93); +insert into help_relation (help_topic_id,help_keyword_id) values (327,93); insert into help_relation (help_topic_id,help_keyword_id) values (282,94); insert into help_relation (help_topic_id,help_keyword_id) values (83,95); insert into help_relation (help_topic_id,help_keyword_id) values (57,95); @@ -1268,7 +1268,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (112,109); insert into help_relation (help_topic_id,help_keyword_id) values (459,109); insert into help_relation (help_topic_id,help_keyword_id) values (38,110); -insert into help_relation (help_topic_id,help_keyword_id) values (116,110); +insert into help_relation (help_topic_id,help_keyword_id) values (117,110); insert into help_relation (help_topic_id,help_keyword_id) values (261,110); insert into help_relation (help_topic_id,help_keyword_id) values (148,110); insert into help_relation (help_topic_id,help_keyword_id) values (122,111); @@ -1340,7 +1340,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (197,137); insert into help_relation (help_topic_id,help_keyword_id) values (310,138); insert into help_relation (help_topic_id,help_keyword_id) values (344,139); -insert into help_relation (help_topic_id,help_keyword_id) values (115,139); +insert into help_relation (help_topic_id,help_keyword_id) values (116,139); insert into help_relation (help_topic_id,help_keyword_id) values (453,139); insert into help_relation (help_topic_id,help_keyword_id) values (48,139); insert into help_relation (help_topic_id,help_keyword_id) values (120,139); @@ -1361,7 +1361,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (353,147); insert into help_relation (help_topic_id,help_keyword_id) values (419,147); insert into help_relation (help_topic_id,help_keyword_id) values (344,148); -insert into help_relation (help_topic_id,help_keyword_id) values (326,148); +insert into help_relation (help_topic_id,help_keyword_id) values (327,148); insert into help_relation (help_topic_id,help_keyword_id) values (464,149); insert into help_relation (help_topic_id,help_keyword_id) values (178,150); insert into help_relation (help_topic_id,help_keyword_id) values (95,151); @@ -1441,7 +1441,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (419,181); insert into help_relation (help_topic_id,help_keyword_id) values (196,181); insert into help_relation (help_topic_id,help_keyword_id) values (301,182); -insert into help_relation (help_topic_id,help_keyword_id) values (115,182); +insert into help_relation (help_topic_id,help_keyword_id) values (116,182); insert into help_relation (help_topic_id,help_keyword_id) values (420,182); insert into help_relation (help_topic_id,help_keyword_id) values (358,182); insert into help_relation (help_topic_id,help_keyword_id) values (353,183); @@ -1597,7 +1597,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (86,251); insert into help_relation (help_topic_id,help_keyword_id) values (184,252); insert into help_relation (help_topic_id,help_keyword_id) values (353,253); -insert into help_relation (help_topic_id,help_keyword_id) values (327,254); +insert into help_relation (help_topic_id,help_keyword_id) values (326,254); insert into help_relation (help_topic_id,help_keyword_id) values (353,254); insert into help_relation (help_topic_id,help_keyword_id) values (360,254); insert into help_relation (help_topic_id,help_keyword_id) values (312,255); @@ -1633,7 +1633,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (87,264); insert into help_relation (help_topic_id,help_keyword_id) values (60,264); insert into help_relation (help_topic_id,help_keyword_id) values (355,264); -insert into help_relation (help_topic_id,help_keyword_id) values (327,265); +insert into help_relation (help_topic_id,help_keyword_id) values (326,265); insert into help_relation (help_topic_id,help_keyword_id) values (191,266); insert into help_relation (help_topic_id,help_keyword_id) values (197,267); insert into help_relation (help_topic_id,help_keyword_id) values (230,268); @@ -1695,7 +1695,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (58,286); insert into help_relation (help_topic_id,help_keyword_id) values (391,286); insert into help_relation (help_topic_id,help_keyword_id) values (325,286); -insert into help_relation (help_topic_id,help_keyword_id) values (326,286); +insert into help_relation (help_topic_id,help_keyword_id) values (327,286); insert into help_relation (help_topic_id,help_keyword_id) values (120,286); insert into help_relation (help_topic_id,help_keyword_id) values (189,286); insert into help_relation (help_topic_id,help_keyword_id) values (449,286); @@ -1742,7 +1742,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (184,308); insert into help_relation (help_topic_id,help_keyword_id) values (420,309); insert into help_relation (help_topic_id,help_keyword_id) values (388,310); -insert into help_relation (help_topic_id,help_keyword_id) values (117,310); +insert into help_relation (help_topic_id,help_keyword_id) values (115,310); insert into help_relation (help_topic_id,help_keyword_id) values (459,311); insert into help_relation (help_topic_id,help_keyword_id) values (25,312); insert into help_relation (help_topic_id,help_keyword_id) values (344,312); @@ -2002,7 +2002,7 @@ insert into help_relation (help_topic_id insert into help_relation (help_topic_id,help_keyword_id) values (474,444); insert into help_relation (help_topic_id,help_keyword_id) values (344,445); insert into help_relation (help_topic_id,help_keyword_id) values (39,446); -insert into help_relation (help_topic_id,help_keyword_id) values (115,446); +insert into help_relation (help_topic_id,help_keyword_id) values (116,446); insert into help_relation (help_topic_id,help_keyword_id) values (266,446); insert into help_relation (help_topic_id,help_keyword_id) values (58,446); insert into help_relation (help_topic_id,help_keyword_id) values (184,446);

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2851)
by knielsen＠knielsen-hq.org 28 Apr '10

28 Apr '10

#At lp:maria 2851 knielsen(a)knielsen-hq.org 2010-04-28 (Hopefully) better fix for Windows warning on redefined TAILQ_EMPTY; the previous attempt broke build on Debian4. modified: extra/libevent/event-internal.h === modified file 'extra/libevent/event-internal.h' --- a/extra/libevent/event-internal.h 2010-04-09 10:39:27 +0000 +++ b/extra/libevent/event-internal.h 2010-04-28 13:00:18 +0000 @@ -69,14 +69,17 @@ struct event_base { }; /* Internal use only: Functions that might be missing from <sys/queue.h> */ -#ifndef HAVE_TAILQFOREACH /* These following macros are copied from BSD sys/queue.h Copyright (c) 1991, 1993, The Regents of the University of California. All rights reserved. */ +#ifndef TAILQ_EMPTY +#define TAILQ_EMPTY(head) ((head)->tqh_first == NULL) #define TAILQ_FIRST(head) ((head)->tqh_first) #define TAILQ_END(head) NULL #define TAILQ_NEXT(elm, field) ((elm)->field.tqe_next) +#endif /* TAILQ_EMPTY */ +#ifndef HAVE_TAILQFOREACH #define TAILQ_FOREACH(var, head, field) \ for((var) = TAILQ_FIRST(head); \ (var) != TAILQ_END(head); \

1 0

Re: [Maria-developers] Where is the wiki page for coordinating semantics of CREATE TABLE options?
by Sergei Golubchik 28 Apr '10

28 Apr '10

Hi, Henrik! On Apr 27, Henrik Ingo wrote: > Serg > > At the storage engine meeting, we decided to host a wiki page where > all engines can document their CREATE TABLE options, so that semantics > of options can remain consistent. > > Does this wiki page exist yet? No, it does not Regards, Sergei

1 0

[Maria-developers] Updated (by Serg): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 26 Apr '10

26 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 26 Apr '10

26 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: Serg CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Serg - Mon, 26 Apr 2010, 14:10)=-=- Observers changed: Serg DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Knielsen): Efficient group commit for binary log (116)
by worklog-noreply＠askmonty.org 26 Apr '10

26 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Efficient group commit for binary log CREATION DATE..: Mon, 26 Apr 2010, 13:28 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 116 (http://askmonty.org/worklog/?tid=116) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Currently, in order to ensure that the server can recover after a crash to a state in which storage engines and binary log are consistent with each other, it is necessary to use XA with durable commits for both storage engines (innodb_flush_log_at_trx_commit=1) and binary log (sync_binlog=1). This is _very_ expensive, since the server needs to do three fsync() operations for every commit, as there is no working group commit when the binary log is enabled. The idea is to - Implement/fix group commit to work properly with the binary log enabled. - (Optionally) avoid the need to fsync() in the engine, and instead rely on replaying any lost transactions from the binary log against the engine during crash recovery. For background see these articles: http://kristiannielsen.livejournal.com/12254.html http://kristiannielsen.livejournal.com/12408.html http://kristiannielsen.livejournal.com/12553.html ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Windows test platforms
by Bo Thorsen 26 Apr '10

26 Apr '10

Hi everyone, I'm going to build a set of Windows installations to run buildbot on. But finding the right set of versions to do this is a challenge. There are so many versions of Windows that it's impossible to get even close to the full set of them. So I'd like to have a good set of systems that pretty much cover most options. The buildbot system will work in three parts. One will build a set of installable packages - for now probably just a zip file. Another will install these packages and test them on clean systems. A third will do compilation tests on OS/Compiler combinations. Windows 2000 or earlier is not going to be done. I'm not sure about Windows Server 2003. I'd like to skip XP, as MS cut support for it, but I think there are too many live installations to do this yet. For building the installable packages, I intend to use XP and Visual Studio 2008. My list of systems to test the packages and do the compile + make test on would be: XP Pro 32 bit Vista 32 bit Windows 7 64 bit Server 2008 64 bit On XP, I'd install VC++ 2005, on the others VC++ 2008. When Visual Studio 2010 is out, I'll install that on Vista and one of the 64 bit systems. This is four systems that does a very thin span of the incredible amount of systems done by MS. (And people say there are a lot of Linux distributions.) Does this make sense to you? Please let me know what you think. Kind regards, Bo Thorsen, Monty Program AB.

2 1

Re: [Maria-developers] Windows test platforms
by Bo Thorsen 26 Apr '10

26 Apr '10

Monday 26 April 2010 11:04:50 Sergey Petrunya wrote: > On Mon, Apr 26, 2010 at 07:52:55AM +0200, Bo Thorsen wrote: > > The buildbot system will work in three parts. One will build a set of > > installable packages - for now probably just a zip file. Another will > > install these packages and test them on clean systems. A third will do > > compilation tests on OS/Compiler combinations. > > And none of them will run mysql-test-run? Why? I didn't write it, but I assume both the test platforms will run the test cases. The package build won't. Is this what you expected? Kind regards, Bo Thorsen, Monty Program AB.

1 0

Re: [Maria-developers] Windows test platforms
by Bo Thorsen 26 Apr '10

26 Apr '10

Monday 26 April 2010 11:25:53 Michael Widenius wrote: > Hi! > > >>>>> "Bo" == Bo Thorsen <bo(a)askmonty.org> writes: > > <cut> > > Bo> For building the installable packages, I intend to use XP and Visual > Studio Bo> 2008. > > We should look at using Visual Studio 2010 ASAP, as this is said to > give significantly better performance. The only reason I didn't choose VS2010 is that it's only beta at this moment. When it's out of beta, I agree we should use this instead. However, having a test buildbot system with the 2010 beta is probably a good idea, to get rid of compiler problems from it. > Bo> My list of systems to test the packages and do the compile + make test > on Bo> would be: > > Bo> XP Pro 32 bit > Bo> Vista 32 bit > Bo> Windows 7 64 bit > Bo> Server 2008 64 bit > > Bo> On XP, I'd install VC++ 2005, on the others VC++ 2008. When Visual > Studio 2010 Bo> is out, I'll install that on Vista and one of the 64 bit > systems. > > Bo> This is four systems that does a very thin span of the incredible > amount of Bo> systems done by MS. (And people say there are a lot of Linux > distributions.) > > Bo> Does this make sense to you? Please let me know what you think. > > Sounds good. > > We should probably start with trying to get packages for Windows 7, 64 > bit. I hope we can have just two packages - 32 and 64 bit - that will install on all Windows systems. There might be differences between the systems that have to be handled in the packages, but I don't want to have two packages per Windows version. That would be *really* bad. Kind regards, Bo Thorsen, Monty Program AB.

1 0

[Maria-developers] Storage Engine Summit 2010
by Sergei Golubchik 25 Apr '10

25 Apr '10

Hi, all! We are happy to announce the *Storage Engine Summit* 2010. This traditional one-day meeting of Storage Engine vendors/authors takes place yearly right after the User Conference. This year it will be held on Friday, April 16h. Invited are all MySQL/MariaDB Storage Engine vendors and anyone interested in the MySQL/MariaDB Storage Engine API. Please register in advance, the space is limited. We will talk about new developments in the area of the MySQL/MariaDB Storage Engine API - what was implemented recently, what we are working on - and about new directions of the development, what needs to be implemented to make the life of Storage Engine authors easier. See http://askmonty.org/wiki/Storage_Engine_Summit_2010 for details. Regards, Serg

2 2

[Maria-developers] New (by Serg): innodb statistics in the slow log (115)
by worklog-noreply＠askmonty.org 25 Apr '10

25 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: innodb statistics in the slow log CREATION DATE..: Sun, 25 Apr 2010, 16:35 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 115 (http://askmonty.org/worklog/?tid=115) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: We need to find some way for innoDB/XtraDB specific information to appear in the slow log. Same effect as in Percona patches, but a with cleaner implementation. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Serg): innodb statistics in the slow log (115)
by worklog-noreply＠askmonty.org 25 Apr '10

25 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: innodb statistics in the slow log CREATION DATE..: Sun, 25 Apr 2010, 16:35 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 115 (http://askmonty.org/worklog/?tid=115) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: We need to find some way for innoDB/XtraDB specific information to appear in the slow log. Same effect as in Percona patches, but a with cleaner implementation. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Serg): insert ignore ha_extra hint (114)
by worklog-noreply＠askmonty.org 25 Apr '10

25 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: insert ignore ha_extra hint CREATION DATE..: Sun, 25 Apr 2010, 16:32 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 114 (http://askmonty.org/worklog/?tid=114) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: There is HA_EXTRA_WRITE_CAN_REPLACE hint that tells the engine that the following ::write_row() calls are part of the REPLACE statement, not INSERT. With this knowledge the engine can execute the replace internally, deleting the conflicting row in the ::write_row() method instead of returning an error. We need a similar HA_EXTRA_WRITE_CAN_IGNORE hint to allow engines to optimize INSERT IGNORE in a similar way. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Serg): insert ignore ha_extra hint (114)
by worklog-noreply＠askmonty.org 25 Apr '10

25 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: insert ignore ha_extra hint CREATION DATE..: Sun, 25 Apr 2010, 16:32 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 114 (http://askmonty.org/worklog/?tid=114) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: There is HA_EXTRA_WRITE_CAN_REPLACE hint that tells the engine that the following ::write_row() calls are part of the REPLACE statement, not INSERT. With this knowledge the engine can execute the replace internally, deleting the conflicting row in the ::write_row() method instead of returning an error. We need a similar HA_EXTRA_WRITE_CAN_IGNORE hint to allow engines to optimize INSERT IGNORE in a similar way. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Serg): support many clustered keys per table (113)
by worklog-noreply＠askmonty.org 25 Apr '10

25 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: support many clustered keys per table CREATION DATE..: Sun, 25 Apr 2010, 16:23 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 113 (http://askmonty.org/worklog/?tid=113) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: The server now assumes that there can be only one clustered key per table, and only primary key can be clustered. It's not true for certain storage engines, we need to remove this limitation ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0