March 2011 - developers - lists.mariadb.org

Re: [Maria-developers] Fwd: [Commits] Rev 2901: BNLH algorithm always used a full table scan over the joined table in file:///home/igor/maria/maria-5.3-mwl128-hashrange/
by Sergey Petrunya 02 Mar '11

02 Mar '11

Hello Igor, Here's the promised proof of deadcode: On Fri, Feb 11, 2011 at 08:49:37PM -0800, Igor Babaev wrote: > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2011-02-06 04:57:03 +0000 > +++ b/sql/sql_select.cc 2011-02-12 04:41:22 +0000 > @@ -8419,6 +8438,11 @@ > sort_by_tab->type= JT_ALL; > sort_by_tab->read_first_record= join_init_read_record; > } > + else if (sort_by_tab->type == JT_HASH_NEXT) > + { > + sort_by_tab->type= JT_HASH; > + sort_by_tab->read_first_record= join_init_read_record; > + } > } > break; > } Statement: the above else-if branch can never be taken, i.e. equality sort_by_tab->type == JT_HASH_NEXT (1) can never be true. Proof: First, let's see what JOIN_TABs can be in sort_by_tab. sort_by_tab is assigned in this statement: JOIN_TAB *sort_by_tab= join->group && join->simple_group && join->group_list ? join->join_tab+join->const_tables : join->get_sort_by_join_tab(); i.e. possible values are 1. join->join_tab+join->const_tables 2. join->get_sort_by_join_tab() get_sort_by_join_tab is defined in sql_select.h as: JOIN_TAB *get_sort_by_join_tab() { return (need_tmp || !sort_by_table || skip_sort_order || ((group || tmp_table_param.sum_func_count) && !group_list)) ? NULL : join_tab+const_tables; } i.e. it can return: 1. NULL (which is not interesting) 2. join->join_tab + const_tables. this draws us to conclusion that sort_by_tab can be - NULL - join->join_tab + const_tables. so equality (1) can be rewritten as join->join_tab[const_tables].type == JT_HASH_NEXT (2) Now, let's explore the question, what join_tab elements can have type==JT_HASH_NEXT. The value JT_HASH_NEXT is assigned in only one place: sql_select.cc|8399| tab->type= tab->type == JT_ALL ? JT_NEXT : JT_HASH_NEXT; this line is inside the case block of code: break; case JT_ALL: case JT_HASH: .... tab->type= tab->type == JT_ALL ? JT_NEXT : JT_HASH_NEXT; which means that tab->type can become JT_HASH_NEXT only if it was JT_HASH before. Let's see when a join_tab can get type==JT_HASH. There's only one assignment that can assign type=JT_HASH. It is at sql_select.cc:8279 : if (tab->cache && tab->cache->get_join_alg() == JOIN_CACHE::BNLH_JOIN_ALG) tab->type= JT_HASH; i.e. tab->type can become JT_HASH only if tab->cache!=NULL. tab->cache can be set to non-NULL value only inside check_join_cache_usage(). However, the first lines in check_join_cache_usage() have this code: uint i= tab - join->join_tab; join->return_tab= 0; if (cache_level == 0 || i == join->const_tables || !prev_tab) return 0; from which we can conclude that join->join_tab[join->const_tables].cache == NULL will always hold. Going back on our chain of reasoning, we get that join->join_tab[join->const_tables].cache == NULL at all times=> join->join_tab[join->const_tables].type != JT_HASH at all times => join->join_tab[join->const_tables].type != JT_HASH_NEXT at all times => equality (2) is false always => original equality (1) is false always, q.e.d. BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Igor please review: BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) in file:///home/psergey/dev2/5.3/]
by Sergey Petrunya 02 Mar '11

02 Mar '11

On Wed, Mar 02, 2011 at 08:04:39AM -0800, Igor Babaev wrote: > > +SELECT alias2.f1 AS field1 , alias1.f2 AS field2 > > +FROM t1 AS alias1 > > +LEFT OUTER JOIN t1 AS alias2 ON alias1.f2 = alias2.f5 > > +WHERE alias2.f3 > 'k' > > +AND alias2.f3 <= 'z' > > +OR alias2.f4 IN ('o', 'y') > > +AND alias2.f4 >= 'g' AND alias2.f4 < 'z'; > > +field1 field2 > > +Warnings: > > +Warning 1292 Truncated incorrect DOUBLE value: 'k' > > Why do we need comparison of a string with an integer that generates the > above warning? Can we do without it? Why not to use char(4) instead > if integer? Got rid of string vs integer comparison, as well as simplified the testcase a bit (further simplification is difficult). Please find the result below: ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: commits(a)mariadb.org X-Mailer: mail (GNU Mailutils 1.2) Date: Wed, 2 Mar 2011 17:06:15 +0300 (MSK) Subject: [Commits] Rev 2929: BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) in file:///home/psergey/dev2/5.3/ At file:///home/psergey/dev2/5.3/ ------------------------------------------------------------ revno: 2929 revision-id: psergey(a)askmonty.org-20110302140613-4qt8xvdvxs4t4m98 parent: psergey(a)askmonty.org-20110301072222-1rmapkelx9l1kw8u committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: 5.3 timestamp: Wed 2011-03-02 17:06:13 +0300 message: BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) - Add debugging aid which maps MRR's range_ids (which are pointers so are different on every execution and hard to track) to range numbers (which allow much easier debugging). The mapper is not compiled unless the developer puts a #define into code to request it. - Fix the bug: the problem was that Mrr_ordered_index_reader's interrupt_read() and resume_read() would save and restore 1) index tuple 2) the rowid (as bytes returned by handler->position()). Clustered primary key columns were not saved/restored. They are not explicitly present in the index tuple (i.e. table->key_info[secondary_key].key_parts doesn't list them), but they are actually there, in particular table->field[clustered_primary_key_member].part_of_key(secondary_key) == 1. Index condition pushdown code [correctly] uses the latter as inidication that pushed index condition can refer to clustered PK members. The fix was to make interrupt_read()/resume_read() to save/restore clustered primary key members as well, so that we get correct values for them when evaluating pushed index condition. [Second attempt, make the code compile without the debugging aid] === modified file 'mysql-test/r/innodb_mrr.result' --- a/mysql-test/r/innodb_mrr.result 2011-01-17 21:26:04 +0000 +++ b/mysql-test/r/innodb_mrr.result 2011-03-02 14:06:13 +0000 @@ -736,3 +736,31 @@ c1 c2 c3 08:29:45 NULL 2009-02-01 drop table `t1`; +# +# BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) +# +set @_save_join_cache_level= @@join_cache_level; +set join_cache_level = 6; +CREATE TABLE t1 ( +f1 int(11), f2 int(11), f3 varchar(1), f4 varchar(1), +PRIMARY KEY (f1), +KEY (f3), +KEY (f2) +) ENGINE=InnoDB; +INSERT INTO t1 VALUES ('11','8','f','f'),('12','5','v','v'),('13','8','s','s'), +('14','8','a','a'),('15','6','p','p'),('16','7','z','z'),('17','2','a','a'), +('18','5','h','h'),('19','7','h','h'),('20','2','v','v'),('21','9','v','v'), +('22','142','b','b'),('23','3','y','y'),('24','0','v','v'),('25','3','m','m'), +('26','5','z','z'),('27','9','n','n'),('28','1','d','d'),('29','107','a','a'); +select count(*) from ( +SELECT alias1.f2 +FROM +t1 AS alias1 JOIN ( +t1 AS alias2 FORCE KEY (f3) JOIN +t1 AS alias3 FORCE KEY (f2) ON alias3.f2 = alias2.f2 AND alias3.f4 = alias2.f3 +) ON alias3.f1 <= alias2.f1 +) X; +count(*) +361 +set join_cache_level=@_save_join_cache_level; +drop table t1; === modified file 'mysql-test/t/innodb_mrr.test' --- a/mysql-test/t/innodb_mrr.test 2011-01-17 21:26:04 +0000 +++ b/mysql-test/t/innodb_mrr.test 2011-03-02 14:06:13 +0000 @@ -426,3 +426,31 @@ SELECT * FROM t1 WHERE c2 <=> NULL ORDER BY c2 LIMIT 2; drop table `t1`; +--echo # +--echo # BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) +--echo # +set @_save_join_cache_level= @@join_cache_level; +set join_cache_level = 6; +CREATE TABLE t1 ( + f1 int(11), f2 int(11), f3 varchar(1), f4 varchar(1), + PRIMARY KEY (f1), + KEY (f3), + KEY (f2) +) ENGINE=InnoDB; +INSERT INTO t1 VALUES ('11','8','f','f'),('12','5','v','v'),('13','8','s','s'), +('14','8','a','a'),('15','6','p','p'),('16','7','z','z'),('17','2','a','a'), +('18','5','h','h'),('19','7','h','h'),('20','2','v','v'),('21','9','v','v'), +('22','142','b','b'),('23','3','y','y'),('24','0','v','v'),('25','3','m','m'), +('26','5','z','z'),('27','9','n','n'),('28','1','d','d'),('29','107','a','a'); + +select count(*) from ( + SELECT alias1.f2 + FROM + t1 AS alias1 JOIN ( + t1 AS alias2 FORCE KEY (f3) JOIN + t1 AS alias3 FORCE KEY (f2) ON alias3.f2 = alias2.f2 AND alias3.f4 = alias2.f3 + ) ON alias3.f1 <= alias2.f1 +) X; + +set join_cache_level=@_save_join_cache_level; +drop table t1; === modified file 'sql/multi_range_read.cc' --- a/sql/multi_range_read.cc 2011-02-18 22:31:01 +0000 +++ b/sql/multi_range_read.cc 2011-03-02 14:06:13 +0000 @@ -387,15 +387,25 @@ bool Mrr_ordered_index_reader::set_interruption_temp_buffer(uint rowid_length, uint key_len, + uint saved_pk_len, uchar **space_start, uchar *space_end) { - if (space_end - *space_start <= (ptrdiff_t)(rowid_length + key_len)) + if (space_end - *space_start <= (ptrdiff_t)(rowid_length + key_len + saved_pk_len)) return TRUE; support_scan_interruptions= TRUE; saved_rowid= *space_start; *space_start += rowid_length; + + if (saved_pk_len) + { + saved_primary_key= *space_start; + *space_start += saved_pk_len; + } + else + saved_primary_key= NULL; + saved_key_tuple= *space_start; *space_start += key_len; @@ -406,17 +416,25 @@ void Mrr_ordered_index_reader::set_no_interruption_temp_buffer() { support_scan_interruptions= FALSE; - saved_key_tuple= saved_rowid= NULL; /* safety */ + saved_key_tuple= saved_rowid= saved_primary_key= NULL; /* safety */ have_saved_rowid= FALSE; } void Mrr_ordered_index_reader::interrupt_read() { DBUG_ASSERT(support_scan_interruptions); + TABLE *table= file->get_table(); /* Save the current key value */ - key_copy(saved_key_tuple, file->get_table()->record[0], - &file->get_table()->key_info[file->active_index], + key_copy(saved_key_tuple, table->record[0], + &table->key_info[file->active_index], keypar.key_tuple_length); + + if (saved_primary_key) + { + key_copy(saved_primary_key, table->record[0], + &table->key_info[table->s->primary_key], + table->key_info[table->s->primary_key].key_length); + } /* Save the last rowid */ memcpy(saved_rowid, file->ref, file->ref_length); @@ -433,9 +451,16 @@ void Mrr_ordered_index_reader::resume_read() { - key_restore(file->get_table()->record[0], saved_key_tuple, - &file->get_table()->key_info[file->active_index], + TABLE *table= file->get_table(); + key_restore(table->record[0], saved_key_tuple, + &table->key_info[file->active_index], keypar.key_tuple_length); + if (saved_primary_key) + { + key_restore(table->record[0], saved_primary_key, + &table->key_info[table->s->primary_key], + table->key_info[table->s->primary_key].key_length); + } } @@ -738,8 +763,8 @@ @retval other Error */ -int DsMrr_impl::dsmrr_init(handler *h_arg, RANGE_SEQ_IF *seq_funcs, - void *seq_init_param, uint n_ranges, uint mode, +int DsMrr_impl::dsmrr_init(handler *h_arg, RANGE_SEQ_IF *seq_funcs_arg, + void *seq_init_param_arg, uint n_ranges, uint mode, HANDLER_BUFFER *buf) { THD *thd= current_thd; @@ -748,6 +773,8 @@ uint key_buff_elem_size; handler *h_idx; Mrr_ordered_rndpos_reader *disk_strategy= NULL; + RANGE_SEQ_IF seq_funcs= *seq_funcs_arg; + void *seq_init_param= seq_init_param_arg; bool do_sort_keys= FALSE; DBUG_ENTER("DsMrr_impl::dsmrr_init"); LINT_INIT(key_buff_elem_size); /* set/used when do_sort_keys==TRUE */ @@ -810,8 +837,8 @@ { /* Pre-calculate some parameters of key sorting */ keypar.use_key_pointers= test(mode & HA_MRR_MATERIALIZED_KEYS); - seq_funcs->get_key_info(seq_init_param, &keypar.key_tuple_length, - &keypar.key_tuple_map); + seq_funcs.get_key_info(seq_init_param, &keypar.key_tuple_length, + &keypar.key_tuple_map); keypar.key_size_in_keybuf= keypar.use_key_pointers? sizeof(char*) : keypar.key_tuple_length; key_buff_elem_size= keypar.key_size_in_keybuf + (int)is_mrr_assoc * sizeof(void*); @@ -819,9 +846,18 @@ /* Ordered index reader needs some space to store an index tuple */ if (strategy != index_strategy) { + /* Normally we don't need to save/restore PK columns*/ + uint saved_pk_length=0; + if (h_idx->primary_key_is_clustered()) + { + uint pk= h_idx->get_table()->s->primary_key; + saved_pk_length= h_idx->get_table()->key_info[pk].key_length; + } + if (reader_factory.ordered_index_reader. set_interruption_temp_buffer(primary_file->ref_length, keypar.key_tuple_length, + saved_pk_length, &full_buf, full_buf_end)) goto use_default_impl; } @@ -845,7 +881,9 @@ if (do_sort_keys && !key_buffer->have_space_for(key_buff_elem_size)) goto use_default_impl; - if ((res= index_strategy->init(primary_file, seq_funcs, seq_init_param, n_ranges, + MRR_DEBUG(setup_range_id_mapping(&seq_funcs, &seq_init_param, n_ranges)); + + if ((res= index_strategy->init(primary_file, &seq_funcs, seq_init_param, n_ranges, mode, &keypar, key_buffer, &buf_manager))) goto error; } @@ -873,7 +911,9 @@ if ((res= setup_two_handlers())) goto error; - if ((res= index_strategy->init(secondary_file, seq_funcs, seq_init_param, + MRR_DEBUG(setup_range_id_mapping(&seq_funcs, &seq_init_param, n_ranges)); + + if ((res= index_strategy->init(secondary_file, &seq_funcs, seq_init_param, n_ranges, mode, &keypar, key_buffer, &buf_manager)) || (res= disk_strategy->init(primary_file, index_strategy, mode, @@ -883,6 +923,7 @@ } } + res= strategy->refill_buffer(TRUE); if (res) { @@ -910,7 +951,7 @@ DBUG_ASSERT(primary_file->inited == handler::INDEX); /* Call correct init function and assign to top level object */ Mrr_simple_index_reader *s= &reader_factory.simple_index_reader; - res= s->init(primary_file, seq_funcs, seq_init_param, n_ranges, mode, NULL, + res= s->init(primary_file, &seq_funcs, seq_init_param, n_ranges, mode, NULL, NULL, NULL); strategy= s; DBUG_RETURN(res); @@ -1033,6 +1074,7 @@ DBUG_ENTER("DsMrr_impl::dsmrr_close"); close_second_handler(); strategy= NULL; + MRR_DEBUG(clear_range_id_mapping()); DBUG_VOID_RETURN; } @@ -1690,7 +1732,110 @@ DBUG_VOID_RETURN; } - +#ifdef DBUG_MRR +range_seq_t DsMrr_impl::hook_init(void *init_params, uint n_ranges, uint flags) +{ + DsMrr_impl *mrr= (DsMrr_impl*)init_params; + mrr->backend_seq= mrr->backend_init(mrr->backend_init_params, n_ranges, flags); + return (range_seq_t)mrr; +} + +bool DsMrr_impl::hook_next(range_seq_t seq, KEY_MULTI_RANGE *range) +{ + DsMrr_impl *mrr= (DsMrr_impl*)seq; + int res; + if (!(res= mrr->backend_next(mrr->backend_seq, range))) + { + DBUG_ASSERT(mrr->cur_range_no < mrr->n_range_ids); + mrr->range_ids[mrr->cur_range_no++]= range->ptr; + } + return res; +} + +void DsMrr_impl::hook_get_key_info(void *init_params, uint *length, key_part_map *map) +{ + DsMrr_impl *mrr= (DsMrr_impl*)init_params; + mrr->backend_get_key_info(mrr->backend_init_params, length, map); +} + +bool DsMrr_impl::hook_skip_index_tuple(range_seq_t seq, char *range_info) +{ + DsMrr_impl *mrr= (DsMrr_impl*)seq; + int rangeno= mrr->range_id_to_number(range_info); + bool res= mrr->backend_skip_index_tuple(mrr->backend_seq, range_info); + fprintf(stderr, "checking index tuple %d -> %d\n", + rangeno, + (int)res); + return res; +} + +bool DsMrr_impl::hook_skip_record(range_seq_t seq, char *range_info, uchar *rowid) +{ + DsMrr_impl *mrr= (DsMrr_impl*)seq; + bool res= mrr->backend_skip_record(mrr->backend_seq, range_info, rowid); + return res; +} + +void DsMrr_impl::setup_range_id_mapping(RANGE_SEQ_IF *rseq, void **init_params, uint n_ranges) +{ + /* + Hook ourselves into RANGE_SEQ_IF so that we can watch ranges as they are + enumerated + */ + backend_init= rseq->init; + rseq->init= hook_init; + + backend_next= rseq->next; + rseq->next= hook_next; + + backend_init_params= *init_params; + *init_params= (void*)this; + + if (rseq->get_key_info) + { + backend_get_key_info= rseq->get_key_info; + rseq->get_key_info= hook_get_key_info; + } + + if (rseq->skip_index_tuple) + { + backend_skip_index_tuple= rseq->skip_index_tuple; + rseq->skip_index_tuple= hook_skip_index_tuple; + } + + if (rseq->skip_record) + { + backend_skip_record= rseq->skip_record; + rseq->skip_record= hook_skip_record; + } + + /* Alloc space for the mapping */ + n_range_ids= n_ranges; + if (n_ranges) + range_ids= (char**)malloc(sizeof(char*) * n_ranges); + else + range_ids= NULL; + cur_range_no= 0; +} + +void DsMrr_impl::clear_range_id_mapping() +{ + if (range_ids) + free(range_ids); + range_ids= NULL; + n_range_ids= 0; +} + +int DsMrr_impl::range_id_to_number(char *range_id) +{ + for (int i=0; i < n_range_ids; i++) + { + if (range_ids[i] == range_id) + return i; + } + return -1; +} +#endif /* ************************************************************************** * DS-MRR implementation ends ***************************************************************************/ === modified file 'sql/multi_range_read.h' --- a/sql/multi_range_read.h 2010-12-20 11:40:12 +0000 +++ b/sql/multi_range_read.h 2011-03-02 14:06:13 +0000 @@ -271,7 +271,8 @@ mrr_funcs.skip_index_tuple(mrr_iter, range_info)); } - bool set_interruption_temp_buffer(uint rowid_length, uint key_len, + bool set_interruption_temp_buffer(uint rowid_length, uint key_len, + uint saved_pk_len, uchar **space_start, uchar *space_end); void set_no_interruption_temp_buffer(); @@ -314,6 +315,7 @@ bool have_saved_rowid; uchar *saved_key_tuple; + uchar *saved_primary_key; static int compare_keys(void* arg, uchar* key1, uchar* key2); static int compare_keys_reverse(void* arg, uchar* key1, uchar* key2); @@ -520,7 +522,12 @@ typedef void (handler::*range_check_toggle_func_t)(bool on); DsMrr_impl() - : secondary_file(NULL) {}; + : secondary_file(NULL) + { +#ifdef DBUG_MRR + range_id_mapping_ctor(); +#endif + } void init(handler *h_arg, TABLE *table_arg) { @@ -615,6 +622,52 @@ int setup_two_handlers(); void close_second_handler(); + +#ifdef DBUG_MRR +#define MRR_DEBUG(x) x + /* + Debugging aid: support for converting range_ids (which are pointers) to + range numbers. Range's number specifies when the range was returned by + RANGE_SEQ_IF::next(). First returned range gets number 0, second gets + number 1, and so on. + */ + char **range_ids; /* range_ids[N] == range_id of range with number N */ + int n_range_ids; /* Total number of range ids */ + + /* "current" range number (makes sense when walking the ranges) */ + int cur_range_no; + + + /* RANGE_SEQ_IF override: "backend" functions */ + range_seq_t (*backend_init)(void *init_params, uint n_ranges, uint flags); + bool (*backend_next)(range_seq_t seq, KEY_MULTI_RANGE *range); + void (*backend_get_key_info)(void *init_params, uint *length, key_part_map *map); + bool (*backend_skip_index_tuple)(range_seq_t seq, char *range_info); + bool (*backend_skip_record)(range_seq_t seq, char *range_info, uchar *rowid); + void *backend_init_params; + range_seq_t backend_seq; + + /* RANGE_SEQ_IF override: our hook-functions */ + static range_seq_t hook_init(void *init_params, uint n_ranges, uint flags); + static bool hook_next(range_seq_t seq, KEY_MULTI_RANGE *range); + static void hook_get_key_info(void *init_params, uint *length, key_part_map *map); + static bool hook_skip_index_tuple(range_seq_t seq, char *range_info); + static bool hook_skip_record(range_seq_t seq, char *range_info, uchar *rowid); + + /* Mapper [de]initialization functions*/ + void range_id_mapping_ctor() + { + range_ids= NULL; + n_range_ids= 0; + } + void setup_range_id_mapping(RANGE_SEQ_IF *rseq, void **init_params, uint n_ranges); + void clear_range_id_mapping(); + + /* The reason of all of the above: function to map range ids to range numbers */ + int range_id_to_number(char *range_id); +#else +#define MRR_DEBUG(x) +#endif }; /** _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message ----- -- BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 2905: Merge Percona patch MWL#47 into mariadb 5.2-percona. in http://bazaar.launchpad.net/~maria-captains/maria/5.2
by Kristian Nielsen 02 Mar '11

02 Mar '11

knielsen(a)knielsen-hq.org writes: > message: > Merge Percona patch MWL#47 into mariadb 5.2-percona. > > This patch adds options to annotate the binlog (and the mysqlbinlog > output) with the original SQL query for queries that are logged > using row-based replication. Serg raised the very relevant issue about what to do about the similar feature in MySQL 5.6: http://forge.mysql.com/worklog/task.php?id=4033 http://forge.mysql.com/worklog/task.php?id=5404 The Oracle team has worked more on this since last I looked, and the two features are now very similar. Here are the main differences, as I see it: - The MySQL 5.6 feature is enabled by --binlog_rows_query_log_events, while the MWL#47 option is called --binlog-annotate-rows-events. - MWL#47 has --replicate-annotate-rows-events which makes the slave receive and binlog the annotations. The MySQL feature always receives annotation events if present in the binlog (so replication 5.6->pre-5.6 will break if annotations are enabled). The MySQL feature binlogs the annotations based on the same --binlog-annotate-rows-events option as the master. - MWL#47 has a mysqlbinlog option --skip-annotate-rows-events, MySQL 5.6 does not. - MySQL 5.6 mysqlbinlog can log the annotations with BINLOG statements so that they are preserved with `mysqlbinlog | mysql`; MWL#47 does not have this. - MWL#47 uses binlog event type 160, MySQL 5.6 uses an ignorable event of type 29. As far as I can tell, the actual event format is identical, except for the LOG_EVENT_IGNORABLE_F flag (which says it's ok for a slave to ignore this event if it doesn't know about it, however there is no server version that knows the LOG_EVENT_IGNORABLE_F but not the MySQL annotation event). - I think there are a couple of issues with the MySQL patch (newlines in queries breaking mysqlbinlog output, duplicated annotations for INSERT DELAYED). There might be issues with the MWL#47 patch also, of course. So the question is how to prepare for when we merge MySQL 5.6. An easy option could be to just ditch MWL#47 (and possibly backport the MySQL 5.6 patch). However, as I understand it, the original sponsor of MWL#47 is already using it, and so we may have to remain compatible with the existing MWL#47 implementation. Can we get some definite information on that? Another option might be to simply change the MWL#47 to use the same event type as MySQL (the event format is the same, and we can add the LOG_EVENT_IGNORABLE_F). And replace the MWL#47 options --binlog-annotate-rows-events and --replicate-annotate-rows-events with the MySQL --binlog_rows_query_log_events option (though I think the MySQL option name is not good). But again, I am not sure if this is a valid option given possible existing usage of MWL#47. The third option I can think of is to leave MWL#47 as it is. Then when we get to merge MySQL 5.6, we will basically drop the MySQL part, doing the following to remain backwards compatible with MySQL: - Make the --binlog_rows_query_log_events option set --binlog-annotate-rows-events and --replicate-annotate-rows-events. - Make a slave read a MySQL rows_query_log_event as a MWL#47 annotation event. - Merge over the bit of the MySQL patch that makes mysqlbinlog output annotation events using a BINLOG '...' statement. This should make upgrade from MySQL 5.6 -> MariaDB fully work. Using MySQL5.6+ with MariaDB will also work, only MySQL slaves will not receive MariaDB master annotation events. The third option seems to preserve the existing MWL#47 most faithfully and does not seem to need much work in terms of the eventual merge, so this is what I suggest as the best way forward. But I am open to other (justified) opinions. - Kristian.

2 1

Re: [Maria-developers] [Commits] Rev 3014: New table creation/renaming block added if old encoded table present. in file:///home/bell/maria/bzr/work-maria-5.1-CREATE/
by Sergei Golubchik 02 Mar '11

02 Mar '11

Hi, sanja! On Jan 13, sanja(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 3014 > revision-id: sanja(a)askmonty.org-20110113180607-e5hc8zuugxxjtn6k > parent: psergey(a)askmonty.org-20110112083018-90rw058f00uslvoz > committer: sanja(a)askmonty.org > branch nick: work-maria-5.1-CREATE > timestamp: Thu 2011-01-13 20:06:07 +0200 > message: > New table creation/renaming block added if old encoded table present. you've extended the fix, but not the test case. please add a test case for the new code and ok to push Regards, Sergei

1 0

[Maria-developers] Igor please review: Rev 2929: BUG#693747: Assertion multi_range_read.cc:908: int DsMrr_impl::dsmrr_init in file:///home/psergey/dev2/5.3-2/]
by Sergey Petrunya 02 Mar '11

02 Mar '11

Hello Igor, Could you please review the below: ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: commits(a)mariadb.org X-Mailer: mail (GNU Mailutils 1.2) Date: Wed, 2 Mar 2011 00:17:53 +0300 (MSK) Subject: [Commits] Rev 2929: BUG#693747: Assertion multi_range_read.cc:908: int DsMrr_impl::dsmrr_init in file:///home/psergey/dev2/5.3-2/ At file:///home/psergey/dev2/5.3-2/ ------------------------------------------------------------ revno: 2929 revision-id: psergey(a)askmonty.org-20110301211747-8wvngodwxzr9i2e9 parent: psergey(a)askmonty.org-20110301072222-1rmapkelx9l1kw8u committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: 5.3-2 timestamp: Wed 2011-03-02 00:17:47 +0300 message: BUG#693747: Assertion multi_range_read.cc:908: int DsMrr_impl::dsmrr_init - Make DsMrr_impl::dsmrr_init() handle the case of 1. 1st MRR scan using DS-MRR strategy (i.e. doing key sorting and rowid sorting) 2. 2nd MRR scan getting a buffer that's too small to fit one key element and one rowid element, and so falling back to default MRR implementation In this case, dsmrr_init() is invoked with {primary_handler, secondary_handler} initialized for DS-MRR scan and have to reset them to be initialized for the default MRR scan. === modified file 'mysql-test/r/maria_mrr.result' --- a/mysql-test/r/maria_mrr.result 2010-12-02 11:10:52 +0000 +++ b/mysql-test/r/maria_mrr.result 2011-03-01 21:17:47 +0000 @@ -403,3 +403,30 @@ count(*) 0 drop table t1, t2; +# +# BUG#693747: Assertion multi_range_read.cc:908: int DsMrr_impl::dsmrr_init( +# +set @_save_join_cache_level= @@join_cache_level; +set @_save_join_buffer_size= @@join_buffer_size; +set join_cache_level=8; +set join_buffer_size=10000; +CREATE TABLE t1 ( +f1 int(11), f2 varchar(32) COLLATE latin1_swedish_ci, +f3 int(11), f4 varchar(1024) COLLATE utf8_bin, +f5 varchar(1024) COLLATE latin1_bin, +KEY (f5) +) ENGINE=Aria TRANSACTIONAL=0 ; +# Fill the table with some data +SELECT alias2.f1 AS field1 , alias1.f2 AS field2 +FROM t1 AS alias1 +LEFT OUTER JOIN t1 AS alias2 ON alias1.f2 = alias2.f5 +WHERE alias2.f3 > 'k' +AND alias2.f3 <= 'z' +OR alias2.f4 IN ('o', 'y') +AND alias2.f4 >= 'g' AND alias2.f4 < 'z'; +field1 field2 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'k' +set join_cache_level=@_save_join_cache_level; +set join_buffer_size=@_save_join_buffer_size; +drop table t1; === modified file 'mysql-test/t/maria_mrr.test' --- a/mysql-test/t/maria_mrr.test 2010-11-09 16:02:08 +0000 +++ b/mysql-test/t/maria_mrr.test 2011-03-01 21:17:47 +0000 @@ -125,3 +125,81 @@ table1.col_varchar_1024_latin1_key = table2.col_varchar_10_latin1 AND table1.pk<>0 ; drop table t1, t2; + +--echo # +--echo # BUG#693747: Assertion multi_range_read.cc:908: int DsMrr_impl::dsmrr_init( +--echo # +set @_save_join_cache_level= @@join_cache_level; +set @_save_join_buffer_size= @@join_buffer_size; + +set join_cache_level=8; +set join_buffer_size=10000; + +CREATE TABLE t1 ( + f1 int(11), f2 varchar(32) COLLATE latin1_swedish_ci, + f3 int(11), f4 varchar(1024) COLLATE utf8_bin, + f5 varchar(1024) COLLATE latin1_bin, + KEY (f5) +) ENGINE=Aria TRANSACTIONAL=0 ; + +--echo # Fill the table with some data +--disable_query_log +INSERT IGNORE INTO t1 VALUES +('0','cueikuirqr','0','okay','hcueikuirqrzflno'),('2009','her','0','ULWAH','ehcueikuirqrzfln'), +('0','YKAOE','0','of','qieehcueikuirqrz'),('2009','you\'re','0','k','nkqieehcueikuirq'), +('2003','b','0','bgsnkqieehcueiku','the'),('0','MGUDG','0','p','m'), +('0','UXAGU','0','elfhjawbwbgsnkqi','HZXVA'),('2004','bwbgsnkqie','0','x','something'), +('2007','s','0','okay','slelfhjawbwbgsnk'),('2007','the','0','or','if'), +('2008','TDLKE','0','mean','MGWNJ'),('2006','do','0','KGFVV','see'), +('2001','why','0','a','mean'),(NULL,'THKCG','0','no','YFLDY'), +(NULL,'x','0','mean','e'),('2007','yncitaeysb','0','WDOSC','tgyncitaeysbgucs'), +(NULL,'ZEOXX','0','awbwbgsnkqieehcu','jawbwbgsnkqieehc'),('2006','hjawbwbgsn','0','just','fhjawbwbgsnkqiee'), +('0','all','0','k','sbgucsgqslelfhja'),('2009','the','0','f','would'), +('2005','mtgyncitae','0','w','ISNQQ'),('2003','KNCUI','0','r','want'), +('2006','is','0','j','i'),('2000','out','0','o','jvcmjlmtgyncitae'), +('2001','it','0','txyjvcmjlmtgynci','you'),('2008','LHDIH','0','ZKFJK','txmtxyjvcmjlmtgy'), +('0','z','0','the','ntxmtxyjvcmjlmtg'),(NULL,'vyhnmvgmcn','0','PWEYL','AIGQK'), +('0','ytvyhnmvgm','0','gbytvyhnmvgmcntx','z'),('2002','t','0','just','on'), +('2002','xqegbytvyh','0','to','ixqegbytvyhnmvgm'),('2004','WGVRU','0','GHBZB','h'), +('0','b','0','yhnmvgmcntxmtxyj','z'),('2003','who','0','ddixqegbytvyhnmv','gddixqegbytvy'), +('0','PMLFL','0','n','vgmcntxmtxyjvcmj'),(NULL,'back','0','FIXPA','n'), +('2005','i','0','I\'ll','PZGUB'),('0','f','0','t','the'), +(NULL,'PNXVP','0','t','v'),('2004','MAKKL','0','u','CGCWF'), +('0','RMDAV','0','noypgddixqegbytv','v'),('0','l','0','ESAVW','n'), +('2008','rhnoypgddi','0','s','VIZNE'),('2007','t','0','XZECS','a'), +(NULL,'like','0','k','JSHPZ'),('2009','pskeywslmk','0','epskeywslmkdrhno','q'), +(NULL,'QZZJJ','0','LOTFB','c'),('0','atlxepskey','0','b','YJRMA'), +(NULL,'YUVOU','0','wslmkdrhnoypgddi','eywslmkdrhnoypgd'),('2006','some','0','can','r'), +('2002','c','0','k','her'),(NULL,'o','0','ALBVZ','EMURT'), +('0','if','0','okay','had'),('2006','when','0','e','CLVWT'), +(NULL,'blfufrcdjm','0','ZAKUQ','IZCZN'),('2007','vutblfufrc','0','MPKMI','how'), +(NULL,'why','0','UBUAG','I'),('2004','IXLYQ','0','ILPBE','weuwuvutblfufrcd'), +('0','here','0','your','m'),(NULL,'ZOCTJ','0','GTPOO','IDSFD'), +('0','kqsweuwuvu','0','CXQLW','oh'),('0','ykqsweuwuv','0','CLJWP','zykqsweuwuvutblf'), +('0','zezykqsweu','0','had','t'),(NULL,'q','0','jisuzezykqsweuwu','o'), +('2007','IBKAU','0','t','oh'),('0','ivjisuzezy','0','y','XHXKE'), +('0','xsivjisuze','0','right','plxsivjisuzezykq'),('2009','have','0','so','uvplxsivjisuzezy'), +('0','on','0','b','me'),('0','ijkfuvplxs','0','RZXDY','OGEHV'), +('0','u','0','all','okay'),('0','i','0','s','pajzbbojshnijkfu'), +('2001','of','0','xpajzbbojshnijkf','g'),('0','for','0','ITBAQ','I\'m'), +('2006','or','0','now','ZOJHX'),('0','n','0','k','you'), +(NULL,'that','0','suzezykqsweuwuvu','just'),('0','bbojshnijk','0','QGYOA','JYGSJ'), +('2009','k','0','ajzbbojshnijkfuv','y'),('2007','k','0','affpegvavdgxpajz','y'), +('2007','be','0','RCTIT','m'),('2000','fnbmxwicrk','0','lcfnbmxwicrkdyma','t'), +('0','yaffpegvav','0','v','have'),('0','crkdymahya','0','icrkdymahyaffpeg','QQWQI'), +('0','t','0','nijkfuvplxsivjis','hnijkfuvplxsivji'),('0','dgxpajzbbo','0','vdgxpajzbbojshni','vavdgxpajzbbojsh'), +('2006','g','0','gvavdgxpajzbbojs','pegvavdgxpajzbbo'),('2007','I\'m','0','g','ffpegvavdgxpajzb'); +--enable_query_log + + +SELECT alias2.f1 AS field1 , alias1.f2 AS field2 +FROM t1 AS alias1 +LEFT OUTER JOIN t1 AS alias2 ON alias1.f2 = alias2.f5 +WHERE alias2.f3 > 'k' +AND alias2.f3 <= 'z' +OR alias2.f4 IN ('o', 'y') +AND alias2.f4 >= 'g' AND alias2.f4 < 'z'; + +set join_cache_level=@_save_join_cache_level; +set join_buffer_size=@_save_join_buffer_size; +drop table t1; === modified file 'sql/multi_range_read.cc' --- a/sql/multi_range_read.cc 2011-02-18 22:31:01 +0000 +++ b/sql/multi_range_read.cc 2011-03-01 21:17:47 +0000 @@ -907,7 +907,26 @@ DBUG_RETURN(res); use_default_impl: - DBUG_ASSERT(primary_file->inited == handler::INDEX); + if (primary_file->inited != handler::INDEX) + { + /* We can get here when + - we've previously successfully done a DS-MRR scan (and so have + secondary_file!= NULL, secondary_file->inited= INDEX, + primary_file->inited=RND) + - for this invocation, we haven't got enough buffer space, and so we + have to use the default MRR implementation. + + note: primary_file->ha_index_end() will call dsmrr_close() which will + close/destroy the secondary_file, this is intentional. + (Yes this is slow, but one can't expect performance with join buffer + so small that it can accomodate one rowid and one index tuple) + */ + if ((res= primary_file->ha_rnd_end()) || + (res= primary_file->ha_index_init(keyno, test(mode & HA_MRR_SORTED)))) + { + DBUG_RETURN(res); + } + } /* Call correct init function and assign to top level object */ Mrr_simple_index_reader *s= &reader_factory.simple_index_reader; res= s->init(primary_file, seq_funcs, seq_init_param, n_ranges, mode, NULL, _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message ----- -- BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Igor please review: BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) in file:///home/psergey/dev2/5.3/]
by Sergey Petrunya 02 Mar '11

02 Mar '11

Hello Igor, Could you please review the below: ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: commits(a)mariadb.org X-Mailer: mail (GNU Mailutils 1.2) Date: Wed, 2 Mar 2011 16:27:43 +0300 (MSK) Subject: [Commits] Rev 2929: BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) in file:///home/psergey/dev2/5.3/ At file:///home/psergey/dev2/5.3/ ------------------------------------------------------------ revno: 2929 revision-id: psergey(a)askmonty.org-20110302132739-s6vasca1nm6eh2le parent: psergey(a)askmonty.org-20110301072222-1rmapkelx9l1kw8u committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: 5.3 timestamp: Wed 2011-03-02 16:27:39 +0300 message: BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) - Add debugging aid which maps MRR's range_ids (which are pointers so are different on every execution and hard to track) to range numbers (which allow much easier debugging). The mapper is not compiled unless the developer puts a #define into code to request it. - Fix the bug: the problem was that Mrr_ordered_index_reader's interrupt_read() and resume_read() would save and restore 1) index tuple 2) the rowid (as bytes returned by handler->position()). Clustered primary key columns were not saved/restored. They are not explicitly present in the index tuple (i.e. table->key_info[secondary_key].key_parts doesn't list them), but they are actually there, in particular table->field[clustered_primary_key_member].part_of_key(secondary_key) == 1. Index condition pushdown code [correctly] uses the latter as inidication that pushed index condition can refer to clustered PK members. The fix was to make interrupt_read()/resume_read() to save/restore clustered primary key members as well, so that we get correct values for them when evaluating pushed index condition. === modified file 'mysql-test/r/innodb_mrr.result' --- a/mysql-test/r/innodb_mrr.result 2011-01-17 21:26:04 +0000 +++ b/mysql-test/r/innodb_mrr.result 2011-03-02 13:27:39 +0000 @@ -736,3 +736,31 @@ c1 c2 c3 08:29:45 NULL 2009-02-01 drop table `t1`; +# +# BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) +# +set @_save_join_cache_level= @@join_cache_level; +set join_cache_level = 6; +CREATE TABLE t1 ( +f1 int(11), f2 int(11), f3 varchar(1), f4 varchar(1), +PRIMARY KEY (f1), +KEY (f3), +KEY (f2) +) ENGINE=InnoDB; +INSERT INTO t1 VALUES ('11','8','f','f'),('12','5','v','v'),('13','8','s','s'), +('14','8','a','a'),('15','6','p','p'),('16','7','z','z'),('17','2','a','a'), +('18','5','h','h'),('19','7','h','h'),('20','2','v','v'),('21','9','v','v'), +('22','142','b','b'),('23','3','y','y'),('24','0','v','v'),('25','3','m','m'), +('26','5','z','z'),('27','9','n','n'),('28','1','d','d'),('29','107','a','a'); +select count(*) from ( +SELECT alias1.f2 +FROM +t1 AS alias1 JOIN ( +t1 AS alias2 FORCE KEY (f3) JOIN +t1 AS alias3 FORCE KEY (f2) ON alias3.f2 = alias2.f2 AND alias3.f4 = alias2.f3 +) ON alias3.f1 <= alias2.f1 +) X; +count(*) +361 +set join_cache_level=@_save_join_cache_level; +drop table t1; === modified file 'mysql-test/t/innodb_mrr.test' --- a/mysql-test/t/innodb_mrr.test 2011-01-17 21:26:04 +0000 +++ b/mysql-test/t/innodb_mrr.test 2011-03-02 13:27:39 +0000 @@ -426,3 +426,31 @@ SELECT * FROM t1 WHERE c2 <=> NULL ORDER BY c2 LIMIT 2; drop table `t1`; +--echo # +--echo # BUG#707925: Wrong result with join_cache_level=6 optimizer_use_mrr = force (incremental, BKA join) +--echo # +set @_save_join_cache_level= @@join_cache_level; +set join_cache_level = 6; +CREATE TABLE t1 ( + f1 int(11), f2 int(11), f3 varchar(1), f4 varchar(1), + PRIMARY KEY (f1), + KEY (f3), + KEY (f2) +) ENGINE=InnoDB; +INSERT INTO t1 VALUES ('11','8','f','f'),('12','5','v','v'),('13','8','s','s'), +('14','8','a','a'),('15','6','p','p'),('16','7','z','z'),('17','2','a','a'), +('18','5','h','h'),('19','7','h','h'),('20','2','v','v'),('21','9','v','v'), +('22','142','b','b'),('23','3','y','y'),('24','0','v','v'),('25','3','m','m'), +('26','5','z','z'),('27','9','n','n'),('28','1','d','d'),('29','107','a','a'); + +select count(*) from ( + SELECT alias1.f2 + FROM + t1 AS alias1 JOIN ( + t1 AS alias2 FORCE KEY (f3) JOIN + t1 AS alias3 FORCE KEY (f2) ON alias3.f2 = alias2.f2 AND alias3.f4 = alias2.f3 + ) ON alias3.f1 <= alias2.f1 +) X; + +set join_cache_level=@_save_join_cache_level; +drop table t1; === modified file 'sql/multi_range_read.cc' --- a/sql/multi_range_read.cc 2011-02-18 22:31:01 +0000 +++ b/sql/multi_range_read.cc 2011-03-02 13:27:39 +0000 @@ -387,15 +387,25 @@ bool Mrr_ordered_index_reader::set_interruption_temp_buffer(uint rowid_length, uint key_len, + uint saved_pk_len, uchar **space_start, uchar *space_end) { - if (space_end - *space_start <= (ptrdiff_t)(rowid_length + key_len)) + if (space_end - *space_start <= (ptrdiff_t)(rowid_length + key_len + saved_pk_len)) return TRUE; support_scan_interruptions= TRUE; saved_rowid= *space_start; *space_start += rowid_length; + + if (saved_pk_len) + { + saved_primary_key= *space_start; + *space_start += saved_pk_len; + } + else + saved_primary_key= NULL; + saved_key_tuple= *space_start; *space_start += key_len; @@ -406,17 +416,25 @@ void Mrr_ordered_index_reader::set_no_interruption_temp_buffer() { support_scan_interruptions= FALSE; - saved_key_tuple= saved_rowid= NULL; /* safety */ + saved_key_tuple= saved_rowid= saved_primary_key= NULL; /* safety */ have_saved_rowid= FALSE; } void Mrr_ordered_index_reader::interrupt_read() { DBUG_ASSERT(support_scan_interruptions); + TABLE *table= file->get_table(); /* Save the current key value */ - key_copy(saved_key_tuple, file->get_table()->record[0], - &file->get_table()->key_info[file->active_index], + key_copy(saved_key_tuple, table->record[0], + &table->key_info[file->active_index], keypar.key_tuple_length); + + if (saved_primary_key) + { + key_copy(saved_primary_key, table->record[0], + &table->key_info[table->s->primary_key], + table->key_info[table->s->primary_key].key_length); + } /* Save the last rowid */ memcpy(saved_rowid, file->ref, file->ref_length); @@ -433,9 +451,16 @@ void Mrr_ordered_index_reader::resume_read() { - key_restore(file->get_table()->record[0], saved_key_tuple, - &file->get_table()->key_info[file->active_index], + TABLE *table= file->get_table(); + key_restore(table->record[0], saved_key_tuple, + &table->key_info[file->active_index], keypar.key_tuple_length); + if (saved_primary_key) + { + key_restore(table->record[0], saved_primary_key, + &table->key_info[table->s->primary_key], + table->key_info[table->s->primary_key].key_length); + } } @@ -738,8 +763,8 @@ @retval other Error */ -int DsMrr_impl::dsmrr_init(handler *h_arg, RANGE_SEQ_IF *seq_funcs, - void *seq_init_param, uint n_ranges, uint mode, +int DsMrr_impl::dsmrr_init(handler *h_arg, RANGE_SEQ_IF *seq_funcs_arg, + void *seq_init_param_arg, uint n_ranges, uint mode, HANDLER_BUFFER *buf) { THD *thd= current_thd; @@ -748,6 +773,8 @@ uint key_buff_elem_size; handler *h_idx; Mrr_ordered_rndpos_reader *disk_strategy= NULL; + RANGE_SEQ_IF seq_funcs= *seq_funcs_arg; + void *seq_init_param= seq_init_param_arg; bool do_sort_keys= FALSE; DBUG_ENTER("DsMrr_impl::dsmrr_init"); LINT_INIT(key_buff_elem_size); /* set/used when do_sort_keys==TRUE */ @@ -810,8 +837,8 @@ { /* Pre-calculate some parameters of key sorting */ keypar.use_key_pointers= test(mode & HA_MRR_MATERIALIZED_KEYS); - seq_funcs->get_key_info(seq_init_param, &keypar.key_tuple_length, - &keypar.key_tuple_map); + seq_funcs.get_key_info(seq_init_param, &keypar.key_tuple_length, + &keypar.key_tuple_map); keypar.key_size_in_keybuf= keypar.use_key_pointers? sizeof(char*) : keypar.key_tuple_length; key_buff_elem_size= keypar.key_size_in_keybuf + (int)is_mrr_assoc * sizeof(void*); @@ -819,9 +846,18 @@ /* Ordered index reader needs some space to store an index tuple */ if (strategy != index_strategy) { + /* Normally we don't need to save/restore PK columns*/ + uint saved_pk_length=0; + if (h_idx->primary_key_is_clustered()) + { + uint pk= h_idx->get_table()->s->primary_key; + saved_pk_length= h_idx->get_table()->key_info[pk].key_length; + } + if (reader_factory.ordered_index_reader. set_interruption_temp_buffer(primary_file->ref_length, keypar.key_tuple_length, + saved_pk_length, &full_buf, full_buf_end)) goto use_default_impl; } @@ -845,7 +881,9 @@ if (do_sort_keys && !key_buffer->have_space_for(key_buff_elem_size)) goto use_default_impl; - if ((res= index_strategy->init(primary_file, seq_funcs, seq_init_param, n_ranges, + MRR_DEBUG(setup_range_id_mapping(&seq_funcs, &seq_init_param, n_ranges)); + + if ((res= index_strategy->init(primary_file, &seq_funcs, seq_init_param, n_ranges, mode, &keypar, key_buffer, &buf_manager))) goto error; } @@ -873,7 +911,9 @@ if ((res= setup_two_handlers())) goto error; - if ((res= index_strategy->init(secondary_file, seq_funcs, seq_init_param, + MRR_DEBUG(setup_range_id_mapping(&seq_funcs, &seq_init_param, n_ranges)); + + if ((res= index_strategy->init(secondary_file, &seq_funcs, seq_init_param, n_ranges, mode, &keypar, key_buffer, &buf_manager)) || (res= disk_strategy->init(primary_file, index_strategy, mode, @@ -883,6 +923,7 @@ } } + res= strategy->refill_buffer(TRUE); if (res) { @@ -910,7 +951,7 @@ DBUG_ASSERT(primary_file->inited == handler::INDEX); /* Call correct init function and assign to top level object */ Mrr_simple_index_reader *s= &reader_factory.simple_index_reader; - res= s->init(primary_file, seq_funcs, seq_init_param, n_ranges, mode, NULL, + res= s->init(primary_file, &seq_funcs, seq_init_param, n_ranges, mode, NULL, NULL, NULL); strategy= s; DBUG_RETURN(res); @@ -1033,6 +1074,7 @@ DBUG_ENTER("DsMrr_impl::dsmrr_close"); close_second_handler(); strategy= NULL; + MRR_DEBUG(clear_range_id_mapping()); DBUG_VOID_RETURN; } @@ -1690,7 +1732,110 @@ DBUG_VOID_RETURN; } - +#ifdef DBUG_MRR +range_seq_t DsMrr_impl::hook_init(void *init_params, uint n_ranges, uint flags) +{ + DsMrr_impl *mrr= (DsMrr_impl*)init_params; + mrr->backend_seq= mrr->backend_init(mrr->backend_init_params, n_ranges, flags); + return (range_seq_t)mrr; +} + +bool DsMrr_impl::hook_next(range_seq_t seq, KEY_MULTI_RANGE *range) +{ + DsMrr_impl *mrr= (DsMrr_impl*)seq; + int res; + if (!(res= mrr->backend_next(mrr->backend_seq, range))) + { + DBUG_ASSERT(mrr->cur_range_no < mrr->n_range_ids); + mrr->range_ids[mrr->cur_range_no++]= range->ptr; + } + return res; +} + +void DsMrr_impl::hook_get_key_info(void *init_params, uint *length, key_part_map *map) +{ + DsMrr_impl *mrr= (DsMrr_impl*)init_params; + mrr->backend_get_key_info(mrr->backend_init_params, length, map); +} + +bool DsMrr_impl::hook_skip_index_tuple(range_seq_t seq, char *range_info) +{ + DsMrr_impl *mrr= (DsMrr_impl*)seq; + int rangeno= mrr->range_id_to_number(range_info); + bool res= mrr->backend_skip_index_tuple(mrr->backend_seq, range_info); + fprintf(stderr, "checking index tuple %d -> %d\n", + rangeno, + (int)res); + return res; +} + +bool DsMrr_impl::hook_skip_record(range_seq_t seq, char *range_info, uchar *rowid) +{ + DsMrr_impl *mrr= (DsMrr_impl*)seq; + bool res= mrr->backend_skip_record(mrr->backend_seq, range_info, rowid); + return res; +} + +void DsMrr_impl::setup_range_id_mapping(RANGE_SEQ_IF *rseq, void **init_params, uint n_ranges) +{ + /* + Hook ourselves into RANGE_SEQ_IF so that we can watch ranges as they are + enumerated + */ + backend_init= rseq->init; + rseq->init= hook_init; + + backend_next= rseq->next; + rseq->next= hook_next; + + backend_init_params= *init_params; + *init_params= (void*)this; + + if (rseq->get_key_info) + { + backend_get_key_info= rseq->get_key_info; + rseq->get_key_info= hook_get_key_info; + } + + if (rseq->skip_index_tuple) + { + backend_skip_index_tuple= rseq->skip_index_tuple; + rseq->skip_index_tuple= hook_skip_index_tuple; + } + + if (rseq->skip_record) + { + backend_skip_record= rseq->skip_record; + rseq->skip_record= hook_skip_record; + } + + /* Alloc space for the mapping */ + n_range_ids= n_ranges; + if (n_ranges) + range_ids= (char**)malloc(sizeof(char*) * n_ranges); + else + range_ids= NULL; + cur_range_no= 0; +} + +void DsMrr_impl::clear_range_id_mapping() +{ + if (range_ids) + free(range_ids); + range_ids= NULL; + n_range_ids= 0; +} + +int DsMrr_impl::range_id_to_number(char *range_id) +{ + for (int i=0; i < n_range_ids; i++) + { + if (range_ids[i] == range_id) + return i; + } + return -1; +} +#endif /* ************************************************************************** * DS-MRR implementation ends ***************************************************************************/ === modified file 'sql/multi_range_read.h' --- a/sql/multi_range_read.h 2010-12-20 11:40:12 +0000 +++ b/sql/multi_range_read.h 2011-03-02 13:27:39 +0000 @@ -271,7 +271,8 @@ mrr_funcs.skip_index_tuple(mrr_iter, range_info)); } - bool set_interruption_temp_buffer(uint rowid_length, uint key_len, + bool set_interruption_temp_buffer(uint rowid_length, uint key_len, + uint saved_pk_len, uchar **space_start, uchar *space_end); void set_no_interruption_temp_buffer(); @@ -314,6 +315,7 @@ bool have_saved_rowid; uchar *saved_key_tuple; + uchar *saved_primary_key; static int compare_keys(void* arg, uchar* key1, uchar* key2); static int compare_keys_reverse(void* arg, uchar* key1, uchar* key2); @@ -520,7 +522,12 @@ typedef void (handler::*range_check_toggle_func_t)(bool on); DsMrr_impl() - : secondary_file(NULL) {}; + : secondary_file(NULL) + { +#ifdef DBUG_MRR + range_id_mapping_ctor(); +#endif + } void init(handler *h_arg, TABLE *table_arg) { @@ -615,6 +622,50 @@ int setup_two_handlers(); void close_second_handler(); + +#ifdef DBUG_MRR +#define MRR_DEBUG(x) x + /* + Debugging aid: support for converting range_ids (which are pointers) to + range numbers. Range's number specifies when the range was returned by + RANGE_SEQ_IF::next(). First returned range gets number 0, second gets + number 1, and so on. + */ + char **range_ids; /* range_ids[N] == range_id of range with number N */ + int n_range_ids; /* Total number of range ids */ + + /* "current" range number (makes sense when walking the ranges) */ + int cur_range_no; + + + /* RANGE_SEQ_IF override: "backend" functions */ + range_seq_t (*backend_init)(void *init_params, uint n_ranges, uint flags); + bool (*backend_next)(range_seq_t seq, KEY_MULTI_RANGE *range); + void (*backend_get_key_info)(void *init_params, uint *length, key_part_map *map); + bool (*backend_skip_index_tuple)(range_seq_t seq, char *range_info); + bool (*backend_skip_record)(range_seq_t seq, char *range_info, uchar *rowid); + void *backend_init_params; + range_seq_t backend_seq; + + /* RANGE_SEQ_IF override: our hook-functions */ + static range_seq_t hook_init(void *init_params, uint n_ranges, uint flags); + static bool hook_next(range_seq_t seq, KEY_MULTI_RANGE *range); + static void hook_get_key_info(void *init_params, uint *length, key_part_map *map); + static bool hook_skip_index_tuple(range_seq_t seq, char *range_info); + static bool hook_skip_record(range_seq_t seq, char *range_info, uchar *rowid); + + /* Mapper [de]initialization functions*/ + void range_id_mapping_ctor() + { + range_ids= NULL; + n_range_ids= 0; + } + void setup_range_id_mapping(RANGE_SEQ_IF *rseq, void **init_params, uint n_ranges); + void clear_range_id_mapping(); + + /* The reason of all of the above: function to map range ids to range numbers */ + int range_id_to_number(char *range_id); +#endif }; /** _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message ----- -- BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] dgcov.pl
by Kristian Nielsen 02 Mar '11

02 Mar '11

Hi Serg, I pushed to lp:mariadb-tools a "fix" for the negative numbers output by gcov which causes dgcov.pl to break. (Sergey Petrunya needed it). It just strips the leading '-' from negative numbers, so it does not help to understand what the problem is, I would still like to figure that out... but meanwhile, I hope you are ok with the workaround, or at least I wanted to make you aware of it. - Kristian.

1 0

Re: [Maria-developers] Release of MariaDB 5.1.55
by Daniel Bartholomew 02 Mar '11

02 Mar '11

On Mon, 28 Feb 2011 12:40:13 +0100 Kristian Nielsen <knielsen(a)knielsen-hq.org> wrote: Kristian> Monty gave the go for releasing MariaDB 5.1.55. Thanks Kristian for all of your work in getting the packages ready, especially the new .deb repository. And thank you to all of the developers who worked on this release! Here are the download, release notes, and changelog pages for 5.1.55: * http://downloads.askmonty.org/mariadb/5.1.55 * http://kb.askmonty.org/v/mariadb-5155-release-notes * http://kb.askmonty.org/v/mariadb-5155-changelog All: If you have a moment, please check over the above pages to see if there are any mistakes or omissions and let me know (or jump in and fix if you have editing rights). One brief note about the new downloads system: I'll also be adding some redirects to the old wiki-based downloads site to direct people to the new downloads site (in case they have the old site bookmarked). Community team: I tested a sample of the mirrors and they all had the files so please announce the release on the announce list, blogs, and so on. After sending out this email I will add an entry to the releases RSS feed. Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

2 1

[Maria-developers] Buildbot Update
by Adam M. Dutko 01 Mar '11

01 Mar '11

All of the buildbots I run should be back online.

2 2